By design of Incognito, there is NO (repeat: NO) way to "find the windows I had open in that Incognito Mode before the computer closed it all down" (as in your question).
And if anybody knows some sort of work around for it, then that is to be considered as a bug ... which has become a feature.
Note: I'm assuming that you're looking for the complete URLs of each of those windows you had open. In that case, IMO running the cmd ipconfig/displaydns on a Windows PC (as suggested in the other answer to this question) does not seem to fully answer your question.
As a sample, just pick a random link, such as this new site proposal (disclosure: it's a proposal I endorse ... just pick any link you prefer to do a similar experiment). Then verify the results you get from the above command. Does it include the complete URL? FYI: not in my verification I just did. At best you get the domain name, but not the entire URL. Here is the relevant part of the result I got (which does not include the complete URL of the sample I mentioned above ...):
area51.stackexchange.com
----------------------------------------
Record Name . . . . . : area51.stackexchange.com
Record Type . . . . . : 1
Time To Live . . . . : 278
Data Length . . . . . : 4
Section . . . . . . . : Answer
A (Host) Record . . . : 151.101.1.69