I had two problems that don't seem to be addressed anywhere.
A website project whose directory I'd renamed from ProjectB to ProjectA kept loading into a VS 2013 solution as ProjectB after the rename. F2, rt-click and long clicking the project name did not enable me to rename it. I have no csproj file. I deleted all non-source files then imported into a new, empty solution. A full text search of the directory showed the name only in the .suo file, which I'd deleted.
I fixed this (solution below), then imported a website in directory ProjectB into the solution. When I did, it was given the name ProjectB(2), ProjectB(3), ProjectB(4), despite that there was nothing called ProjectB any more. Again I couldn't manually rename to just ProjectB. There was no folder or file that conflicted. I even restarted Windows after deleting the temporary files, but the next time I imported the project, it came in as ProjectB(8). Solution below.
I then re-opened the same solution on another computer, and ProjectB (in directory ProjectB) was renamed to ProjectA(2), despite having never been named that anywhere. Solution below.