Your program needs some .dlls to run: some are the system ones, some are shipped with the compiler, and some come from the libraries you use (SFML). You need to ship all those .dlls (except the system ones) with your .exe, and they should be in the same directory.
It doesn't really matter if you make a proper installer or send your friend a zip archive. (If it's an archive, they might have to manually extract it before running the .exe.)
The question is how to figure out which .dlls to ship. There are several approaches:
Open the console, cd to where your .exe is, do set PATH= and try running the executable by typing its name. Since the compiler installation is no longer in the PATH, it shouldn't see the .dlls in there, and it should complain about them being missing. After you provide one .dll, it will ask for the next one.
A more civilized approach is to use a tool like ntdll to list all .dlls your app uses. Then copy them, ignoring the system ones (located in C:\Windows or subdirectories).
Note that both approaches rely on there being no extraneous .dlls in C:\Windows or subdirectories; some poorly written installers like to put their own .dlls in there. To check for that, make a list of all .dlls that come with your compiler (they should be in the same directory as the gcc.exe), and the ones that come from your libraries (SFML). Then look for the .dlls with the same name in C:\Windows and subdirectories, and if you find any, remove them.