Well I have been through this..
I actually followed this Q&A on Superuser, which took me to searching and installing an "MP4 ID3 tag editor". Well this basically ruined my video. It did produce the desired result of the thumbnail, but the sound was lost. Later I found it to be a stupid option to try and edit MP4 tag using an MP3 tag editor (mind it, the software description did list MP4 compatibility).
After lots of trial and error, I stumbled across MediaMonkey, it a Video Player and MP4 tag editor. It works simply well. There are two versions, one paid and free. The free works well for your purpose. It even automatically detects a image / cover in the folder of the video.
One tip, try find an image or crop the image into a square to get the desired output as illustrated in your question.