I struggled with this for a long time, but finally found solutions.
According to users, this Reddit post solved it for many users, but that's not what I used.
https://www.reddit.com/r/firefox/comments/s2k5tt/have_multiple_firefox_profile_and_taskbar_icons/?rdt=63276
Kenny's answer (just above mine, atm) looks quite interesting too, and I'd advise checking it.
Anyway, here's what I personally did, and wanted to share in case it may help :
Create a 256*256 image (Googled what I need, edited with Gimp...)
Create a custom icon using https://icoconvert.com/
I selected "custom sizes / multi size in one icon" and created a single icon file with all four 16x16, 32x32, 48x48 & 256x256 sized icons from the image I wanted to use.
Then, to use it in FF, I went to Firefox's directory, created some copies of the exe, and changed the icons in copies using Resource Hacker (found on Wikihow, probably here).
It's safe, but we're on Internet, so I'd advise you to check it with Virustotal anyway !
Operation was really simple. Just open the exe, go look for the "icon groups" and replace the icon in groups 32512 (taskbar icon) and 1 (file explorer icon).
Then save it (You should always work on copies, but resource hacker will also make a backup file for you).
Once you have as many different customized exes as you need for your sessions, just use usual shortcuts and launch them with "path-to-FF-exe.exe" -no-remote -P <yourProfileName> as you would usually do for using several session.
I assume a limitation of this technique is it may get messed-up by auto updates (I didn't test it), but I've been disabling them and updating manually at pretty long intervals for quite some time now.
Also, at the time I write this, I can't go past FF ESR 115 on Win7, so updates are definitely not an issue.
Maybe resource hacker has a silent mode which could allow automating this too ?
Anyway, if you care about this, the Reddit I posted above or Kenny's answer might be more suited to your needs.
I just wish I found something like this before, and prefer sharing it anyway. (I Looked for this several times in the past and never found a solution until today)