I'm sure that they were not file system errors: reading it at the block level (pv /dev/sdb > /dev/null) gave a really low 1.4 MB/s and then at some point it stopped with "input/output" error, both from Linux (I/O error) and from Windows + Cygwin (event id 153 in Event Viewer, "disk" source).
Trying to copy the files elsewhere, almost all of them could be read and some of them gave an I/O error (part of the file could be saved, like pictures or videos cut in half).
It was used in an Android phone as secondary storage. Android reported that the card was failing and suggested to replace it. After that, I did the above tests on the PC with the built-in card reader.
Then just for a test, before throwing it away, I formatted it on Windows (with exFAT). It took 6 hours (which gives also 1.4 MB/s), but it ran until the end. chkdsk d: showed 0 bad sectors
After that, the full card can be read at the block level at 18 MB/s, with no errors (tried on two different PCs). I filled it with files and also no errors when writing them or reading them afterwards.
Was it fixed when formatting? Or because of exFAT? Was it Android or the phone that wrongly changed some setting on the card?
