(Answer due to @johnwhitington)
Use a PDF command-line manipulation tool which supports such (lossess) scaling. One such tool is cpdf, short for "Coherent PDF".
Get the binary distribution from community.coherentpdf.com, or from the GitHub repo releases section. You can also get the source and build if you like.
In the ZIP archive, select the directory relevant to your platform, and decompress the binary any there into anyway on your path.
Assuming your input file is original.pdf and you want to create scaled.pdf , write:
cpdf -scale-to-fit a4portrait original.pdf -o scaled.pdf
Note that if your PDF contains some rasterized images (such as PNG or JPEG) - its resolution on the scaled page will be lower: not because of any degradation by the scaling tool, but because the same number of pixels take up more space. That may appear less pleasing to the viewer.