As it turns out, Apache 2.4's Require matches the IP exactly. If you have multiple IP addresses aliasing localhost, you need to list all of them (or use a special alias, if one exists, as explained below).
In this particular case, the error.log entry reveals it all: The client connected through the IPv6 interface (ip == ::1). That needs to be white-listed as well:
<RequireAll>
Require all granted
# require localhost
<RequireAny>
Require ip 127.0.0.1
Require ip ::1
</RequireAny>
</RequireAll>
Any suggestions as to whether there is a simpler/safer method to get this done, are very welcome!
Update
As Helge Klein suggests, Require local is a more concise alternative:
<RequireAll>
Require all granted
# require localhost
Require local
</RequireAll>