We have a very old PHP application that needs PHP 4 to run.  We're decommissioning the old server and so I've built PHP 4 on the new server (Ubuntu 13.04 32 bit).  When I did ./configure I made sure to do --with-config-file-path=/etc/php4/php.ini.  I'm trying to enable error logging, but my changes to the php.ini don't seem to be taking hold.
Yes, the file exists and is owned by the Apache user
nick@server:/etc/php4$ ls
php.ini
Yes, I restarted Apache.  And yes, it shows up in the phpinfo():
Configuration File (php.ini) Path   /etc/php4/php.ini
Does the --with-config-file-path only take hold at compilation time and thus in order to make changes I have to recompile?  Or does it work the same as newer versions of PHP?
For example, in the php.ini I've turned off display_errors, but phpinfo() shows:
display_errors  On
Note that it shows On for both the local and master values for this server.
Also, I'm sorry for perpetuating PHP 4
My php.ini and screenshot of phpinfo to prove I'm not lying
 
    