You have a group oliver which is the current group of your home folder. You need to add apache to this group. Use usermod -a -G oliver apache as root (probably using sudo) to add the apache user to the group oliver. You will need to restart apache (or your machine) to apply the change.
After this you can control the permissions with the group permission. Write access isn't usually given by default to other group members, so you may want to run chmod -R g+w /home/oliver. That adds write permission (+w) for other members of the group (g), recursively (-R). You own the folder, so you do not need sudo for that.
Keep in mind that apache may not run under the username apache. It may be something such as www, www-data, or webuser. Either check top/ps to see what it is running as, or run cat /etc/passwd and see which user looks like apache uses it. More specific instruction would require what distribution you are running and what version.