PHP Thumbnail Image Generator Caching: How to set If-Last-Modified/Max-Age/Last-Modified HEADERS correctly in PHP?

Question

Even after a very high score of Google PageSpeed(97) & Yahoo! YSlow(92) the PHP generated thumbnails don't seem to be coming passively from an old cache: they seem to be generated every time again...and again... freshly baked consuming lots of waisted time.

This question will focus only & specifically on how to solve the CACHE problem of the PHP Code that generates the thumbs:

Just have a look at these tiny puny little thumbnails measuring only 3 ~ 5 kb each!

Waterfall in detail: http://www.webpagetest.org/result/110328_AM_8T00/1/details/

Any & all suggestons are +1 help to me and warmly welcome, for I have grown quite desperate on this issue for the last months. Thanx a Thousand!

Using or not Modrewrite does not influence speed: both are same. I use these rewrite conditions: RewriteCond %{REQUEST_URI} ^/IMG-.*$ & RewriteCond %{REQUEST_FILENAME} !-f

Both the original default URL as well as the beautified rewritten URL produce the same delays!! So let us not point the fault to the lightning fast Apache: its the PHP Cache / headers that are somehow wrongly coded...

---

Warning by webpagetest.org: Leverage browser caching of static assets: 69/100

FAILED - (No max-age or expires): http://aster.nu/imgcpu?src=aster_bg/124.jpg&w=1400&h=100&c=p

---

After each refresh, you will see either of these two warnings appear on random at REDbot.org

---

Relevant Portions of The Code:

// Script is directly called
if(isset($_GET['src']) && (isset($_GET['w']) || isset($_GET['h']) || isset($_GET['m']) || isset($_GET['f']) || isset($_GET['q']))){
    $ImageProcessor = new ImageProcessor(true);
    $ImageProcessor->Load($_GET['src'], true);
    $ImageProcessor->EnableCache("/var/www/vhosts/blabla.org/httpdocs/tmp/", 345600);
    $ImageProcessor->Parse($quality);
}

/* Images processing class
 * - create image thumbnails on the fly
 * - Can be used with direct url imgcpu.php?src=
 * - Cache images for efficiency 
 */
class ImageProcessor
{
    private $_image_path;      # Origninal image path
    protected $_image_name;    # Image name   string
    private $_image_type;      # Image type  int    
    protected $_mime;          # Image mime type  string    
    private $_direct_call = false;   # Is it a direct url call?  boolean        
    private $_image_resource;  # Image resource   var Resource      
    private $_cache_folder;    # Cache folder strig
    private $_cache_ttl;        # Cache time to live  int
    private $_cache = false;    # Cache on   boolean
    private $_cache_skip = false;   # Cache skip   var boolean

    private function cleanUrl($image){   # Cleanup url
        $cimage = str_replace("\\", "/", $image);
        return $cimage;
    }   

    /** Get image resource
     * @access private, @param string $image, @param string $extension, @return resource  */
    private function GetImageResource($image, $extension){
        switch($extension){
            case "jpg":
                @ini_set('gd.jpeg_ignore_warning', 1);
                $resource = imagecreatefromjpeg($image);
                break;
        }
        return $resource;
    }


    /* Save image to cache folder
     * @access private, @return void  */
    private function cacheImage($name, $content){

        # Write content file
        $path = $this->_cache_folder . $name;
        $fh = fopen($path, 'w') or die("can't open file");
        fwrite($fh, $content);
        fclose($fh);

        # Delete expired images
        foreach (glob($this->_cache_folder . "*") as $filename) {
            if(filemtime($filename) < (time() - $this->_cache_ttl)){
                unlink( $filename );
            }
        }
    }

    /* Get an image from cache
     * @access public, @param string $name, @return void */
    private function cachedImage($name){
        $file = $this->_cache_folder . $name;
        $fh = fopen($file, 'r');
        $content = fread($fh,  filesize($file));
        fclose($fh);
        return $content;
    }

    /* Get name of the cache file
     * @access private, @return string  */
    private function generateCacheName(){
        $get = implode("-", $_GET);
        return md5($this->_resize_mode . $this->_image_path . $this->_old_width . $this->_old_height . $this->_new_width . $this->_new_height . $get) . "." . $this->_extension;
    }

    /* Check if a cache file is expired
     * @access private,  @return bool  */
    private function cacheExpired(){
        $path = $this->_cache_folder . $this->generateCacheName();
        if(file_exists($path)){
            $filetime = filemtime($path);
            return $filetime < (time() - $this->_cache_ttl);
        }else{
            return true;
        }
    }

    /* Lazy load the image resource needed for the caching to work
     * @return void */
    private function lazyLoad(){
        if(empty($this->_image_resource)){
            if($this->_cache && !$this->cacheExpired()){
                $this->_cache_skip = true;
                return;
            }
            $resource = $this->GetImageResource($this->_image_path, $this->_extension);
            $this->_image_resource = $resource;
        }    
    }

    /* Constructor
     * @access public, @param bool $direct_call, @return void */
    public function __construct($direct_call=false){

    # Check if GD extension is loaded
        if (!extension_loaded('gd') && !extension_loaded('gd2')) {
            $this->showError("GD is not loaded");
        }

        $this->_direct_call = $direct_call;
    }

    /* Resize
     * @param int $width, @param int $height, @param define $mode
     * @param bool $auto_orientation houd rekening met orientatie wanneer er een resize gebeurt */
    public function Resize($width=100, $height=100, $mode=RESIZE_STRETCH, $auto_orientation=false){

        // Validate resize mode
        $valid_modes = array("f", "p");
        }
                     // .... omitted .....

        // Set news size vars because these are used for the
        // cache name generation
                 // .... omitted .....          
        $this->_old_width = $width;
        $this->_old_height = $height;

        // Lazy load for the directurl cache to work
        $this->lazyLoad();
        if($this->_cache_skip) return true;

        // Create canvas for the new image
        $new_image = imagecreatetruecolor($width, $height);

        imagecopyresampled($new_image, $this->_image_resource, $dst_x, $dst_y, $src_x, $src_y, $dst_w, $dst_h, $src_w, $src_h);

             // .... omitted .....

        $this->_image_resource = $new_image;
    }

    /* Create image resource from path or url
     * @access public, @param string $location, @param bool $lazy_load, @return */
    public function Load($image,$lazy_load=false){

        // Cleanup image url
        $image = $this->cleanUrl($image);

        // Check if it is a valid image
        if(isset($mimes[$extension]) && ((!strstr($image, "http://") && file_exists($image)) || strstr($image, "http://")) ){

            // Urlencode if http
            if(strstr($image, "http://")){
                $image = str_replace(array('http%3A%2F%2F', '%2F'), array('http://', '/'), urlencode($image));
            }
            $image = str_replace("+", "%20", $image);

            $this->_extension = $extension;
            $this->_mime = $mimes[$extension];
            $this->_image_path = $image;
            $parts = explode("/", $image);
            $this->_image_name = str_replace("." . $this->_extension, "", end($parts));

            // Get image size
            list($width, $height, $type) = getimagesize($image);
            $this->_old_width = $width;
            $this->_old_height = $height;
            $this->_image_type = $type;
        }else{
            $this->showError("Wrong image type or file does not exists.");
        }
        if(!$lazy_load){
            $resource = $this->GetImageResource($image, $extension);
            $this->_image_resource = $resource;
        }           
    }

    /* Save image to computer
     * @access public, @param string $destination, @return void  */
    public function Save($destination, $quality=60){
        if($this->_extension == "png" || $this->_extension == "gif"){
            imagesavealpha($this->_image_resource, true); 
        }
        switch ($this->_extension) {
            case "jpg": imagejpeg($this->_image_resource,$destination, $quality);   break;
            case "gif": imagegif($this->_image_resource,$destination);      break;
            default: $this->showError('Failed to save image!');             break;
        }           
    }

    /* Print image to screen
     * @access public, @return void */
    public function Parse($quality=60){
        $name = $this->generateCacheName();
        $content = "";
        if(!$this->_cache || ($this->_cache && $this->cacheExpired())){
            ob_start();
            header ("Content-type: " . $this->_mime);
            if($this->_extension == "png" || $this->_extension == "gif"){
                imagesavealpha($this->_image_resource, true); 
            }

            switch ($this->_extension) {
                case "jpg": imagejpeg($this->_image_resource, "", $quality);    break;
                case "gif": imagegif($this->_image_resource);   break;
                default: $this->showError('Failed to save image!');             break;
            }

            $content = ob_get_contents();
            ob_end_clean();
        }else{

            if (isset ($_SERVER['HTTP_IF_MODIFIED_SINCE'])) {
                if (strtotime ($_SERVER['HTTP_IF_MODIFIED_SINCE']) < strtotime('now')) {
                    header ('HTTP/1.1 304 Not Modified');
                    die ();
                }
            }

            // change the modified headers
            $gmdate_expires = gmdate ('D, d M Y H:i:s', strtotime ('now +10 days')) . ' GMT';
            $gmdate_modified = gmdate ('D, d M Y H:i:s') . ' GMT';

            header ("Content-type: " . $this->_mime);
            header ('Accept-Ranges: bytes');
            header ('Last-Modified: ' . $gmdate_modified);
            header ('Cache-Control: max-age=864000, must-revalidate');
            header ('Expires: ' . $gmdate_expires);

            echo $this->cachedImage($name);
            exit();
        }

        // Save image content
        if(!empty($content) && $this->_cache){
            $this->cacheImage($name, $content);
        }

        // Destroy image
        $this->Destroy();

        echo $content;
        exit();
    }

    /* Destroy resources
     * @access public,  @return void */
    public function Destroy(){
        imagedestroy($this->_image_resource); 
    }


    /* Get image resources
     * @access public,  @return resource */
    public function GetResource(){
        return $this->_image_resource;
    }

    /* Set image resources
     * @access public, @param resource $image, @return resource */
    public function SetResource($image){
        $this->_image_resource = $image;
    }

    /* Enable caching
     * @access public, @param string $folder, @param int $ttl,   * @return void */
    public function EnableCache($folder="/var/www/vhosts/blabla.org/httpdocs/tmp/", $ttl=345600){
        if(!is_dir($folder)){
            $this->showError("Directory '" . $folder . "' does'nt exist");
        }else{
            $this->_cache           = true;
            $this->_cache_folder    = $folder;
            $this->_cache_ttl       = $ttl;
        }
        return false;
    }
}

The original author granted me permission for placing parts of code in here for solving this issue.

---

Answer 1

If I'm understanding the question correctly, this is entirely to be expected. Image manipulation is slow.

The yellow is your browser sending the request. The green is your browser waiting on the server to actually create the thumbnail, which takes a very significant amount of time, no matter what library the server is using. The blue is the server sending the response, which, unlike the previous steps, is affected by filesize.

There's not much to be done about the inherent slowness of image manipulation. It would be wise to cache these thumbnails so that they are only generated once and are then served statically. That way, very few of your users will ever have to sit through that green delay, and your server will be happy, too.

EDIT: If the issue is that the files exist at those URLs, but your RewriteRule is kicking in anyway, bear in mind that, by default, rules run without checking if the file exists.

Use the following condition above your RewriteRule to make sure the file exists.

RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule # ...etc...

Answer 2

imgcpu.php?src=foo/foo.jpg&w=100&h=100

so imgcpu.php is running for every image request?

In that case, if you are worried about performance,:

• the script needs to do some caching of the thumbnails it creates. If it resizes stuff on every request, that's your problem right there.

• the script needs to send some caching headers to the browser - a pure PHP script won't do that, and will be refreshed on every page load

• a session_start() call inside the PHP script could lead to concurrency issues because of session locking.

you will need to show some PHP code. Maybe in a separate question, though.

Answer 3

Apache can serve up files from your hard disk a lot faster than PHP can, and it appears that you're doing the latter to handle caching:

 /**
     * Get an image from cache
     * 
     * @access public
     * @param string $name
     * @return void
     */
    private function cachedImage($name){
        $file = $this->_cache_folder . $name;
        $fh = fopen($file, 'r');
        $content = fread($fh,  filesize($file));
        fclose($fh);
        return $content;
    }

There's a better way of doing what that function is doing (passthru), but the best option is to setup a regex that'll only rewrite a request to your thumbnailing script if the file doesn't already exist:

RewriteEngine On
RewriteCond %{REQUEST_FILENAME} -s [OR]
RewriteCond %{REQUEST_FILENAME} -l [OR]
RewriteCond %{REQUEST_FILENAME} -d
RewriteRule ^images/.*$ - [NC,L]
RewriteRule ^images/(.*)$ /imgcpu.php/$1 [NC,L]

And then introduce logic to parse the request to an image and format it accordingly. For example, you could say thumbs should be named after the original file and have the W x H dimensions appended like "stackoverflow_logo_100x100.jpg".

Make sense?

---

Per request (in the comment), the description of the "s", "l", and "d" flags is as follows (quoting the docs):

'-d' (is directory) Treats the TestString as a pathname and tests whether or not it exists, and is a directory.

'-s' (is regular file, with size) Treats the TestString as a pathname and tests whether or not it exists, and is a regular file with size greater than zero.

'-l' (is symbolic link) Treats the TestString as a pathname and tests whether or not it exists, and is a symbolic link.

Answer 4

Your checking your HTTP_IF_MODIFIED_SINCE header and cache AFTER you generate the image so the image is getting generated and cached every time you load the page. You would get a considerable decrease in time if you move these checks closer to the start of execution, before you start processing the image.

Answer 5

Matchu gave you answer why. If you want to fix it, save the created thumbnails, so they are not recreated on each request. I use simple 404 page that catches request to thumbnails that haven't been created, that script figures out the required dimensions and file from url - eg /thumbs/100x100/cat.png means create 100x100 thumbnail from /images/cat.png.