Tag Archives: google closure compiler

Optimizing and Compiling js/css files in php

In the last month, my team  in the company have been working on applying new theme to old project that we have, this project is more than 3 years old and it is written in relatively old technology (symfony 1.4/Propel ORM).

I wanted to find an automated method to optimize the javascript and stylesheet files that are served for this project (similar to the functionality of assetic in symfony2) , so I write a couple of files to automate this optimization, which do the following:

  1. Scan stylesheet folder and Optimize files using CssMin project:
    which compress the css file by removing whitespaces and comments, then minify.
  2. Scan javascript folder and Optimize  using google closure compiler:
    which parses javascript files, and convert it into better optimized form, as the closure page states:

    It parses your JavaScript, analyzes it, removes dead code and rewrites and minimizes what’s left. It also checks syntax, variable references, and types, and warns about common JavaScript pitfalls.

    note: I used CSSMin and google closure compiler, since they have the least dependencies, so I can utilize without installing additional packages, however, other options like Grunt or UglifyJs are really powerful but require npm to be installed.
  3. Creating new unique file names, through md5(resource_file_size) and copy it to the destination folder with the new name.
    This will prevent the browser caching for the modified files.

    note: other method for preventing browser cache is “cache busting” where you append changable query string to the resource file like

    <link rel="stylesheet" type="text/css" media="screen" href="/css/style.css?v=8" />
  4. Adding the association between the original file and the compiled file name to an array that will be used for rendering the resource path.

 

and here is the code of the task that performs the optimization:

<?php
include('CSSMinify.php'); //download CSS Min from http://code.google.com/p/cssmin/
class optimizeResourcesTask{

    private static $RESOURCES_ASSOCIATION = array();  //array to hold  mapping between original files and compiled ones
    
    private $source_js_folder = 'js/'; //the relative path of the source files for javascript directory, it will be scanned and its individual js files will be optimized
    private $target_js_folder = 'js-compiled/';    //result js files will be stored in this directory
    
    //target and source CSS folders preferred to be on the same folder level, 
    //otherwise path rewrite should be handled in the contents of the css files
    private $source_css_folder = 'css/'; //the relative path of the source files for stylesheet directory, it will be scanned and its individual css files will be optimized
    private $target_css_folder = 'css-compiled/';  //result css files will be stored in this directory   
    
    //path of the file that will hold associative array containing mapping between original files and compiled ones
    private $resource_map_file = '_resource_map.php';
    


    public function run() {
        // initialize the database connection

        $css_dir = __DIR__ .'/'. $this->source_css_folder;
        $this->optimizeCSSResources($css_dir);

        $js_dir = __DIR__.'/'.$this->source_js_folder;
        $this->optimizeJSResources($js_dir);

        $this->writeMappingData();
        
        
        $this->cleanupOldData($this->target_css_folder, 'css');
        $this->cleanupOldData($this->target_js_folder, 'js');
    }

    /**
     * iterating over the CSS directory and optimizing all of its contents
     * every single CSS file found it this directory will be passed to optimizeOneCSS() method in order to optimize
     * @param string $dir
     */
    protected function optimizeCSSResources($dir = null) {
        if (is_null($dir)) {
            $dir = __DIR__ . '/'.$this->source_css_folder;
        }

        if ($handle = opendir($dir)) {
            while (false !== ($entry = readdir($handle))) {
                if ($entry != "." && $entry != "..") {
					
                    if (is_dir($dir . $entry)) {
                        $this->optimizeCSSResources($dir . $entry);
                    } else {
                        $this->optimizeOptimizeOneCSS($dir . $entry);
                    }
                }
            }
        }
    }


    /**
     * optimize one CSS file by using CSSMin library to minify the contents of the file
     * generate new file name using hash of its file contents
     * add the new file name association to $RESOURCES_ASSOCIATION static variable in order to write resource association array later
     * @link "http://code.google.com/p/cssmin/" CSSMin documentation
     * @param string $file css file absolute path to minify
     */
    protected function optimizeOptimizeOneCSS($file) {
	
        print('trying to optimize css file ' . $file. chr(10));
        $info = pathinfo($file);
        if ($info['extension'] == 'css') {
            $optimized_css = CssMin::minify(file_get_contents($file));

            $target_css_dir_absolute = __DIR__ . '/' . $this->target_css_folder;
            if (!is_dir($target_css_dir_absolute)) {
                mkdir($target_css_dir_absolute);
                chmod($target_css_dir_absolute, 0777);
            }

            $new_name = md5($optimized_css) . '.css';
            file_put_contents($target_css_dir_absolute .  $new_name, $optimized_css);


            $file_relative_path = str_replace(__DIR__ , '', $file);
			
            self::$RESOURCES_ASSOCIATION[$file_relative_path] = '/' . $this->target_css_folder .  $new_name;

            print('CSS FILE: ' . $file . ' has been optimized to ' . $target_css_dir_absolute .  $new_name. chr(10));
			
        } else {
            print("skipping $file from optimization, not stylesheet file, just copying it". chr(10));
            
            $file_relative_path = str_replace(__DIR__ . $this->source_css_folder, '/', $file);
            
            $target_css_dir_absolute = __DIR__ . '/' . $this->target_css_folder .dirname($file_relative_path);
            
            if (!is_dir($target_css_dir_absolute)) {
                mkdir($target_css_dir_absolute);
                chmod($target_css_dir_absolute, 0777);
            }
            
            copy($file, $target_css_dir_absolute.'/'.basename($file));
        }
    }
	
	
    /**
     * iterating over the JS directory and optimizing all of its files contents'
     * every single JS file found it this directory will be passed to optimizeOneJS() method in order to optimize/minimize
     * @param string $dir
     */
    protected function optimizeJSResources($dir = null) {

        if (is_null($dir)) {
            $dir = __DIR__ . '/'.$this->source_js_folder;
        }
        print('getting JS inside ' . $dir. chr(10));

        if ($handle = opendir($dir)) {
            while (false !== ($entry = readdir($handle))) {
                
                if ($entry != "." && $entry != "..") {

                    if (is_dir($dir . $entry)) {
                        $this->optimizeJSResources($dir .  $entry);
                    } else {
                        $file_path = $dir . $entry;
                        $pathinfo = pathinfo($file_path);
                        if($pathinfo['extension']=='js'){
                            $this->optimizeOneJS($file_path);
                        }else{
                            print($file_path.' is not passed to optimization, its not a valid js file'. chr(10));
                        }
                    }
                }
            }
        }
    }

    /**
     * optimize one JS File using "Google Closure Compiler", 
     * store the optimized file in target directory named as hash of the file contents
     * add the new file name association to $RESOURCES_ASSOCIATION static variable in order to write resource association array later
     * @link  "https://developers.google.com/closure/compiler/docs/gettingstarted_api" "Google Closure Compiler API"
     * @param string $file js file absolute path to optimize/minify
     */
    protected function optimizeOneJS($file) {
	
        print("trying to optimize js ". $file. chr(10));

        $post_fields = array(
            'js_code' => file_get_contents($file),
            'compilation_level' => 'SIMPLE_OPTIMIZATIONS',
            'output_format' => 'text',
            'output_info' => 'compiled_code',
        );



        $ch = curl_init("http://closure-compiler.appspot.com/compile");
        curl_setopt($ch, CURLOPT_POST, count($post_fields));
        curl_setopt($ch, CURLOPT_POSTFIELDS, http_build_query($post_fields));
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);

        $optimized_js = curl_exec($ch);
        
        if(strpos($optimized_js,'Error(22): Too many compiles performed recently.') !==false){ //Google Closure API returned error on too many compilation
            trigger_error($file.' is failed to be compiled, skipped...');
            return;
        }
        
        curl_close($ch);

        $target_js_dir_absolute = __DIR__ . '/' . $this->target_js_folder;
        if (!is_dir($target_js_dir_absolute)) {
            mkdir($target_js_dir_absolute);
            chmod($target_js_dir_absolute, 0777);
        }

        $new_name = md5($optimized_js) . '.js';
        file_put_contents($target_js_dir_absolute . '/' . $new_name, $optimized_js);


        $file_relative_path = str_replace(__DIR__ , '', $file);
        self::$RESOURCES_ASSOCIATION[$file_relative_path] = '/' . $this->target_js_folder . $new_name;

        print('JS FILE: ' . $file . ' has been optimized to ' . $new_name. chr(10));
    }

    /**
     * write $resources_map array stored in $RESOURCES_ASSOCIATION into $resource_map_file file that will be used in generating
     * the association between original JS/CSS files and the optimized/minimized ones
     */
    protected function writeMappingData() {
        $str = "<?php \$resources_map = array(";
        foreach (self::$RESOURCES_ASSOCIATION as $original_file => $optimized_file) {
            $str .= "'$original_file'=>'$optimized_file', " . chr(10);
        }
        $str .= "); ";

        $f = fopen(__DIR__ . '/' . $this->resource_map_file, 'w+');
        fwrite($f, $str);
        fclose($f);

        echo 'mapping data written to ' . $this->resource_map_file . chr(10);
    }

    /**
     * this function will remove any file that exists $target_js_folder and $target_css_folder
     * and doesnot exist in $RESOURCES_ASSOCIATION array, most probably that were generated from old builds and not used anymore
     * @param $dir the relative path of the directory to cleanup
     * @param $extension_to_filter the extension that is going to be cleaned (either css or js), the idea is to ignore cleaning static resources like font files, ex. woff, eot
     */
    protected function cleanupOldData($dir, $extension_to_filter){
        $dir_absolute = __DIR__.'/'.$dir;
		
        if ($handle = opendir($dir_absolute)) {
            while (false !== ($entry = readdir($handle))) {
                if ($entry != "." && $entry != "..") {
                    
                    if (is_dir($dir_absolute .  $entry)) {
                        $this->cleanupOldData($dir.$entry, $extension_to_filter);
                    }else{
                        $file_path = $dir_absolute .  $entry;
                        $pathinfo = pathinfo($file_path);
                        print('examining   /'.$dir .  $entry. chr(10));
                        
                        //including the packup files deletion
                        if(in_array($pathinfo['extension'], array($extension_to_filter, $extension_to_filter.'~')) && !in_array('/'.$dir . $entry, self::$RESOURCES_ASSOCIATION)){                            
                            unlink($file_path);
                            print($file_path.' is deleted....'. chr(10));
                        }
                    }
                }
            }
        }        
    }
}


$task = new OptimizeResourcesTask();
$task->run();
echo 'optimization done...';

after running this class, it will generate “_resource_map.php” file that contains array to store mapping between original resources and compiled ones, its contents will be similar to this:

<?php $resources_map = array('/css/main.css'=>'/css-compiled/d41d8cd98f00b204e9800998ecf8427e.css', 
'/css/redmondjquery-ui-1.8.14.custom.css'=>'/css-compiled/d41d8cd98f00b204e9800998ecf8427e.css', 
'/css/style.css'=>'/css-compiled/f866be09baee73d596cb578b02d37d29.css', 
'/js/jquery-1.5.1.min.js'=>'/js-compiled/6c1b3f8d121bfefdad82fb4854a8f254.js', 
'/js/jquery-ui-1.8.14.custom.min.js'=>'/js-compiled/e34d1750b1305e35327964b7f0ea6bb9.js', 
'/js/jquery.cookie.js'=>'/js-compiled/08bf7e471064522f8e45c382b2b93550.js', 
'/js/jquery.easing-1.3.pack.js'=>'/js-compiled/0301f5ff89729b3c0fc5622b7633f4b8.js', 
'/js/jquery.fancybox-1.3.4.js'=>'/js-compiled/cb707a9b340d624510e1fa27d3692f0e.js', 
'/js/jquery.fancybox-1.3.4.pack.js'=>'/js-compiled/f58ec8d752b6148925d6a3f14061c269.js', 
'/js/jquery.min.js'=>'/js-compiled/5ee7bdd2dbbdec528925cb61c3010598.js', 
'/js/jquery.validate.min.js'=>'/js-compiled/9d28b87b0ec7b4e3195665adbd6918be.js', 
); 

now we need a function to get the optimized version of the files (in production environment only):

<?php 
function resource_path($file){
    global $config;
	if($config['env'] == 'prod'){ //serve compiled resource only on production environment
		include '_resource_map.php';
		if(isset($resources_map[$file])){
			return $resources_map[$file]; //return compiled version of the file
		}
	}
	return $file;
} ?>

and here is the use of example css/js file in “header.php”:

<link href="<?php echo resource_path('/css/style.css') ?>" rel="stylesheet" type="text/css" />
<script src="<?php echo resource_path('/js/jquery.min.js') ?>" type="text/javascript" ></script>

now once you render the page in production environment, the optimized css/js will be served instead of the original ones, as follows:
<link href="/css-compiled/f866be09baee73d596cb578b02d37d29.css" rel="stylesheet" type="text/css" />
<script src="/js-compiled/5ee7bdd2dbbdec528925cb61c3010598.js" type="text/javascript" ></script>

Now everything works good and you can serve optimized versions of your resource files with minimal effort upon each update on your website. Whenever there is some amendments to the website theme,  I would only run optimizeResourcesTask to optimize files and serve them automatically in production environments.

I used this code for my project s that written in native php or old symfony version, but as I mentioned  earlier there is some frameworks like symfony2 assetic that perform similar functionality with long list of optimizers available.