Speeding up thumbnail generation with Paperclip

Friday, April 09, 2010

Speeding up thumbnail generation with Paperclip

On the heels of my other Paperclip patch- I noticed that generating four styles from a large original and auto orienting them takes a lot of time (about 12 seconds on average). Trying to cut that time down I wondered if it would be possible to generate the smaller thumbnails from the larger ones. That is to say, generate :huge from :original, :large from :huge, :medium from :large, and :thumb from :medium. Paperclip can’t do this by default as the order in which it generates the styles is random, but with some patches and a special processor it can.

In my tests on my laptop, processing 267 images took 53 minutes originally. With these patches in place it takes 19 minutes. Per image it’s a savings of about 8 seconds. Not bad.

Here’s what the relevant portion of has_attached_file looks like now. Note that :pre_convert_options is another Paperclip patch (see my other article for more).

has_attached_file :image,
  :style_order => [:huge, :large, :medium, :thumb],
  :styles => {:thumb => {:geometry => '67x50>', :format => :jpg,
                          :processors => [:recursive_thumbnail], :thumbnail => :medium},
              :medium => {:geometry => '500x375>', :format => :jpg,
                          :processors => [:recursive_thumbnail], :thumbnail => :large},
              :large => {:geometry => 'x600>', :format => :jpg,
                         :processors => [:recursive_thumbnail], :thumbnail => :huge},
              :huge => {:geometry => '1000x1000>', :format => :jpg,
                        :pre_convert_options => '-auto-orient'},
             },

Notice the :recursive_thumbnail processor. That goes into lib/paperclip_processors/recursive_thumbnail.rb and looks like this:

module Paperclip
  class RecursiveThumbnail < Thumbnail
    def initialize file, options = {}, attachment = nil
      super attachment.to_file(options[:thumbnail] || :original), options, attachment
    end
  end
end

All it does is switch the source file from the original to the one specified by the :thumbnail option in my model.

Finally, here are the necessary patches to Paperclip to make it process the styles in the order specified by :style_order:

diff --git a/vendor/plugins/paperclip/lib/paperclip/attachment.rb b/vendor/plugins/paperclip/lib/paperclip/attachment.rb
index 51c20eb..aed5eca 100644
- a/vendor/plugins/paperclip/lib/paperclip/attachment.rb
+ b/vendor/plugins/paperclip/lib/paperclip/attachment.rb
@ -9,6 +9,7 @ module Paperclip
       @default_options ||= {
         :url           => "/system/:attachment/:id/:style/:filename,
         :path          =>",
+        :style_order   => [],
         :styles        => {},
         :default_url   => "/:attachment/:style/missing.png",
         :default_style => :original,
@ -18,7 +19,7 @ module Paperclip
       }
     end

-    attr_reader :name, :instance, :styles, :default_style, :pre_convert_options, :convert_options, :queued_for_write, :options
+    attr_reader :name, :instance, :style_order, :styles, :default_style, :pre_convert_options, :convert_options, :queued_for_write, :options
@ -33,6 +34,8 @ module Paperclip
       @url                  = @url.call(self) if @url.is_a?(Proc)
       @path                 = options[:path]
       @path                 = @path.call(self) if @path.is_a?(Proc)
+      @style_order          = options[:style_order]
+      @style_order          = @style_order.call(self) if @style_order.is_a?(Proc)
       @styles               = options[:styles]
       @styles               = @styles.call(self) if @styles.is_a?(Proc)
       @default_url          = options[:default_url]
@ -387,7 +390,7 @ module Paperclip
     end
-      @styles.each do |name, args|
+      styles_in_order.each do |name, args|
         begin
           raise RuntimeError.new(“Style #{name} has no processors defined.”) if args[:processors].blank?
           @queued_for_write[name] = args[:processors].inject(@queued_for_write[:original]) do |file, processor|
@ -421,6 +424,10 @ module Paperclip
       end
     end

def post_process_styles #:nodoc:

+    def styles_in_order #:nodoc:
+      @style_order.empty? ? @styles : @styles.sort_by{|s| @style_order.index(s.first)}
+    end
+
   end
 end

In case the formatting gets messed up, a diff of my project containing all of these changes is available here .

If you find this useful, please vote for it.

phil•ip hall•strom |'filip 'hôlsträm|

noun:
one who works with the Internet