Some of the file meta and entity store file operations are not atomic and could lead to cache corruption. Understand these problems better and move to truly atomic ops where possible.

rcrowley has collected a list of atomic operations here:

http://rcrowley.org/2010/01/06/things-unix-can-do-atomically

Passenger's Union Station feature introduced in version 3.0.4 brokes rack-cache. It uses rack.env variable to store state or something and refers unix socket which fails when dumping with Marshal.

I've added Marshal.dump check to MetaStore to verify that object is marshallable, but it could be performance issue.

After moving some of my middleware before Rack Cache, I noticed that Rack Cache was filtering out any Rails-related rack variables, such as action_controller.request.path_parameters. I found that by doing a merge! on the environment instead of creating a new Hash, the Rails rack variables were preserved.

My fix: http://github.com/winton/rack-cache/commit/05ac431b9d7db3c9f61fa1512956e4eb7b6fa52f

I tested this on a bare copy of Rails 2.3.5 with Ruby 1.9.1. I could probably come up with a test if you cannot replicate.

I tried to execute the test suite of rack-cache 1.0.2 together with rack 1.3.0. Unfortunately the test suite fails. I am not sure where the problem comes from :/

$ bacon -q -I.:lib:test test/context_test.rb
memcached library not available. related tests will be skipped.
dalli library not available. related tests will be skipped.
........FF.............................................................
Bacon::Error: not {"X-Rack-Cache"=>"miss, store", "X-Content-Digest"=>"0a4d55a8d778e5022fab701977c5d840bbc486d0", "Date"=>"Fri, 08 Jul 2011 14:02:46 GMT", "Content-Type"=>"text/html", "Age"=>"0"}.include?("Content-Type") failed
    ./test/context_test.rb:92: Rack::Cache::Context - responds with 304 when If-Modified-Since matches Last-Modified
    ./test/context_test.rb:78
    ./test/context_test.rb:4

Bacon::Error: not {"ETag"=>"12345", "X-Rack-Cache"=>"miss, store", "X-Content-Digest"=>"0a4d55a8d778e5022fab701977c5d840bbc486d0", "Date"=>"Fri, 08 Jul 2011 14:02:46 GMT", "Content-Type"=>"text/html", "Age"=>"0"}.include?("Content-Type") failed
    ./test/context_test.rb:111: Rack::Cache::Context - responds with 304 when If-None-Match matches ETag
    ./test/context_test.rb:98
    ./test/context_test.rb:4

71 tests, 449 assertions, 2 failures, 0 errors

Hi,

I have a project running Rails 3.2.5 with Rack::Cache activated. The application_controller handles the detection of the user's preferred language and set I18n.locale accordingly.

When the first request arrives Rack::Cache caches the page (let's say in english) properly, but then if the next request, from another web browser in french, Rack::Cache serves the english version. The solution to fix this issue is to add a custom cache_key but I can't get it working. Here the two codes I've tried in production.rb:

config.action_dispatch.rack_cache[:cache_key] = Proc.new { |request|
  [I18n.locale, ':', Rack::Cache::Key.new(request).generate].join
}

config.action_dispatch.rack_cache = { 
  :metastore => "file:tmp/rack-cache/meta",
  :entitystore => "file:tmp/rack-cache/body",
  :verbose => true,
  :private_headers  => ['Cookie', 'Set-Cookie', 'Authorization'],
  :cache_key => Proc.new { |request|
      [I18n.locale, ':', Rack::Cache::Key.new(request).generate].join
      }
}

Am I missing something? Thanks for your help.
Bastien

PS: I currently deactivated Rack::Cache in order to serve the right language

This is going to be a pain by I'm trying to move all my projects away from the more complex test frameworks.

/Users/fredrik/.rvm/gems/ruby-1.9.2-p0/gems/rack-client-0.3.0/lib/rack/client/parser/body_collection.rb:4: warning: undefining `object_id' may cause serious problems

To get Rack::Cache working with Devise/Warden you must insert Rack::Cache before Warden::Manager, or else you will get "undefined method `camelize' for nil:NilClass" when loggin in your application -

config.middleware.insert_before Warden::Manager, Rack::Cache, ...

And also that when sendfile_header is set (this is now set by default in production environment), Rack::Cache will return empty body:

config.action_dispatch.x_sendfile_header = "X-Sendfile"

I spent couple of hours on these two issues.
Maybe this should be somehow fixed or documented somewhere.

I'd like to "bring my own cache".

• Why are there so many cache-implementation-reinventions ?
• Why do I have to build something rack-specific just to get some caching ?
• Why do I have to even bother with entity/metastore ? (it may be nice for advanced users, but lets start simple)

can we just focus on support a simple interface like []/[]=/clear moneta style or read/write/fetch/clear and be done with this ?

• removes a ton of code/bugs/indirection
• make it easier to use
• makes it easier to debug

I don't want to learn how rack-cache works internally, does it really have to be more complicated then:
use Rack::Cache, :cache => Rails.cache
use Rack::Cache, :cache => MyOwnCachenew

maybe at most:
use Rack::Cache, :enititystore => Rails.cache, :metastore => ActiveSupport::Cache::Memorystore.new

would also solve support for mongodb/redis/multi-server-memcache etc and surely a few issues.

The title says everything. When I send a GET request to the server, containing an If-None-Match header that matches the ETag, I get a 304 Not Modified response as expected. When I do the same for POST-Requests, I get a full response, even though it has the given ETag.

iblue@raven ~ $ curl "http://testing.example.org/resource.json" -H 'If-None-Match: "11263794d27103747aa3048b8d1ca76d" ' -i
HTTP/1.1 304 Not Modified
Server: nginx/0.7.67
Date: Tue, 01 May 2012 12:33:23 GMT
Connection: keep-alive
Status: 304 Not Modified
X-UA-Compatible: IE=Edge,chrome=1
ETag: "11263794d27103747aa3048b8d1ca76d"
Cache-Control: must-revalidate, private, max-age=0
Set-Cookie: _session_id=82d7a4d488e4bd295902cca2b0e3f0b6; path=/; expires=Wed, 01-May-2013 12:33:23 GMT; HttpOnly
X-Request-Id: 4fc3e3bc9f7cd4c334d815ff6958d2d3
X-Runtime: 0.950690
X-Rack-Cache: miss

iblue@raven ~ $ curl -X POST "http://testing.example.org/resource.json" -H 'If-None-Match: "11263794d27103747aa3048b8d1ca76d" ' -i
HTTP/1.1 200 OK
Server: nginx/0.7.67
Date: Tue, 01 May 2012 12:33:36 GMT
Content-Type: application/json; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
Status: 200 OK
X-UA-Compatible: IE=Edge,chrome=1
ETag: "11263794d27103747aa3048b8d1ca76d"
Cache-Control: max-age=0, private, must-revalidate
Set-Cookie: _session_id=82d7a4d488e4bd295902cca2b0e3f0b6; path=/; expires=Wed, 01-May-2013 12:33:36 GMT; HttpOnly
X-Request-Id: b4121c1ba148f4a1212bee429b418a40
X-Runtime: 0.954076
X-Rack-Cache: invalidate, pass

[... Content here ... ]

Hello.
I use Windows and had weird issue with Radiant CMS, which uses rack-cache gem.
The following code causes the issue:

https://github.com/rtomayko/rack-cache/blob/master/lib/rack/cache/metastore.rb#L216

def initialize(root="/tmp/rack-cache/meta-#{ARGV[0]}")
   @root = File.expand_path(root)
   FileUtils.mkdir_p(root, :mode => 0755)
end

You see, FileUtils.mkdir_p( '/tmp/foo/bar' ) creates C:/tmp/foo/bar on Windows (If my current drive is C).
Of course that’s not possible on hosting environments. There are no permissions.
Generally on shared-hosting environment you have access to application folder only. It would be nice to change that behaviour so that folders are created inside Rails app root.

Please let me know if you need any help. I could submit a patch.

Thank you.

In certain circumstances, rack-cache returns 304 Not Modified on POST and PUT requests. The specific example I am seeing:

1. iPhone app does a POST to a Rails server to the path /login

2. The response is returned with a http status code of 200 and an etag added by rack/etag.rb. Note the response has a custom auth token http header, but rack only takes into consideration the response body. Often times POST to login will do a redirect, but in this case its a JSON api so no reason to redirect.

3. User closes app

4. User opens app, logs in again. The iphone app makes the same request as before, but this time includes the etag it got back the first time.

5. rack-cache, in context.rb, notices this is a post and invalidates the cache (not that there is anything in the cache), then passes the request along to rest of rack stack and rails.

6. Once Rails has fulfilled the request, on the return code path rack-cache in context.rb, around line 98 does this:

   response.not_modified! if not_modified?(response)

That changes the response to 304 because the etag from the client matches the etag figured out by rack (see #2 above), although this is a POST.
7. iphone client sees a 304, looks in its cache, pulls out the previous reponse. But that response has an old auth token that has been invalidated. So the next request the iPhone app makes will fail due to an authentication error.

I believe that rack-cache is incorrect here, it should only change the response status to 304 for GET and HEAD requests. Although it seems strange for the server to return an etag on a POST/PUT and the client to then return it, there is a use case in WebDav:

http://www.webdav.org/specs/rfc4918.html#rfc.section.12.1

The etag is used for a server to detect if a resource about to be updated has been updated by someone else. If it has, then instead of a 304 a 412 should be returned. Of course that is outside of the realm of rack-cache, so it should just return the response with out changing its status to 304.

Thoughts?

I see this error when trying to serve a webm video from rails 3.2.3 using Rack::Cache 1.2:

cache: [GET /media/video.webm] miss
couldn't decode video - got incompatible marshal file format (can't be read)
    format version 4.8 required; 110.184 given

The mimetype vor webm is registered using an initializer:

Rack::Mime::MIME_TYPES.merge!({
  ".webm"    => "video/webm"
}

After deactivating rack-cache I can serve the video without any problem.

When using puma with Rack::Cache, the assets do not get loaded correctly, this issue is described on the puma project: puma/puma#57

I found out that puma sets env['rack.run_once'] to true and rack-cache uses this information to determine to run the request on a non threadsafe mode, i.e. in /lib/rack/cache/context.rb:48.

def call(env)
  if env['rack.run_once'] 
    call! env
  else
    clone.call! env
  end
end

As per rack specs (http://rack.rubyforge.org/doc/files/SPEC.html), rack.run_once is not a guaranty that the application will be served only once. The specs also describe rack.multithread if the application object may be simultaneously invoked by another thread in the same process. I'll send a pull request to add test that rack.multithread is not set to true i.e.

def call(env)
  if env['rack.run_once'] && !env['rack.multithread']
    call! env
  else
    clone.call! env
  end
end

Is there a way to define how etags are generated and what key is being used to store it?

It would make expiration much easier when fine granular expiration is necessary without Rails.cache.clear.

Right now Etags are not being used for the memcache store, but something called X-Content-Digest.
Why is that?

EDIT; I found out about cache_key. Wondering if there is a way to do this in the controller/model level instead of a configuration.

Example;
fresh_when :etag => @jobs.maximum(:updated_at)
So before an update, I can clear memcache with @jobs.maximum(:updated_at)

According to the HTTP specification §14.26 a server must ignore the if-not-modified-since if no etag matched:

"If none of the entity tags match, then the server MAY perform the requested method as if the If-None-Match header field did not exist, but MUST also ignore any If-Modified-Since header field(s) in the request. That is, if no entity tags match, then the server MUST NOT return a 304 (Not Modified) response."

I will write a patch

Is there any interest in optionally passing 'max-age' to the underlying datastore on write to automatically expire stale data? It has potential for memory saving if long-tail entries get cleaned up before waiting for a subsequent request to be invalidated & replaced.

Hi,

In your .gemspec file [1], there is specified project homepage which is apparently invalid. Could you please fix it? Thank you

[1] https://github.com/rtomayko/rack-cache/blob/master/rack-cache.gemspec#L71

I am currently on Rails 3.0.14 and trying to use rack cache 1.2

I have the following in my application.rb:

config.middleware.insert_before ActionDispatch::Static, Rack::Cache,
  :metastore   => 'file:/var/cache/rack/meta',
  :entitystore => 'file:/var/cache/rack/body'
  :verbose => true

When I start the app locally, rack cache does not appear to be doing anything for pages with:

expires_in 3.minutes, :public => true

Is there anything else required to get rack cache working with Rails 3.0.14? (at the moment upgrading to Rails 3.1+ is not an option) Should I be using an older version of rack cache?

It should be possible to purge and/or invalidate cache entries explicitly using an API or response header.

I noticed that I get server failures when my EntityStore is unable or unwilling to save a response. Two examples: if memcache goes down, any writes through Dalli will fail and any reads will return nil. Or when I'm using a simple NilStore for testing in Rails, my writes will always do nothing and reads will always return nil (which can be useful for exercising an app's cache code without firing up an instance of memcached).

In both cases, Rack::Cache gets a valid response from Rails but then replaces the body with nil. This happens because MetaStore#store writes the response's body to the entity store (which fails), immediately reads it back from the store (getting nil), and then replaces the response's body with that nil.

Is there any reason for the metastore to replace the response body? Deleting the entity_store.open line from MetaStore#store would fix the two cases I'm having trouble with. That is, the server would still return a response, even if that response can't be cached. This would also eliminate an unnecessary read and get rid of the minute possibility that even a working store could throw away the entry between the write and subsequent read. But maybe I'm overlooking something? Does reading the newly stored body have some beneficial side-effect?

Here's the relevant section in 1.0.3 (and it looks similar in 1.1):

  # write the response body to the entity store if this is the
  # original response.
  if response.headers['X-Content-Digest'].nil?
    if request.env['rack-cache.use_native_ttl'] && response.fresh?
      digest, size = entity_store.write(response.body, response.ttl)
    else
      digest, size = entity_store.write(response.body)
    end
    response.headers['X-Content-Digest'] = digest
    response.headers['Content-Length'] = size.to_s unless response.headers['Transfer-Encoding']
    response.body = entity_store.open(digest)
  end

Thoughts?

Are there any docs or documented API for writing a new back-end store?

For my own odd purposes, I would like to use a store backed by ActiveRecord (yes,obviously not very performant, but suits my needs).

I could write it up and submit a pull request... but this is probably not of general interest. Is there any way for me to create a Rack::Cache store that lives in it's own gem, and still works with Rack::Cache?

As far as I can tell, Rack::Cache::Key only takes into account protocol, domain, path, and query parameters. Different user-agents might request the same URL and expect to get different results. I see two major factors that Key ignores: the Accepts header and the Accepts-Language header. That is, if a German-speaking user requests the HTML version of a resource and then a Portuguese-speaking person requests the JSON version, the Portuguese person will get the German HTML.

Additionally, it might be important to also take the HTTP verb into account. With only the five basic verbs (HEAD, GET, POST, PUT, and DELETE) there's probably not much of an issue -- Rack::Cache should only be caching GETs anyway. Once you try to put WebDAV or CalDAV behind Rack::Cache, though, things might get a little hairier.

Rescuing from Errno::ENOENT does not work for JRuby. JRuby raises an IOError which needs to be handled as well. For (some) more information, please take a look at the following discussion:

http://groups.google.com/group/rack-cache/browse_thread/thread/1bb880ea57037620/5255700919043264?lnk=gst&q=jruby#5255700919043264

Rack::Bug has a nice little framework for displaying firebug-style panels with request information. It should be possible to hook into Rack::Cache so that the number of cache hits, misses, validates, etc. can be recorded and displayed.

There's a typo in the code used to determine which memcached gem to use. The patch in the gist fixes this

http://gist.github.com/117610

NO matter how I require the rack-cache in enviornment.rb file, cant' get beyond this error. THis is on REE 1.8.7, rails 2.3.5.

I have tried doing a require 'rack/cache' and I have also tried config.gem 'rack-cache', :lib => 'rack/cache'.

Am I missing something? There is nothing in the docs about this.

Repo points to http://tomayko.com/src/rack-cache/

We're using redis-rack-cache for our store. We found that our Redis install was quickly taking all available memory, even though our pages are Cache-Control: public, max-age=180.

What I discovered is that rack-cache.use_native_ttl is not documented and not enabled by default; the Redis driver sets the content with no expiration at all meaning Redis will keep it around forever.

1. Could you document this flag? I was going off of this page: http://rtomayko.github.com/rack-cache/configuration
2. Does it make sense to enable this flag by default so it doesn't happen to others?

I posted this issue to rack, but got answer that problems might be in rack-cache.

can you please check - https://github.com/rack/rack/issues/#issue/105

Rack lint is complaining because HEAD requests are returning the content length of what the body would have been. Should Context.call! set the content length header to 0 when it sets the body to [] or is it normal to have another middleware fix the content length?

We're using memcache as our metastore in production, but we're noticing alot of timeout errors. The reason for the errors isn't rack-cache's problem but rack-cache is letting these unrescuable exceptions surface into 500s.

It would be nice if rack-cache rescued from any internal exceptions and logged them, but passed the request along to the app.

http://github.com/josh/rack-cache/commit/7bcd0cec2723b91ee0d557153134d457219895ba

I have been attempting to get streaming working in Rails 3.2 on Heroku (see my SO post here: http://stackoverflow.com/questions/9915704/rails-3-2-streaming).

I am coming to the conclusion that rack-cache is causing the problem. Disabling it using config.middleware.delete(Rack::Cache) in production.rb seems to fix it. This, obviously, disables it for my entire app.

I only want it disabled for the one streaming request (which is on the admin side and will be used infrequently). It'd be a major bummer to lose caching for the sake of one small (but required) admin feature.

Are there any work-arounds for this? Thanks!

I'm running a Rails 3.2 app on unicorn in production. The Rack::Cache middleware is enabled by default using the Rails stores which is memcached in my case.

From time to time unicorn reports an app error in the form "body is nil/does not respond to each". After investigating this issue for a long time, I found the issue to lie within Rack::Cache:

Rack::Cache tries to store the response body in the cache, then rereads it from there; this happens in the following piece of code (I added two debugging statements):

    if response.headers['X-Content-Digest'].nil?
      if request.env['rack-cache.use_native_ttl'] && response.fresh?
        digest, size = entity_store.write(response.body, response.ttl)
      else
        digest, size = entity_store.write(response.body)
      end
      Rails.logger.debug "XXX wrote #{response.body.class.inspect} (size #{size}) to #{digest}"
      response.headers['X-Content-Digest'] = digest
      response.headers['Content-Length'] = size.to_s unless response.headers['Transfer-Encoding']
      response.body = entity_store.open(digest)
      Rails.logger.debug "XXX read #{response.body.class.inspect} from #{digest}"
    end 

(from https://github.com/rtomayko/rack-cache/blob/1.2/lib/rack/cache/metastore.rb#L57)

This leads to the following output from time to time:

2012-10-10 17:01:04 [DEBUG] XXX wrote Rack::BodyProxy (size 11937) to 349f1e49d69a5112f28480c647fb0aa73e3a573c
2012-10-10 17:01:04 [DEBUG] XXX read NilClass from 349f1e49d69a5112f28480c647fb0aa73e3a573c

So, what happens? The response body is written to memcached and then response.body is set to the result of a cache read, which sometimes happens to be nil...

I checked the memcached log and found no mentions of errors or problems, so memcached seems to work properly... It also works properly everywhere else where it is used in the application.

So, two questions remain with me:

• Why is the response body set to the value from the cache; isn't this a bit dangerous? For example the cache could crash after the write and return nil for the read, leading to the same problem that I have.
• How can I help to debug this issue more? ;)

Versions of gems involved:

• rails 3.2.6
• rack-cache 1.2
• memcache-client 1.8.5

Rack::Cache setup:
Rails default a.k.a. {:metastore=>"rails:/", :entitystore=>"rails:/", :verbose=>false}

Rails cache setup:

config.cache_store = :mem_cache_store, 'brain'

It seems Rack::Cache will swallow If-None-Match and If-Modified-Since headers. This seems to be a problem if I want to use Rack::Cache but have parts of my application that only allows private caching. In this case private caching will not reduce the strain on my server.

I've made a fresh Rails3 app and added Rack::Cache.

I'm calling my application with the following command:

$ curl -H "If-Modified-Since: Sat, 29 Jan 2011 11:14:37 GMT" -H "If-None-Match: \"03273f2f207cb7864f217458f0f85e4e\"" -q -v "http://127.0.0.1:3000/users"

And this is what I'm seeing in the log:

Started GET "/users" for 127.0.0.1 at 2011-01-29 14:15:02 +0100
  Processing by UsersController#index as */*
HTTP_IF_MODIFIED_SINCE: Sat, 29 Jan 2011 11:14:37 GMT
HTTP_IF_NONE_MATCH: "03273f2f207cb7864f217458f0f85e4e"
  User Load (0.9ms)  SELECT "users".* FROM "users"
Completed 304 Not Modified in 26ms

After adding Rack::Cache to application.rb and restarting my server this is what I get with the same curl command:

cache: [GET /users] miss

Started GET "/users" for 127.0.0.1 at 2011-01-29 14:24:22 +0100
  Processing by UsersController#index as */*
HTTP_IF_MODIFIED_SINCE: 
HTTP_IF_NONE_MATCH: 
Rendered users/index.html.erb within layouts/application (30.7ms)
Completed 200 OK in 91ms (Views: 43.8ms | ActiveRecord: 1.9ms)

In both cases I do get a 304 back from the server as reported by curl - I'm guessing that might be due to Rack::ConditionalGet though.

This is a repost of an issue I recently created in the rails/rails repository. I'm not 100% sure who's responsibility it is to resolve the problem (rails or rack-cache), but I wanted to be sure you knew about it. Specifically, on the Rack::Cache side, the bug occurs in the Metastore where it performs a hash.delete('X-Status').to_i. Using the Rails MemoryStore for caching through the RailsMetaStore, the in-memory object is handed back to Rack::Cache, which manipulates the hash directly on the first successful cache lookup. Since the Status header is removed any "successful" request after the first fails out with HTTP 0 responses.

This bug causes cached asset responses to be invalid after two successful requests. This means that with ActionController caching enabled in production, assets will not be returned correctly to clients.

In the Rails 3.1.0.beta1, using:

• rails (3.1.0.beta1)
• rack (1.3.0.beta)
• rack-cache (1.0.1)

There is a reproducible bug when ActionController caching is enabled (in production), where ActionDispatch::RailsMetaStore, along with ActiveSupport::Cache::MemoryStore, will ultimately cause Rack::Cache to return an HTTP 0 (invalid!) response to Rack.

The implementation of Rack::Cache works by storing hashes into the cache to retrieve on future requests. Those hashes are written and pulled as necessary for invalidation.

The basic lifecycle looks like this:

1. First request, cache empty, find file, generate fingerprint, append to file name, store request response hash information to cache, serve content back to client with a miss, store header response on the cache,
2. Second request, identify matching file request based on name and fingerprint, read from cache, read and drop the X-Status header from the cached hash and return that hash forward to Rack.
3. Third request, identify matching file request based on name and fingerprint, read from cache, read and drop the X-Status header (except it's already been DELETED!), find no status, return HTTP 0 to rack.
4. All subsequent matches follow #3.

So, it seems to be that because the MemoryStore cache is handing back the hash object directly, Rack::Cache is then manipulating it before handing it upstream to Rack. In most cases, that's fine, except that because MemoryStore is handing back the actual object, and Rack::Cache is manipulating it, the cache object is ultimately, unknowingly being manipulated. So, subsequent finds are returning the "bad" object.

I've been stuck trying to get Rack::Cache to use Rack::Sendfile without any luck. I've sent a message to the mailing list but it seems that there's no activity there.

The details are in this StackOverflow question, but in summary, even with Rack::Sendfile in front of Rack::Cache, my images don't seem to be sent using Rack::Sendfile.

What do I do?

Hi Ryan,

Could I request a 1.0.3 release? I'm eager to get my fix pushed so I can use my non-blocking async code with rack-cache.

Thanks!

It would be nice if there were an option to disable 304 not modified responses in the case where you want to do post-processing of cached content (in my case, ESI in non-production environments a la Varnish in production).

As far as I can tell, the HTTP specification does not prevent returning 200 with content even if the if-modified-since or if-match freshness condition is satisfied. This allows upstream processors to work their mojo and make the final determination as to whether 304 is appropriate or not.

In the case of ESI processing, this would allow me to catch edge caching bugs before production.

Thoughts?

I'm seeing a lot of these errors in our logs:

cache error: invalid byte sequence in UTF-8
/data/apps/production/web/shared/bundle/ruby/1.9.1/gems/rack-cache-1.1/lib/rack/cache/key.rb:46:in `split'
/data/apps/production/web/shared/bundle/ruby/1.9.1/gems/rack-cache-1.1/lib/rack/cache/key.rb:46:in `block in query_string'
/data/apps/production/web/shared/bundle/ruby/1.9.1/gems/rack-cache-1.1/lib/rack/cache/key.rb:45:in `map'
/data/apps/production/web/shared/bundle/ruby/1.9.1/gems/rack-cache-1.1/lib/rack/cache/key.rb:45:in `query_string'

    def query_string
      return nil if @request.query_string.nil?

      @request.query_string.split(/[&;] */n).
46:        map { |p| unescape(p).split('=', 2) }.
        sort.
        map { |k,v| "#{escape(k)}=#{escape(v)}" }.
        join('&')
    end

Rack::Cache::Key#generate might generate a key which is too long
to be used as a key in memcache. Would it be better to hash the URL?

This ticket is a request to add the Infinispan backed TorqueBoxStore (http://torquebox.org/documentation/1.0.1/web.html#caching) caching system to Rack::Cache for jruby users running on Torquebox.

The Torquebox devs have already created a back-end for for ActiveSupport caching (ActiveSupport::Cache::TorqueBoxStore) which could probably be adapted / used as a base for this.

I am having a look at doing this myself ($deity have mercy on us all) but since Rack::Cache will be included and enabled by default on Rails 3.1 this might be done by the Torquebox devs.

Either way I thought I would add a ticket here for feedback purposes.

Please delete

What about a Moneta store? Some of the internal stores could be replaced and the caching implementation could be simplified. There is some stuff like EntityStore.open which could not be provided by the moneta api but which has to be emulated so or so for memcached or similar backends.

I'm setting up a sinatra app and a client library for accessing the app, both using Rack::Cache for caching (the client using faraday_middleware: https://github.com/pengwynn/faraday_middleware/wiki/Caching).

The sinatra app sets the "X-Content-Digest" header and sends it with the rest of the request. When the client library gets the request and starts handling it via Rack::Cache, it notices that the X-Content-Digest header is set (https://github.com/rtomayko/rack-cache/blob/master/lib/rack/cache/metastore.rb#L59), so it doesn't store the entity on the client side, but still stores the meta information. Future requests from the client for cached information find the meta entry, try to open the entity, fail to open it (raising an error), which then causes Rack::Cache on the client side to pass on to the server.

Is this intended functionality to avoid caching an entity multiple times, or a bug?

I can imagine possibly wanting to be able to stack multiple Rack::Cache instances not only for this use case, but also on the server side (maybe, for instance, one with a memcached entity store and a secondary file based entity store).

It is possible that multiple etags are valid. rack-cache should therefore support multiple etags per uri.

Shouldn't verbose logging go somewhere other than @rack.errors? This pollutes STDERR with debug info that'd be better-served in STDOUT; on Apache/Passenger/Rails 3.1, these all go to the error_log.

This issue isn't strictly a bug in Rack::Cache, but it's a problem that can very easily come up when using Rack::Cache with Rails. Not sure what the best solution is, but thought I'd raise the issue and see what you thought.

It's very easy to recreate the problem:

1. Create a new Rails app. Use the asset pipeline. Put something into one of your assets.

2. Add Rack::Cache as directed in config/application.yml:

      config.middleware.use(Rack::Cache)
   

3. Start your Rails server in development mode.

4. Load a page, receive asset.

5. In a fresh cache (i.e. a new Chrome incognito window), load the same page a second time. This time the asset request wrongly returns a 304 and empty body, thus the asset is not received.

Here's why it happens (to the best of my understanding):

The Rails middleware stack includes Rack::ConditionalGet, and this resides higher on the stack than Rack::Cache (which is added near the end). ConditionalGet sees the request for the asset and forwards it along.

Rack::Cache receives the request (because of the asset pipeline, asset requests travel the whole stack in development). It needs to check if its cached version is fresh, so it adds HTTP_IF_MODIFIED_SINCE and HTTP_IF_NONE_MATCH headers to the request env and forwards it along.

Rails agrees that yeah, that's fresh, so Rack::Cache correctly builds a 200 response using its cached copy. It passes things back up the stack to Rack::ConditionalGet, which now looks at the env, sees that the caching headers are set (because Rack::Cache set them earlier), and thinks those are browser headers. So it decides to return a 304 to the poor browser, who never indicated that it had anything cached.

Here's how I fixed / worked around the issue:

config.middleware.insert_before(Rack::ConditionalGet, Rack::Cache)

If Rack::Cache gets to do its business first, everything goes according to plan, and the two middlewares play much nicer together.

So... is this Rack::Cache's fault for dirtying up @env? Or Rack::ConditionalGet's fault for looking at the env at a weird time and assuming it came from the browser? Or is this just a documentation issue?

I'm running into an issue caching responses with headers like this:

Cache-Control: max-age=300, public
ETag: "5c564d692c85b70068fd76ea25f2be2f"

While fresh, this works fine, but once stale, Net::HTTP is raising this error:

NoMethodError: undefined method `strip' for nil:NilClass
        from ~/.rbenv/versions/1.9.3-p448/lib/ruby/1.9.1/net/http.rb:1436:in `block in initialize_http_header'

The issue appears to be in lib/rack/cache/context.rb. This change fixes it:

197c197
<       @env['HTTP_IF_MODIFIED_SINCE'] = entry.last_modified

---
>       @env['HTTP_IF_MODIFIED_SINCE'] = entry.last_modified if entry.last_modified

It's currently not possible to use more than a single memcached server for storage. Seth Faxon did some work on this and sent me a patch. Here's his email:

I spent some time this weekend modifying your Rack::Cache gem to work with the memcache-client gem. I needed it to work with multiple memcache servers, so I broke the URI convention you were using. I'm using it in a Rails app and the new config lines look like:

config.middleware.use(Rack::Cache,
 :verbose =>     true,
 :metastore =>   {:memcache => {:server => ['127.0.0.1:11211', '127.0.0.2:11211'], :namespace => 'nui_meta'}},
 :entitystore => {:memcache => {:server => ['127.0.0.1:11211', '127.0.0.2:11211'], :namespace => 'nui_entity'}})

The entity store and cachecontrol changes are pretty straight forward. I did change the MEMCACHE static variable to use the MemCache gem.

The other significant change was to metastore.rb, if the memcache server is down the response body returns nil. I added a check at line 63 of metastore.rb to put the body back after the read fails.

I'm planning on writing tests this afternoon, and just wanted to get some feedback from you to see if this is a direction you wanted to go. I have no desire to fork this, and if you have some feedback I would like to incorporate it.

Thanks for the great gem, this was my first time playing with Rack and I was able to jump in pretty quickly with the stuff you provided.