The magic happens in Rack::ETag and Rack::ConditionalGet. (Mine are located at $GEM_HOME/rack-1.5.5/lib/rack/conditionalget.rb and $GEM_HOME/rack-1.5.5/lib/rack/etag.rb.)
These are middlewares.
Rack::ETag makes a digest out of the body (the very same body which gets returned by the rack function @app.call) and sets the response ETag header to this value. For Rails 4.0, at least, it's an MD5 hex digest.
Rack::ConditionalGet compares the request's If-None-Match header against the response's ETag header. If they match, then it returns a 304.