6

For this setup with default (unspecified) search_data:

class Item < ActiveRecord::Base
 searchkick
 has_many :quantities, dependent: :destroy
 scope :search_import, -> { includes(:quantities) }

end

When importing all database records by running Item.reindex, each "batch" eager loads the quantities for all of the item ids as expected.

However, if I want to specify the index model document differently than the default attributes using the search_data method including the associated model data with something like this:

class Item < ActiveRecord::Base
 searchkick
 has_many :quantities, dependent: :destroy

 def search_data
   {
     part_number: standard_part_number,
     category_id: category_id.to_i,
     content_source_name: content_source_name,
     standard_price: standard_price.to_i,
     locations: quantities.map {|q| [q.location_code,
       quantity: q.value.to_i,
       bins: q.location_bins]
     }.to_h
   }
 end

 scope :search_import, -> { includes(:quantities) }

end

where I am operating on the quantities using map to define a locations attribute, returning to import using Item.reindex I see that it not only eager loads all of the associated quantities each batch, but it also then loads all quantities per item with a hit to the database again for each item record it is indexing.

Am I missing something here to get Searchkick to eager load the quantities models and be able to customize the search data record using that already loaded associated model without it doing another pull from the database again per item?

Update I've determined there is something interfering with the way the ActiveRecord normally responds to the method name used for the association with the eager loaded models in our app and may not be exclusively related to us using Searchkick, although it did reveal the issue. I am not sure what it is that is interfering at this time but it has something to do with the target method on the association. Perhaps a gem that is installed is causing this problem. I did manage to find a way to work around this (for now) using item.association(:quantities).target as a replacement for item.quantities and now when running the reindex it makes use of the already eager loaded quantities and doesn't hit the db again for each item

Streamline
  • 2,040
  • 4
  • 37
  • 56

1 Answers1

0

I see that it not only eager loads all of the associated quantities each batch

This is expected behaviour (and very likely performant), since each batch you will get different quantities to load, since they are connecting to different items, so you don't need to keep all quantities in memory.

each batch, but it also then loads all quantities per item with a hit to the database again for each item record it is indexing.

This is actually unexpected, but my guess here is, that in one of the methods in Quantity (#location_code #value or #location_bins) or even one of the methods on Item that you call (#standard_part_number, #category_id, #content_source_name, #standard_price), there is some implementation, that requires reloading of records. Without knowing the code of that methods it is purely speculative, but the presented part of the code looks fine.

smallbutton
  • 3,377
  • 15
  • 27
  • Hmm. I am not sure what is happening here. All of those other fields referenced for `Item` and `Quantity` are just attributes, not specifically defined methods. As a test I commented out all code in the `Quantity` model except the association definition and reduced the `Item` `search_data` to just `standard_part_number` and the `locations` : `quantities.map` and I still get the same unexpected results. – Streamline Sep 01 '18 at 17:34
  • I further reduced the `Item` `search_data` method to `{ standard_part_number: standard_part_number, quantities: quantities.map{|q| q.value} }` and still get the same unexpected loading of all associated `quantities` record per `item` record. This rails app was just migrated from rails 3 to rails 4. I wonder if there are settings or something in place left over from rails 3 that is signaling the already eager loaded model data to be ignored if this is not an artifact of `Searchkick`. Not sure yet. Need to figure out how to confirm through further investigation. – Streamline Sep 01 '18 at 17:59
  • I've determined there is something interfering with the way the activerecord normally responds to the method name with the eager loaded models in the app. I am not sure what it is that is interfering but it has something to do with the target method on the association. Perhaps a gem that is installed is causing this problem. I did manage to find a way to work around this (for now) using `item.association(:quantities).target` as a replacement for `item.quantities` and now when running the `reindex` it makes use of the already eager loaded quantities and doesn't hit the db again for each `item`. – Streamline Sep 02 '18 at 07:55