How to set Django queryset to GKE memcached

Question

Somehow I thought, pymemcached should work out of the box like django cache.

But what I am realizing is it doesn't.

Going through the docs, there are methods like set_multi, get_multi. This should work well with django dictionaries(key/value pairs).

Given that Django querysets are not tranditionally dictionaries, what would be the method to use with pymemcached.

Here is how am using it for the moment:

if not settings.DEBUG:
    from pymemcache.client.base import Client

    NODE_NAME = os.environ['NODE_NAME']
    client = Client((NODE_NAME, 5000))

class ProductsListCacheAPIView(ListAPIView):
    
    model = Product
    
    serializer_class = TestProductSerializer
    filter_backends= [SearchFilter, OrderingFilter]
    permission_classes = [AllowAny]
    search_fields = ['product_title', 'product_description', 'user__first_name', 'product_type', 'product_price', 'product_city']
    pagination_class = ProductPageNumberPagination
    

    def get_queryset(self, *args, **kwargs):
 
        query = self.request.GET.get("search")
        lookups = ()

        .....

            queryset_list = queryset_list.filter(lookups).distinct()

        cache_key = 'my_api_marketplace_cache' 
        data = client.get(cache_key)
        
        if data is None:
            print('Getting marketplace fresh from database!')
            data = queryset_list
            client.set(cache_key, queryset_list, cache_time)
        return data

This doesn't work ofcourse. Seems unlike django cache

from django.core.cache import cache (django.core.cache.backends.memcached.PyLibMCCache)

The set method seems to work only with strings for pymemcached.

How can I use pymemcached here to cache django querysets?

score 1 · Answer 1 · answered Dec 06 '20 at 18:36

The short answer is that not each cache is a query cache. The Django Cache is optimized to be a response cache (page cache) and works with minimal configuration to cache completely rendered pages.

Additionally, with decorators or calling caching functions correctly, you can cache things that can be serialized to a string as you've discovered.

The usefulness of a queryset cache is minimal:

First you can cache a queryset for views (both DRF and Django views) at declaration time, if it's not depending on runtime data. This saves the building of the queryset structure, but doesn't evaluate it
Second, databases have query caches, outside of Django's scope that you can leverage.
Third, there are very few cases, where caching the evaluated queryset is a better approach because it can be used to render different outputs. In general, the final queryset is either dependant on request context or the same queryset generates the same output, so why not cache the output.

Looking at your case, you fall in the 3rd category: you can simply cache the view and let it depend on the search parameter. There will be no upside to caching just the queryset.

How to set Django queryset to GKE memcached

1 Answers1