Django rest api - searching a method field with the search filter

Question

im trying to filter search a rest api page and want to use a method field as one of the search fields, however when I do this I get an error stating the field is not valid and it then lists the field in my model as the only valid source

serialiser:

class SubnetDetailsSerializer(QueryFieldsMixin, serializers.HyperlinkedModelSerializer):
    subnet = serializers.SerializerMethodField()
    device = serializers.ReadOnlyField(
        source='device.hostname',
    )
    circuit_name = serializers.ReadOnlyField(
        source='circuit.name',
    )
    subnet_name = serializers.ReadOnlyField(
        source='subnet.description',
    )
    safe_subnet = serializers.SerializerMethodField()

    def get_safe_subnet(self, obj):
        return '{}{}'.format(obj.subnet.subnet, obj.subnet.mask.replace('/','_')) 

    def get_subnet(self, obj):
        return '{}{}'.format(obj.subnet.subnet, obj.subnet.mask) 

    class Meta:
        model = DeviceCircuitSubnets   
        fields = ('id','device_id','subnet_id','circuit_id','subnet','safe_subnet','subnet_name','device','circuit_name')

views:

class SubnetDetailsSet(viewsets.ReadOnlyModelViewSet):
    queryset = DeviceCircuitSubnets.objects.all().select_related('circuit','subnet','device')
    serializer_class = SubnetDetailsSerializer
    permission_classes = (IsAdminUser,)
    filter_class = DeviceCircuitSubnets
    filter_backends = (filters.SearchFilter,)
    search_fields = (
        'device__hostname',
        'circuit__name',
        'subnet__subnet',
        'safe_subnet'
    )

how can include the safe_subnet in the search fields?

Thanks

EDIT This is the code now

views.py

class SubnetDetailsSet(viewsets.ReadOnlyModelViewSet):
    queryset = DeviceCircuitSubnets.objects.all()
    serializer_class = SubnetDetailsSerializer
    permission_classes = (IsAdminUser,)
    filter_class = DeviceCircuitSubnets
    filter_backends = (filters.SearchFilter,)
    search_fields = (
        'device__hostname',
        'circuit__name',
        'subnet__subnet',
        'safe_subnet'
    )

    def get_queryset(self):
        return (
            super().get_queryset()
            .select_related('circuit','subnet','device')
            .annotate(
                safe_subnet=Concat(
                    F('subnet__subnet'),
                    Replace(F('subnet__mask'), V('/'), V('_')),
                    output_field=CharField()
                )
            )
        )

serializer.py

class SubnetDetailsSerializer(QueryFieldsMixin, serializers.HyperlinkedModelSerializer):
    subnet = serializers.SerializerMethodField()
    device = serializers.ReadOnlyField(
        source='device.hostname',
    )
    circuit_name = serializers.ReadOnlyField(
        source='circuit.name',
    )
    subnet_name = serializers.ReadOnlyField(
        source='subnet.description',
    )
    def get_safe_subnet(self, obj):
        return getattr(obj, 'safe_subnet', None)

    def get_subnet(self, obj):
        return '{}{}'.format(obj.subnet.subnet, obj.subnet.mask) 

    class Meta:
        model = DeviceCircuitSubnets   
        fields = ('id','device_id','subnet_id','circuit_id','subnet','safe_subnet','subnet_name','device','circuit_name')

Model:

class DeviceCircuitSubnets(models.Model):
    device = models.ForeignKey(Device, on_delete=models.CASCADE)
    circuit = models.ForeignKey(Circuit, on_delete=models.CASCADE, blank=True, null=True)
    subnet = models.ForeignKey(Subnet, on_delete=models.CASCADE)
    active_link = models.BooleanField(default=False, verbose_name="Active Link?")
    active_link_timestamp = models.DateTimeField(auto_now=True, blank=True, null=True)

Error:

Exception Type: ImproperlyConfigured at /api/subnets/
Exception Value: Field name `safe_subnet` is not valid for model `DeviceCircuitSubnets`.

searches are performed using queries, you can't search on a method field. — dirkgroten, Aug 30 '19 at 11:20
Do you *need* to use a ?search= for all of these, or can you use a custom filter that supports `?subnet=192.168/24&device=django&circut=31` style queries? The latter is preferable, in this kind of situation. — Andrew, Sep 23 '19 at 13:21

bdoubleu · Answer 1 · 2019-08-30T11:43:39.727

5

You need to annotate your queryset with the safe_subnet attribute so it becomes searchable.

from django.db.models import F, Value as V
from django.db.models.functions import Concat, Replace

class SubnetDetailsSet(viewsets.ReadOnlyModelViewSet):
    queryset = DeviceCircuitSubnets.objects.all()
    serializer_class = SubnetDetailsSerializer
    permission_classes = (IsAdminUser,)
    filter_class = DeviceCircuitSubnets
    filter_backends = (filters.SearchFilter,)
    search_fields = (
        'device__hostname',
        'circuit__name',
        'subnet__subnet',
        'safe_subnet'
    )

    def get_queryset(self):
        return (
            super().get_queryset()
            .select_related('circuit','subnet','device')
            .annotate(
                safe_subnet=Concat(
                    F('subnet__subnet'),
                    Replace(F('subnet__mask'), V('/'), V('_')),
                    output_field=CharField()
                )
            )
        )

Then in your serializer you can use the following.

def get_safe_subnet(self, obj):
    return obj.safe_subnet

edited Aug 30 '19 at 11:43

answered Aug 30 '19 at 11:31

bdoubleu

5,568
2
20
53

Im seeing the error Field name `safe_subnet` is not valid for model `DeviceCircuitSubnets`. stil – AlexW Aug 30 '19 at 12:27
If the error is caused by the filter then it's possibly a result of this [DRF issue](https://github.com/encode/django-rest-framework/issues/6094) and you should be able to solve by upgrading. Otherwise, if the serializer is used outside `SubnetDetailsSet` and the queryset isn't annotated then you would also get this error. You could change to `return getattr(obj, 'safe_subnet', None)` if you don't want to annotate for some reason. – bdoubleu Sep 03 '19 at 11:43
I've tried upgrading and adding what was suggested but I still get the same error – AlexW Sep 12 '19 at 10:47
@AlexW does the annotation work when you use it in the shell? You need to look at the traceback and see what's causing the error. Likely `SearchFilter` doesn't call the view's `.get_queryset()` method before filtering so you need to figure out how to do that. – bdoubleu Sep 17 '19 at 09:55
the annotation works through shell yes, I have no idea how to edit the search filter to include that, would you happen to know how to do this? Thanks – AlexW Sep 18 '19 at 14:14

Julien Kieffer · Answer 2 · 2019-09-27T10:01:10.553

Previous answer with annotate is a really good start:

from .rest_filters import DeviceCircuitSubnetsFilter  

class SubnetDetailsSet(viewsets.ReadOnlyModelViewSet):
    queryset = DeviceCircuitSubnets.objects.all()
    serializer_class = SubnetDetailsSerializer
    permission_classes = (IsAdminUser,)
    # That's where hint lays
    filter_class = DeviceCircuitSubnetsFilter
    #filter_backends = (filters.SearchFilter,)
    search_fields = (
        'device__hostname',
        'circuit__name',
        'subnet__subnet',
        'safe_subnet'
    )

    #No need to override your queryset

Now in rest_filters.py

from django_filters import rest_framework as filters
from django.db.models import F, Value as V
from django.db.models.functions import Concat, Replace
#.... import models

class DeviceCircuitSubnets(filters.FilterSet):

    safe_subnet = filters.CharFilter(
        name='safe_subnet',
        method='safe_subnet_filter')

    def safe_subnet_filter(self, queryset, name, value):
        """
        Those line will make ?safe_subnet=your_pk available
        """
        return  queryset.annotate(
                    safe_subnet=Concat(
                        F('subnet__subnet'),
                        Replace(F('subnet__mask'), V('/'), V('_')),
                        output_field=CharField()
                )
            ).filter(safe_subnet=value)
        )
    class Meta:
        model = DeviceCircuitSubnets
        # See https://django-filter.readthedocs.io/en/master/guide/usage.html#generating-filters-with-meta-fields
        # This pattern is definitely a killer!
        fields = {
            'device': ['exact', 'in'],
            'circuit': ['exact', 'in'],
            'subnet': ['exact', 'in'],
            'active_link': ['exact'],
            'active_link_timestamp': ['lte', 'gte']
        }

Please note: I'm annotating safe_subnet within the filer, depending on how much you use this, you might want to set this up in your model's manager!

I'm getting Error: safe_subnet = django_filters.IntegerField( AttributeError: module 'django_filters' has no attribute 'IntegerField' — AlexW, Sep 26 '19 at 14:53
I still see the same AttributeError: module 'django_filters.rest_framework' has no attribute 'IntegerField' — AlexW, Sep 27 '19 at 09:45
Sorry... so used to Serializers/Models... Correct name was `NumberFilter`... response is updated with `CharFilter`: If filter does not match your data type, you can find full list here: https://django-filter.readthedocs.io/en/master/ref/filters.html#filters My guess is you'll need CharFilter as `safe_subnet` is CharField — Julien Kieffer, Sep 27 '19 at 10:02
ok its gone back to the original error now for some reason "Cannot resolve keyword 'safe_subnet' into field" — AlexW, Sep 27 '19 at 11:28
Did you keep `safe_subnet` from your APIView `search_fields`? If yes can you try without this ? — Julien Kieffer, Sep 28 '19 at 06:48
I now get __init__() got an unexpected keyword argument 'name' which I think is erring under kwargs {'label': '[invalid name]', 'name': 'safe_subnet', 'required': False} — AlexW, Sep 30 '19 at 07:14

score 0 · Accepted Answer · answered Sep 26 '19 at 20:19

Going in a completely different direction from the other (excellent) answers. Since you want to be able to filter frequently on the safe_subnet field, why not just let it be an actual database field in your model? You could calculate and populate/update the value during one of your save methods and then just let django-filters do it's thing. This also has the advantage of allowing the filtering to be done directly through SQL which would theoretically provide better performance.

I did end up going with this, overriding the form valid method and creating it on the fly in the DB — AlexW, Oct 04 '19 at 11:33

Django rest api - searching a method field with the search filter

3 Answers3