Elasticsearch query match multiple values to single field

Question

Trying to get documents matching values X1 or Y1 for field ABC. Tried both must or should queries, but not getting expected result. Can someone suggest what kind of query should I try? Using HighLevelRestClient.

{
  "bool" : {
    "must" : [
      {
        "term" : {
          "ABC" : {
            "value" : "X1",
            "boost" : 1.0
          }
        }
      },
      {
        "term" : {
          "ABC" : {
            "value" : "Y1",
            "boost" : 1.0
          }
        }
      }
    ]
  }
}

OR

{
  "bool" : {
    "should" : [
      {
        "term" : {
          "ABC" : {
            "value" : "X1",
            "boost" : 1.0
          }
        }
      },
      {
        "term" : {
          "ABC" : {
            "value" : "Y1",
            "boost" : 1.0
          }
        }
      }
    ]
  }
}

Mapping


{
  "mappings": {
    "properties": {
      "ABC": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 2
          }
        }
      },

mustNot condition works fine. If I just reverse the condition and ignore fields values then I get the result.

X1 and Y1 are exact field values (Think enums)

BoolQueryBuilder x = QueryBuilders.boolQuery();
for (SomeEnum enum : enums) { 
   x.should(QueryBuilders.termQuery("ABC",enum.name());
}

Still query returns all documents. This should have filtered the documents to matching values

Sample Doc


{
  "_index": "some_index",
  "_type": "_doc",
  "_id": "uyeuyeuryweoyqweo",
  "_score": 1.0,
  "_source": {
    "A": true
    "ABC": "X1"
    "WS": "E"
  }
}, 
{
  "_index" : "some_index",
  "_type" : "_doc",
  "_id" : "uyeuyeuryweoyqweo1",
  "_score" : 1.0,
  "_source" : {
    "A" : true,
    "ABC" : "Y1",
    "WS" : "MMM"
  }
}

can you show your mapping, sample docs and expected output? – Amit Aug 09 '20 at 09:08 — Amit, Aug 09 '20 at 09:08

Amit · Accepted Answer · 2020-08-09T09:33:08.300

2

As you have not provided your mapping, possible cause is due to the mismatch of search time tokens to index-tokens.

As you are using term query which are not analyzed as mentioned in doc

Returns documents that contain an exact term in a provided field.

It means that you documents in index must conatins the exact tokens as X1 and Y1 and if these fields are text fields and you have not defined any anlyzer than elasticsearch uses standard analyzer which lowercases the tokens, hence in index x1 and y1 would be stored and nothing will match.

EDIT : As suspected, issue was due to term query used on text fields, below query would give the expected results

{
  "bool" : {
    "should" : [
      {
        "term" : {
          "ABC.keyword" : {
            "value" : "X1",
            "boost" : 1.0
          }
        }
      },
      {
        "term" : {
          "ABC.keyword" : {
            "value" : "Y1",
            "boost" : 1.0
          }
        }
      }
    ]
  }
}

edited Aug 09 '20 at 09:33

answered Aug 09 '20 at 09:12

Amit

30,756
6
57
88

I have added mapping – java_dude Aug 09 '20 at 09:24
@java_dude got it, you just need to use the `should` and `ABC.keyword` in your query to get the expected output :) – Amit Aug 09 '20 at 09:29
Result returns all documents when using should. Added rest client code – java_dude Aug 09 '20 at 09:50
@java_dude, could you show your sample and expected docs? you need both X1 and Y1 present or any one would suffice? – Amit Aug 09 '20 at 09:53
Document can have either. Need to pick all the documents matching values in the fiel. Added sample document – java_dude Aug 09 '20 at 10:00
Added two sample document. Query should pick both of these. – java_dude Aug 09 '20 at 10:07
@java_dude should query should bring both of your sample docs, can you try the query I provided in answer using postman or curl request against your index? – Amit Aug 09 '20 at 10:12
The query works using curl. Will try implementing it using HighLevelRestClient. Will update this post in 8 hours. Much appreciated your help. – java_dude Aug 09 '20 at 10:38
Curl works but HighLevelRest Client does not. I will post a new question and refer it here – java_dude Aug 09 '20 at 22:36
@java_dude thanks for marking answer , were u able to solve using JHLRC? – Amit Aug 10 '20 at 01:35

Elasticsearch query match multiple values to single field

1 Answers1