How to fix ElasticSearch ‘Fielddata is disabled on text fields by default’ for keyword field

Question

I'm getting the "Fielddata is disabled on text fields by default" on a keyword field. Below is the code.

{
 "aggs": {
   "agg_terms_user": {
     "terms": {
       "field": "user"
     }
   }
 }
}

The mapping for the user field is as below

user: { type: "keyword" }

Since the user field has type set as keyword I shouldn't get the error. However, the error is still thrown.

[illegal_argument_exception] Fielddata is disabled on text fields by default. Set fielddata=true on [user] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.

I don't know what to try now.

That won't work. The user field is already set as keyword. Had I mapped it as user: {type: "text",fields: { keyword: { type: "keyword" } then it would have made sense to use "user.keyword" — Nikhil, Dec 12 '19 at 06:00
Make sure that `user` (or the field you are using) is of type `keyword` because this error is the one that ES rises when doing a term aggregation on a `text` field. — moliware, Dec 12 '19 at 06:52
Also, I've a doubt. How do I do range aggregation over a long field if it mandatory to keep it keyword — Nikhil, Dec 12 '19 at 08:28
It's an intermittent issue. It works sometime while sometimes it fails. Also, it works just fine without error in one of the instance while it throws intermittent errors in the other. Both the instances are on AWS and have the exact same code. I checked the AWS ES configuration and I don't find anything suspicious. Am I missing anything — Nikhil, Dec 12 '19 at 08:36
NO, your user field is text datatype, is written in the error stack Fielddata is disabled on text fields by default. Set fielddata=true on [user] in order to load fielddata — Lupanoide, Dec 12 '19 at 08:47
maybe you have a subfield called user.keyword of type keyword — Lupanoide, Dec 12 '19 at 08:47
That's what I'm saying, This error is unexpected. It's keyword in my code and I don't have any subfield. Also, it works just fine in one instance of app while throws error in the other instance. — Nikhil, Dec 12 '19 at 08:51
Are you sure that you haven't subfield? Could you try { "aggs": { "agg_terms_user": { "terms": { "field": "user.keyword" } } } } — Lupanoide, Dec 12 '19 at 09:17
How do you have taken your mapping? Could you please post the output of GET /_mapping — Lupanoide, Dec 12 '19 at 09:18

score 34 · Answer 1 · edited Jan 18 '21 at 06:06

34

The comment of @Andrey Borisko was correct

I used

"field": "user.keyword"

instead of

"field": "user"

based on Nikhil's example and it worked for me.

edited Jan 18 '21 at 06:06

Gino Mempin

25,369
29
96
135

answered Oct 20 '20 at 16:01

Investigator

1,431
2
17
24

1

If you don't have access/permissions to change the index mappings and/or to re-index, this is the way to go. – Gino Mempin Jan 18 '21 at 06:08

score 9 · Accepted Answer · answered Dec 12 '19 at 09:52

9

I found the reason behind the unexpected error. The ES wasn't getting re-indexed properly. Once I deleted the indexed first and then recreated it then it started working like a charm.

answered Dec 12 '19 at 09:52

Nikhil

665
2
11
25

2

Can you provide more details on how you did that? (v6.8) – rtrigo Jun 12 '20 at 17:05
1

Delete the index and recreate it. – Nikhil Jul 20 '20 at 06:53
2

Deleting indexes really helped and you can do in your server terminal using this command `curl -X DELETE 'http://localhost:9200/your_index_name'` and just in case you don't know the name of your index, you can use `curl -X GET 'http://localhost:9200/_search'` and you will see your index names. Replace your domain name and port if any changes in default setup. – djmzfKnm Aug 06 '20 at 09:23

score 2 · Answer 3 · answered Jul 20 '21 at 07:28

Check out the documentation in the elastic which clearly mentions that we cannot use the text field for aggregations, sorting, or scripting

https://www.elastic.co/guide/en/elasticsearch/reference/current/text.html#fielddata-mapping-param

This can be achieved with help of keyword usage. Try searching by

"field" : "user.keyword"

Ndm · Answer 4 · 2022-05-18T05:34:09.847

1

Facing the same issue in heartbeat application.I have solved this issue by following below steps.

stop the application [ in my case I stopped the heartbeat application]
Delete the index related to apps.
Start the application

edited May 18 '22 at 05:34

answered May 18 '22 at 05:33

Ndm

11
1
5

How to fix ElasticSearch ‘Fielddata is disabled on text fields by default’ for keyword field

4 Answers4