3

I'm not getting the expected results from my Analyzer and would like to test the tokenization process.

The answer this question: How to use a Lucene Analyzer to tokenize a String?

List<String> result = new ArrayList<String>();
TokenStream stream  = analyzer.tokenStream(field, new StringReader(keywords));

try {
    while(stream.incrementToken()) {
        result.add(stream.getAttribute(TermAttribute.class).term());
    }
}
catch(IOException e) {
    // not thrown b/c we're using a string reader...
}

return result;

Uses the TermAttribute to extract the tokens from the stream. The problem is that TermAttribute is no longer in Lucene 6.

What has it been replaced by?

What would the equivalent be with Lucene 6.6.0?

Martinffx
  • 2,426
  • 4
  • 33
  • 60
  • As [this answer](https://stackoverflow.com/a/2638252/981744) describes, TermAttribute was replaced with `CharTermAttribute`. I'm not a Lucene expert but this is what you get a from a little Googling. – Erwin Bolwidt Jun 21 '17 at 09:06
  • 1
    Thanks @ErwinBolwidt, that answer was exactly what I was looking for... my lucene vocabulary is clearly lacking. Can you you throw it in answer? – Martinffx Jun 21 '17 at 09:44

1 Answers1

0

I'm pretty sure it was replaced by CharTermAttribute javadoc

The ticket is pretty old, but maybe the code was kept around a bit longer: https://issues.apache.org/jira/browse/LUCENE-2372

Slomo
  • 1,224
  • 8
  • 11