lucene Analysis Iterating manually through analyzed tokens


Example

TokenStream stream = myAnalyzer.tokenStream("myField", textToAnalyze);
stream.addAttribute(CharTermAttribute.class);
stream.reset();
while(stream.incrementToken()) {
    CharTermAttribute token = stream.getAttribute(CharTermAttribute.class);
    System.out.println(token.toString());
}

stream.close();

A number of Attributes are available. The most common is CharTermAttribute, which is used to get the analyzed term as a String.