thinkmap.datasource.util
Class ReservedFieldStandardAnalyzer

java.lang.Object
  extended by org.apache.lucene.analysis.Analyzer
      extended by thinkmap.datasource.util.ReservedFieldStandardAnalyzer
All Implemented Interfaces:
Closeable

public class ReservedFieldStandardAnalyzer
extends org.apache.lucene.analysis.Analyzer

Filters StandardTokenizer with StandardFilter, LowerCaseFilter and StopFilter, using a list of English stop words.

You must specify the required Version compatibility when creating StandardAnalyzer:


Field Summary
static int DEFAULT_MAX_TOKEN_LENGTH
          Default maximum allowed token length
static String[] RESERVED_FIELDS
           
 Set<String> reservedFields
           
static Set<?> STOP_WORDS_SET
          An unmodifiable set containing some common English words that are usually not useful for searching.
 
Constructor Summary
ReservedFieldStandardAnalyzer()
          Builds an analyzer with the default stop words (STOP_WORDS_SET).
ReservedFieldStandardAnalyzer(org.apache.lucene.util.Version matchVersion)
           
ReservedFieldStandardAnalyzer(org.apache.lucene.util.Version matchVersion, Set<?> stopWords)
          Builds an analyzer with the given stop words.
 
Method Summary
 int getMaxTokenLength()
           
 org.apache.lucene.analysis.TokenStream reusableTokenStream(String fieldName, Reader reader)
           
 void setMaxTokenLength(int length)
          Set maximum allowed token length.
 org.apache.lucene.analysis.TokenStream tokenStream(String fieldName, Reader reader)
          Constructs a StandardTokenizer filtered by a StandardFilter, a LowerCaseFilter and a StopFilter.
 
Methods inherited from class org.apache.lucene.analysis.Analyzer
close, getOffsetGap, getPositionIncrementGap, getPreviousTokenStream, setPreviousTokenStream
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

STOP_WORDS_SET

public static final Set<?> STOP_WORDS_SET
An unmodifiable set containing some common English words that are usually not useful for searching.


RESERVED_FIELDS

public static final String[] RESERVED_FIELDS

reservedFields

public final Set<String> reservedFields

DEFAULT_MAX_TOKEN_LENGTH

public static final int DEFAULT_MAX_TOKEN_LENGTH
Default maximum allowed token length

See Also:
Constant Field Values
Constructor Detail

ReservedFieldStandardAnalyzer

public ReservedFieldStandardAnalyzer()
Builds an analyzer with the default stop words (STOP_WORDS_SET).

Parameters:
matchVersion - Lucene version to match See above

ReservedFieldStandardAnalyzer

public ReservedFieldStandardAnalyzer(org.apache.lucene.util.Version matchVersion)

ReservedFieldStandardAnalyzer

public ReservedFieldStandardAnalyzer(org.apache.lucene.util.Version matchVersion,
                                     Set<?> stopWords)
Builds an analyzer with the given stop words.

Parameters:
matchVersion - Lucene version to match See above
stopWords - stop words
Method Detail

tokenStream

public org.apache.lucene.analysis.TokenStream tokenStream(String fieldName,
                                                          Reader reader)
Constructs a StandardTokenizer filtered by a StandardFilter, a LowerCaseFilter and a StopFilter.

Specified by:
tokenStream in class org.apache.lucene.analysis.Analyzer

setMaxTokenLength

public void setMaxTokenLength(int length)
Set maximum allowed token length. If a token is seen that exceeds this length then it is discarded. This setting only takes effect the next time tokenStream or reusableTokenStream is called.


getMaxTokenLength

public int getMaxTokenLength()
See Also:
setMaxTokenLength(int)

reusableTokenStream

public org.apache.lucene.analysis.TokenStream reusableTokenStream(String fieldName,
                                                                  Reader reader)
                                                           throws IOException
Overrides:
reusableTokenStream in class org.apache.lucene.analysis.Analyzer
Throws:
IOException


Copyright © 2013 Thinkmap, Inc. All Rights Reserved.