org.apache.lucene.analysis.nl
public class DutchAnalyzer extends org.apache.lucene.analysis.Analyzer
Analyzer
for Dutch language.
Supports an external list of stopwords (words that will not be indexed at all), an external list of exclusions (word that will not be stemmed, but indexed) and an external list of word-stem pairs that overrule the algorithm (dictionary stemming). A default set of stopwords is used unless an alternative list is specified, but the exclusion list is empty by default.
NOTE: This class uses the same Version
dependent settings as StandardAnalyzer
.
Modifier and Type | Field and Description |
---|---|
static java.lang.String[] |
DUTCH_STOP_WORDS
List of typical Dutch stopwords.
|
Constructor and Description |
---|
DutchAnalyzer()
Deprecated.
Use
DutchAnalyzer(Version) instead |
DutchAnalyzer(java.io.File stopwords)
Deprecated.
Use
DutchAnalyzer(Version, File) instead |
DutchAnalyzer(java.util.HashSet stopwords)
Deprecated.
Use
DutchAnalyzer(Version, HashSet) instead |
DutchAnalyzer(java.lang.String[] stopwords)
Deprecated.
Use
DutchAnalyzer(Version, String[]) instead |
DutchAnalyzer(org.apache.lucene.util.Version matchVersion)
Builds an analyzer with the default stop words (
DUTCH_STOP_WORDS )
and a few default entries for the stem exclusion table. |
DutchAnalyzer(org.apache.lucene.util.Version matchVersion,
java.io.File stopwords)
Builds an analyzer with the given stop words.
|
DutchAnalyzer(org.apache.lucene.util.Version matchVersion,
java.util.HashSet stopwords)
Builds an analyzer with the given stop words.
|
DutchAnalyzer(org.apache.lucene.util.Version matchVersion,
java.lang.String[] stopwords)
Builds an analyzer with the given stop words.
|
Modifier and Type | Method and Description |
---|---|
org.apache.lucene.analysis.TokenStream |
reusableTokenStream(java.lang.String fieldName,
java.io.Reader reader)
Returns a (possibly reused)
TokenStream which tokenizes all the
text in the provided Reader . |
void |
setStemDictionary(java.io.File stemdictFile)
Reads a stemdictionary file , that overrules the stemming algorithm
This is a textfile that contains per line
word\tstem, i.e: two tab seperated words
|
void |
setStemExclusionTable(java.io.File exclusionlist)
Builds an exclusionlist from the words contained in the given file.
|
void |
setStemExclusionTable(java.util.HashSet exclusionlist)
Builds an exclusionlist from a Hashtable.
|
void |
setStemExclusionTable(java.lang.String[] exclusionlist)
Builds an exclusionlist from an array of Strings.
|
org.apache.lucene.analysis.TokenStream |
tokenStream(java.lang.String fieldName,
java.io.Reader reader)
Creates a
TokenStream which tokenizes all the text in the
provided Reader . |
public static final java.lang.String[] DUTCH_STOP_WORDS
public DutchAnalyzer()
DutchAnalyzer(Version)
insteadDUTCH_STOP_WORDS
)
and a few default entries for the stem exclusion table.public DutchAnalyzer(org.apache.lucene.util.Version matchVersion)
DUTCH_STOP_WORDS
)
and a few default entries for the stem exclusion table.public DutchAnalyzer(java.lang.String[] stopwords)
DutchAnalyzer(Version, String[])
insteadstopwords
- public DutchAnalyzer(org.apache.lucene.util.Version matchVersion, java.lang.String[] stopwords)
matchVersion
- stopwords
- public DutchAnalyzer(java.util.HashSet stopwords)
DutchAnalyzer(Version, HashSet)
insteadstopwords
- public DutchAnalyzer(org.apache.lucene.util.Version matchVersion, java.util.HashSet stopwords)
stopwords
- public DutchAnalyzer(java.io.File stopwords)
DutchAnalyzer(Version, File)
insteadstopwords
- public DutchAnalyzer(org.apache.lucene.util.Version matchVersion, java.io.File stopwords)
stopwords
- public void setStemExclusionTable(java.lang.String[] exclusionlist)
exclusionlist
- public void setStemExclusionTable(java.util.HashSet exclusionlist)
public void setStemExclusionTable(java.io.File exclusionlist)
public void setStemDictionary(java.io.File stemdictFile)
public org.apache.lucene.analysis.TokenStream tokenStream(java.lang.String fieldName, java.io.Reader reader)
TokenStream
which tokenizes all the text in the
provided Reader
.tokenStream
in class org.apache.lucene.analysis.Analyzer
TokenStream
built from a StandardTokenizer
filtered with StandardFilter
, StopFilter
,
and DutchStemFilter
public org.apache.lucene.analysis.TokenStream reusableTokenStream(java.lang.String fieldName, java.io.Reader reader) throws java.io.IOException
TokenStream
which tokenizes all the
text in the provided Reader
.reusableTokenStream
in class org.apache.lucene.analysis.Analyzer
TokenStream
built from a StandardTokenizer
filtered with StandardFilter
, StopFilter
,
and DutchStemFilter
java.io.IOException
Copyright © 2000-2014 Apache Software Foundation. All Rights Reserved.