org.apache.lucene.benchmark.byTask.feeds
public abstract class ContentSource extends java.lang.Object
ContentSource
is responsible for creating DocData
objects for
its documents to be consumed by DocMaker
. It also keeps track
of various statistics, such as how many documents were generated, size in
bytes etc.
Supports the following configuration parameters:
Modifier and Type | Field and Description |
---|---|
protected static int |
BUFFER_SIZE |
protected java.lang.String |
encoding |
protected boolean |
forever |
protected int |
logStep |
protected boolean |
verbose |
Constructor and Description |
---|
ContentSource() |
Modifier and Type | Method and Description |
---|---|
protected void |
addBytes(long numBytes) |
protected void |
addDoc() |
abstract void |
close()
Called when reading from this content source is no longer required.
|
protected void |
collectFiles(java.io.File dir,
java.util.ArrayList files)
A convenience method for collecting all the files of a content source from
a given directory.
|
long |
getBytesCount()
Returns the number of bytes generated since last reset.
|
Config |
getConfig() |
int |
getDocsCount()
Returns the number of generated documents since last reset.
|
protected java.io.InputStream |
getInputStream(java.io.File file)
Returns an
InputStream over the requested file. |
abstract DocData |
getNextDocData(DocData docData)
Returns the next
DocData from the content source. |
long |
getTotalBytesCount()
Returns the total number of bytes that were generated by this source.
|
int |
getTotalDocsCount()
Returns the total number of generated documents.
|
void |
resetInputs()
Resets the input for this content source, so that the test would behave as
if it was just started, input-wise.
|
void |
setConfig(Config config)
Sets the
Config for this content source. |
protected boolean |
shouldLog()
Returns true whether it's time to log a message (depending on verbose and
the number of documents generated).
|
protected static final int BUFFER_SIZE
protected boolean forever
protected int logStep
protected boolean verbose
protected java.lang.String encoding
protected final void addBytes(long numBytes)
protected final void addDoc()
protected final void collectFiles(java.io.File dir, java.util.ArrayList files)
File
instances are stored in the
given files
.protected java.io.InputStream getInputStream(java.io.File file) throws java.io.IOException
InputStream
over the requested file. This method
attempts to identify the appropriate InputStream
instance to return
based on the file name (e.g., if it ends with .bz2 or .bzip, return a
'bzip' InputStream
).java.io.IOException
protected final boolean shouldLog()
public abstract void close() throws java.io.IOException
java.io.IOException
public final long getBytesCount()
public final int getDocsCount()
public final Config getConfig()
public abstract DocData getNextDocData(DocData docData) throws NoMoreDataException, java.io.IOException
DocData
from the content source.NoMoreDataException
java.io.IOException
public final long getTotalBytesCount()
public final int getTotalDocsCount()
public void resetInputs() throws java.io.IOException
NOTE: the default implementation resets the number of bytes and documents generated since the last reset, so it's important to call super.resetInputs in case you override this method.
java.io.IOException
Copyright © 2000-2014 Apache Software Foundation. All Rights Reserved.