Class IntersectBlockReader
- java.lang.Object
-
- org.apache.lucene.index.TermsEnum
-
- org.apache.lucene.index.BaseTermsEnum
-
- org.apache.lucene.codecs.uniformsplit.BlockReader
-
- org.apache.lucene.codecs.uniformsplit.IntersectBlockReader
-
- All Implemented Interfaces:
Accountable
,BytesRefIterator
- Direct Known Subclasses:
STIntersectBlockReader
public class IntersectBlockReader extends BlockReader
The "intersect"TermsEnum
response toUniformSplitTerms.intersect(CompiledAutomaton, BytesRef)
, intersecting the terms with an automaton.By design of the UniformSplit block keys, it is less efficient than
org.apache.lucene.backward_codecs.lucene40.blocktree.IntersectTermsEnum
forFuzzyQuery
(-37%). It is slightly slower forWildcardQuery
(-5%) and slightly faster forPrefixQuery
(+5%).- WARNING: This API is experimental and might change in incompatible ways in the next release.
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description protected class
IntersectBlockReader.AutomatonNextTermCalculator
This is mostly a copy of AutomatonTermsEnum.protected static class
IntersectBlockReader.BlockIteration
Block iteration order.-
Nested classes/interfaces inherited from class org.apache.lucene.index.TermsEnum
TermsEnum.SeekStatus
-
-
Field Summary
Fields Modifier and Type Field Description protected Automaton
automaton
protected IntersectBlockReader.BlockIteration
blockIteration
Block iteration order determined when scanning the terms in the current block.protected BytesRef
commonSuffix
protected boolean
finite
protected int
minTermLength
protected IntersectBlockReader.AutomatonNextTermCalculator
nextStringCalculator
protected int
NUM_CONSECUTIVELY_REJECTED_TERMS_THRESHOLD
Threshold that controls when to attempt to jump to a block away.protected int
numConsecutivelyRejectedTerms
Counter of the number of consecutively rejected terms.protected int
numMatchedBytes
Number of bytes accepted by the automaton when validating the current term.protected ByteRunAutomaton
runAutomaton
protected BytesRef
seekTerm
Set this when our current mode is seeking to this term.protected int[]
states
Automaton states reached when validating the current term, from 0 tonumMatchedBytes
- 1.-
Fields inherited from class org.apache.lucene.codecs.uniformsplit.BlockReader
blockDecoder, blockFirstLineStart, blockHeader, blockHeaderReader, blockInput, blockLine, blockLineReader, blockReadBuffer, blockStartFP, dictionaryBrowser, dictionaryBrowserSupplier, fieldMetadata, forcedTerm, lineIndexInBlock, postingsReader, scratchBlockBytes, scratchBlockLine, scratchTermState, termState, termStateForced, termStateSerializer, termStatesReadBuffer
-
Fields inherited from interface org.apache.lucene.util.Accountable
NULL_ACCOUNTABLE
-
-
Constructor Summary
Constructors Modifier Constructor Description protected
IntersectBlockReader(CompiledAutomaton compiled, BytesRef startTerm, IndexDictionary.BrowserSupplier dictionaryBrowserSupplier, IndexInput blockInput, PostingsReaderBase postingsReader, FieldMetadata fieldMetadata, BlockDecoder blockDecoder)
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description protected boolean
endsWithCommonSuffix(byte[] termBytes, int termLength)
Indicates whether the given term ends with the automaton common suffix.protected int
getMinTermLength()
Computes the minimal length of the terms accepted by the automaton.BytesRef
next()
protected boolean
nextBlock()
Opens the next block.protected BytesRef
nextTermInBlockMatching()
Finds the next block line that matches (accepted by the automaton), or null when at end of block.TermsEnum.SeekStatus
seekCeil(BytesRef text)
void
seekExact(long ord)
Not supported.boolean
seekExact(BytesRef text)
void
seekExact(BytesRef term, TermState state)
Positions thisBlockReader
without re-seeking the term dictionary.protected boolean
seekFirstBlock()
-
Methods inherited from class org.apache.lucene.codecs.uniformsplit.BlockReader
clearTermState, compareToMiddleAndJump, createBlockHeaderSerializer, createBlockLineSerializer, createDeltaBaseTermStateSerializer, decodeBlockBytesIfNeeded, docFreq, getOrCreateDictionaryBrowser, impacts, initializeBlockReadLazily, initializeHeader, isBeyondLastTerm, isCurrentTerm, newCorruptIndexException, nextTerm, ord, postings, ramBytesUsed, readHeader, readLineInBlock, readTermState, readTermStateIfNotRead, seekInBlock, seekInBlock, term, termState, totalTermFreq
-
Methods inherited from class org.apache.lucene.index.BaseTermsEnum
attributes
-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
Methods inherited from interface org.apache.lucene.util.Accountable
getChildResources
-
-
-
-
Field Detail
-
NUM_CONSECUTIVELY_REJECTED_TERMS_THRESHOLD
protected final int NUM_CONSECUTIVELY_REJECTED_TERMS_THRESHOLD
Threshold that controls when to attempt to jump to a block away.This counter is 0 when entering a block. It is incremented each time a term is rejected by the automaton. When the counter is greater than or equal to this threshold, then we compute the next term accepted by the automaton, with
IntersectBlockReader.AutomatonNextTermCalculator
, and we jump to a block away if the next term accepted is greater than the immediate next term in the block.A low value, for example 1, improves the performance of automatons requiring many jumps, for example
FuzzyQuery
and mostWildcardQuery
. A higher value improves the performance of automatons with less or no jump, for examplePrefixQuery
. A threshold of 4 seems to be a good balance.- See Also:
- Constant Field Values
-
automaton
protected final Automaton automaton
-
runAutomaton
protected final ByteRunAutomaton runAutomaton
-
finite
protected final boolean finite
-
commonSuffix
protected final BytesRef commonSuffix
-
minTermLength
protected final int minTermLength
-
nextStringCalculator
protected final IntersectBlockReader.AutomatonNextTermCalculator nextStringCalculator
-
seekTerm
protected BytesRef seekTerm
Set this when our current mode is seeking to this term. Set to null after.
-
numMatchedBytes
protected int numMatchedBytes
Number of bytes accepted by the automaton when validating the current term.
-
states
protected int[] states
Automaton states reached when validating the current term, from 0 tonumMatchedBytes
- 1.
-
blockIteration
protected IntersectBlockReader.BlockIteration blockIteration
Block iteration order determined when scanning the terms in the current block.
-
numConsecutivelyRejectedTerms
protected int numConsecutivelyRejectedTerms
Counter of the number of consecutively rejected terms. Depending onNUM_CONSECUTIVELY_REJECTED_TERMS_THRESHOLD
, this may trigger a jump to a block away.
-
-
Constructor Detail
-
IntersectBlockReader
protected IntersectBlockReader(CompiledAutomaton compiled, BytesRef startTerm, IndexDictionary.BrowserSupplier dictionaryBrowserSupplier, IndexInput blockInput, PostingsReaderBase postingsReader, FieldMetadata fieldMetadata, BlockDecoder blockDecoder) throws IOException
- Throws:
IOException
-
-
Method Detail
-
getMinTermLength
protected int getMinTermLength()
Computes the minimal length of the terms accepted by the automaton. This speeds up the term scanning for automatons accepting a finite language.
-
next
public BytesRef next() throws IOException
- Specified by:
next
in interfaceBytesRefIterator
- Overrides:
next
in classBlockReader
- Throws:
IOException
-
seekFirstBlock
protected boolean seekFirstBlock() throws IOException
- Throws:
IOException
-
nextTermInBlockMatching
protected BytesRef nextTermInBlockMatching() throws IOException
Finds the next block line that matches (accepted by the automaton), or null when at end of block.- Returns:
- The next term in the current block that is accepted by the automaton; or null if none.
- Throws:
IOException
-
endsWithCommonSuffix
protected boolean endsWithCommonSuffix(byte[] termBytes, int termLength)
Indicates whether the given term ends with the automaton common suffix. This allows to quickly skip terms that the automaton would reject eventually.
-
nextBlock
protected boolean nextBlock() throws IOException
Opens the next block. Depending on theblockIteration
order, it may be the very next block, or a block away that may containseekTerm
.- Returns:
- true if the next block is opened; false if there is no blocks anymore and the iteration is over.
- Throws:
IOException
-
seekExact
public boolean seekExact(BytesRef text)
- Overrides:
seekExact
in classBlockReader
-
seekExact
public void seekExact(long ord)
Description copied from class:BlockReader
Not supported.- Overrides:
seekExact
in classBlockReader
-
seekExact
public void seekExact(BytesRef term, TermState state)
Description copied from class:BlockReader
Positions thisBlockReader
without re-seeking the term dictionary.The block containing the term is not read by this method. It will be read lazily only if needed, for example if
BlockReader.next()
is called. CallingBlockReader.postings(org.apache.lucene.index.PostingsEnum, int)
after this method does require the block to be read.- Overrides:
seekExact
in classBlockReader
-
seekCeil
public TermsEnum.SeekStatus seekCeil(BytesRef text)
- Overrides:
seekCeil
in classBlockReader
-
-