Package org.apache.lucene.tests.index

Class BaseCompoundFormatTestCase

java.lang.Object

org.junit.Assert

org.apache.lucene.tests.util.LuceneTestCase

org.apache.lucene.tests.index.BaseCompoundFormatTestCase

public abstract class BaseCompoundFormatTestCase
extends LuceneTestCase

Abstract class to do basic tests for a compound format. NOTE: This test focuses on the compound impl, nothing else. The [stretch] goal is for this test to be so thorough in testing a new CompoundFormat that if this test passes, then all Lucene/Solr tests should also pass. Ie, if there is some bug in a given CompoundFormat that this test fails to catch then this test needs to be improved!

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.lucene.tests.util.LuceneTestCase

LuceneTestCase.AwaitsFix, LuceneTestCase.BadApple, LuceneTestCase.Monster, LuceneTestCase.Nightly, LuceneTestCase.SuppressCodecs, LuceneTestCase.SuppressFileSystems, LuceneTestCase.SuppressFsync, LuceneTestCase.SuppressReproduceLine, LuceneTestCase.SuppressSysoutChecks, LuceneTestCase.SuppressTempFileChecks, LuceneTestCase.ThrowingConsumer<T>, LuceneTestCase.ThrowingRunnable, LuceneTestCase.Weekly

Field Summary

Fields inherited from class org.apache.lucene.tests.util.LuceneTestCase

assertsAreEnabled, classRules, DEFAULT_LINE_DOCS_FILE, INFOSTREAM, JENKINS_LARGE_LINE_DOCS_FILE, LEAVE_TEMPORARY, MAYBE_CACHE_POLICY, RANDOM_MULTIPLIER, ruleChain, suiteFailureMarker, SYSPROP_AWAITSFIX, SYSPROP_BADAPPLES, SYSPROP_FAILFAST, SYSPROP_MAXFAILURES, SYSPROP_MONSTER, SYSPROP_NIGHTLY, SYSPROP_WEEKLY, TEST_ASSERTS_ENABLED, TEST_AWAITSFIX, TEST_BADAPPLES, TEST_CODEC, TEST_DIRECTORY, TEST_DOCVALUESFORMAT, TEST_LINE_DOCS_FILE, TEST_MONSTER, TEST_NIGHTLY, TEST_POSTINGSFORMAT, TEST_THROTTLING, TEST_WEEKLY, VERBOSE

Constructor Summary

Constructors

Constructor	Description
BaseCompoundFormatTestCase()

Method Summary

Modifier and Type	Method	Description
protected void	addRandomFields(Document doc)	Add random fields to the provided document.
protected final <D extends Directory> D	applyCreatedVersionMajor(D d)	Set the created version of the given Directory and return it.
protected static void	assertEqualArrays(String msg, byte[] expected, byte[] test, int start, int len)
protected static void	assertSameSeekBehavior(String msg, IndexInput expected, IndexInput actual)
protected static void	assertSameStreams(String msg, IndexInput expected, IndexInput test)
protected static void	assertSameStreams(String msg, IndexInput expected, IndexInput actual, long seekTo)
protected static Directory	createLargeCFS(Directory dir)	Setup a large compound file with a number of components, each of which is a sequential file (so that we can easily tell that we are reading in the right byte).
protected static void	createRandomFile(Directory dir, String name, int size, byte[] segId)	Creates a file of the specified size with random data.
protected static void	createSequenceFile(Directory dir, String name, byte start, int size, byte[] segID, String segSuffix)	Creates a file of the specified size with sequential data.
protected Collection<String>	excludedExtensionsFromByteCounts()	Return the list of extensions that should be excluded from byte counts when comparing indices that store the same content.
protected abstract Codec	getCodec()	Returns the codec to run tests against
protected int	getCreatedVersionMajor()	Returns the major version that this codec is compatible with.
protected final DirectoryReader	maybeWrapWithMergingReader(DirectoryReader r)
protected boolean	mergeIsStable()
protected static SegmentInfo	newSegmentInfo(Directory dir, String name)	Returns a new fake segment
void	setUp()	For subclasses to override.
protected boolean	shouldTestMergeInstance()	Returns false if only the regular fields reader should be tested, and true if only the merge instance should be tested.
void	tearDown()	For subclasses to override.
void	testCheckIntegrity()
void	testCheckIntegrityReadsAllBytes()	This test is a best effort at verifying that checkIntegrity doesn't miss any files.
void	testClonedStreamsClosing()
void	testCorruptFilesAreCaught()
void	testCreateOutputDisabled()
void	testDeleteFileDisabled()
void	testDoubleClose()
void	testEmpty()
void	testFileNotFound()
void	testLargeCFS()
void	testListAll()
void	testMakeLockDisabled()
void	testManySubFiles()
void	testMergeStability()	The purpose of this test is to make sure that bulk merge doesn't accumulate useless data over runs.
void	testMissingCodecHeadersAreCaught()
void	testMultiClose()	Calls close multiple times on closeable codec apis
void	testPassIOContext()
void	testRandomAccess()	This test opens two files from a compound stream and verifies that their file positions are independent of each other.
void	testRandomAccessClones()	This test opens two files from a compound stream and verifies that their file positions are independent of each other.
void	testRandomExceptions()	Tests exception handling on write and openInput/createOutput
void	testRandomFiles()	This test creates a compound file based on a large number of files of various length.
void	testReadPastEOF()
void	testRenameFileDisabled()
void	testResourceNameInsideCompoundFile()
void	testSingleFile()	This test creates compound file based on a single file.
void	testSyncDisabled()
void	testTwoFiles()	This test creates compound file based on two files.

Methods inherited from class org.apache.lucene.tests.util.LuceneTestCase

addVirusChecker, assertDeletedDocsEquals, assertDocsAndPositionsEnumEquals, assertDocsEnumEquals, assertDocsSkippingEquals, assertDocValuesEquals, assertDocValuesEquals, assertFieldInfosEquals, assertNormsEquals, assertPointsEquals, assertPositionsSkippingEquals, assertReaderEquals, assertReaderStatisticsEquals, assertStoredFieldEquals, assertStoredFieldsEquals, assertTermsEnumEquals, assertTermsEquals, assertTermsEquals, assertTermsStatisticsEquals, assertTermStatsEquals, assertTermVectorsEquals, asSet, assumeFalse, assumeNoException, assumeTrue, assumeWorkingMMapOnWindows, atLeast, atLeast, callStackContains, callStackContains, callStackContainsAnyOf, closeAfterSuite, closeAfterTest, collate, createTempDir, createTempDir, createTempFile, createTempFile, dumpArray, dumpIterator, ensureSaneIWCOnNightly, expectThrows, expectThrows, expectThrows, expectThrowsAnyOf, expectThrowsAnyOf, getDataInputStream, getDataPath, getJvmForkArguments, getOnlyLeafReader, getTestClass, getTestName, hasWorkingMMapOnWindows, isTestThread, localeForLanguageTag, maybeChangeLiveIndexWriterConfig, maybeWrapReader, newAlcoholicMergePolicy, newAlcoholicMergePolicy, newBytesRef, newBytesRef, newBytesRef, newBytesRef, newBytesRef, newBytesRef, newDirectory, newDirectory, newDirectory, newDirectory, newDirectory, newField, newField, newFSDirectory, newFSDirectory, newIndexWriterConfig, newIndexWriterConfig, newIndexWriterConfig, newIOContext, newIOContext, newLogMergePolicy, newLogMergePolicy, newLogMergePolicy, newLogMergePolicy, newLogMergePolicy, newMaybeVirusCheckingDirectory, newMaybeVirusCheckingFSDirectory, newMergePolicy, newMergePolicy, newMergePolicy, newMockDirectory, newMockDirectory, newMockDirectory, newMockFSDirectory, newMockFSDirectory, newSearcher, newSearcher, newSearcher, newSearcher, newSnapshotIndexWriterConfig, newStringField, newStringField, newStringField, newStringField, newTextField, newTextField, newTieredMergePolicy, newTieredMergePolicy, overrideDefaultQueryCache, overrideTestDefaultQueryCache, random, randomLocale, randomTimeZone, rarely, rarely, replaceMaxFailureRule, resetDefaultQueryCache, restoreCPUCoreCount, restoreIndexWriterMaxDocs, runWithRestrictedPermissions, setIndexWriterMaxDocs, setupCPUCoreCount, setUpExecutorService, shutdownExecutorService, slowFileExists, usually, usually, wrapReader

Methods inherited from class org.junit.Assert

assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertFalse, assertFalse, assertNotEquals, assertNotEquals, assertNotEquals, assertNotEquals, assertNotEquals, assertNotEquals, assertNotEquals, assertNotEquals, assertNotNull, assertNotNull, assertNotSame, assertNotSame, assertNull, assertNull, assertSame, assertSame, assertThat, assertThat, assertThrows, assertThrows, assertTrue, assertTrue, fail, fail

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

BaseCompoundFormatTestCase

public BaseCompoundFormatTestCase()

Method Details

testEmpty

public void testEmpty()
               throws IOException

Throws:

IOException

testSingleFile

public void testSingleFile()
                    throws IOException

This test creates compound file based on a single file. Files of different sizes are tested: 0, 1, 10, 100 bytes.

Throws:

IOException

testTwoFiles

public void testTwoFiles()
                  throws IOException

This test creates compound file based on two files.

Throws:

IOException

testDoubleClose

public void testDoubleClose()
                     throws IOException

Throws:

IOException

testPassIOContext

public void testPassIOContext()
                       throws IOException

Throws:

IOException

testLargeCFS

public void testLargeCFS()
                  throws IOException

Throws:

IOException

testListAll

public void testListAll()
                 throws Exception

Throws:

Exception

testCreateOutputDisabled

public void testCreateOutputDisabled()
                              throws IOException

Throws:

IOException

testDeleteFileDisabled

public void testDeleteFileDisabled()
                            throws IOException

Throws:

IOException

testRenameFileDisabled

public void testRenameFileDisabled()
                            throws IOException

Throws:

IOException

testSyncDisabled

public void testSyncDisabled()
                      throws IOException

Throws:

IOException

testMakeLockDisabled

public void testMakeLockDisabled()
                          throws IOException

Throws:

IOException

testRandomFiles

public void testRandomFiles()
                     throws IOException

This test creates a compound file based on a large number of files of various length. The file content is generated randomly. The sizes range from 0 to 1Mb. Some of the sizes are selected to test the buffering logic in the file reading code. For this the chunk variable is set to the length of the buffer used internally by the compound file logic.

Throws:

IOException

testManySubFiles

public void testManySubFiles()
                      throws IOException

Throws:

IOException

testClonedStreamsClosing

public void testClonedStreamsClosing()
                              throws IOException

Throws:

IOException

testRandomAccess

public void testRandomAccess()
                      throws IOException

This test opens two files from a compound stream and verifies that their file positions are independent of each other.

Throws:

IOException

testRandomAccessClones

public void testRandomAccessClones()
                            throws IOException

This test opens two files from a compound stream and verifies that their file positions are independent of each other.

Throws:

IOException

testFileNotFound

public void testFileNotFound()
                      throws IOException

Throws:

IOException

testReadPastEOF

public void testReadPastEOF()
                     throws IOException

Throws:

IOException

newSegmentInfo

protected static SegmentInfo newSegmentInfo(Directory dir,
 String name)

Returns a new fake segment

createRandomFile

protected static void createRandomFile(Directory dir,
 String name,
 int size,
 byte[] segId)
                                throws IOException

Creates a file of the specified size with random data.

Throws:

IOException

createSequenceFile

protected static void createSequenceFile(Directory dir,
 String name,
 byte start,
 int size,
 byte[] segID,
 String segSuffix)
                                  throws IOException

Creates a file of the specified size with sequential data. The first byte is written as the start byte provided. All subsequent bytes are computed as start + offset where offset is the number of the byte.

Throws:

IOException

assertSameStreams

protected static void assertSameStreams(String msg,
 IndexInput expected,
 IndexInput test)
                                 throws IOException

Throws:

IOException

assertSameStreams

protected static void assertSameStreams(String msg,
 IndexInput expected,
 IndexInput actual,
 long seekTo)
                                 throws IOException

Throws:

IOException

assertSameSeekBehavior

protected static void assertSameSeekBehavior(String msg,
 IndexInput expected,
 IndexInput actual)
                                      throws IOException

Throws:

IOException

assertEqualArrays

protected static void assertEqualArrays(String msg,
 byte[] expected,
 byte[] test,
 int start,
 int len)

createLargeCFS

protected static Directory createLargeCFS(Directory dir)
                                   throws IOException

Setup a large compound file with a number of components, each of which is a sequential file (so that we can easily tell that we are reading in the right byte). The methods sets up 20 files - _123.0 to _123.19, the size of each file is 1000 bytes.

Throws:

IOException

addRandomFields

protected void addRandomFields(Document doc)

Add random fields to the provided document.

testMergeStability

public void testMergeStability()
                        throws Exception

The purpose of this test is to make sure that bulk merge doesn't accumulate useless data over runs.

Throws:

Exception

testResourceNameInsideCompoundFile

public void testResourceNameInsideCompoundFile()
                                        throws Exception

Throws:

Exception

testMissingCodecHeadersAreCaught

public void testMissingCodecHeadersAreCaught()
                                      throws Exception

Throws:

Exception

testCorruptFilesAreCaught

public void testCorruptFilesAreCaught()
                               throws Exception

Throws:

Exception

testCheckIntegrity

public void testCheckIntegrity()
                        throws IOException

Throws:

IOException

getCodec

protected abstract Codec getCodec()

Returns the codec to run tests against

getCreatedVersionMajor

protected int getCreatedVersionMajor()

Returns the major version that this codec is compatible with.

applyCreatedVersionMajor

protected final <D extends Directory> D applyCreatedVersionMajor(D d)
                                                          throws IOException

Set the created version of the given Directory and return it.

Throws:

IOException

setUp

public void setUp()
           throws Exception

Description copied from class: LuceneTestCase

For subclasses to override. Overrides must call super.setUp().

Overrides:

setUp in class LuceneTestCase

Throws:

Exception

tearDown

public void tearDown()
              throws Exception

Description copied from class: LuceneTestCase

For subclasses to override. Overrides must call super.tearDown().

Overrides:

tearDown in class LuceneTestCase

Throws:

Exception

excludedExtensionsFromByteCounts

protected Collection<String> excludedExtensionsFromByteCounts()

Return the list of extensions that should be excluded from byte counts when comparing indices that store the same content.

mergeIsStable

protected boolean mergeIsStable()

testMultiClose

public void testMultiClose()
                    throws IOException

Calls close multiple times on closeable codec apis

Throws:

IOException

testRandomExceptions

public void testRandomExceptions()
                          throws Exception

Tests exception handling on write and openInput/createOutput

Throws:

Exception

shouldTestMergeInstance

protected boolean shouldTestMergeInstance()

Returns false if only the regular fields reader should be tested, and true if only the merge instance should be tested.

maybeWrapWithMergingReader

protected final DirectoryReader maybeWrapWithMergingReader(DirectoryReader r)
                                                    throws IOException

Throws:

IOException

testCheckIntegrityReadsAllBytes

public void testCheckIntegrityReadsAllBytes()
                                     throws Exception

This test is a best effort at verifying that checkIntegrity doesn't miss any files. It tests that the combination of opening a reader and calling checkIntegrity on it reads all bytes of all files.

Throws:

Exception