Class SegmentInfos
- java.lang.Object
-
- org.apache.lucene.index.SegmentInfos
-
- All Implemented Interfaces:
java.lang.Cloneable
,java.lang.Iterable<SegmentCommitInfo>
public final class SegmentInfos extends java.lang.Object implements java.lang.Cloneable, java.lang.Iterable<SegmentCommitInfo>
A collection of segmentInfo objects with methods for operating on those segments in relation to the file system.The active segments in the index are stored in the segment info file, segments_N. There may be one or more segments_N files in the index; however, the one with the largest generation is the active one (when older segments_N files are present it's because they temporarily cannot be deleted, or, a writer is in the process of committing, or a custom
IndexDeletionPolicy
is in use). This file lists each segment by name and has details about the codec and generation of deletes.There is also a file segments.gen. This file contains the current generation (the _N in segments_N) of the index. This is used only as a fallback in case the current generation cannot be accurately determined by directory listing alone (as is the case for some NFS clients with time-based directory cache expiration). This file simply contains an
Int32
version header (FORMAT_SEGMENTS_GEN_CURRENT
), followed by the generation recorded asInt64
, written twice.Files:
- segments.gen: GenHeader, Generation, Generation
- segments_N: Header, Version, NameCounter, SegCount, <SegName, SegCodec, DelGen, DeletionCount, FieldInfosGen, UpdatesFiles>SegCount, CommitUserData, Checksum
- Header -->
CodecHeader
- GenHeader, NameCounter, SegCount, DeletionCount -->
Int32
- Generation, Version, DelGen, Checksum, FieldInfosGen -->
Int64
- SegName, SegCodec -->
String
- CommitUserData -->
Map<String,String>
- UpdatesFiles -->
Set<String>
- Version counts how often the index has been changed by adding or deleting documents.
- NameCounter is used to generate names for new segment files.
- SegName is the name of the segment, and is used as the file name prefix for all of the files that compose the segment's index.
- DelGen is the generation count of the deletes file. If this is -1,
there are no deletes. Anything above zero means there are deletes
stored by
LiveDocsFormat
. - DeletionCount records the number of deleted documents in this segment.
- Checksum contains the CRC32 checksum of all bytes in the segments_N file up until the checksum. This is used to verify integrity of the file on opening the index.
- SegCodec is the
name
of the Codec that encoded this segment. - CommitUserData stores an optional user-supplied opaque
Map<String,String> that was passed to
IndexWriter.setCommitData(java.util.Map)
. - FieldInfosGen is the generation count of the fieldInfos file. If this is -1,
there are no updates to the fieldInfos in that segment. Anything above zero
means there are updates to fieldInfos stored by
FieldInfosFormat
. - UpdatesFiles stores the list of files that were updated in that segment.
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static class
SegmentInfos.FindSegmentsFile
Utility class for executing code that needs to do something with the current segments file.
-
Field Summary
Fields Modifier and Type Field Description int
counter
Used to name new segments.static int
FORMAT_SEGMENTS_GEN_CURRENT
Used for the segments.gen file only! Whenever you add a new format, make it 1 smaller (negative version logic)!java.util.Map<java.lang.String,java.lang.String>
userData
Opaque Map<String, String> that user can specify during IndexWriter.commitlong
version
Counts how often the index has been changed.static int
VERSION_40
The file format version for the segments_N codec header, up to 4.5.static int
VERSION_46
The file format version for the segments_N codec header, since 4.6+.
-
Constructor Summary
Constructors Constructor Description SegmentInfos()
Sole constructor.
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Deprecated Methods Modifier and Type Method Description void
add(SegmentCommitInfo si)
Appends the providedSegmentCommitInfo
.void
addAll(java.lang.Iterable<SegmentCommitInfo> sis)
Appends the providedSegmentCommitInfo
s.java.util.List<SegmentCommitInfo>
asList()
Returns all contained segments as an unmodifiableList
view.void
changed()
Call this before committing if changes have been made to the segments.void
clear()
Clear allSegmentCommitInfo
s.SegmentInfos
clone()
Returns a copy of this instance, also copying each SegmentInfo.java.util.Collection<java.lang.String>
files(Directory dir, boolean includeSegmentsFile)
Returns all file names referenced by SegmentInfo instances matching the provided Directory (ie files associated with any "external" segments are skipped).static long
generationFromSegmentsFileName(java.lang.String fileName)
Parse the generation off the segments file name and return it.static int
getDefaultGenLookahedCount()
Returns thedefaultGenLookaheadCount
.long
getGeneration()
Returns current generation.static java.io.PrintStream
getInfoStream()
ReturnsinfoStream
.static long
getLastCommitGeneration(java.lang.String[] files)
Get the generation of the most recent commit to the list of index files (N in the segments_N file).static long
getLastCommitGeneration(Directory directory)
Get the generation of the most recent commit to the index in this directory (N in the segments_N file).static java.lang.String
getLastCommitSegmentsFileName(java.lang.String[] files)
Get the filename of the segments_N file for the most recent commit in the list of index files.static java.lang.String
getLastCommitSegmentsFileName(Directory directory)
Get the filename of the segments_N file for the most recent commit to the index in this Directory.long
getLastGeneration()
Returns last succesfully read or written generation.java.lang.String
getNextSegmentFileName()
Get the next segments_N filename that will be written.java.lang.String
getSegmentsFileName()
Get the segments_N filename in use by this segment infos.java.util.Map<java.lang.String,java.lang.String>
getUserData()
ReturnuserData
saved with this commit.long
getVersion()
version number when this SegmentInfos was generated.SegmentCommitInfo
info(int i)
ReturnsSegmentCommitInfo
at the provided index.java.util.Iterator<SegmentCommitInfo>
iterator()
Returns an unmodifiableIterator
of contained segments in order.void
read(Directory directory)
Find the latest commit (segments_N file
) and load allSegmentCommitInfo
s.void
read(Directory directory, java.lang.String segmentFileName)
Read a particular segmentFileName.void
remove(SegmentCommitInfo si)
Remove the providedSegmentCommitInfo
.static void
setDefaultGenLookaheadCount(int count)
Advanced: set how many times to try incrementing the gen when loading the segments file.static void
setInfoStream(java.io.PrintStream infoStream)
If non-null, information about retries when loading the segments file will be printed to this.int
size()
Returns number ofSegmentCommitInfo
s.java.lang.String
toString(Directory directory)
Returns readable description of this segment.int
totalDocCount()
Returns sum of all segment's docCounts.static java.lang.String
write3xInfo(Directory dir, SegmentInfo si, IOContext context)
Deprecated.static void
writeSegmentsGen(Directory dir, long generation)
A utility for writing theIndexFileNames.SEGMENTS_GEN
file to aDirectory
.
-
-
-
Field Detail
-
VERSION_40
public static final int VERSION_40
The file format version for the segments_N codec header, up to 4.5.- See Also:
- Constant Field Values
-
VERSION_46
public static final int VERSION_46
The file format version for the segments_N codec header, since 4.6+.- See Also:
- Constant Field Values
-
FORMAT_SEGMENTS_GEN_CURRENT
public static final int FORMAT_SEGMENTS_GEN_CURRENT
Used for the segments.gen file only! Whenever you add a new format, make it 1 smaller (negative version logic)!- See Also:
- Constant Field Values
-
counter
public int counter
Used to name new segments.
-
version
public long version
Counts how often the index has been changed.
-
userData
public java.util.Map<java.lang.String,java.lang.String> userData
Opaque Map<String, String> that user can specify during IndexWriter.commit
-
-
Constructor Detail
-
SegmentInfos
public SegmentInfos()
Sole constructor. Typically you call this and then useor #read(Directory,String)
to populate eachSegmentCommitInfo
. Alternatively, you can add/remove your ownSegmentCommitInfo
s.
-
-
Method Detail
-
info
public SegmentCommitInfo info(int i)
ReturnsSegmentCommitInfo
at the provided index.
-
getLastCommitGeneration
public static long getLastCommitGeneration(java.lang.String[] files)
Get the generation of the most recent commit to the list of index files (N in the segments_N file).- Parameters:
files
- -- array of file names to check
-
getLastCommitGeneration
public static long getLastCommitGeneration(Directory directory) throws java.io.IOException
Get the generation of the most recent commit to the index in this directory (N in the segments_N file).- Parameters:
directory
- -- directory to search for the latest segments_N file- Throws:
java.io.IOException
-
getLastCommitSegmentsFileName
public static java.lang.String getLastCommitSegmentsFileName(java.lang.String[] files)
Get the filename of the segments_N file for the most recent commit in the list of index files.- Parameters:
files
- -- array of file names to check
-
getLastCommitSegmentsFileName
public static java.lang.String getLastCommitSegmentsFileName(Directory directory) throws java.io.IOException
Get the filename of the segments_N file for the most recent commit to the index in this Directory.- Parameters:
directory
- -- directory to search for the latest segments_N file- Throws:
java.io.IOException
-
getSegmentsFileName
public java.lang.String getSegmentsFileName()
Get the segments_N filename in use by this segment infos.
-
generationFromSegmentsFileName
public static long generationFromSegmentsFileName(java.lang.String fileName)
Parse the generation off the segments file name and return it.
-
writeSegmentsGen
public static void writeSegmentsGen(Directory dir, long generation)
A utility for writing theIndexFileNames.SEGMENTS_GEN
file to aDirectory
.NOTE: this is an internal utility which is kept public so that it's accessible by code from other packages. You should avoid calling this method unless you're absolutely sure what you're doing!
-
getNextSegmentFileName
public java.lang.String getNextSegmentFileName()
Get the next segments_N filename that will be written.
-
read
public final void read(Directory directory, java.lang.String segmentFileName) throws java.io.IOException
Read a particular segmentFileName. Note that this may throw an IOException if a commit is in process.- Parameters:
directory
- -- directory containing the segments filesegmentFileName
- -- segment file to load- Throws:
CorruptIndexException
- if the index is corruptjava.io.IOException
- if there is a low-level IO error
-
read
public final void read(Directory directory) throws java.io.IOException
Find the latest commit (segments_N file
) and load allSegmentCommitInfo
s.- Throws:
java.io.IOException
-
write3xInfo
@Deprecated public static java.lang.String write3xInfo(Directory dir, SegmentInfo si, IOContext context) throws java.io.IOException
Deprecated.- Throws:
java.io.IOException
-
clone
public SegmentInfos clone()
Returns a copy of this instance, also copying each SegmentInfo.
-
getVersion
public long getVersion()
version number when this SegmentInfos was generated.
-
getGeneration
public long getGeneration()
Returns current generation.
-
getLastGeneration
public long getLastGeneration()
Returns last succesfully read or written generation.
-
setInfoStream
public static void setInfoStream(java.io.PrintStream infoStream)
If non-null, information about retries when loading the segments file will be printed to this.
-
setDefaultGenLookaheadCount
public static void setDefaultGenLookaheadCount(int count)
Advanced: set how many times to try incrementing the gen when loading the segments file. This only runs if the primary (listing directory) and secondary (opening segments.gen file) methods fail to find the segments file.
-
getDefaultGenLookahedCount
public static int getDefaultGenLookahedCount()
Returns thedefaultGenLookaheadCount
.- See Also:
setDefaultGenLookaheadCount(int)
-
getInfoStream
public static java.io.PrintStream getInfoStream()
ReturnsinfoStream
.- See Also:
setInfoStream(java.io.PrintStream)
-
files
public java.util.Collection<java.lang.String> files(Directory dir, boolean includeSegmentsFile) throws java.io.IOException
Returns all file names referenced by SegmentInfo instances matching the provided Directory (ie files associated with any "external" segments are skipped). The returned collection is recomputed on each invocation.- Throws:
java.io.IOException
-
toString
public java.lang.String toString(Directory directory)
Returns readable description of this segment.
-
getUserData
public java.util.Map<java.lang.String,java.lang.String> getUserData()
ReturnuserData
saved with this commit.- See Also:
IndexWriter.commit()
-
totalDocCount
public int totalDocCount()
Returns sum of all segment's docCounts. Note that this does not include deletions
-
changed
public void changed()
Call this before committing if changes have been made to the segments.
-
iterator
public java.util.Iterator<SegmentCommitInfo> iterator()
Returns an unmodifiableIterator
of contained segments in order.- Specified by:
iterator
in interfacejava.lang.Iterable<SegmentCommitInfo>
-
asList
public java.util.List<SegmentCommitInfo> asList()
Returns all contained segments as an unmodifiableList
view.
-
size
public int size()
Returns number ofSegmentCommitInfo
s.
-
add
public void add(SegmentCommitInfo si)
Appends the providedSegmentCommitInfo
.
-
addAll
public void addAll(java.lang.Iterable<SegmentCommitInfo> sis)
Appends the providedSegmentCommitInfo
s.
-
clear
public void clear()
Clear allSegmentCommitInfo
s.
-
remove
public void remove(SegmentCommitInfo si)
Remove the providedSegmentCommitInfo
.WARNING: O(N) cost
-
-