org.apache.iotdb.db.tools

类 TsFileRewriteTool

java.lang.Object

org.apache.iotdb.db.tools.TsFileRewriteTool

所有已实现的接口:

AutoCloseable

直接已知子类:

TsFileOnlineUpgradeTool

public class TsFileRewriteTool
extends Object
implements AutoCloseable

字段概要

字段

限定符和类型	字段和说明
protected Decoder	defaultTimeDecoder
protected long	maxPlanIndex Maximum index of plans executed within this TsFile.
protected long	minPlanIndex Minimum index of plans executed within this TsFile.
protected Iterator<Modification>	modsIterator
protected List<Modification>	oldModification
protected File	oldTsFile
protected TsFileResource	oldTsFileResource
protected Map<Long,TsFileIOWriter>	partitionWriterMap PartitionId -> TsFileIOWriter
protected TsFileSequenceReader	reader
protected Decoder	valueDecoder

构造器概要

构造器

构造器和说明
TsFileRewriteTool(TsFileResource resourceToBeRewritten) Create a file reader of the given file.
TsFileRewriteTool(TsFileResource resourceToBeRewritten, boolean needReaderForV2)

方法概要

限定符和类型	方法和说明
protected boolean	checkIfNeedToDecode(MeasurementSchema schema, String deviceId, PageHeader pageHeader, long chunkHeaderOffset) If the page have no statistics or crosses multi partitions, will return true, otherwise return false.
void	close()
protected void	decodeAndWritePage(String deviceId, MeasurementSchema schema, ByteBuffer pageData, Map<Long,ChunkWriterImpl> partitionChunkWriterMap, long chunkHeaderOffset)
protected void	endChunkGroup()
protected TsFileResource	endFileAndGenerateResource(TsFileIOWriter tsFileIOWriter)
protected boolean	fileCheck() check if the file has correct magic strings and version number
protected TsFileIOWriter	getOrDefaultTsFileIOWriter(File oldTsFile, long partition)
void	parseAndRewriteFile(List<TsFileResource> rewrittenResources) Parse the old files and generate some new files according to the time partition interval.
protected void	reWriteChunk(String deviceId, boolean firstChunkInChunkGroup, MeasurementSchema schema, List<PageHeader> pageHeadersInChunk, List<ByteBuffer> pageDataInChunk, List<Boolean> needToDecodeInfoInChunk, long chunkHeaderOffset) This method is for rewriting the Chunk which data is in the different time partitions.
protected void	rewritePageIntoFiles(BatchData batchData, MeasurementSchema schema, Map<Long,ChunkWriterImpl> partitionChunkWriterMap)
static void	rewriteTsFile(TsFileResource resourceToBeRewritten, List<TsFileResource> rewrittenResources) Rewrite an old file to the latest version
String	upgradeTsFileName(String oldTsFileName)
protected void	writePage(MeasurementSchema schema, PageHeader pageHeader, ByteBuffer pageData, Map<Long,ChunkWriterImpl> partitionChunkWriterMap)

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

字段详细资料

reader

protected TsFileSequenceReader reader

oldTsFile

protected File oldTsFile

oldModification

protected List<Modification> oldModification

oldTsFileResource

protected TsFileResource oldTsFileResource

modsIterator

protected Iterator<Modification> modsIterator

defaultTimeDecoder

protected Decoder defaultTimeDecoder

valueDecoder

protected Decoder valueDecoder

partitionWriterMap

protected Map<Long,TsFileIOWriter> partitionWriterMap

PartitionId -> TsFileIOWriter

maxPlanIndex

protected long maxPlanIndex

Maximum index of plans executed within this TsFile.

minPlanIndex

protected long minPlanIndex

Minimum index of plans executed within this TsFile.

构造器详细资料

TsFileRewriteTool

public TsFileRewriteTool(TsFileResource resourceToBeRewritten)
                  throws IOException

Create a file reader of the given file. The reader will read the real data and rewrite to some new tsFiles.

抛出:

IOException - If some I/O error occurs

TsFileRewriteTool

public TsFileRewriteTool(TsFileResource resourceToBeRewritten,
                         boolean needReaderForV2)
                  throws IOException

抛出:

IOException

方法详细资料

rewriteTsFile

public static void rewriteTsFile(TsFileResource resourceToBeRewritten,
                                 List<TsFileResource> rewrittenResources)
                          throws IOException,
                                 WriteProcessException,
                                 IllegalPathException

Rewrite an old file to the latest version

参数:

resourceToBeRewritten - the tsfile which to be rewrite

rewrittenResources - the rewritten files

抛出:

IOException

WriteProcessException

IllegalPathException

close

public void close()
           throws IOException

指定者:

close 在接口中 AutoCloseable

抛出:

IOException

parseAndRewriteFile

public void parseAndRewriteFile(List<TsFileResource> rewrittenResources)
                         throws IOException,
                                WriteProcessException,
                                IllegalPathException

Parse the old files and generate some new files according to the time partition interval.

抛出:

IOException - WriteProcessException

WriteProcessException

IllegalPathException

checkIfNeedToDecode

protected boolean checkIfNeedToDecode(MeasurementSchema schema,
                                      String deviceId,
                                      PageHeader pageHeader,
                                      long chunkHeaderOffset)
                               throws IllegalPathException

If the page have no statistics or crosses multi partitions, will return true, otherwise return false.

抛出:

IllegalPathException

reWriteChunk

protected void reWriteChunk(String deviceId,
                            boolean firstChunkInChunkGroup,
                            MeasurementSchema schema,
                            List<PageHeader> pageHeadersInChunk,
                            List<ByteBuffer> pageDataInChunk,
                            List<Boolean> needToDecodeInfoInChunk,
                            long chunkHeaderOffset)
                     throws IOException,
                            PageException,
                            IllegalPathException

This method is for rewriting the Chunk which data is in the different time partitions. In this case, we have to decode the data to points, and then rewrite the data points to different chunkWriters, finally write chunks to their own upgraded TsFiles.

抛出:

IOException

PageException

IllegalPathException

endChunkGroup

protected void endChunkGroup()
                      throws IOException

抛出:

IOException

upgradeTsFileName

public String upgradeTsFileName(String oldTsFileName)

getOrDefaultTsFileIOWriter

protected TsFileIOWriter getOrDefaultTsFileIOWriter(File oldTsFile,
                                                    long partition)

writePage

protected void writePage(MeasurementSchema schema,
                         PageHeader pageHeader,
                         ByteBuffer pageData,
                         Map<Long,ChunkWriterImpl> partitionChunkWriterMap)
                  throws PageException

抛出:

PageException

decodeAndWritePage

protected void decodeAndWritePage(String deviceId,
                                  MeasurementSchema schema,
                                  ByteBuffer pageData,
                                  Map<Long,ChunkWriterImpl> partitionChunkWriterMap,
                                  long chunkHeaderOffset)
                           throws IOException,
                                  IllegalPathException

抛出:

IOException

IllegalPathException

rewritePageIntoFiles

protected void rewritePageIntoFiles(BatchData batchData,
                                    MeasurementSchema schema,
                                    Map<Long,ChunkWriterImpl> partitionChunkWriterMap)

fileCheck

protected boolean fileCheck()
                     throws IOException

check if the file has correct magic strings and version number

抛出:

IOException

endFileAndGenerateResource

protected TsFileResource endFileAndGenerateResource(TsFileIOWriter tsFileIOWriter)
                                             throws IOException

抛出:

IOException