java.lang.Object
com.google.common.util.concurrent.AbstractService
org.yamcs.AbstractYamcsService
org.yamcs.parameterarchive.ParameterArchive
- All Implemented Interfaces:
com.google.common.util.concurrent.Service
,YamcsService
The parameter archive stores data in partitions(optional) -> intervals -> segments.
A partition covers one year/month/day and each partition has its own RocksDB database.
An interval covers 2^23 millisec (=~ 139 minutes) - so for any timestamp (Yamcs time) we know exactly in which interval it falls.
A segment covers at most maxSegmentSize samples for one parameter. The segments do not cover a fixed period of time; we use them to avoid intervals getting very large; usually (1Hz or less frequency data) there is only one segment in an interval.
Segments cannot span across intervals.
When new data has been received in the past, the whole interval has to be re-created (by doing a replay); that likely means a new split of the respective interval into segments.
-
Nested Class Summary
Nested ClassesNested classes/interfaces inherited from interface com.google.common.util.concurrent.Service
com.google.common.util.concurrent.Service.Listener, com.google.common.util.concurrent.Service.State
-
Field Summary
FieldsModifier and TypeFieldDescriptionstatic final long
static final int
static final boolean
static final int
Fields inherited from class org.yamcs.AbstractYamcsService
config, log, serviceName, yamcsInstance
-
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptionvoid
compact()
createAndGetPartition
(long segStart) get partition for segment, creating it if it doesn't existdeletePartitions
(long start, long stop) Delete all partitions that overlap with [start, stop) segment.protected void
doStart()
protected void
doStop()
static long
getInterval
(long instant) static long
duration in milliseconds of one segmentstatic long
getIntervalEnd
(long instant) returns the end of the interval where the instant fitsstatic long
getIntervalStart
(long instant) returns the interval (instant) where this instant could fit.org.rocksdb.RocksIterator
getPartitions
(long start, long stop, boolean ascending) a copy of the partitions from start to stop inclusivegetSpec()
Returns the valid configuration options for this service.getTimeSegment
(ParameterArchive.Partition p, long segmentStart, int parameterGroupId) void
init
(String yamcsInstance, String serviceName, YConfiguration config) Initialize this service.void
printKeys
(PrintStream out) Future<?>
reprocess
(long start, long stop) void
writeToArchive
(long segStart, Collection<PGSegment> pgList) void
writeToArchive
(PGSegment pgs) Methods inherited from class org.yamcs.AbstractYamcsService
getConfig, getYamcsInstance
Methods inherited from class com.google.common.util.concurrent.AbstractService
addListener, awaitRunning, awaitRunning, awaitRunning, awaitTerminated, awaitTerminated, awaitTerminated, doCancelStart, failureCause, isRunning, notifyFailed, notifyStarted, notifyStopped, startAsync, state, stopAsync, toString
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
Methods inherited from interface com.google.common.util.concurrent.Service
addListener, awaitRunning, awaitRunning, awaitRunning, awaitTerminated, awaitTerminated, awaitTerminated, failureCause, isRunning, startAsync, state, stopAsync
-
Field Details
-
STORE_RAW_VALUES
public static final boolean STORE_RAW_VALUES- See Also:
-
NUMBITS_MASK
public static final int NUMBITS_MASK- See Also:
-
TIMESTAMP_MASK
public static final int TIMESTAMP_MASK- See Also:
-
INTERVAL_MASK
public static final long INTERVAL_MASK- See Also:
-
-
Constructor Details
-
ParameterArchive
public ParameterArchive()
-
-
Method Details
-
getSpec
Description copied from interface:YamcsService
Returns the valid configuration options for this service.- Returns:
- the argument specification, or
null
if the args should not be validated.
-
init
public void init(String yamcsInstance, String serviceName, YConfiguration config) throws InitException Description copied from interface:YamcsService
Initialize this service. This is called before the service is started. All operations should finish fast.- Specified by:
init
in interfaceYamcsService
- Overrides:
init
in classAbstractYamcsService
- Parameters:
yamcsInstance
- The yamcs instance, ornull
if this is a global service.serviceName
- The service name.config
- The configured arguments for this service. IfYamcsService.getSpec()
is implemented then this contains the arguments after being validated (including any defaults).- Throws:
InitException
- When something goes wrong during the execution of this method.
-
getPartitioningSchema
-
getParameterIdDb
-
getParameterGroupIdDb
-
writeToArchive
- Throws:
org.rocksdb.RocksDBException
IOException
-
writeToArchive
public void writeToArchive(long segStart, Collection<PGSegment> pgList) throws org.rocksdb.RocksDBException, IOException - Throws:
org.rocksdb.RocksDBException
IOException
-
createAndGetPartition
public ParameterArchive.Partition createAndGetPartition(long segStart) throws org.rocksdb.RocksDBException get partition for segment, creating it if it doesn't exist- Parameters:
segStart
-- Throws:
org.rocksdb.RocksDBException
-
reprocess
-
getPartitions
a copy of the partitions from start to stop inclusive- Parameters:
start
-stop
-- Returns:
- a sorted list of partitions
-
doStart
protected void doStart()- Specified by:
doStart
in classcom.google.common.util.concurrent.AbstractService
-
doStop
protected void doStop()- Specified by:
doStop
in classcom.google.common.util.concurrent.AbstractService
-
printKeys
public void printKeys(PrintStream out) throws DecodingException, org.rocksdb.RocksDBException, IOException - Throws:
DecodingException
org.rocksdb.RocksDBException
IOException
-
deletePartitions
public List<ParameterArchive.Partition> deletePartitions(long start, long stop) throws org.rocksdb.RocksDBException Delete all partitions that overlap with [start, stop) segment.- Parameters:
start
-stop
-- Returns:
- all the partitions removed
- Throws:
org.rocksdb.RocksDBException
-
getIterator
public org.rocksdb.RocksIterator getIterator(ParameterArchive.Partition p) throws org.rocksdb.RocksDBException, IOException - Throws:
org.rocksdb.RocksDBException
IOException
-
getTimeSegment
public SortedTimeSegment getTimeSegment(ParameterArchive.Partition p, long segmentStart, int parameterGroupId) throws org.rocksdb.RocksDBException, IOException - Throws:
org.rocksdb.RocksDBException
IOException
-
getIntervalStart
public static long getIntervalStart(long instant) returns the interval (instant) where this instant could fit.- Parameters:
instant
-- Returns:
-
getInterval
public static long getInterval(long instant) -
getIntervalEnd
public static long getIntervalEnd(long instant) returns the end of the interval where the instant fits- Parameters:
instant
-- Returns:
-
getIntervalDuration
public static long getIntervalDuration()duration in milliseconds of one segment- Returns:
-
getTablespace
-
getRealtimeFiller
-
compact
public void compact()
-