org.broadinstitute.hellbender.engine

Class FeatureManager

java.lang.Object

org.broadinstitute.hellbender.engine.FeatureManager

All Implemented Interfaces:

java.lang.AutoCloseable

public final class FeatureManager
extends java.lang.Object
implements java.lang.AutoCloseable

Handles discovery of available codecs and Feature arguments, file format detection and codec selection, and creation/management/querying of FeatureDataSources for each source of Features. At startup, walks the packages specified in GATKConfig.codec_packages() in the config file to discover what codecs are available to decode Feature-containing files. Then, given a tool instance, it discovers what FeatureInput argument fields are declared in the tool's class hierarchy (and associated ArgumentCollections), and for each argument actually specified by the user on the command line, determines the type of the file and the codec required to decode it, creates a FeatureDataSource for that file, and adds it to a query-able resource pool. Clients can then call #getFeatures(FeatureInput, SimpleInterval) to query the data source for a particular FeatureInput over a specific interval.

Constructor Summary

Constructors

Constructor and Description
FeatureManager(CommandLineProgram toolInstance) Create a FeatureManager given a CommandLineProgram tool instance, discovering all FeatureInput arguments in the tool and creating query-able FeatureDataSources for them.
FeatureManager(CommandLineProgram toolInstance, int featureQueryLookahead) Create a FeatureManager given a CommandLineProgram tool instance, discovering all FeatureInput arguments in the tool and creating query-able FeatureDataSources for them.
FeatureManager(CommandLineProgram toolInstance, int featureQueryLookahead, int cloudPrefetchBuffer, int cloudIndexPrefetchBuffer) Create a FeatureManager given a CommandLineProgram tool instance, discovering all FeatureInput arguments in the tool and creating query-able FeatureDataSources for them.
FeatureManager(CommandLineProgram toolInstance, int featureQueryLookahead, int cloudPrefetchBuffer, int cloudIndexPrefetchBuffer, GenomicsDBOptions gdbOptions) Create a FeatureManager given a CommandLineProgram tool instance, discovering all FeatureInput arguments in the tool and creating query-able FeatureDataSources for them.

Method Summary

Modifier and Type	Method and Description
void	close() Permanently closes this manager by closing all backing data sources
void	dumpAllFeatureCacheStats()
java.util.List<htsjdk.samtools.SAMSequenceDictionary>	getAllSequenceDictionaries() Returns the sequence dictionaries associated with all feature sources.
java.util.List<htsjdk.variant.vcf.VCFHeader>	getAllVariantHeaders() This method finds and returns all of the variant headers from the feature sources.
static htsjdk.tribble.FeatureCodec<? extends htsjdk.tribble.Feature,?>	getCodecForFile(java.nio.file.Path featurePath) Utility method that determines the correct codec to use to read Features from the provided file.
static htsjdk.tribble.FeatureCodec<? extends htsjdk.tribble.Feature,?>	getCodecForFile(java.nio.file.Path featurePath, java.lang.Class<? extends htsjdk.tribble.Feature> featureType) Utility method that determines the correct codec to use to read Features from the provided file, optionally considering only codecs that produce a particular type of Feature.
<T extends htsjdk.tribble.Feature> java.util.Iterator<T>	getFeatureIterator(FeatureInput<T> featureDescriptor) Given a FeatureInput argument field from our tool, returns an iterator to its features starting from the first one.
<T extends htsjdk.tribble.Feature> java.util.List<T>	getFeatures(FeatureInput<T> featureDescriptor, htsjdk.samtools.util.Locatable interval) Given a FeatureInput argument field from our tool, queries the data source for that FeatureInput over the specified interval, and returns a List of the Features overlapping that interval from that data source.
<T extends htsjdk.tribble.Feature> java.lang.Object	getHeader(FeatureInput<T> featureDescriptor) Get the header associated with a particular FeatureInput
java.util.List<htsjdk.samtools.SAMSequenceDictionary>	getVariantSequenceDictionaries() Returns the list of sequence dictionaries retrieved from the VCF headers of variant Feature inputs.
boolean	isEmpty() Does this manager have no sources of Features to query?
static boolean	isFeatureFile(java.nio.file.Path file)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FeatureManager

public FeatureManager(CommandLineProgram toolInstance)

Create a FeatureManager given a CommandLineProgram tool instance, discovering all FeatureInput arguments in the tool and creating query-able FeatureDataSources for them. Uses the default caching behavior of FeatureDataSource.

Parameters:

toolInstance - Instance of the tool to be run (potentially containing one or more FeatureInput arguments) Must have undergone command-line argument parsing and argument value injection already.

FeatureManager

public FeatureManager(CommandLineProgram toolInstance,
                      int featureQueryLookahead)

Create a FeatureManager given a CommandLineProgram tool instance, discovering all FeatureInput arguments in the tool and creating query-able FeatureDataSources for them. Allows control over how much caching is performed by each FeatureDataSource.

Parameters:

toolInstance - Instance of the tool to be run (potentially containing one or more FeatureInput arguments) Must have undergone command-line argument parsing and argument value injection already.

featureQueryLookahead - When querying FeatureDataSources, cache this many extra bases of context beyond the end of query intervals in anticipation of future queries (>= 0).

FeatureManager

public FeatureManager(CommandLineProgram toolInstance,
                      int featureQueryLookahead,
                      int cloudPrefetchBuffer,
                      int cloudIndexPrefetchBuffer)

Create a FeatureManager given a CommandLineProgram tool instance, discovering all FeatureInput arguments in the tool and creating query-able FeatureDataSources for them. Allows control over how much caching is performed by each FeatureDataSource.

Parameters:

toolInstance - Instance of the tool to be run (potentially containing one or more FeatureInput arguments) Must have undergone command-line argument parsing and argument value injection already.

featureQueryLookahead - When querying FeatureDataSources, cache this many extra bases of context beyond the end of query intervals in anticipation of future queries (>= 0).

cloudPrefetchBuffer - MB size of caching/prefetching wrapper for the data, if on Google Cloud (0 to disable).

cloudIndexPrefetchBuffer - MB size of caching/prefetching wrapper for the index, if on Google Cloud (0 to disable).

FeatureManager

public FeatureManager(CommandLineProgram toolInstance,
                      int featureQueryLookahead,
                      int cloudPrefetchBuffer,
                      int cloudIndexPrefetchBuffer,
                      GenomicsDBOptions gdbOptions)

Create a FeatureManager given a CommandLineProgram tool instance, discovering all FeatureInput arguments in the tool and creating query-able FeatureDataSources for them. Allows control over how much caching is performed by each FeatureDataSource.

Parameters:

toolInstance - Instance of the tool to be run (potentially containing one or more FeatureInput arguments) Must have undergone command-line argument parsing and argument value injection already.

featureQueryLookahead - When querying FeatureDataSources, cache this many extra bases of context beyond the end of query intervals in anticipation of future queries (>= 0).

cloudPrefetchBuffer - MB size of caching/prefetching wrapper for the data, if on Google Cloud (0 to disable).

cloudIndexPrefetchBuffer - MB size of caching/prefetching wrapper for the index, if on Google Cloud (0 to disable).

gdbOptions - settings for GenomicsDB to use when reading from a GenomicsDB workspace

Method Detail

dumpAllFeatureCacheStats

public void dumpAllFeatureCacheStats()

isEmpty

public boolean isEmpty()

Does this manager have no sources of Features to query?

Returns:

true if there are no Feature sources available to query, otherwise false

getAllVariantHeaders

public java.util.List<htsjdk.variant.vcf.VCFHeader> getAllVariantHeaders()

This method finds and returns all of the variant headers from the feature sources.

Returns:

A list of all variant headers for features.

getVariantSequenceDictionaries

public java.util.List<htsjdk.samtools.SAMSequenceDictionary> getVariantSequenceDictionaries()

Returns the list of sequence dictionaries retrieved from the VCF headers of variant Feature inputs. Note: this method returns an empty list if the variant inputs happen not to have sequence dictionaries (since they are optional in the VCF format).

getAllSequenceDictionaries

public java.util.List<htsjdk.samtools.SAMSequenceDictionary> getAllSequenceDictionaries()

Returns the sequence dictionaries associated with all feature sources. This method will return an empty List if none of the feature sources have dictionaries.

getFeatures

public <T extends htsjdk.tribble.Feature> java.util.List<T> getFeatures(FeatureInput<T> featureDescriptor,
                                                                        htsjdk.samtools.util.Locatable interval)

Given a FeatureInput argument field from our tool, queries the data source for that FeatureInput over the specified interval, and returns a List of the Features overlapping that interval from that data source. Will throw an exception if the provided FeatureInput did not come from the tool that this FeatureManager was initialized with, or was not an @Argument-annotated field in the tool (or parent classes).

Type Parameters:

T - type of Feature in the source represented by featureDescriptor

Parameters:

featureDescriptor - FeatureInput argument from our tool representing the Feature source to query

interval - interval to query over (returned Features will overlap this interval)

Returns:

A List of all Features in the backing data source for the provided FeatureInput that overlap the provided interval (may be empty if there are none, but never null)

getFeatureIterator

public <T extends htsjdk.tribble.Feature> java.util.Iterator<T> getFeatureIterator(FeatureInput<T> featureDescriptor)

Given a FeatureInput argument field from our tool, returns an iterator to its features starting from the first one.

Warning!: calling this method a second time on the same FeatureInput on the same FeatureManager instance will invalidate (close) the iterator returned from the first call.

An exception will be thrown if the FeatureInput provided did not come from the tool that this manager was initialized with, or was not an @Argument-annotated field in the tool (or parent classes).

Type Parameters:

T - type of Feature in the source represented by featureDescriptor

Parameters:

featureDescriptor - FeatureInput argument from our tool representing the Feature source to query

Returns:

never null, a iterator to all the features in the backing data source.

Throws:

GATKException - if the feature-descriptor is not found in the manager or is null.

getHeader

public <T extends htsjdk.tribble.Feature> java.lang.Object getHeader(FeatureInput<T> featureDescriptor)

Get the header associated with a particular FeatureInput

Type Parameters:

T - type of Feature in our FeatureInput

Parameters:

featureDescriptor - the FeatureInput whose header we want to retrieve

Returns:

header for the provided FeatureInput

getCodecForFile

public static htsjdk.tribble.FeatureCodec<? extends htsjdk.tribble.Feature,?> getCodecForFile(java.nio.file.Path featurePath)

Utility method that determines the correct codec to use to read Features from the provided file. Codecs MUST correctly implement the FeatureCodec.canDecode(String) method in order to be considered as candidates for decoding the file. Throws an exception if no suitable codecs are found (this is a user error, since the file is of an unsupported format), or if more than one codec claims to be able to decode the file (this is a configuration error on the codec authors' part).

Parameters:

featurePath - path for which to find the right codec

Returns:

the codec suitable for decoding the provided file

getCodecForFile

public static htsjdk.tribble.FeatureCodec<? extends htsjdk.tribble.Feature,?> getCodecForFile(java.nio.file.Path featurePath,
                                                                                              java.lang.Class<? extends htsjdk.tribble.Feature> featureType)

Utility method that determines the correct codec to use to read Features from the provided file, optionally considering only codecs that produce a particular type of Feature. Codecs MUST correctly implement the FeatureCodec.canDecode(String) method in order to be considered as candidates for decoding the file, and must produce Features of the specified type if featureType is non-null. Throws an exception if no suitable codecs are found (this is a user error, since the file is of an unsupported format), or if more than one codec claims to be able to decode the file (this is a configuration error on the codec authors' part).

Parameters:

featurePath - Path for which to find the right codec

featureType - If specified, consider only codecs that produce Features of this type. May be null, in which case all codecs are considered.

Returns:

the codec suitable for decoding the provided file

isFeatureFile

public static boolean isFeatureFile(java.nio.file.Path file)

Parameters:

file - file to check

Returns:

True if the file exists and contains Features (ie., we have a FeatureCodec that can decode it), otherwise false

close

public void close()

Permanently closes this manager by closing all backing data sources

Specified by:

close in interface java.lang.AutoCloseable