package rdd
- Alphabetic
- Public
- All
Type Members
-
class
ADAMContext extends Serializable with Logging
The ADAMContext provides functions on top of a SparkContext for loading genomic data.
-
trait
ADAMSaveAnyArgs extends SaveArgs
Argument configuration for saving any output format.
-
abstract
class
AvroGenomicDataset[T, U <: Product, V <: AvroGenomicDataset[T, U, V]] extends GenomicDataset[T, U, V]
An abstract class that extends GenomicDataset and where the underlying data are Avro IndexedRecords.
An abstract class that extends GenomicDataset and where the underlying data are Avro IndexedRecords. This abstract class provides methods for saving Avro to Parquet.
-
abstract
class
AvroReadGroupGenomicDataset[T, U <: Product, V <: AvroReadGroupGenomicDataset[T, U, V]] extends AvroGenomicDataset[T, U, V] with GenomicDatasetWithLineage[T, U, V]
An abstract class describing a GenomicDataset where:
An abstract class describing a GenomicDataset where:
* The data are Avro IndexedRecords. * The data are associated to read groups (i.e., they are reads or fragments).
- case class DatasetBoundGenericGenomicDataset[T, U <: Product](dataset: Dataset[U], sequences: SequenceDictionary, converter: GenericConverter[T, U], tagHolder: TagHolder[T, U]) extends GenericGenomicDataset[T, U] with Product with Serializable
-
trait
DatasetBoundGenomicDataset[T, U <: Product, V <: GenomicDataset[T, U, V]] extends GenomicDataset[T, U, V]
A trait describing a GenomicDataset that is physically backed by a Dataset.
- case class FullOuterShuffleRegionJoin[T, U](leftRdd: RDD[(ReferenceRegion, T)], rightRdd: RDD[(ReferenceRegion, U)])(implicit evidence$11: ClassTag[T], evidence$12: ClassTag[U]) extends ShuffleRegionJoin[T, U, Option[T], Option[U]] with SortedIntervalPartitionJoinWithVictims[T, U, Option[T], Option[U]] with Product with Serializable
- case class GenericConverter[T, U] extends Product with Serializable
- sealed abstract class GenericGenomicDataset[T, U <: Product] extends GenomicDataset[T, U, GenericGenomicDataset[T, U]]
-
case class
GenomeBins(binSize: Long, seqLengths: Map[String, Long]) extends Serializable with Product
Partition a genome into a set of bins.
Partition a genome into a set of bins.
Note that this class will not tolerate invalid input, so filter in advance if you use it.
- binSize
The size of each bin in nucleotides
- seqLengths
A map containing the length of each contig
- case class GenomicBroadcast[T, U <: Product, V <: GenomicDataset[T, U, V]] extends Product with Serializable
-
trait
GenomicDataset[T, U <: Product, V <: GenomicDataset[T, U, V]] extends Logging
A trait that wraps an RDD or Dataset of genomic data with helpful metadata.
A trait that wraps an RDD or Dataset of genomic data with helpful metadata.
- T
The type of the data in the wrapped RDD or Dataset.
- U
The type of this GenomicDataset.
- trait GenomicDatasetConversion[T, U <: Product, V <: GenomicDataset[T, U, V], X, Y <: Product, Z <: GenomicDataset[X, Y, Z]] extends Function2[V, Dataset[Y], Z]
- trait GenomicDatasetWithLineage[T, U <: Product, V <: GenomicDatasetWithLineage[T, U, V]] extends GenomicDataset[T, U, V]
-
case class
GenomicPositionPartitioner(numParts: Int, seqLengths: Map[String, Long]) extends Partitioner with Logging with Product with Serializable
GenomicPositionPartitioner partitions ReferencePosition objects into separate, spatially-coherent regions of the genome.
GenomicPositionPartitioner partitions ReferencePosition objects into separate, spatially-coherent regions of the genome.
This can be used to organize genomic data for computation that is spatially distributed (e.g. GATK and Queue's "scatter-and-gather" for locus-parallelizable walkers).
- numParts
The number of equally-sized regions into which the total genomic space is partitioned; the total number of partitions is numParts + 1, with the "+1" resulting from one extra partition that is used to capture null or UNMAPPED values of the ReferencePosition type.
- seqLengths
a map relating sequence-name to length and indicating the set and length of all extant sequences in the genome.
-
case class
GenomicRegionPartitioner(partitionSize: Long, seqLengths: Map[String, Long], start: Boolean = true) extends Partitioner with Logging with Product with Serializable
A partitioner for ReferenceRegion-keyed data.
A partitioner for ReferenceRegion-keyed data.
- partitionSize
The number of bases per partition.
- seqLengths
A map between contig names and contig lengths.
- start
If true, use the start position (instead of the end position) to decide which partition a key belongs to.
-
trait
InFormatter[T, U <: Product, V <: GenomicDataset[T, U, V], W <: InFormatter[T, U, V, W]] extends Serializable
Formats data going into a pipe to an invoked process.
Formats data going into a pipe to an invoked process.
- T
The type of records being formatted.
-
trait
InFormatterCompanion[T, U <: Product, V <: GenomicDataset[T, U, V], W <: InFormatter[T, U, V, W]] extends AnyRef
A trait for singleton objects that build an InFormatter from a GenomicDataset.
A trait for singleton objects that build an InFormatter from a GenomicDataset.
Often, when creating an outputstream, we need to add metadata to the output that is not attached to individual records. An example of this is writing a header with contig/read group/format info, as is done with SAM/BAM/VCF.
- T
The type of the records this InFormatter writes out.
- U
The type of the GenomicDataset this companion object understands.
- V
The type of InFormatter this companion object creates.
- case class InnerShuffleRegionJoin[T, U](leftRdd: RDD[(ReferenceRegion, T)], rightRdd: RDD[(ReferenceRegion, U)])(implicit evidence$3: ClassTag[T], evidence$4: ClassTag[U]) extends ShuffleRegionJoin[T, U, T, U] with VictimlessSortedIntervalPartitionJoin[T, U, T, U] with Product with Serializable
- case class InnerShuffleRegionJoinAndGroupByLeft[T, U](leftRdd: RDD[(ReferenceRegion, T)], rightRdd: RDD[(ReferenceRegion, U)])(implicit evidence$5: ClassTag[T], evidence$6: ClassTag[U]) extends ShuffleRegionJoin[T, U, T, Iterable[U]] with VictimlessSortedIntervalPartitionJoin[T, U, T, Iterable[U]] with Product with Serializable
-
case class
InnerTreeRegionJoin[T, U]()(implicit evidence$1: ClassTag[T], evidence$2: ClassTag[U]) extends RegionJoin[T, U, T, U] with TreeRegionJoin[T, U, T, U] with Product with Serializable
Implements an inner region join where the left side of the join is broadcast.
-
case class
InnerTreeRegionJoinAndGroupByRight[T, U]()(implicit evidence$5: ClassTag[T], evidence$6: ClassTag[U]) extends RegionJoin[T, U, Iterable[T], U] with TreeRegionJoin[T, U, Iterable[T], U] with Product with Serializable
Performs an inner region join, followed logically by grouping by the right value.
Performs an inner region join, followed logically by grouping by the right value. This is implemented without any shuffling; the join naturally returns values on the left grouped by the right value.
- case class LeftOuterShuffleRegionJoin[T, U](leftRdd: RDD[(ReferenceRegion, T)], rightRdd: RDD[(ReferenceRegion, U)])(implicit evidence$7: ClassTag[T], evidence$8: ClassTag[U]) extends ShuffleRegionJoin[T, U, T, Option[U]] with VictimlessSortedIntervalPartitionJoin[T, U, T, Option[U]] with Product with Serializable
- case class LeftOuterShuffleRegionJoinAndGroupByLeft[T, U](leftRdd: RDD[(ReferenceRegion, T)], rightRdd: RDD[(ReferenceRegion, U)])(implicit evidence$9: ClassTag[T], evidence$10: ClassTag[U]) extends ShuffleRegionJoin[T, U, T, Iterable[U]] with VictimlessSortedIntervalPartitionJoin[T, U, T, Iterable[U]] with Product with Serializable
-
abstract
class
MultisampleAvroGenomicDataset[T, U <: Product, V <: MultisampleAvroGenomicDataset[T, U, V]] extends AvroGenomicDataset[T, U, V] with MultisampleGenomicDataset[T, U, V]
An abstract class that extends the MultisampleGenomicDataset trait, where the data are Avro IndexedRecords.
-
trait
MultisampleGenomicDataset[T, U <: Product, V <: MultisampleGenomicDataset[T, U, V]] extends GenomicDataset[T, U, V]
A trait describing a GenomicDataset with data from multiple samples.
-
trait
OutFormatter[T] extends Serializable
Deserializes data coming out of a pipe from an invoked process.
Deserializes data coming out of a pipe from an invoked process.
- T
The type of records being formatted.
- case class RDDBoundGenericGenomicDataset[T, U <: Product](rdd: RDD[T], sequences: SequenceDictionary, converter: GenericConverter[T, U], tagHolder: TagHolder[T, U], optPartitionMap: Option[Array[Option[(ReferenceRegion, ReferenceRegion)]]] = None) extends GenericGenomicDataset[T, U] with Product with Serializable
-
case class
ReferencePartitioner(sd: SequenceDictionary) extends Partitioner with Product with Serializable
Repartitions objects that are keyed by a ReferencePosition or ReferenceRegion into a single partition per contig.
-
abstract
class
RegionJoin[T, U, RT, RU] extends Serializable
A trait describing a join in the genomic coordinate space between two RDDs where the values are keyed by a ReferenceRegion.
A trait describing a join in the genomic coordinate space between two RDDs where the values are keyed by a ReferenceRegion.
- T
The type of the left RDD.
- U
The type of the right RDD.
- RT
The type of data yielded by the left RDD at the output of the join. This may not match T if the join is an outer join, etc.
- RU
The type of data yielded by the right RDD at the output of the join.
- case class RightOuterShuffleRegionJoinAndGroupByLeft[T, U](leftRdd: RDD[(ReferenceRegion, T)], rightRdd: RDD[(ReferenceRegion, U)])(implicit evidence$13: ClassTag[T], evidence$14: ClassTag[U]) extends ShuffleRegionJoin[T, U, Option[T], Iterable[U]] with SortedIntervalPartitionJoinWithVictims[T, U, Option[T], Iterable[U]] with Product with Serializable
-
case class
RightOuterTreeRegionJoin[T, U]()(implicit evidence$3: ClassTag[T], evidence$4: ClassTag[U]) extends RegionJoin[T, U, Option[T], U] with TreeRegionJoin[T, U, Option[T], U] with Product with Serializable
Implements a right outer region join where the left side of the join is broadcast.
-
case class
RightOuterTreeRegionJoinAndGroupByRight[T, U]()(implicit evidence$7: ClassTag[T], evidence$8: ClassTag[U]) extends RegionJoin[T, U, Iterable[T], U] with TreeRegionJoin[T, U, Iterable[T], U] with Product with Serializable
Performs a right outer region join, followed logically by grouping by the right value.
Performs a right outer region join, followed logically by grouping by the right value. This is implemented without any shuffling; the join naturally returns values on the left grouped by the right value. In this implementation, empty collections on the left side of the join are kept.
-
sealed abstract
class
ShuffleRegionJoin[T, U, RT, RU] extends RegionJoin[T, U, RT, RU]
A trait describing join implementations that are based on a sort-merge join.
A trait describing join implementations that are based on a sort-merge join.
- T
The type of the left RDD.
- U
The type of the right RDD.
- RT
The type of data yielded by the left RDD at the output of the join. This may not match T if the join is an outer join, etc.
- RU
The type of data yielded by the right RDD at the output of the join.
- sealed trait SortedIntervalPartitionJoinWithVictims[T, U, RT, RU] extends ShuffleRegionJoin[T, U, RT, RU]
-
trait
TreeRegionJoin[T, U, RT, RU] extends RegionJoin[T, U, RT, RU]
Implements a shuffle free (broadcast) region join.
Implements a shuffle free (broadcast) region join.
The broadcast values are stored in a sorted array. It was going to be an ensemble of interval trees, but, that didn't work out.
- sealed trait VictimlessSortedIntervalPartitionJoin[T, U, RT, RU] extends ShuffleRegionJoin[T, U, RT, RU]
Value Members
-
object
ADAMContext extends Serializable
This singleton provides an implicit conversion from a SparkContext to the ADAMContext, as well as implicit functions for the Pipe API.
-
object
GenomicPositionPartitioner extends Serializable
Helper for creating genomic position partitioners.
-
object
GenomicRegionPartitioner extends Serializable
Helper object for creating GenomicRegionPartitioners.