Distinct (core 2.44.0 API)

java.lang.Object
- org.apache.beam.sdk.transforms.PTransform<PCollection<T>,PCollection<T>>
- - org.apache.beam.sdk.transforms.Distinct<T>

Type Parameters:

T - the type of the elements of the input and output PCollections

All Implemented Interfaces:

java.io.Serializable, HasDisplayData
```
public class Distinct<T>
extends PTransform<PCollection<T>,PCollection<T>>
```
Distinct<T> takes a PCollection<T> and returns a PCollection<T> that has all distinct elements of the input. Thus, each element is unique within each window.
Two values of type T are compared for equality not by regular Java Object.equals(java.lang.Object), but instead by first encoding each of the elements using the PCollection's Coder, and then comparing the encoded bytes. This admits efficient parallel evaluation.
Optionally, a function may be provided that maps each element to a representative value. In this case, two elements will be considered duplicates if they have equal representative values, with equality being determined as above.
By default, the Coder of the output PCollection is the same as the Coder of the input PCollection.
Each output element is in the same window as its corresponding input element, and has the timestamp of the end of that window. The output PCollection has the same WindowFn as the input.
Does not preserve any order the input PCollection might have had.
Example of use:
```
 PCollection<String> words = ...;
 PCollection<String> uniqueWords =
     words.apply(Distinct.<String>create());
 
```
See Also:

Serialized Form

Nested Class Summary

Nested Classes
Modifier and Type	Class	Description
`static class`	`Distinct.WithRepresentativeValues<T,IdT>`	A `Distinct` `PTransform` that uses a `SerializableFunction` to obtain a representative value for each input element.

Field Summary
- Fields inherited from class org.apache.beam.sdk.transforms.PTransform
  name, resourceHints

Constructor Summary

Constructors
Constructor Description

Distinct()

Method Summary

All Methods Static Methods Instance Methods Concrete Methods
Modifier and Type	Method	Description
`static <T> Distinct<T>`	`create()`	Returns a `Distinct<T>` `PTransform`.
`PCollection<T>`	`expand(PCollection<T> in)`	Override this method to specify how this `PTransform` should be expanded on the given `InputT`.
`static <T,IdT> Distinct.WithRepresentativeValues<T,IdT>`	`withRepresentativeValueFn(SerializableFunction<T,IdT> fn)`	Returns a `Distinct<T, IdT>` `PTransform`.

Methods inherited from class org.apache.beam.sdk.transforms.PTransform
compose, compose, getAdditionalInputs, getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, getResourceHints, populateDisplayData, setResourceHints, toString, validate, validate

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

- Constructor Detail
  - Distinct
```
public Distinct()
```
- Method Detail
  - create
```
public static <T> Distinct<T> create()
```
    Returns a Distinct<T> PTransform.
    
    Type Parameters:
    
    T - the type of the elements of the input and output PCollections
  - withRepresentativeValueFn
```
public static <T,IdT> Distinct.WithRepresentativeValues<T,IdT> withRepresentativeValueFn(SerializableFunction<T,IdT> fn)
```
    Returns a Distinct<T, IdT> PTransform.
    
    Type Parameters:
    
    T - the type of the elements of the input and output PCollections
    
    IdT - the type of the representative value used to dedup
  - expand
```
public PCollection<T> expand(PCollection<T> in)
```
    Description copied from class: PTransform
    
    Override this method to specify how this PTransform should be expanded on the given InputT.
    NOTE: This method should not be called directly. Instead apply the PTransform should be applied to the InputT using the apply method.
    Composite transforms, which are defined in terms of other transforms, should return the output of one of the composed transforms. Non-composite transforms, which do not apply any transforms internally, should return a new unbound output and register evaluators (via backend-specific registration methods).
    
    Specified by:
    
    expand in class PTransform<PCollection<T>,PCollection<T>>