Class MapKeys<K1,K2,V>
- java.lang.Object
-
- org.apache.beam.sdk.transforms.PTransform<PCollection<KV<K1,V>>,PCollection<KV<K2,V>>>
-
- org.apache.beam.sdk.transforms.MapKeys<K1,K2,V>
-
- Type Parameters:
K1
- the type of the keys in the inputPCollection
K2
- the type of the keys in the outputPCollection
- All Implemented Interfaces:
java.io.Serializable
,HasDisplayData
public class MapKeys<K1,K2,V> extends PTransform<PCollection<KV<K1,V>>,PCollection<KV<K2,V>>>
MapKeys
maps aSerializableFunction<K1,K2>
over keys of aPCollection<KV<K1,V>>
and returns aPCollection<KV<K2, V>>
.Example of use:
PCollection<KV<Integer, String>> input = ...; PCollection<KV<Double, String> output = input.apply(MapKeys.into(TypeDescriptors.doubles()).via(Integer::doubleValue));
See also
MapValues
.- See Also:
- Serialized Form
-
-
Field Summary
-
Fields inherited from class org.apache.beam.sdk.transforms.PTransform
name, resourceHints
-
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description <FailureT> org.apache.beam.sdk.transforms.SimpleMapWithFailures<KV<K1,V>,KV<K2,V>,FailureT>
exceptionsInto(TypeDescriptor<FailureT> failureTypeDescriptor)
Returns a newSimpleMapWithFailures
transform that catches exceptions raised while mapping elements, with the given type descriptor used for the failure collection but the exception handler yet to be specified usingSimpleMapWithFailures.exceptionsVia(ProcessFunction)
.<FailureT> org.apache.beam.sdk.transforms.SimpleMapWithFailures<KV<K1,V>,KV<K2,V>,FailureT>
exceptionsVia(InferableFunction<WithFailures.ExceptionElement<KV<K1,V>>,FailureT> exceptionHandler)
Returns a newSimpleMapWithFailures
transform that catches exceptions raised while mapping elements, passing the raised exception instance and the input element being processed through the givenexceptionHandler
and emitting the result to a failure collection.PCollection<KV<K2,V>>
expand(PCollection<KV<K1,V>> input)
Override this method to specify how thisPTransform
should be expanded on the givenInputT
.static <K2> MapKeys<?,K2,?>
into(TypeDescriptor<K2> outputType)
Returns a newMapKeys
transform with the given type descriptor for the output type, but the mapping function yet to be specified usingvia(SerializableFunction)
.<NewValueT,NewKeyT>
MapKeys<NewKeyT,K2,NewValueT>via(SerializableFunction<NewKeyT,K2> fn)
Returns aMapKeys<K1, K2, V>
PTransform
for aProcessFunction<NewK1, K2>
with predefinedoutputType
.-
Methods inherited from class org.apache.beam.sdk.transforms.PTransform
compose, compose, getAdditionalInputs, getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, getResourceHints, populateDisplayData, setResourceHints, toString, validate, validate
-
-
-
-
Method Detail
-
via
public <NewValueT,NewKeyT> MapKeys<NewKeyT,K2,NewValueT> via(SerializableFunction<NewKeyT,K2> fn)
Returns aMapKeys<K1, K2, V>
PTransform
for aProcessFunction<NewK1, K2>
with predefinedoutputType
.- Type Parameters:
NewKeyT
- the type of the keys in the inputPCollection
NewValueT
- the type of the values in the input and outputPCollection
s
-
into
public static <K2> MapKeys<?,K2,?> into(TypeDescriptor<K2> outputType)
Returns a newMapKeys
transform with the given type descriptor for the output type, but the mapping function yet to be specified usingvia(SerializableFunction)
.
-
exceptionsInto
@RequiresNonNull("fn") public <FailureT> org.apache.beam.sdk.transforms.SimpleMapWithFailures<KV<K1,V>,KV<K2,V>,FailureT> exceptionsInto(TypeDescriptor<FailureT> failureTypeDescriptor)
Returns a newSimpleMapWithFailures
transform that catches exceptions raised while mapping elements, with the given type descriptor used for the failure collection but the exception handler yet to be specified usingSimpleMapWithFailures.exceptionsVia(ProcessFunction)
.See
WithFailures
documentation for usage patterns of the returnedWithFailures.Result
.Example usage:
Result<PCollection<KV<Integer, String>>, String> result = input.apply( MapKeys.into(TypeDescriptors.integers()) .<String, String>via(word -> 1 / word.length) // Could throw ArithmeticException .exceptionsInto(TypeDescriptors.strings()) .exceptionsVia(ee -> ee.exception().getMessage())); PCollection<KV<Integer, String>> output = result.output(); PCollection<String> failures = result.failures();
-
exceptionsVia
@RequiresNonNull("fn") public <FailureT> org.apache.beam.sdk.transforms.SimpleMapWithFailures<KV<K1,V>,KV<K2,V>,FailureT> exceptionsVia(InferableFunction<WithFailures.ExceptionElement<KV<K1,V>>,FailureT> exceptionHandler)
Returns a newSimpleMapWithFailures
transform that catches exceptions raised while mapping elements, passing the raised exception instance and the input element being processed through the givenexceptionHandler
and emitting the result to a failure collection.This method takes advantage of the type information provided by
InferableFunction
, meaning that a call toexceptionsInto(TypeDescriptor)
may not be necessary.See
WithFailures
documentation for usage patterns of the returnedWithFailures.Result
.Example usage:
Result<PCollection<KV<Integer, String>>, String> result = input.apply( MapKeys.into(TypeDescriptors.integers()) .<String, String>via(word -> 1 / word.length) // Could throw ArithmeticException .exceptionsVia( new InferableFunction<ExceptionElement<KV<String, String>>, String>() { @Override public String apply(ExceptionElement<KV<String, String>> input) { return input.exception().getMessage(); } })); PCollection<KV<Integer, String>> output = result.output(); PCollection<String> failures = result.failures();
-
expand
public PCollection<KV<K2,V>> expand(PCollection<KV<K1,V>> input)
Description copied from class:PTransform
Override this method to specify how thisPTransform
should be expanded on the givenInputT
.NOTE: This method should not be called directly. Instead apply the
PTransform
should be applied to theInputT
using theapply
method.Composite transforms, which are defined in terms of other transforms, should return the output of one of the composed transforms. Non-composite transforms, which do not apply any transforms internally, should return a new unbound output and register evaluators (via backend-specific registration methods).
- Specified by:
expand
in classPTransform<PCollection<KV<K1,V>>,PCollection<KV<K2,V>>>
-
-