Class RampupThrottlingFn<T>
- java.lang.Object
-
- org.apache.beam.sdk.transforms.DoFn<T,T>
-
- org.apache.beam.sdk.io.gcp.datastore.RampupThrottlingFn<T>
-
- All Implemented Interfaces:
java.io.Serializable
,org.apache.beam.sdk.transforms.display.HasDisplayData
public class RampupThrottlingFn<T> extends org.apache.beam.sdk.transforms.DoFn<T,T> implements java.io.Serializable
An implementation of a client-side throttler that enforces a gradual ramp-up, broadly in line with Datastore best practices. See also https://cloud.google.com/datastore/docs/best-practices#ramping_up_traffic.- See Also:
- Serialized Form
-
-
Nested Class Summary
-
Nested classes/interfaces inherited from class org.apache.beam.sdk.transforms.DoFn
org.apache.beam.sdk.transforms.DoFn.AlwaysFetched, org.apache.beam.sdk.transforms.DoFn.BoundedPerElement, org.apache.beam.sdk.transforms.DoFn.BundleFinalizer, org.apache.beam.sdk.transforms.DoFn.Element, org.apache.beam.sdk.transforms.DoFn.FieldAccess, org.apache.beam.sdk.transforms.DoFn.FinishBundle, org.apache.beam.sdk.transforms.DoFn.FinishBundleContext, org.apache.beam.sdk.transforms.DoFn.GetInitialRestriction, org.apache.beam.sdk.transforms.DoFn.GetInitialWatermarkEstimatorState, org.apache.beam.sdk.transforms.DoFn.GetRestrictionCoder, org.apache.beam.sdk.transforms.DoFn.GetSize, org.apache.beam.sdk.transforms.DoFn.GetWatermarkEstimatorStateCoder, org.apache.beam.sdk.transforms.DoFn.Key, org.apache.beam.sdk.transforms.DoFn.MultiOutputReceiver, org.apache.beam.sdk.transforms.DoFn.NewTracker, org.apache.beam.sdk.transforms.DoFn.NewWatermarkEstimator, org.apache.beam.sdk.transforms.DoFn.OnTimer, org.apache.beam.sdk.transforms.DoFn.OnTimerContext, org.apache.beam.sdk.transforms.DoFn.OnTimerFamily, org.apache.beam.sdk.transforms.DoFn.OnWindowExpiration, org.apache.beam.sdk.transforms.DoFn.OnWindowExpirationContext, org.apache.beam.sdk.transforms.DoFn.OutputReceiver<T extends java.lang.Object>, org.apache.beam.sdk.transforms.DoFn.ProcessContext, org.apache.beam.sdk.transforms.DoFn.ProcessContinuation, org.apache.beam.sdk.transforms.DoFn.ProcessElement, org.apache.beam.sdk.transforms.DoFn.RequiresStableInput, org.apache.beam.sdk.transforms.DoFn.RequiresTimeSortedInput, org.apache.beam.sdk.transforms.DoFn.Restriction, org.apache.beam.sdk.transforms.DoFn.Setup, org.apache.beam.sdk.transforms.DoFn.SideInput, org.apache.beam.sdk.transforms.DoFn.SplitRestriction, org.apache.beam.sdk.transforms.DoFn.StartBundle, org.apache.beam.sdk.transforms.DoFn.StartBundleContext, org.apache.beam.sdk.transforms.DoFn.StateId, org.apache.beam.sdk.transforms.DoFn.Teardown, org.apache.beam.sdk.transforms.DoFn.TimerFamily, org.apache.beam.sdk.transforms.DoFn.TimerId, org.apache.beam.sdk.transforms.DoFn.Timestamp, org.apache.beam.sdk.transforms.DoFn.TruncateRestriction, org.apache.beam.sdk.transforms.DoFn.UnboundedPerElement, org.apache.beam.sdk.transforms.DoFn.WatermarkEstimatorState, org.apache.beam.sdk.transforms.DoFn.WindowedContext
-
-
Constructor Summary
Constructors Constructor Description RampupThrottlingFn(int numWorkers, org.apache.beam.sdk.values.PCollectionView<org.joda.time.Instant> timestampSideInput)
RampupThrottlingFn(org.apache.beam.sdk.options.ValueProvider<java.lang.Integer> numWorkers, org.apache.beam.sdk.values.PCollectionView<org.joda.time.Instant> firstInstantSideInput)
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description void
populateDisplayData(org.apache.beam.sdk.transforms.display.DisplayData.Builder builder)
void
processElement(org.apache.beam.sdk.transforms.DoFn.ProcessContext c)
Emit only as many elements as the exponentially increasing budget allows.void
setup()
-
-
-
Constructor Detail
-
RampupThrottlingFn
public RampupThrottlingFn(org.apache.beam.sdk.options.ValueProvider<java.lang.Integer> numWorkers, org.apache.beam.sdk.values.PCollectionView<org.joda.time.Instant> firstInstantSideInput)
-
RampupThrottlingFn
public RampupThrottlingFn(int numWorkers, org.apache.beam.sdk.values.PCollectionView<org.joda.time.Instant> timestampSideInput)
-
-
Method Detail
-
setup
@Setup public void setup()
-
processElement
@ProcessElement public void processElement(org.apache.beam.sdk.transforms.DoFn.ProcessContext c) throws java.io.IOException, java.lang.InterruptedException
Emit only as many elements as the exponentially increasing budget allows.- Throws:
java.io.IOException
java.lang.InterruptedException
-
populateDisplayData
public void populateDisplayData(org.apache.beam.sdk.transforms.display.DisplayData.Builder builder)
-
-