A B C D E F G H I J K L M N O P R S T U V W
All Classes All Packages
All Classes All Packages
All Classes All Packages
A
- abort() - Method in interface org.apache.beam.runners.fnexecution.state.StateDelegator.Registration
-
De-registers the handler for all future requests for state for the registered process bundle instruction id.
- accessPattern() - Method in class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors.SideInputSpec
- addLengthPrefixedCoder(String, RunnerApi.Components.Builder, boolean) - Static method in class org.apache.beam.runners.fnexecution.wire.LengthPrefixUnknownCoders
-
Recursively traverses the coder tree and wraps the first unknown coder in every branch with a
LengthPrefixCoder
unless an ancestor coder is itself aLengthPrefixCoder
. - addRunnerWireCoder(PipelineNode.PCollectionNode, RunnerApi.Components.Builder, RunnerApi.ExecutableStagePayload.WireCoderSetting) - Static method in class org.apache.beam.runners.fnexecution.wire.WireCoders
-
Creates a runner-side wire coder for a port read/write for the given PCollection.
- addSdkWireCoder(PipelineNode.PCollectionNode, RunnerApi.Components.Builder, RunnerApi.ExecutableStagePayload.WireCoderSetting) - Static method in class org.apache.beam.runners.fnexecution.wire.WireCoders
-
Creates an SDK-side wire coder for a port read/write for the given PCollection.
- append(K, W, Iterator<V>) - Method in interface org.apache.beam.runners.fnexecution.state.StateRequestHandlers.BagUserStateHandler
-
Appends the values to the bag user state for the given key and window.
- ArtifactDestination() - Constructor for class org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService.ArtifactDestination
- ArtifactRetrievalService - Class in org.apache.beam.runners.fnexecution.artifact
-
An
ArtifactRetrievalService
that usesFileSystems
as its backing storage. - ArtifactRetrievalService() - Constructor for class org.apache.beam.runners.fnexecution.artifact.ArtifactRetrievalService
- ArtifactRetrievalService(int) - Constructor for class org.apache.beam.runners.fnexecution.artifact.ArtifactRetrievalService
- ArtifactRetrievalService(ArtifactResolver) - Constructor for class org.apache.beam.runners.fnexecution.artifact.ArtifactRetrievalService
- ArtifactRetrievalService(ArtifactResolver, int) - Constructor for class org.apache.beam.runners.fnexecution.artifact.ArtifactRetrievalService
- ArtifactStagingService - Class in org.apache.beam.runners.fnexecution.artifact
- ArtifactStagingService(ArtifactStagingService.ArtifactDestinationProvider) - Constructor for class org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService
- ArtifactStagingService.ArtifactDestination - Class in org.apache.beam.runners.fnexecution.artifact
-
A pairing of a newly created artifact type and an ouptut stream that will be readable at that type.
- ArtifactStagingService.ArtifactDestinationProvider - Interface in org.apache.beam.runners.fnexecution.artifact
-
Provides a concrete location to which artifacts can be staged on retrieval.
- asResponseObserver() - Method in class org.apache.beam.runners.fnexecution.control.FnApiControlClient
B
- BagUserStateSpec() - Constructor for class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors.BagUserStateSpec
- BatchSideInputHandlerFactory - Class in org.apache.beam.runners.fnexecution.translation
-
StateRequestHandler
that uses aBatchSideInputHandlerFactory.SideInputGetter
to access side inputs. - BatchSideInputHandlerFactory.SideInputGetter - Interface in org.apache.beam.runners.fnexecution.translation
-
Returns the value for the side input with the given PCollection id from the runner.
- beamFilesystemArtifactDestinationProvider(String) - Static method in class org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService
-
An ArtifactDestinationProvider that places new artifacts as files in a Beam filesystem.
- BeamWorkerStatusGrpcService - Class in org.apache.beam.runners.fnexecution.status
-
A Fn Status service which can collect run-time status information from SDK harnesses for debugging purpose.
- BundleCheckpointHandler - Interface in org.apache.beam.runners.fnexecution.control
-
A handler which is invoked when the SDK returns
BeamFnApi.DelayedBundleApplication
s as part of the bundle completion. - BundleCheckpointHandlers - Class in org.apache.beam.runners.fnexecution.control
-
Utility methods for creating
BundleCheckpointHandler
s. - BundleCheckpointHandlers() - Constructor for class org.apache.beam.runners.fnexecution.control.BundleCheckpointHandlers
- BundleCheckpointHandlers.StateAndTimerBundleCheckpointHandler<T> - Class in org.apache.beam.runners.fnexecution.control
-
A
BundleCheckpointHandler
which usesTimerInternals.TimerData
andValueState
to rescheduleBeamFnApi.DelayedBundleApplication
. - BundleFinalizationHandler - Interface in org.apache.beam.runners.fnexecution.control
-
A handler for the runner when a finalization request has been received.
- BundleFinalizationHandlers - Class in org.apache.beam.runners.fnexecution.control
-
Utility methods for creating
BundleFinalizationHandler
s. - BundleFinalizationHandlers() - Constructor for class org.apache.beam.runners.fnexecution.control.BundleFinalizationHandlers
- BundleFinalizationHandlers.InMemoryFinalizer - Class in org.apache.beam.runners.fnexecution.control
- BundleProgressHandler - Interface in org.apache.beam.runners.fnexecution.control
-
A handler for bundle progress messages, both during bundle execution and on its completion.
- BundleSplitHandler - Interface in org.apache.beam.runners.fnexecution.control
-
A handler which is invoked whenever an active bundle is split.
- ByteStringCoder - Class in org.apache.beam.runners.fnexecution.wire
-
A duplicate of
ByteStringCoder
that uses the Apache Beam vendored protobuf.
C
- clear(K, W) - Method in interface org.apache.beam.runners.fnexecution.state.StateRequestHandlers.BagUserStateHandler
-
Clears the bag user state for the given key and window.
- close() - Method in class org.apache.beam.runners.fnexecution.artifact.ArtifactRetrievalService
- close() - Method in class org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService
- close() - Method in class org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext
- close() - Method in class org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory
- close() - Method in class org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.WrappedSdkHarnessClient
- close() - Method in class org.apache.beam.runners.fnexecution.control.FnApiControlClient
- close() - Method in class org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService
- close() - Method in interface org.apache.beam.runners.fnexecution.control.RemoteBundle
-
Closes this bundle.
- close() - Method in class org.apache.beam.runners.fnexecution.control.SdkHarnessClient.BundleProcessor.ActiveBundle
-
Blocks until bundle processing is finished.
- close() - Method in class org.apache.beam.runners.fnexecution.control.SdkHarnessClient
- close() - Method in class org.apache.beam.runners.fnexecution.control.SingleEnvironmentInstanceJobBundleFactory
-
Deprecated.
- close() - Method in class org.apache.beam.runners.fnexecution.data.GrpcDataService
- close() - Method in class org.apache.beam.runners.fnexecution.environment.ProcessEnvironment
- close() - Method in interface org.apache.beam.runners.fnexecution.environment.RemoteEnvironment
-
.
- close() - Method in class org.apache.beam.runners.fnexecution.environment.StaticRemoteEnvironment
- close() - Method in class org.apache.beam.runners.fnexecution.logging.GrpcLoggingService
- close() - Method in class org.apache.beam.runners.fnexecution.provisioning.StaticGrpcProvisionService
- close() - Method in class org.apache.beam.runners.fnexecution.state.GrpcStateService
- close() - Method in class org.apache.beam.runners.fnexecution.status.BeamWorkerStatusGrpcService
- coder() - Method in class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors.TimerSpec
- consistentWithEquals() - Method in class org.apache.beam.runners.fnexecution.wire.ByteStringCoder
- control(StreamObserver<BeamFnApi.InstructionRequest>) - Method in class org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService
-
Called by gRPC for each incoming connection from an SDK harness, and enqueue an available SDK harness client.
- ControlClientPool - Interface in org.apache.beam.runners.fnexecution.control
-
A pool of control clients that brokers incoming SDK harness connections (in the form of
InstructionRequestHandlers
. - ControlClientPool.Sink - Interface in org.apache.beam.runners.fnexecution.control
-
A sink for
InstructionRequestHandlers
keyed by worker id. - ControlClientPool.Source - Interface in org.apache.beam.runners.fnexecution.control
-
A source of
InstructionRequestHandlers
. - create() - Static method in class org.apache.beam.runners.fnexecution.control.MapControlClientPool
-
Creates a
MapControlClientPool
. - create() - Static method in class org.apache.beam.runners.fnexecution.environment.ProcessManager
- create() - Static method in class org.apache.beam.runners.fnexecution.state.GrpcStateService
-
Create a new
GrpcStateService
. - create(String) - Method in interface org.apache.beam.runners.fnexecution.control.OutputReceiverFactory
-
Get a new
FnDataReceiver
for an output PCollection. - create(String, String) - Method in class org.apache.beam.runners.fnexecution.control.TimerReceiverFactory
- create(String, String, String, Struct) - Static method in class org.apache.beam.runners.fnexecution.provisioning.JobInfo
- create(String, ByteString, OutputStream) - Static method in class org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService.ArtifactDestination
- create(ProvisionApi.ProvisionInfo, HeaderAccessor) - Static method in class org.apache.beam.runners.fnexecution.provisioning.StaticGrpcProvisionService
- create(Endpoints.ApiServiceDescriptor, HeaderAccessor) - Static method in class org.apache.beam.runners.fnexecution.status.BeamWorkerStatusGrpcService
-
Create new instance of
BeamWorkerStatusGrpcService
. - create(ReferenceCountingExecutableStageContextFactory.Creator, SerializableFunction<Object, Boolean>) - Static method in class org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory
- create(EnvironmentFactory, GrpcFnServer<GrpcDataService>, GrpcFnServer<GrpcStateService>, IdGenerator) - Static method in class org.apache.beam.runners.fnexecution.control.SingleEnvironmentInstanceJobBundleFactory
-
Deprecated.
- create(ProcessManager, RunnerApi.Environment, String, InstructionRequestHandler) - Static method in class org.apache.beam.runners.fnexecution.environment.ProcessEnvironment
- create(ProcessManager, GrpcFnServer<StaticGrpcProvisionService>, ControlClientPool.Source, IdGenerator, PipelineOptions) - Static method in class org.apache.beam.runners.fnexecution.environment.ProcessEnvironmentFactory
- create(JobInfo) - Static method in class org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext
- create(JobInfo) - Static method in class org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory
- create(JobInfo, Map<String, EnvironmentFactory.Provider>) - Static method in class org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory
- create(GrpcFnServer<FnApiControlClientPoolService>, GrpcFnServer<GrpcLoggingService>, GrpcFnServer<ArtifactRetrievalService>, GrpcFnServer<StaticGrpcProvisionService>, ControlClientPool.Source, IdGenerator) - Static method in class org.apache.beam.runners.fnexecution.environment.ExternalEnvironmentFactory
- create(PipelineOptions, ExecutorService, OutboundObserverFactory) - Static method in class org.apache.beam.runners.fnexecution.data.GrpcDataService
- create(PipelineOptions, GrpcFnServer<GrpcLoggingService>, GrpcFnServer<FnApiControlClientPoolService>, ControlClientPool.Source) - Static method in class org.apache.beam.runners.fnexecution.environment.EmbeddedEnvironmentFactory
- createEnvironment(RunnerApi.Environment, String) - Method in class org.apache.beam.runners.fnexecution.environment.DockerEnvironmentFactory
-
Creates a new, active
RemoteEnvironment
backed by a local Docker container. - createEnvironment(RunnerApi.Environment, String) - Method in class org.apache.beam.runners.fnexecution.environment.EmbeddedEnvironmentFactory
- createEnvironment(RunnerApi.Environment, String) - Method in interface org.apache.beam.runners.fnexecution.environment.EnvironmentFactory
-
Creates an active
RunnerApi.Environment
and returns a handle to it. - createEnvironment(RunnerApi.Environment, String) - Method in class org.apache.beam.runners.fnexecution.environment.ExternalEnvironmentFactory
-
Creates a new, active
RemoteEnvironment
backed by an unmanaged worker. - createEnvironment(RunnerApi.Environment, String) - Method in class org.apache.beam.runners.fnexecution.environment.ProcessEnvironmentFactory
-
Creates a new, active
RemoteEnvironment
backed by a forked process. - createEnvironment(RunnerApi.Environment, String) - Method in class org.apache.beam.runners.fnexecution.environment.StaticRemoteEnvironmentFactory
- createEnvironmentFactory(GrpcFnServer<FnApiControlClientPoolService>, GrpcFnServer<GrpcLoggingService>, GrpcFnServer<ArtifactRetrievalService>, GrpcFnServer<StaticGrpcProvisionService>, ControlClientPool, IdGenerator) - Method in class org.apache.beam.runners.fnexecution.environment.DockerEnvironmentFactory.Provider
- createEnvironmentFactory(GrpcFnServer<FnApiControlClientPoolService>, GrpcFnServer<GrpcLoggingService>, GrpcFnServer<ArtifactRetrievalService>, GrpcFnServer<StaticGrpcProvisionService>, ControlClientPool, IdGenerator) - Method in class org.apache.beam.runners.fnexecution.environment.EmbeddedEnvironmentFactory.Provider
- createEnvironmentFactory(GrpcFnServer<FnApiControlClientPoolService>, GrpcFnServer<GrpcLoggingService>, GrpcFnServer<ArtifactRetrievalService>, GrpcFnServer<StaticGrpcProvisionService>, ControlClientPool, IdGenerator) - Method in interface org.apache.beam.runners.fnexecution.environment.EnvironmentFactory.Provider
-
Creates
EnvironmentFactory
for the provided GrpcServices. - createEnvironmentFactory(GrpcFnServer<FnApiControlClientPoolService>, GrpcFnServer<GrpcLoggingService>, GrpcFnServer<ArtifactRetrievalService>, GrpcFnServer<StaticGrpcProvisionService>, ControlClientPool, IdGenerator) - Method in class org.apache.beam.runners.fnexecution.environment.ExternalEnvironmentFactory.Provider
- createEnvironmentFactory(GrpcFnServer<FnApiControlClientPoolService>, GrpcFnServer<GrpcLoggingService>, GrpcFnServer<ArtifactRetrievalService>, GrpcFnServer<StaticGrpcProvisionService>, ControlClientPool, IdGenerator) - Method in class org.apache.beam.runners.fnexecution.environment.ProcessEnvironmentFactory.Provider
- createEnvironmentFactory(GrpcFnServer<FnApiControlClientPoolService>, GrpcFnServer<GrpcLoggingService>, GrpcFnServer<ArtifactRetrievalService>, GrpcFnServer<StaticGrpcProvisionService>, ControlClientPool, IdGenerator) - Method in class org.apache.beam.runners.fnexecution.environment.StaticRemoteEnvironmentFactory.Provider
- createOutputMap(Iterable<String>) - Static method in class org.apache.beam.runners.fnexecution.translation.PipelineTranslatorUtils
-
Creates a mapping from PCollection id to output tag integer.
D
- data(StreamObserver<BeamFnApi.Elements>) - Method in class org.apache.beam.runners.fnexecution.data.GrpcDataService
- decode(InputStream) - Method in class org.apache.beam.runners.fnexecution.wire.ByteStringCoder
- decode(InputStream, Coder.Context) - Method in class org.apache.beam.runners.fnexecution.wire.ByteStringCoder
- decodeTimerDataTimerId(String) - Static method in class org.apache.beam.runners.fnexecution.control.TimerReceiverFactory
-
Decodes a string into the transform and timer family ids.
- DEFAULT_BUFFER_SIZE - Static variable in class org.apache.beam.runners.fnexecution.artifact.ArtifactRetrievalService
- DefaultExecutableStageContext - Class in org.apache.beam.runners.fnexecution.control
-
Implementation of a
ExecutableStageContext
. - DefaultJobBundleFactory - Class in org.apache.beam.runners.fnexecution.control
-
A
JobBundleFactory
for which the implementation can specify a customEnvironmentFactory
for environment management. - DefaultJobBundleFactory.ServerInfo - Class in org.apache.beam.runners.fnexecution.control
-
A container for EnvironmentFactory and its corresponding Grpc servers.
- DefaultJobBundleFactory.WrappedSdkHarnessClient - Class in org.apache.beam.runners.fnexecution.control
-
Holder for an
SdkHarnessClient
along with its associated state and data servers. - delegateBasedUponType(EnumMap<BeamFnApi.StateKey.TypeCase, StateRequestHandler>) - Static method in class org.apache.beam.runners.fnexecution.state.StateRequestHandlers
-
Returns a
StateRequestHandler
which delegates to the supplied handler depending on theBeamFnApi.StateRequest
stype
. - deregister() - Method in interface org.apache.beam.runners.fnexecution.state.StateDelegator.Registration
-
De-registers the handler for all future requests for state for the registered process bundle instruction id.
- DockerEnvironmentFactory - Class in org.apache.beam.runners.fnexecution.environment
-
An
EnvironmentFactory
that creates docker containers by shelling out to docker. - DockerEnvironmentFactory.Provider - Class in org.apache.beam.runners.fnexecution.environment
-
Provider for DockerEnvironmentFactory.
E
- elementCoder() - Method in class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors.SideInputSpec
- elementCoder() - Method in interface org.apache.beam.runners.fnexecution.state.StateRequestHandlers.IterableSideInputHandler
-
Returns the
Coder
to use for the elements of the resulting values iterable. - EMBEDDED_ARTIFACT_URN - Static variable in class org.apache.beam.runners.fnexecution.artifact.ArtifactRetrievalService
- EmbeddedEnvironmentFactory - Class in org.apache.beam.runners.fnexecution.environment
-
An
EnvironmentFactory
that communicates to aFnHarness
which is executing in the same process. - EmbeddedEnvironmentFactory.Provider - Class in org.apache.beam.runners.fnexecution.environment
-
Provider of EmbeddedEnvironmentFactory.
- encode(ByteString, OutputStream) - Method in class org.apache.beam.runners.fnexecution.wire.ByteStringCoder
- encode(ByteString, OutputStream, Coder.Context) - Method in class org.apache.beam.runners.fnexecution.wire.ByteStringCoder
- encodeToTimerDataTimerId(String, String) - Static method in class org.apache.beam.runners.fnexecution.control.TimerReceiverFactory
-
Encodes transform and timer family ids into a single string which retains the human readable format
len(transformId):transformId:timerId
. - EnvironmentFactory - Interface in org.apache.beam.runners.fnexecution.environment
-
Creates
environments
which communicate to anSdkHarnessClient
. - EnvironmentFactory.Provider - Interface in org.apache.beam.runners.fnexecution.environment
-
Provider for a
EnvironmentFactory
andServerFactory
for the environment. - ExecutableProcessBundleDescriptor() - Constructor for class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors.ExecutableProcessBundleDescriptor
- ExecutableStageContext - Interface in org.apache.beam.runners.fnexecution.control
-
The context required in order to execute
stages
. - ExecutableStageContext.Factory - Interface in org.apache.beam.runners.fnexecution.control
-
Creates
ExecutableStageContext
instances. - ExternalEnvironmentFactory - Class in org.apache.beam.runners.fnexecution.environment
-
An
EnvironmentFactory
which requests workers via the given URL in the Environment. - ExternalEnvironmentFactory.Provider - Class in org.apache.beam.runners.fnexecution.environment
-
Provider of ExternalEnvironmentFactory.
F
- FILE_ARTIFACT_URN - Static variable in class org.apache.beam.runners.fnexecution.artifact.ArtifactRetrievalService
- finalizeAllOutstandingBundles() - Method in class org.apache.beam.runners.fnexecution.control.BundleFinalizationHandlers.InMemoryFinalizer
-
All finalization requests will be sent without waiting for the responses.
- fireEligibleTimers(InMemoryTimerInternals, Map<KV<String, String>, FnDataReceiver<Timer>>, Object) - Static method in class org.apache.beam.runners.fnexecution.translation.PipelineTranslatorUtils
-
Fires all timers which are ready to be fired.
- FnApiControlClient - Class in org.apache.beam.runners.fnexecution.control
-
A client for the control plane of an SDK harness, which can issue requests to it over the Fn API.
- FnApiControlClientPoolService - Class in org.apache.beam.runners.fnexecution.control
-
A Fn API control service which adds incoming SDK harness connections to a sink.
- FnDataService - Interface in org.apache.beam.runners.fnexecution.data
-
The
FnDataService
is able to forward inbound elements to a consumer and is also a consumer of outbound elements. - forBagUserStateHandlerFactory(ProcessBundleDescriptors.ExecutableProcessBundleDescriptor, StateRequestHandlers.BagUserStateHandlerFactory) - Static method in class org.apache.beam.runners.fnexecution.state.StateRequestHandlers
-
Returns an adapter which converts a
StateRequestHandlers.BagUserStateHandlerFactory
to aStateRequestHandler
. - forHandler(RunnerApi.Environment, InstructionRequestHandler) - Static method in interface org.apache.beam.runners.fnexecution.environment.RemoteEnvironment
-
Create a new
RemoteEnvironment
for the providedRunnerApi.Environment
andAutoCloseable
InstructionRequestHandler
. - forIterableSideInput(String, String, Coder<V>, Coder<W>) - Method in interface org.apache.beam.runners.fnexecution.state.StateRequestHandlers.SideInputHandlerFactory
- forIterableSideInput(String, String, Coder<V>, Coder<W>) - Method in class org.apache.beam.runners.fnexecution.translation.BatchSideInputHandlerFactory
- forIterableSideInput(String, String, Coder<V>, Coder<W>) - Method in class org.apache.beam.runners.fnexecution.translation.StreamingSideInputHandlerFactory
- forMultimapSideInput(String, String, KvCoder<K, V>, Coder<W>) - Method in interface org.apache.beam.runners.fnexecution.state.StateRequestHandlers.SideInputHandlerFactory
- forMultimapSideInput(String, String, KvCoder<K, V>, Coder<W>) - Method in class org.apache.beam.runners.fnexecution.translation.BatchSideInputHandlerFactory
- forMultimapSideInput(String, String, KvCoder<K, V>, Coder<W>) - Method in class org.apache.beam.runners.fnexecution.translation.StreamingSideInputHandlerFactory
- forRequestObserver(String, StreamObserver<BeamFnApi.InstructionRequest>, ConcurrentMap<String, BeamFnApi.ProcessBundleDescriptor>) - Static method in class org.apache.beam.runners.fnexecution.control.FnApiControlClient
-
Returns a
FnApiControlClient
which will submit its requests to the provided observer. - forService(InstructionRequestHandler) - Static method in class org.apache.beam.runners.fnexecution.environment.StaticRemoteEnvironmentFactory
- forSideInputHandlerFactory(Map<String, Map<String, ProcessBundleDescriptors.SideInputSpec>>, StateRequestHandlers.SideInputHandlerFactory) - Static method in class org.apache.beam.runners.fnexecution.state.StateRequestHandlers
-
Returns an adapter which converts a
StateRequestHandlers.SideInputHandlerFactory
to aStateRequestHandler
. - forStage(ExecutableStage) - Method in class org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory
- forStage(ExecutableStage) - Method in interface org.apache.beam.runners.fnexecution.control.JobBundleFactory
- forStage(ExecutableStage) - Method in class org.apache.beam.runners.fnexecution.control.SingleEnvironmentInstanceJobBundleFactory
-
Deprecated.
- forStage(ExecutableStage, Map<RunnerApi.ExecutableStagePayload.SideInputId, PCollectionView<?>>, SideInputHandler) - Static method in class org.apache.beam.runners.fnexecution.translation.StreamingSideInputHandlerFactory
-
Creates a new state handler for the given stage.
- forStage(ExecutableStage, BatchSideInputHandlerFactory.SideInputGetter) - Static method in class org.apache.beam.runners.fnexecution.translation.BatchSideInputHandlerFactory
-
Creates a new state handler for the given stage.
- forUserState(String, String, Coder<K>, Coder<V>, Coder<W>) - Method in class org.apache.beam.runners.fnexecution.state.InMemoryBagUserStateFactory
- forUserState(String, String, Coder<K>, Coder<V>, Coder<W>) - Method in interface org.apache.beam.runners.fnexecution.state.StateRequestHandlers.BagUserStateHandlerFactory
- forWriter(LogWriter) - Static method in class org.apache.beam.runners.fnexecution.logging.GrpcLoggingService
- fromExecutableStage(String, ExecutableStage, Endpoints.ApiServiceDescriptor) - Static method in class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors
- fromExecutableStage(String, ExecutableStage, Endpoints.ApiServiceDescriptor, Endpoints.ApiServiceDescriptor) - Static method in class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors
-
Note that the
BeamFnApi.ProcessBundleDescriptor
is constructed by: Adding gRPC read and write nodes wiring them to the specified data endpoint. - fromFile(String) - Static method in class org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService.ArtifactDestination
- fromFile(String, OutputStream) - Static method in class org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService.ArtifactDestination
G
- get(K, W) - Method in interface org.apache.beam.runners.fnexecution.state.StateRequestHandlers.BagUserStateHandler
-
Returns an
Iterable
of values representing the bag user state for the given key and window. - get(K, W) - Method in interface org.apache.beam.runners.fnexecution.state.StateRequestHandlers.MultimapSideInputHandler
-
Returns an
Iterable
of values representing the side input for the given key and window. - get(JobInfo) - Method in interface org.apache.beam.runners.fnexecution.control.ExecutableStageContext.Factory
-
Get or create
ExecutableStageContext
for givenJobInfo
. - get(JobInfo) - Method in class org.apache.beam.runners.fnexecution.control.ReferenceCountingExecutableStageContextFactory
- get(W) - Method in interface org.apache.beam.runners.fnexecution.state.StateRequestHandlers.IterableSideInputHandler
-
Returns an
Iterable
of values representing the side input for the given window. - get(W) - Method in interface org.apache.beam.runners.fnexecution.state.StateRequestHandlers.MultimapSideInputHandler
-
Returns an
Iterable
of keys representing the side input for the given window. - getAllWorkerStatuses(long, TimeUnit) - Method in class org.apache.beam.runners.fnexecution.status.BeamWorkerStatusGrpcService
-
Get all the statuses from all connected SDK harnesses within specified timeout.
- getArtifact(ArtifactApi.GetArtifactRequest, StreamObserver<ArtifactApi.GetArtifactResponse>) - Method in class org.apache.beam.runners.fnexecution.artifact.ArtifactRetrievalService
- getArtifact(RunnerApi.ArtifactInformation) - Static method in class org.apache.beam.runners.fnexecution.artifact.ArtifactRetrievalService
- getBagUserStateSpecs() - Method in class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors.ExecutableProcessBundleDescriptor
-
Get a mapping from PTransform id to user state input id to
bag user states
that are used during execution. - getBundle(OutputReceiverFactory, TimerReceiverFactory, StateRequestHandler, BundleProgressHandler) - Method in interface org.apache.beam.runners.fnexecution.control.StageBundleFactory
-
Get a new
bundle
for processing the data in an executable stage. - getBundle(OutputReceiverFactory, TimerReceiverFactory, StateRequestHandler, BundleProgressHandler, BundleFinalizationHandler) - Method in interface org.apache.beam.runners.fnexecution.control.StageBundleFactory
- getBundle(OutputReceiverFactory, TimerReceiverFactory, StateRequestHandler, BundleProgressHandler, BundleFinalizationHandler, BundleCheckpointHandler) - Method in interface org.apache.beam.runners.fnexecution.control.StageBundleFactory
- getBundle(OutputReceiverFactory, StateRequestHandler, BundleProgressHandler) - Method in interface org.apache.beam.runners.fnexecution.control.StageBundleFactory
-
Get a new
bundle
for processing the data in an executable stage. - getBundle(OutputReceiverFactory, StateRequestHandler, BundleProgressHandler, BundleFinalizationHandler) - Method in interface org.apache.beam.runners.fnexecution.control.StageBundleFactory
- getBundle(OutputReceiverFactory, StateRequestHandler, BundleProgressHandler, BundleFinalizationHandler, BundleCheckpointHandler) - Method in interface org.apache.beam.runners.fnexecution.control.StageBundleFactory
- getCacheTokens() - Method in interface org.apache.beam.runners.fnexecution.state.StateRequestHandler
-
Retrieves a list of valid cache tokens.
- getCoder() - Method in class org.apache.beam.runners.fnexecution.control.RemoteOutputReceiver
- getCoder() - Method in class org.apache.beam.runners.fnexecution.data.RemoteInputDestination
- getDefault() - Static method in class org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter
- getDestination(String, String) - Method in interface org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService.ArtifactDestinationProvider
- getEncodedElementByteSize(ByteString) - Method in class org.apache.beam.runners.fnexecution.wire.ByteStringCoder
- getEncodedTypeDescriptor() - Method in class org.apache.beam.runners.fnexecution.wire.ByteStringCoder
- getEnvironment() - Method in class org.apache.beam.runners.fnexecution.environment.ProcessEnvironment
- getEnvironment() - Method in interface org.apache.beam.runners.fnexecution.environment.RemoteEnvironment
-
Return the environment that the remote handles.
- getEnvironment() - Method in class org.apache.beam.runners.fnexecution.environment.RemoteEnvironment.SimpleRemoteEnvironment
- getEnvironment() - Method in class org.apache.beam.runners.fnexecution.environment.StaticRemoteEnvironment
- getExecutableStageIntermediateId(PipelineNode.PTransformNode) - Static method in class org.apache.beam.runners.fnexecution.translation.PipelineTranslatorUtils
- getId() - Method in interface org.apache.beam.runners.fnexecution.control.RemoteBundle
-
Get an id used to represent this bundle.
- getId() - Method in class org.apache.beam.runners.fnexecution.control.SdkHarnessClient.BundleProcessor.ActiveBundle
-
Returns an id used to represent this bundle.
- getInputId(PipelineNode.PTransformNode) - Static method in class org.apache.beam.runners.fnexecution.translation.PipelineTranslatorUtils
- getInputReceivers() - Method in interface org.apache.beam.runners.fnexecution.control.RemoteBundle
-
Get a map of PCollection ids to
receiver
s which consume input elements, forwarding them to the remote environment. - getInputReceivers() - Method in class org.apache.beam.runners.fnexecution.control.SdkHarnessClient.BundleProcessor.ActiveBundle
-
Get a map of PCollection ids to
receiver
s which consume input elements, forwarding them to the remote environment. - getInstructionRequestHandler() - Method in class org.apache.beam.runners.fnexecution.control.SdkHarnessClient
- getInstructionRequestHandler() - Method in interface org.apache.beam.runners.fnexecution.control.StageBundleFactory
- getInstructionRequestHandler() - Method in class org.apache.beam.runners.fnexecution.environment.ProcessEnvironment
- getInstructionRequestHandler() - Method in interface org.apache.beam.runners.fnexecution.environment.RemoteEnvironment
-
Return an
InstructionRequestHandler
which can communicate with the environment. - getInstructionRequestHandler() - Method in class org.apache.beam.runners.fnexecution.environment.RemoteEnvironment.SimpleRemoteEnvironment
- getInstructionRequestHandler() - Method in class org.apache.beam.runners.fnexecution.environment.StaticRemoteEnvironment
- getOutputId(PipelineNode.PTransformNode) - Static method in class org.apache.beam.runners.fnexecution.translation.PipelineTranslatorUtils
- getOutputStream() - Method in class org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService.ArtifactDestination
- getProcessBundleDescriptor() - Method in class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors.ExecutableProcessBundleDescriptor
- getProcessBundleDescriptor() - Method in interface org.apache.beam.runners.fnexecution.control.StageBundleFactory
- getProcessBundleDescriptor(String) - Method in class org.apache.beam.runners.fnexecution.control.FnApiControlClient
- getProcessBundleDescriptor(BeamFnApi.GetProcessBundleDescriptorRequest, StreamObserver<BeamFnApi.ProcessBundleDescriptor>) - Method in class org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService
- getProcessor(BeamFnApi.ProcessBundleDescriptor, List<RemoteInputDestination>) - Method in class org.apache.beam.runners.fnexecution.control.SdkHarnessClient
-
Provides
SdkHarnessClient.BundleProcessor
that is capable of processing bundles not containing timers or state accesses such as: Side inputs User state Remote references - getProcessor(BeamFnApi.ProcessBundleDescriptor, List<RemoteInputDestination>, StateDelegator) - Method in class org.apache.beam.runners.fnexecution.control.SdkHarnessClient
-
Provides
SdkHarnessClient.BundleProcessor
that is capable of processing bundles not containing timers. - getProcessor(BeamFnApi.ProcessBundleDescriptor, List<RemoteInputDestination>, StateDelegator, Map<String, Map<String, ProcessBundleDescriptors.TimerSpec>>) - Method in class org.apache.beam.runners.fnexecution.control.SdkHarnessClient
-
Provides
SdkHarnessClient.BundleProcessor
that is capable of processing bundles containing timers and state accesses such as: Side inputs User state Remote references - getProvisionInfo(ProvisionApi.GetProvisionInfoRequest, StreamObserver<ProvisionApi.GetProvisionInfoResponse>) - Method in class org.apache.beam.runners.fnexecution.provisioning.StaticGrpcProvisionService
- getPTransformId() - Method in class org.apache.beam.runners.fnexecution.data.RemoteInputDestination
- getReceiver() - Method in class org.apache.beam.runners.fnexecution.control.RemoteOutputReceiver
- getRemoteInputDestinations() - Method in class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors.ExecutableProcessBundleDescriptor
-
Get
RemoteInputDestination
s that input data are sent to theBeamFnApi.ProcessBundleDescriptor
over. - getRemoteOutputCoders() - Method in class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors.ExecutableProcessBundleDescriptor
-
Get all of the transforms materialized by this
ProcessBundleDescriptors.ExecutableProcessBundleDescriptor
and the JavaCoder
for the wire format of that transform. - getServerFactory() - Method in class org.apache.beam.runners.fnexecution.environment.DockerEnvironmentFactory.Provider
- getServerFactory() - Method in class org.apache.beam.runners.fnexecution.environment.EmbeddedEnvironmentFactory.Provider
- getServerFactory() - Method in interface org.apache.beam.runners.fnexecution.environment.EnvironmentFactory.Provider
-
Create the
ServerFactory
applicable to this environment. - getServerFactory() - Method in class org.apache.beam.runners.fnexecution.environment.ExternalEnvironmentFactory.Provider
- getSideInput(String) - Method in interface org.apache.beam.runners.fnexecution.translation.BatchSideInputHandlerFactory.SideInputGetter
- getSideInputs(ExecutableStage) - Static method in class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors
- getSideInputSpecs() - Method in class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors.ExecutableProcessBundleDescriptor
-
Get a mapping from PTransform id to side input id to
side inputs
that are used during execution. - getSingleWorkerStatus(String, long, TimeUnit) - Method in class org.apache.beam.runners.fnexecution.status.BeamWorkerStatusGrpcService
-
Get the latest SDK worker status from the client's corresponding SDK harness.
- getSink() - Method in interface org.apache.beam.runners.fnexecution.control.ControlClientPool
-
Sink for control clients.
- getSink() - Method in class org.apache.beam.runners.fnexecution.control.MapControlClientPool
- getSource() - Method in interface org.apache.beam.runners.fnexecution.control.ControlClientPool
-
Source of control clients.
- getSource() - Method in class org.apache.beam.runners.fnexecution.control.MapControlClientPool
- getStageBundleFactory(ExecutableStage) - Method in class org.apache.beam.runners.fnexecution.control.DefaultExecutableStageContext
- getStageBundleFactory(ExecutableStage) - Method in interface org.apache.beam.runners.fnexecution.control.ExecutableStageContext
- getStagedArtifacts(String) - Method in class org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService
-
Returns the rewritten artifacts associated with this job, keyed by environment.
- getTimerReceivers() - Method in interface org.apache.beam.runners.fnexecution.control.RemoteBundle
-
Get a map of (transform id, timer id) to
receiver
s which consume timers, forwarding them to the remote environment. - getTimerReceivers() - Method in class org.apache.beam.runners.fnexecution.control.SdkHarnessClient.BundleProcessor.ActiveBundle
- getTimerSpec() - Method in class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors.TimerSpec
- getTimerSpecs() - Method in class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors.ExecutableProcessBundleDescriptor
-
Get a mapping from PTransform id to timer id to
timer specs
that are used during execution. - getTypePayload() - Method in class org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService.ArtifactDestination
- getTypeUrn() - Method in class org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService.ArtifactDestination
- getWindowedValueCoder(String, RunnerApi.Components) - Static method in class org.apache.beam.runners.fnexecution.translation.PipelineTranslatorUtils
- getWindowingStrategy(String, RunnerApi.Components) - Static method in class org.apache.beam.runners.fnexecution.translation.PipelineTranslatorUtils
- getWorkerId() - Method in class org.apache.beam.runners.fnexecution.control.FnApiControlClient
- GrpcDataService - Class in org.apache.beam.runners.fnexecution.data
-
A
FnDataService
implemented via gRPC. - GrpcDataService() - Constructor for class org.apache.beam.runners.fnexecution.data.GrpcDataService
-
Deprecated.This constructor is for migrating Dataflow purpose only.
- GrpcLoggingService - Class in org.apache.beam.runners.fnexecution.logging
-
An implementation of the Beam Fn Logging Service over gRPC.
- GrpcStateService - Class in org.apache.beam.runners.fnexecution.state
-
An implementation of the Beam Fn State service.
H
- handle(BeamFnApi.InstructionRequest) - Method in class org.apache.beam.runners.fnexecution.control.FnApiControlClient
- handle(BeamFnApi.InstructionRequest) - Method in interface org.apache.beam.runners.fnexecution.control.InstructionRequestHandler
- handle(BeamFnApi.StateRequest) - Method in interface org.apache.beam.runners.fnexecution.state.StateRequestHandler
-
Handle a
BeamFnApi.StateRequest
asynchronously. - hasUnboundedPCollections(RunnerApi.Pipeline) - Static method in class org.apache.beam.runners.fnexecution.translation.PipelineTranslatorUtils
-
Indicates whether the given pipeline has any unbounded PCollections.
I
- ignored() - Static method in interface org.apache.beam.runners.fnexecution.control.BundleProgressHandler
-
Returns a handler that ignores metrics.
- INHERIT_IO_FILE - Static variable in class org.apache.beam.runners.fnexecution.environment.ProcessManager
-
A symbolic file to indicate that we want to inherit I/O of parent process.
- InMemoryBagUserStateFactory<K,V,W extends org.apache.beam.sdk.transforms.windowing.BoundedWindow> - Class in org.apache.beam.runners.fnexecution.state
-
Holds user state in memory.
- InMemoryBagUserStateFactory() - Constructor for class org.apache.beam.runners.fnexecution.state.InMemoryBagUserStateFactory
- inMemoryFinalizer(InstructionRequestHandler) - Static method in class org.apache.beam.runners.fnexecution.control.BundleFinalizationHandlers
-
A bundle finalizer that stores all bundle finalization requests in memory.
- instantiateCoder(String, RunnerApi.Components) - Static method in class org.apache.beam.runners.fnexecution.translation.PipelineTranslatorUtils
-
Creates a coder for a given PCollection id from the Proto definition.
- instantiateRunnerWireCoder(PipelineNode.PCollectionNode, RunnerApi.Components) - Static method in class org.apache.beam.runners.fnexecution.wire.WireCoders
-
Instantiates a runner-side wire coder for the given PCollection.
- instantiateRunnerWireCoder(PipelineNode.PCollectionNode, RunnerApi.Components, RunnerApi.ExecutableStagePayload.WireCoderSetting) - Static method in class org.apache.beam.runners.fnexecution.wire.WireCoders
-
Instantiates a runner-side wire coder for the given PCollection.
- InstructionRequestHandler - Interface in org.apache.beam.runners.fnexecution.control
-
Interface for any function that can handle a Fn API
BeamFnApi.InstructionRequest
. - isAliveOrThrow() - Method in class org.apache.beam.runners.fnexecution.environment.ProcessManager.RunningProcess
-
Checks if the underlying process is still running.
- isRegisterByteSizeObserverCheap(ByteString) - Method in class org.apache.beam.runners.fnexecution.wire.ByteStringCoder
- isSdfTimer(String) - Static method in class org.apache.beam.runners.fnexecution.control.BundleCheckpointHandlers.StateAndTimerBundleCheckpointHandler
-
A helper function to help check whether the given timer is the timer which is set for rescheduling
BeamFnApi.DelayedBundleApplication
.
J
- JobBundleFactory - Interface in org.apache.beam.runners.fnexecution.control
-
A factory that has all job-scoped information, and can be combined with stage-scoped information to create a
StageBundleFactory
. - jobId() - Method in class org.apache.beam.runners.fnexecution.provisioning.JobInfo
- JobInfo - Class in org.apache.beam.runners.fnexecution.provisioning
-
A subset of
ProvisionApi.ProvisionInfo
that specifies a unique job, while omitting fields that are not known to the runner operator. - JobInfo() - Constructor for class org.apache.beam.runners.fnexecution.provisioning.JobInfo
- jobName() - Method in class org.apache.beam.runners.fnexecution.provisioning.JobInfo
K
- keyCoder() - Method in class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors.BagUserStateSpec
- keyCoder() - Method in interface org.apache.beam.runners.fnexecution.state.StateRequestHandlers.MultimapSideInputHandler
-
Returns the
Coder
to use for the elements of the resulting keys iterable.
L
- LengthPrefixUnknownCoders - Class in org.apache.beam.runners.fnexecution.wire
-
Utilities for replacing or wrapping unknown coders with
LengthPrefixCoder
. - LengthPrefixUnknownCoders() - Constructor for class org.apache.beam.runners.fnexecution.wire.LengthPrefixUnknownCoders
- log(BeamFnApi.LogEntry) - Method in interface org.apache.beam.runners.fnexecution.logging.LogWriter
-
Write the contents of the Log Entry to some logging backend.
- log(BeamFnApi.LogEntry) - Method in class org.apache.beam.runners.fnexecution.logging.Slf4jLogWriter
- logging(StreamObserver<BeamFnApi.LogControl>) - Method in class org.apache.beam.runners.fnexecution.logging.GrpcLoggingService
- LogWriter - Interface in org.apache.beam.runners.fnexecution.logging
-
A consumer of
Beam Log Entries
.
M
- MapControlClientPool - Class in org.apache.beam.runners.fnexecution.control
-
A
ControlClientPool
backed by a client map.
N
- NeedsDocker - Interface in org.apache.beam.runners.fnexecution.environment.testing
-
Category for integration tests that require Docker.
- newBundle(Map<String, RemoteOutputReceiver<?>>, Map<KV<String, String>, RemoteOutputReceiver<Timer<?>>>, StateRequestHandler, BundleProgressHandler, BundleFinalizationHandler, BundleCheckpointHandler) - Method in class org.apache.beam.runners.fnexecution.control.SdkHarnessClient.BundleProcessor
-
Start a new bundle for the given
BeamFnApi.ProcessBundleDescriptor
identifier. - newBundle(Map<String, RemoteOutputReceiver<?>>, Map<KV<String, String>, RemoteOutputReceiver<Timer<?>>>, StateRequestHandler, BundleProgressHandler, BundleSplitHandler, BundleCheckpointHandler, BundleFinalizationHandler) - Method in class org.apache.beam.runners.fnexecution.control.SdkHarnessClient.BundleProcessor
-
Start a new bundle for the given
BeamFnApi.ProcessBundleDescriptor
identifier. - newBundle(Map<String, RemoteOutputReceiver<?>>, BundleProgressHandler) - Method in class org.apache.beam.runners.fnexecution.control.SdkHarnessClient.BundleProcessor
-
Start a new bundle for the given
BeamFnApi.ProcessBundleDescriptor
identifier. - newBundle(Map<String, RemoteOutputReceiver<?>>, StateRequestHandler, BundleProgressHandler) - Method in class org.apache.beam.runners.fnexecution.control.SdkHarnessClient.BundleProcessor
-
Start a new bundle for the given
BeamFnApi.ProcessBundleDescriptor
identifier.
O
- of() - Static method in class org.apache.beam.runners.fnexecution.wire.ByteStringCoder
- of(String, String, RunnerApi.FunctionSpec, Coder<T>, Coder<W>) - Static method in class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors.SideInputSpec
- of(BeamFnApi.ProcessBundleDescriptor, List<RemoteInputDestination>, Map<String, Coder>, Map<String, Map<String, ProcessBundleDescriptors.SideInputSpec>>, Map<String, Map<String, ProcessBundleDescriptors.BagUserStateSpec>>, Map<String, Map<String, ProcessBundleDescriptors.TimerSpec>>) - Static method in class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors.ExecutableProcessBundleDescriptor
- of(Coder<T>, String) - Static method in class org.apache.beam.runners.fnexecution.data.RemoteInputDestination
- of(Coder<T>, FnDataReceiver<T>) - Static method in class org.apache.beam.runners.fnexecution.control.RemoteOutputReceiver
- offer(ArtifactRetrievalService, ArtifactStagingServiceGrpc.ArtifactStagingServiceStub, String) - Static method in class org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService
-
Lazily stages artifacts by letting an ArtifactStagingService resolve and request artifacts.
- offeringClientsToPool(ControlClientPool.Sink, HeaderAccessor) - Static method in class org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService
-
Creates a new
FnApiControlClientPoolService
which will enqueue and vend new SDK harness connections. - onCheckpoint(BeamFnApi.ProcessBundleResponse) - Method in interface org.apache.beam.runners.fnexecution.control.BundleCheckpointHandler
- onCheckpoint(BeamFnApi.ProcessBundleResponse) - Method in class org.apache.beam.runners.fnexecution.control.BundleCheckpointHandlers.StateAndTimerBundleCheckpointHandler
- onClose(Consumer<FnApiControlClient>) - Method in class org.apache.beam.runners.fnexecution.control.FnApiControlClient
- onCompleted(BeamFnApi.ProcessBundleResponse) - Method in interface org.apache.beam.runners.fnexecution.control.BundleProgressHandler
-
Handles the bundle's completion report.
- onProgress(BeamFnApi.ProcessBundleProgressResponse) - Method in interface org.apache.beam.runners.fnexecution.control.BundleProgressHandler
-
Handles a progress report from the bundle while it is executing.
- org.apache.beam.runners.fnexecution.artifact - package org.apache.beam.runners.fnexecution.artifact
-
Pipeline execution-time artifact-management services, including abstract implementations of the Artifact Retrieval Service.
- org.apache.beam.runners.fnexecution.control - package org.apache.beam.runners.fnexecution.control
-
Utilities for a Beam runner to interact with the Fn API
Control Service
via java abstractions. - org.apache.beam.runners.fnexecution.data - package org.apache.beam.runners.fnexecution.data
-
Utilities for a Beam runner to interact with the Fn API
Data Service
via java abstractions. - org.apache.beam.runners.fnexecution.environment - package org.apache.beam.runners.fnexecution.environment
-
Classes used to instantiate and manage SDK harness environments.
- org.apache.beam.runners.fnexecution.environment.testing - package org.apache.beam.runners.fnexecution.environment.testing
-
Test utilities for the environment management package.
- org.apache.beam.runners.fnexecution.logging - package org.apache.beam.runners.fnexecution.logging
-
Classes used to log informational messages over the
Beam Fn Logging Service
. - org.apache.beam.runners.fnexecution.provisioning - package org.apache.beam.runners.fnexecution.provisioning
-
Provision api services.
- org.apache.beam.runners.fnexecution.state - package org.apache.beam.runners.fnexecution.state
-
State API services.
- org.apache.beam.runners.fnexecution.status - package org.apache.beam.runners.fnexecution.status
-
Worker Status API services.
- org.apache.beam.runners.fnexecution.translation - package org.apache.beam.runners.fnexecution.translation
-
Shared utilities for a Beam runner to translate portable pipelines.
- org.apache.beam.runners.fnexecution.wire - package org.apache.beam.runners.fnexecution.wire
-
Wire coders for communications between runner and SDK harness.
- OutputReceiverFactory - Interface in org.apache.beam.runners.fnexecution.control
-
A factory that can create output receivers during an executable stage.
P
- pipelineOptions() - Method in class org.apache.beam.runners.fnexecution.provisioning.JobInfo
- PipelineTranslatorUtils - Class in org.apache.beam.runners.fnexecution.translation
-
Utilities for pipeline translation.
- ProcessBundleDescriptors - Class in org.apache.beam.runners.fnexecution.control
-
Utility methods for creating
BeamFnApi.ProcessBundleDescriptor
instances. - ProcessBundleDescriptors() - Constructor for class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors
- ProcessBundleDescriptors.BagUserStateSpec<K,V,W extends org.apache.beam.sdk.transforms.windowing.BoundedWindow> - Class in org.apache.beam.runners.fnexecution.control
-
A container type storing references to the key, value, and window
Coder
used when handling bag user state requests. - ProcessBundleDescriptors.ExecutableProcessBundleDescriptor - Class in org.apache.beam.runners.fnexecution.control
- ProcessBundleDescriptors.SideInputSpec<T,W extends org.apache.beam.sdk.transforms.windowing.BoundedWindow> - Class in org.apache.beam.runners.fnexecution.control
-
A container type storing references to the value, and window
Coder
used when handling side input state requests. - ProcessBundleDescriptors.TimerSpec<K,V,W extends org.apache.beam.sdk.transforms.windowing.BoundedWindow> - Class in org.apache.beam.runners.fnexecution.control
-
A container type storing references to the key, timer and payload coders and the remote input destination used when handling timer requests.
- ProcessEnvironment - Class in org.apache.beam.runners.fnexecution.environment
-
Environment for process-based execution.
- ProcessEnvironmentFactory - Class in org.apache.beam.runners.fnexecution.environment
-
An
EnvironmentFactory
which forks processes based on the parameters in the Environment. - ProcessEnvironmentFactory.Provider - Class in org.apache.beam.runners.fnexecution.environment
-
Provider of ProcessEnvironmentFactory.
- ProcessManager - Class in org.apache.beam.runners.fnexecution.environment
-
A simple process manager which forks processes and kills them if necessary.
- ProcessManager.RunningProcess - Class in org.apache.beam.runners.fnexecution.environment
- Provider() - Constructor for class org.apache.beam.runners.fnexecution.environment.ExternalEnvironmentFactory.Provider
- Provider(InstructionRequestHandler) - Constructor for class org.apache.beam.runners.fnexecution.environment.StaticRemoteEnvironmentFactory.Provider
- Provider(PipelineOptions) - Constructor for class org.apache.beam.runners.fnexecution.environment.DockerEnvironmentFactory.Provider
- Provider(PipelineOptions) - Constructor for class org.apache.beam.runners.fnexecution.environment.EmbeddedEnvironmentFactory.Provider
- Provider(PipelineOptions) - Constructor for class org.apache.beam.runners.fnexecution.environment.ProcessEnvironmentFactory.Provider
- put(String, InstructionRequestHandler) - Method in interface org.apache.beam.runners.fnexecution.control.ControlClientPool.Sink
-
Puts an
InstructionRequestHandler
into a client pool.
R
- receive(LogicalEndpoint, Coder<T>, FnDataReceiver<T>) - Method in interface org.apache.beam.runners.fnexecution.data.FnDataService
-
Registers a receiver to be notified upon any incoming elements.
- receive(LogicalEndpoint, Coder<T>, FnDataReceiver<T>) - Method in class org.apache.beam.runners.fnexecution.data.GrpcDataService
- ReferenceCountingExecutableStageContextFactory - Class in org.apache.beam.runners.fnexecution.control
-
ExecutableStageContext.Factory
which counts ExecutableStageContext reference for book keeping. - ReferenceCountingExecutableStageContextFactory.Creator - Interface in org.apache.beam.runners.fnexecution.control
-
Interface for creator which extends Serializable.
- registerEnvironment(String, RunnerApi.Environment) - Method in class org.apache.beam.runners.fnexecution.provisioning.StaticGrpcProvisionService
- registerForProcessBundleInstructionId(String, StateRequestHandler) - Method in class org.apache.beam.runners.fnexecution.state.GrpcStateService
- registerForProcessBundleInstructionId(String, StateRequestHandler) - Method in interface org.apache.beam.runners.fnexecution.state.StateDelegator
-
Registers the supplied handler for the given process bundle instruction id for all
BeamFnApi.StateRequest
s with a matching id. - registerJob(String, Map<String, List<RunnerApi.ArtifactInformation>>) - Method in class org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService
-
Registers a set of artifacts to be staged with this service.
- registerProcessBundleDescriptor(BeamFnApi.ProcessBundleDescriptor) - Method in class org.apache.beam.runners.fnexecution.control.FnApiControlClient
- registerProcessBundleDescriptor(BeamFnApi.ProcessBundleDescriptor) - Method in interface org.apache.beam.runners.fnexecution.control.InstructionRequestHandler
- RemoteBundle - Interface in org.apache.beam.runners.fnexecution.control
-
A bundle capable of handling input data elements for a
bundle descriptor
by forwarding them to a remote environment for processing. - RemoteEnvironment - Interface in org.apache.beam.runners.fnexecution.environment
-
A handle to an available remote
RunnerApi.Environment
. - RemoteEnvironment.SimpleRemoteEnvironment - Class in org.apache.beam.runners.fnexecution.environment
-
A
RemoteEnvironment
which uses the defaultRemoteEnvironment.close()
behavior. - RemoteInputDestination<T> - Class in org.apache.beam.runners.fnexecution.data
-
A pair of
Coder
andBeamFnApi.Target
which specifies the arguments to aFnDataService
to send data to a remote harness. - RemoteInputDestination() - Constructor for class org.apache.beam.runners.fnexecution.data.RemoteInputDestination
- RemoteOutputReceiver<T> - Class in org.apache.beam.runners.fnexecution.control
-
A pair of
Coder
andFnDataReceiver
which can be registered to receive elements for aLogicalEndpoint
. - RemoteOutputReceiver() - Constructor for class org.apache.beam.runners.fnexecution.control.RemoteOutputReceiver
- removeStagedArtifacts(String) - Method in interface org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService.ArtifactDestinationProvider
- removeStagedArtifacts(String) - Method in class org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService
- requestProgress() - Method in interface org.apache.beam.runners.fnexecution.control.RemoteBundle
-
Ask the remote bundle for progress.
- requestProgress() - Method in class org.apache.beam.runners.fnexecution.control.SdkHarnessClient.BundleProcessor.ActiveBundle
- requestsFinalization(String) - Method in interface org.apache.beam.runners.fnexecution.control.BundleFinalizationHandler
-
This callback is invoked whenever an inflight bundle that is being processed requests finalization.
- requestsFinalization(String) - Method in class org.apache.beam.runners.fnexecution.control.BundleFinalizationHandlers.InMemoryFinalizer
- resetForNewKey() - Method in class org.apache.beam.runners.fnexecution.state.InMemoryBagUserStateFactory
-
Prepares previous emitted state handlers for processing a new key.
- resolveArtifacts(ArtifactApi.ResolveArtifactsRequest, StreamObserver<ArtifactApi.ResolveArtifactsResponse>) - Method in class org.apache.beam.runners.fnexecution.artifact.ArtifactRetrievalService
- retrievalToken() - Method in class org.apache.beam.runners.fnexecution.provisioning.JobInfo
- reverseArtifactRetrievalService(StreamObserver<ArtifactApi.ArtifactRequestWrapper>) - Method in class org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService
S
- SDF_PREFIX - Static variable in class org.apache.beam.runners.fnexecution.control.BundleCheckpointHandlers.StateAndTimerBundleCheckpointHandler
- SdkHarnessClient - Class in org.apache.beam.runners.fnexecution.control
-
A high-level client for an SDK harness.
- SdkHarnessClient.BundleProcessor - Class in org.apache.beam.runners.fnexecution.control
-
A processor capable of creating bundles for some registered
BeamFnApi.ProcessBundleDescriptor
. - SdkHarnessClient.BundleProcessor.ActiveBundle - Class in org.apache.beam.runners.fnexecution.control
-
An active bundle for a particular
BeamFnApi.ProcessBundleDescriptor
. - send(LogicalEndpoint, Coder<T>) - Method in interface org.apache.beam.runners.fnexecution.data.FnDataService
-
Deprecated.Migrate to use
BeamFnDataOutboundAggregator
directly for sending outbound data. - send(LogicalEndpoint, Coder<T>) - Method in class org.apache.beam.runners.fnexecution.data.GrpcDataService
- ServerInfo() - Constructor for class org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.ServerInfo
- sideInputId() - Method in class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors.SideInputSpec
- SideInputSpec() - Constructor for class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors.SideInputSpec
- SimpleRemoteEnvironment() - Constructor for class org.apache.beam.runners.fnexecution.environment.RemoteEnvironment.SimpleRemoteEnvironment
- SingleEnvironmentInstanceJobBundleFactory - Class in org.apache.beam.runners.fnexecution.control
-
Deprecated.replace with a
DefaultJobBundleFactory
when appropriate if theEnvironmentFactory
is aDockerEnvironmentFactory
, or create anInProcessJobBundleFactory
and inline the creation of the environment if appropriate. - Slf4jLogWriter - Class in org.apache.beam.runners.fnexecution.logging
-
A
LogWriter
which uses anSLF4J Logger
as the underlying log backend. - split(double) - Method in interface org.apache.beam.runners.fnexecution.control.RemoteBundle
-
Ask the remote bundle to split its current processing based upon its knowledge of remaining work.
- split(double) - Method in class org.apache.beam.runners.fnexecution.control.SdkHarnessClient.BundleProcessor.ActiveBundle
- split(BeamFnApi.ProcessBundleSplitResponse) - Method in interface org.apache.beam.runners.fnexecution.control.BundleSplitHandler
- StageBundleFactory - Interface in org.apache.beam.runners.fnexecution.control
-
A bundle factory scoped to a particular
ExecutableStage
, which has all of the resources it needs to provide newRemoteBundles
. - STAGING_TO_ARTIFACT_URN - Static variable in class org.apache.beam.runners.fnexecution.artifact.ArtifactRetrievalService
- startProcess(String, String, List<String>, Map<String, String>) - Method in class org.apache.beam.runners.fnexecution.environment.ProcessManager
-
Forks a process with the given command, arguments, and additional environment variables.
- startProcess(String, String, List<String>, Map<String, String>, File) - Method in class org.apache.beam.runners.fnexecution.environment.ProcessManager
- state(StreamObserver<BeamFnApi.StateResponse>) - Method in class org.apache.beam.runners.fnexecution.state.GrpcStateService
- StateAndTimerBundleCheckpointHandler(TimerInternalsFactory<T>, StateInternalsFactory<T>, Coder<WindowedValue<T>>, Coder) - Constructor for class org.apache.beam.runners.fnexecution.control.BundleCheckpointHandlers.StateAndTimerBundleCheckpointHandler
- StateDelegator - Interface in org.apache.beam.runners.fnexecution.state
-
The
StateDelegator
is able to delegateBeamFnApi.StateRequest
s to a set of registered handlers. - StateDelegator.Registration - Interface in org.apache.beam.runners.fnexecution.state
-
Allows callers to deregister from receiving further state requests.
- StateRequestHandler - Interface in org.apache.beam.runners.fnexecution.state
-
Handler for
StateRequests
. - StateRequestHandlers - Class in org.apache.beam.runners.fnexecution.state
-
A set of utility methods which construct
StateRequestHandler
s. - StateRequestHandlers() - Constructor for class org.apache.beam.runners.fnexecution.state.StateRequestHandlers
- StateRequestHandlers.BagUserStateHandler<K,V,W extends org.apache.beam.sdk.transforms.windowing.BoundedWindow> - Interface in org.apache.beam.runners.fnexecution.state
-
A handler for bag user state.
- StateRequestHandlers.BagUserStateHandlerFactory<K,V,W extends org.apache.beam.sdk.transforms.windowing.BoundedWindow> - Interface in org.apache.beam.runners.fnexecution.state
-
A factory which constructs
StateRequestHandlers.BagUserStateHandler
s. - StateRequestHandlers.IterableSideInputHandler<V,W extends org.apache.beam.sdk.transforms.windowing.BoundedWindow> - Interface in org.apache.beam.runners.fnexecution.state
-
A handler for iterable side inputs.
- StateRequestHandlers.MultimapSideInputHandler<K,V,W extends org.apache.beam.sdk.transforms.windowing.BoundedWindow> - Interface in org.apache.beam.runners.fnexecution.state
-
A handler for multimap side inputs.
- StateRequestHandlers.SideInputHandler - Interface in org.apache.beam.runners.fnexecution.state
-
Marker interface that denotes some type of side input handler.
- StateRequestHandlers.SideInputHandlerFactory - Interface in org.apache.beam.runners.fnexecution.state
-
A factory which constructs
StateRequestHandlers.MultimapSideInputHandler
s. - StaticGrpcProvisionService - Class in org.apache.beam.runners.fnexecution.provisioning
-
A
provision service
that returns a static response to all calls. - StaticRemoteEnvironment - Class in org.apache.beam.runners.fnexecution.environment
-
A
RemoteEnvironment
that connects to Dataflow runner harness. - StaticRemoteEnvironmentFactory - Class in org.apache.beam.runners.fnexecution.environment
-
An
EnvironmentFactory
that creates StaticRemoteEnvironment used by a runner harness that would like to use an existing InstructionRequestHandler. - StaticRemoteEnvironmentFactory.Provider - Class in org.apache.beam.runners.fnexecution.environment
-
Provider for StaticRemoteEnvironmentFactory.
- stopProcess(String) - Method in class org.apache.beam.runners.fnexecution.environment.ProcessManager
-
Stops a previously started process identified by its unique id.
- StreamingSideInputHandlerFactory - Class in org.apache.beam.runners.fnexecution.translation
-
StateRequestHandler
that usesSideInputHandler
to access the broadcast state that represents side inputs.
T
- take(String, Duration) - Method in interface org.apache.beam.runners.fnexecution.control.ControlClientPool.Source
-
Retrieves the
InstructionRequestHandler
for the given worker id, blocking until available or the request times out. - timerId() - Method in class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors.TimerSpec
- TimerReceiverFactory - Class in org.apache.beam.runners.fnexecution.control
-
A factory that passes timers to
TimerReceiverFactory.timerDataConsumer
. - TimerReceiverFactory(StageBundleFactory, BiConsumer<Timer<?>, TimerInternals.TimerData>, Coder) - Constructor for class org.apache.beam.runners.fnexecution.control.TimerReceiverFactory
- TimerSpec() - Constructor for class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors.TimerSpec
- toProvisionInfo() - Method in class org.apache.beam.runners.fnexecution.provisioning.JobInfo
- transformId() - Method in class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors.BagUserStateSpec
- transformId() - Method in class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors.SideInputSpec
- transformId() - Method in class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors.TimerSpec
U
- unsupported() - Static method in interface org.apache.beam.runners.fnexecution.control.BundleSplitHandler
-
Returns a bundle split handler that throws on any split response.
- unsupported() - Static method in interface org.apache.beam.runners.fnexecution.state.StateRequestHandler
- unsupported() - Static method in interface org.apache.beam.runners.fnexecution.state.StateRequestHandlers.BagUserStateHandlerFactory
-
Throws a
UnsupportedOperationException
on the first access. - unsupported() - Static method in interface org.apache.beam.runners.fnexecution.state.StateRequestHandlers.SideInputHandlerFactory
-
Throws a
UnsupportedOperationException
on the first access. - URL_ARTIFACT_URN - Static variable in class org.apache.beam.runners.fnexecution.artifact.ArtifactRetrievalService
- userStateId() - Method in class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors.BagUserStateSpec
- usingFnApiClient(InstructionRequestHandler, FnDataService) - Static method in class org.apache.beam.runners.fnexecution.control.SdkHarnessClient
-
Creates a client for a particular SDK harness.
V
- valueCoder() - Method in class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors.BagUserStateSpec
- valueCoder() - Method in interface org.apache.beam.runners.fnexecution.state.StateRequestHandlers.MultimapSideInputHandler
-
Returns the
Coder
to use for the elements of the resulting values iterable. - verifyDeterministic() - Method in class org.apache.beam.runners.fnexecution.wire.ByteStringCoder
W
- windowCoder() - Method in class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors.BagUserStateSpec
- windowCoder() - Method in class org.apache.beam.runners.fnexecution.control.ProcessBundleDescriptors.SideInputSpec
- WireCoders - Class in org.apache.beam.runners.fnexecution.wire
-
Helpers to construct coders for gRPC port reads and writes.
- withIdGenerator(IdGenerator) - Method in class org.apache.beam.runners.fnexecution.control.SdkHarnessClient
- workerStatus(StreamObserver<BeamFnApi.WorkerStatusRequest>) - Method in class org.apache.beam.runners.fnexecution.status.BeamWorkerStatusGrpcService
All Classes All Packages