org.apache.flink.runtime.resourcemanager

Class ActiveResourceManager<WorkerType extends ResourceIDRetrievable>

java.lang.Object

org.apache.flink.runtime.rpc.RpcEndpoint

org.apache.flink.runtime.rpc.FencedRpcEndpoint<ResourceManagerId>

org.apache.flink.runtime.resourcemanager.ResourceManager<WorkerType>

org.apache.flink.runtime.resourcemanager.ActiveResourceManager<WorkerType>

All Implemented Interfaces:

AutoCloseable, ClusterPartitionManager, LeaderContender, ResourceManagerGateway, FencedRpcGateway<ResourceManagerId>, RpcGateway, org.apache.flink.util.AutoCloseableAsync

public abstract class ActiveResourceManager<WorkerType extends ResourceIDRetrievable>
extends ResourceManager<WorkerType>

Base class for ResourceManager implementations which contains some common variables and methods.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected static class	ActiveResourceManager.PendingWorkerNums Number of workers pending for allocation/registration.

Nested classes/interfaces inherited from class org.apache.flink.runtime.rpc.RpcEndpoint

RpcEndpoint.MainThreadExecutor

Field Summary

Fields

Modifier and Type	Field and Description
protected Map<String,String>	env The process environment variables.
protected org.apache.flink.configuration.Configuration	flinkClientConfig Flink configuration uploaded by client.
protected org.apache.flink.configuration.Configuration	flinkConfig The updated Flink configuration.

Fields inherited from class org.apache.flink.runtime.resourcemanager.ResourceManager

RESOURCE_MANAGER_NAME

Fields inherited from class org.apache.flink.runtime.rpc.RpcEndpoint

log, rpcServer

Constructor Summary

Constructors

Constructor and Description

ActiveResourceManager(org.apache.flink.configuration.Configuration flinkConfig, Map<String,String> env, RpcService rpcService, ResourceID resourceId, HighAvailabilityServices highAvailabilityServices, HeartbeatServices heartbeatServices, SlotManager slotManager, ResourceManagerPartitionTrackerFactory clusterPartitionTrackerFactory, JobLeaderIdService jobLeaderIdService, ClusterInformation clusterInformation, FatalErrorHandler fatalErrorHandler, ResourceManagerMetricGroup resourceManagerMetricGroup)

Method Summary

Modifier and Type	Method and Description
protected int	getNumRequestedNotAllocatedWorkers()
protected int	getNumRequestedNotAllocatedWorkersFor(WorkerResourceSpec workerResourceSpec)
protected int	getNumRequestedNotRegisteredWorkers()
protected int	getNumRequestedNotRegisteredWorkersFor(WorkerResourceSpec workerResourceSpec)
protected CompletableFuture<Void>	getStopTerminationFutureOrCompletedExceptionally(Throwable exception)
protected abstract org.apache.flink.configuration.Configuration	loadClientConfiguration()
protected void	notifyAllocatedWorkerStopped(ResourceID resourceID) Notify that a worker with the given resource spec has been stopped.
protected ActiveResourceManager.PendingWorkerNums	notifyNewWorkerAllocated(WorkerResourceSpec workerResourceSpec, ResourceID resourceID) Notify that a worker with the given resource spec has been allocated.
protected ActiveResourceManager.PendingWorkerNums	notifyNewWorkerAllocationFailed(WorkerResourceSpec workerResourceSpec) Notify that allocation of a worker with the given resource spec has failed.
protected ActiveResourceManager.PendingWorkerNums	notifyNewWorkerRequested(WorkerResourceSpec workerResourceSpec) Notify that a worker with the given resource spec has been requested.
protected void	onTaskManagerRegistration(WorkerRegistration<WorkerType> workerTypeWorkerRegistration)

Methods inherited from class org.apache.flink.runtime.resourcemanager.ResourceManager

cancelSlotRequest, clearStateAsync, closeJobManagerConnection, closeTaskManagerConnection, deregisterApplication, disconnectJobManager, disconnectTaskManager, getNumberOfRegisteredTaskManagers, getNumberRequiredTaskManagers, getRequiredResources, grantLeadership, handleError, heartbeatFromJobManager, heartbeatFromTaskManager, initialize, internalDeregisterApplication, jobLeaderLostLeadership, listDataSets, notifySlotAvailable, onFatalError, onStart, onStop, prepareLeadershipAsync, registerJobManager, registerTaskExecutor, releaseClusterPartitions, releaseResource, removeJob, requestResourceOverview, requestSlot, requestTaskManagerFileUploadByName, requestTaskManagerFileUploadByType, requestTaskManagerInfo, requestTaskManagerInfo, requestTaskManagerLogList, requestTaskManagerMetricQueryServiceAddresses, requestThreadDump, revokeLeadership, sendSlotReport, setFailUnfulfillableRequest, startNewWorker, startServicesOnLeadership, stopWorker, workerStarted

Methods inherited from class org.apache.flink.runtime.rpc.FencedRpcEndpoint

callAsyncWithoutFencing, getFencingToken, getMainThreadExecutor, getUnfencedMainThreadExecutor, runAsyncWithoutFencing, setFencingToken

Methods inherited from class org.apache.flink.runtime.rpc.RpcEndpoint

callAsync, closeAsync, getAddress, getEndpointId, getHostname, getRpcService, getSelfGateway, getTerminationFuture, internalCallOnStart, internalCallOnStop, isRunning, runAsync, scheduleRunAsync, scheduleRunAsync, start, stop, validateRunsInMainThread

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.runtime.rpc.FencedRpcGateway

getFencingToken

Methods inherited from interface org.apache.flink.runtime.rpc.RpcGateway

getAddress, getHostname

Methods inherited from interface org.apache.flink.runtime.leaderelection.LeaderContender

getDescription

Methods inherited from interface org.apache.flink.util.AutoCloseableAsync

close

Field Detail

env

protected final Map<String,String> env

The process environment variables.

flinkConfig

protected final org.apache.flink.configuration.Configuration flinkConfig

The updated Flink configuration. The client uploaded configuration may be updated before passed on to ResourceManager. For example, TaskManagerOptions.MANAGED_MEMORY_SIZE.

flinkClientConfig

protected final org.apache.flink.configuration.Configuration flinkClientConfig

Flink configuration uploaded by client.

Constructor Detail

ActiveResourceManager

public ActiveResourceManager(org.apache.flink.configuration.Configuration flinkConfig,
                             Map<String,String> env,
                             RpcService rpcService,
                             ResourceID resourceId,
                             HighAvailabilityServices highAvailabilityServices,
                             HeartbeatServices heartbeatServices,
                             SlotManager slotManager,
                             ResourceManagerPartitionTrackerFactory clusterPartitionTrackerFactory,
                             JobLeaderIdService jobLeaderIdService,
                             ClusterInformation clusterInformation,
                             FatalErrorHandler fatalErrorHandler,
                             ResourceManagerMetricGroup resourceManagerMetricGroup)

Method Detail

getStopTerminationFutureOrCompletedExceptionally

protected CompletableFuture<Void> getStopTerminationFutureOrCompletedExceptionally(@Nullable
                                                                                   Throwable exception)

loadClientConfiguration

protected abstract org.apache.flink.configuration.Configuration loadClientConfiguration()

onTaskManagerRegistration

protected void onTaskManagerRegistration(WorkerRegistration<WorkerType> workerTypeWorkerRegistration)

Overrides:

onTaskManagerRegistration in class ResourceManager<WorkerType extends ResourceIDRetrievable>

getNumRequestedNotAllocatedWorkers

protected int getNumRequestedNotAllocatedWorkers()

getNumRequestedNotAllocatedWorkersFor

protected int getNumRequestedNotAllocatedWorkersFor(WorkerResourceSpec workerResourceSpec)

getNumRequestedNotRegisteredWorkers

protected int getNumRequestedNotRegisteredWorkers()

getNumRequestedNotRegisteredWorkersFor

protected int getNumRequestedNotRegisteredWorkersFor(WorkerResourceSpec workerResourceSpec)

notifyNewWorkerRequested

protected ActiveResourceManager.PendingWorkerNums notifyNewWorkerRequested(WorkerResourceSpec workerResourceSpec)

Notify that a worker with the given resource spec has been requested.

Parameters:

workerResourceSpec - resource spec of the requested worker

Returns:

updated number of pending workers for the given resource spec

notifyNewWorkerAllocated

protected ActiveResourceManager.PendingWorkerNums notifyNewWorkerAllocated(WorkerResourceSpec workerResourceSpec,
                                                                           ResourceID resourceID)

Notify that a worker with the given resource spec has been allocated.

Parameters:

workerResourceSpec - resource spec of the requested worker

resourceID - id of the allocated resource

Returns:

updated number of pending workers for the given resource spec

notifyNewWorkerAllocationFailed

protected ActiveResourceManager.PendingWorkerNums notifyNewWorkerAllocationFailed(WorkerResourceSpec workerResourceSpec)

Notify that allocation of a worker with the given resource spec has failed.

Parameters:

workerResourceSpec - resource spec of the requested worker

Returns:

updated number of pending workers for the given resource spec

notifyAllocatedWorkerStopped

protected void notifyAllocatedWorkerStopped(ResourceID resourceID)

Notify that a worker with the given resource spec has been stopped.

Parameters:

resourceID - id of the stopped worker resource