All Classes
-
All Classes Interface Summary Class Summary Enum Summary Exception Summary Class Description Adapter An adapter is a modification producing a variation of a model that can be used during prediction.Job<I,O> A class represents an inference job.JobFunction<I,O> A function describing the action to take in aJob
.LmiUtils A utility class to detect optimal engine for LMI model.ModelInfo<I,O> A class represent a loaded model and it's metadata.PermanentBatchAggregator<I,O> a batch aggregator that never terminates by itself.PyAdapter An overload ofAdapter
for the python engine.SageMakerUtils A utility class to detect optimal engine for SageMaker saved model.TemporaryBatchAggregator<I,O> a batch aggregator that terminates after a maximum idle time.WlmCapacityException Thrown to throttle when a job is run but the job queue capacity is exceeded.WlmConfigManager This manages some configurations used by theWorkLoadManager
.WlmException Thrown when an exception occurs inside theWorkLoadManager
.WlmOutOfMemoryException Thrown when no enough memory to load the model.WlmShutdownException Thrown when a job is run but all workers are shutdown.WorkerGroup<I,O> WorkerIdGenerator class to generate an unique worker id.WorkerJob<I,O> AJob
containing metadata from theWorkLoadManager
.WorkerPool<I,O> Manages the work load for a single model.WorkerPoolConfig<I,O> AWorkerPoolConfig
represents a task that could be run in theWorkLoadManager
.WorkerPoolConfig.Status An enum represents state of a worker type.WorkerPoolConfig.ThreadConfig<I,O> The part of theWorkerPoolConfig
for an individualWorkerThread
.WorkerState An enum represents state of a worker.WorkerThread<I,O> TheWorkerThread
is the worker managed by theWorkLoadManager
.WorkerThread.Builder<I,O> A Builder to construct aWorkerThread
.WorkLoadManager WorkLoadManager is responsible to manage the work load of worker thread.