See: Description
Class | Description |
---|---|
Job |
A class represents an inference job.
|
ModelInfo |
A class represent a loaded model and it's metadata.
|
PermanentBatchAggregator |
a batch aggregator that never terminates by itself.
|
TemporaryBatchAggregator |
a batch aggregator that terminates after a maximum idle time.
|
WorkerIdGenerator |
class to generate an unique worker id.
|
WorkerThread |
The
WorkerThread is the worker managed by the WorkLoadManager . |
WorkerThread.Builder |
A Builder to construct a
WorkerThread . |
WorkLoadManager |
WorkLoadManager is responsible to manage the work load of worker thread.
|
Enum | Description |
---|---|
WorkerState |
An enum represents state of a worker.
|
WorkLoadManager