public interface NDManager
extends java.lang.AutoCloseable
NDManager is implemented in each deep learning framework Engine
. NDArray
s are
resources that are allocated in each deep learning framework's native memory space. NDManager is
the key class that manages these native resources.
NDArray can only be created through NDManager. By default, NDArray's lifecycle is attached to
the creator NDManager. NDManager itself implements AutoCloseable
. When NDManager is
closed, all the resource associated with it will be closed as well.
A typical place to obtain NDManager is in PreProcessor.processInput(TranslatorContext,
Object)
or PostProcessor.processOutput(TranslatorContext, NDList)
.
The following is an example of how to use NDManager:
public class MyTranslator implements Translator<FloatBuffer, String> { @Override public NDList processInput(TranslatorContext ctx, FloatBuffer input) { NDManager manager = ctx.getNDManager(); NDArray array = manager.create(shape); array.set(input); return new NDList(array); } // NDArrays created in this method will be closed after method return. }
NDManager has a hierarchical structure; it has a single parent NDManager and has child NDManagers. When the parent NDManager is closed, all children will be closed as well.
The DJL framework manages NDManager's lifecycle by default. You only need to manage the user created child NDManager. The child NDManager becomes useful when you create a large number of temporary NDArrays and want to free the resources earlier than the parent NDManager's lifecycle.
The following is an example of such a use case:
public class MyTranslator implements Translator<List<FloatBuffer>>, String> { @Override public NDList processInput(TranslatorContext ctx, List<FloatBuffer> input) { NDManager manager = ctx.getNDManager(); NDArray array = manager.create(shape, dataType); for (int i = 0; i < input.size(); ++i) { try (NDManager childManager = manager.newSubManager()) { NDArray tmp = childManager.create(itemShape); tmp.put(input.get(i); array.put(i, tmp); } // NDArray tmp will be closed here } return new NDList(array); } }
You can also close an individual NDArray. NDManager won't close an NDArray that's already been closed. In certain use cases, you might want to return an NDArray outside of NDManager's scope.
Modifier and Type | Method and Description |
---|---|
java.nio.ByteBuffer |
allocateDirect(int capacity)
Allocates a new engine specific direct byte buffer.
|
default NDArray |
arange(float stop)
Returns evenly spaced values starting from 0.
|
default NDArray |
arange(float start,
float stop)
Returns evenly spaced values within a given interval with step 1.
|
default NDArray |
arange(float start,
float stop,
float step)
Returns evenly spaced values within a given interval.
|
NDArray |
arange(float start,
float stop,
float step,
DataType dataType)
Returns evenly spaced values within a given interval.
|
default NDArray |
arange(float start,
float stop,
float step,
DataType dataType,
Device device)
Returns evenly spaced values within a given interval.
|
default NDArray |
arange(int stop)
Returns evenly spaced values starting from 0.
|
default NDArray |
arange(int start,
int stop)
Returns evenly spaced values within a given interval with step 1.
|
default NDArray |
arange(int start,
int stop,
int step)
Returns evenly spaced values within a given interval.
|
default NDArray |
arange(int start,
int stop,
int step,
DataType dataType)
Returns evenly spaced values within a given interval.
|
void |
attach(java.lang.String resourceId,
java.lang.AutoCloseable resource)
|
void |
close() |
default NDArray |
create(boolean data)
Creates and initializes a scalar
NDArray . |
default NDArray |
create(boolean[] data)
Creates and initializes a 1D
NDArray . |
default NDArray |
create(boolean[][] data)
Creates and initializes a 2-D
NDArray . |
default NDArray |
create(boolean[] data,
Shape shape)
|
default NDArray |
create(java.nio.Buffer data,
Shape shape)
|
default NDArray |
create(java.nio.Buffer data,
Shape shape,
DataType dataType)
|
default NDArray |
create(byte data)
Creates and initializes a scalar
NDArray . |
default NDArray |
create(byte[] data)
Creates and initializes a 1D
NDArray . |
default NDArray |
create(byte[][] data)
Creates and initializes a 2-D
NDArray . |
default NDArray |
create(byte[] data,
Shape shape)
|
default NDArray |
create(double data)
Creates and initializes a scalar
NDArray . |
default NDArray |
create(double[] data)
Creates and initializes a 1D
NDArray . |
default NDArray |
create(double[][] data)
Creates and initializes a 2D
NDArray . |
default NDArray |
create(double[] data,
Shape shape)
|
default NDArray |
create(float data)
Creates and initializes a scalar
NDArray . |
default NDArray |
create(float[] data)
Creates and initializes a 1D
NDArray . |
default NDArray |
create(float[][] data)
Creates and initializes a 2D
NDArray . |
default NDArray |
create(float[] data,
Shape shape)
|
default NDArray |
create(int data)
Creates and initializes a scalar
NDArray . |
default NDArray |
create(int[] data)
Creates and initializes a 1D
NDArray . |
default NDArray |
create(int[][] data)
Creates and initializes a 2D
NDArray . |
default NDArray |
create(int[] data,
Shape shape)
|
default NDArray |
create(long data)
Creates and initializes a scalar
NDArray . |
default NDArray |
create(long[] data)
Creates and initializes a 1D
NDArray . |
default NDArray |
create(long[][] data)
Creates and initializes a 2-D
NDArray . |
default NDArray |
create(long[] data,
Shape shape)
|
default NDArray |
create(java.lang.Number data)
Creates and initializes a scalar
NDArray . |
default NDArray |
create(Shape shape)
|
NDArray |
create(Shape shape,
DataType dataType)
|
default NDArray |
create(Shape shape,
DataType dataType,
Device device)
|
NDArray |
createCSR(java.nio.Buffer data,
long[] indptr,
long[] indices,
Shape shape)
Creates a Compressed Sparse Row Storage (CSR) Format Matrix.
|
default NDArray |
createCSR(java.nio.Buffer data,
long[] indptr,
long[] indices,
Shape shape,
Device device)
Creates a Compressed Sparse Row Storage (CSR) Format Matrix.
|
NDArray |
createRowSparse(java.nio.Buffer data,
Shape dataShape,
long[] indices,
Shape shape)
Stores the matrix in row sparse format.
|
default NDArray |
createRowSparse(java.nio.Buffer data,
Shape dataShape,
long[] indices,
Shape shape,
Device device)
Stores the matrix in row sparse format.
|
default NDArray |
decode(byte[] bytes)
Decodes
NDArray through byte array. |
default NDArray |
decode(java.io.InputStream is)
Decodes
NDArray through DataInputStream . |
void |
detach(java.lang.String resourceId)
Detaches a
NDArray from this NDManager 's lifecycle. |
default NDArray |
eye(int rows)
Returns a 2-D array with ones on the diagonal and zeros elsewhere.
|
default NDArray |
eye(int rows,
int k)
Returns a 2-D array with ones on the diagonal and zeros elsewhere.
|
default NDArray |
eye(int rows,
int cols,
int k)
Returns a 2-D array with ones on the diagonal and zeros elsewhere.
|
NDArray |
eye(int rows,
int cols,
int k,
DataType dataType)
Returns a 2-D array with ones on the diagonal and zeros elsewhere.
|
default NDArray |
eye(int rows,
int cols,
int k,
DataType dataType,
Device device)
Returns a 2-D array with ones on the diagonal and zeros elsewhere.
|
Device |
getDevice()
Returns the default
Device of this NDManager . |
Engine |
getEngine()
Returns the
Engine associated with this manager. |
NDManager |
getParentManager()
Returns the parent
NDManager . |
void |
invoke(java.lang.String operation,
NDArray[] src,
NDArray[] dest,
ai.djl.util.PairList<java.lang.String,?> params)
An engine specific generic invocation to native operation.
|
NDList |
invoke(java.lang.String operation,
NDList src,
ai.djl.util.PairList<java.lang.String,?> params)
An engine specific generic invocation to native operation.
|
boolean |
isOpen()
Check if the manager is still valid.
|
default NDArray |
linspace(float start,
float stop,
int num)
Returns evenly spaced numbers over a specified interval.
|
NDArray |
linspace(float start,
float stop,
int num,
boolean endpoint)
Returns evenly spaced numbers over a specified interval.
|
default NDArray |
linspace(float start,
float stop,
int num,
boolean endpoint,
Device device)
Returns evenly spaced numbers over a specified interval.
|
default NDArray |
linspace(int start,
int stop,
int num)
Returns evenly spaced numbers over a specified interval.
|
default NDArray |
linspace(int start,
int stop,
int num,
boolean endpoint)
Returns evenly spaced numbers over a specified interval.
|
NDList |
load(java.nio.file.Path path)
Loads the NDArrays saved to a file.
|
default NDList |
load(java.nio.file.Path path,
Device device)
Loads the NDArrays saved to a file.
|
static NDManager |
newBaseManager()
Creates a new top-level
NDManager . |
static NDManager |
newBaseManager(Device device)
Creates a new top-level
NDManager with specified Device . |
static NDManager |
newBaseManager(Device device,
java.lang.String engineName)
Creates a new top-level
NDManager with specified Device and engine. |
NDManager |
newSubManager()
Creates a child
NDManager . |
NDManager |
newSubManager(Device device)
Creates a child
NDManager with specified default Device . |
default NDArray |
ones(Shape shape)
|
NDArray |
ones(Shape shape,
DataType dataType)
|
default NDArray |
ones(Shape shape,
DataType dataType,
Device device)
|
NDArray |
randomMultinomial(int n,
NDArray pValues)
Draw samples from a multinomial distribution.
|
NDArray |
randomMultinomial(int n,
NDArray pValues,
Shape shape)
Draw samples from a multinomial distribution.
|
NDArray |
randomNormal(float loc,
float scale,
Shape shape,
DataType dataType)
Draws random samples from a normal (Gaussian) distribution.
|
default NDArray |
randomNormal(float loc,
float scale,
Shape shape,
DataType dataType,
Device device)
Draws random samples from a normal (Gaussian) distribution.
|
default NDArray |
randomNormal(Shape shape)
Draws random samples from a normal (Gaussian) distribution with mean 0 and standard deviation
1.
|
default NDArray |
randomNormal(Shape shape,
DataType dataType)
Draws random samples from a normal (Gaussian) distribution with mean 0 and standard deviation
1.
|
default NDArray |
randomUniform(float low,
float high,
Shape shape)
Draws samples from a uniform distribution.
|
NDArray |
randomUniform(float low,
float high,
Shape shape,
DataType dataType)
Draws samples from a uniform distribution.
|
default NDArray |
randomUniform(float low,
float high,
Shape shape,
DataType dataType,
Device device)
Draws samples from a uniform distribution.
|
default NDArray |
zeros(Shape shape)
|
NDArray |
zeros(Shape shape,
DataType dataType)
|
default NDArray |
zeros(Shape shape,
DataType dataType,
Device device)
|
static NDManager newBaseManager()
NDManager
.
NDManager
will inherit default Device
.
NDManager
static NDManager newBaseManager(Device device)
NDManager
with specified Device
.device
- the default Device
NDManager
static NDManager newBaseManager(Device device, java.lang.String engineName)
NDManager
with specified Device
and engine.device
- the default Device
engineName
- the name of the engineNDManager
java.nio.ByteBuffer allocateDirect(int capacity)
capacity
- the new buffer's capacity, in bytesdefault NDArray create(java.lang.Number data)
NDArray
.data
- the Number
that needs to be setNDArray
default NDArray create(float data)
NDArray
.data
- the float that needs to be setNDArray
default NDArray create(int data)
NDArray
.data
- the float data that needs to be setNDArray
default NDArray create(double data)
NDArray
.data
- the double data that needs to be setNDArray
default NDArray create(long data)
NDArray
.data
- the long data that needs to be setNDArray
default NDArray create(byte data)
NDArray
.data
- the byte data that needs to be setNDArray
default NDArray create(boolean data)
NDArray
.data
- the boolean data that needs to be setNDArray
default NDArray create(float[] data)
NDArray
.data
- the float array that needs to be setNDArray
default NDArray create(int[] data)
NDArray
.data
- the float array that needs to be setNDArray
default NDArray create(double[] data)
NDArray
.data
- the float array that needs to be setNDArray
default NDArray create(long[] data)
NDArray
.data
- the float array that needs to be setNDArray
default NDArray create(byte[] data)
NDArray
.data
- the float array that needs to be setNDArray
default NDArray create(boolean[] data)
NDArray
.data
- the bool array that needs to be setNDArray
default NDArray create(float[][] data)
NDArray
.data
- the float array that needs to be setNDArray
default NDArray create(int[][] data)
NDArray
.data
- the float array that needs to be setNDArray
default NDArray create(double[][] data)
NDArray
.data
- the float array that needs to be setNDArray
default NDArray create(long[][] data)
NDArray
.data
- the float array that needs to be setNDArray
default NDArray create(byte[][] data)
NDArray
.data
- the float array that needs to be setNDArray
default NDArray create(boolean[][] data)
NDArray
.data
- the boolean array that needs to be setNDArray
default NDArray createCSR(java.nio.Buffer data, long[] indptr, long[] indices, Shape shape, Device device)
data
- the data to set for the CSR Matrixindptr
- the indptr array is what will help identify the rows where the data appearsindices
- the indices array stores the column index for each non-zero element in datashape
- the Shape
of the NDArray
device
- the Device
of the NDArray
NDArray
NDArray createCSR(java.nio.Buffer data, long[] indptr, long[] indices, Shape shape)
default NDArray createRowSparse(java.nio.Buffer data, Shape dataShape, long[] indices, Shape shape, Device device)
NDArray createRowSparse(java.nio.Buffer data, Shape dataShape, long[] indices, Shape shape)
default NDArray decode(byte[] bytes)
NDArray
through byte array.bytes
- byte array to load fromNDArray
default NDArray decode(java.io.InputStream is) throws java.io.IOException
NDArray
through DataInputStream
.is
- input stream data to load fromNDArray
java.io.IOException
- data is not readableNDList load(java.nio.file.Path path)
path
- the path to the filedefault NDList load(java.nio.file.Path path, Device device)
path
- the path to the filedevice
- the device to use for the loaded arraysdefault NDArray zeros(Shape shape)
shape
- the Shape
of the NDArray
NDArray
zeros(Shape, DataType, Device)
default NDArray arange(int stop)
Values are generated within the half-open interval [start, stop) (in other words, the
interval including start but excluding stop). For integer arguments, the function is
equivalent to the Python built-in range function, but returns an instance of NDArray
rather than a list.
stop
- the end of the interval. The interval does not include this valueNDArray
default NDArray arange(float stop)
Values are generated within the half-open interval [start, stop) (in other words, the
interval including start but excluding stop). For integer arguments, the function is
equivalent to the Python built-in range function, but returns an instance of NDArray
rather than a list.
stop
- the end of the interval. The interval does not include this valueNDArray
default NDArray arange(int start, int stop)
Values are generated within the half-open interval [start, stop) (in other words, the
interval including start but excluding stop). For integer arguments, the function is
equivalent to the Python built-in range function, but returns an instance of NDArray
rather than a list.
start
- the start of interval. The interval includes this valuestop
- the end of interval. The interval does not include this valueNDArray
default NDArray arange(float start, float stop)
Values are generated within the half-open interval [start, stop) (in other words, the
interval including start but excluding stop). For integer arguments, the function is
equivalent to the Python built-in range function, but returns an instance of NDArray
rather than a list.
start
- the start of interval. The interval includes this valuestop
- the end of interval. The interval does not include this valueNDArray
default NDArray arange(int start, int stop, int step)
Values are generated within the half-open interval [start, stop) (in other words, the
interval including start but excluding stop). For integer arguments, the function is
equivalent to the Python built-in range function, but returns an instance of NDArray
rather than a list.
start
- the start of interval. The interval includes this valuestop
- the end of interval. The interval does not include this valuestep
- the spacing between valuesNDArray
default NDArray arange(float start, float stop, float step)
Values are generated within the half-open interval [start, stop) (in other words, the
interval including start but excluding stop). For integer arguments, the function is
equivalent to the Python built-in range function, but returns an instance of NDArray
rather than a list.
start
- the start of interval. The interval includes this valuestop
- the end of interval. The interval does not include this valuestep
- the spacing between valuesNDArray
default NDArray arange(int start, int stop, int step, DataType dataType)
Values are generated within the half-open interval [start, stop) (in other words, the
interval including start but excluding stop). For integer arguments, the function is
equivalent to the Python built-in range function, but returns an instance of NDArray
rather than a list.
NDArray arange(float start, float stop, float step, DataType dataType)
Values are generated within the half-open interval [start, stop) (in other words, the
interval including start but excluding stop). For integer arguments, the function is
equivalent to the Python built-in range function, but returns an instance of NDArray
rather than a list.
default NDArray arange(float start, float stop, float step, DataType dataType, Device device)
Values are generated within the half-open interval [start, stop) (in other words, the
interval including start but excluding stop). For integer arguments, the function is
equivalent to the Python built-in range function, but returns an instance of NDArray
rather than a list.
default NDArray eye(int rows)
rows
- the number of rows and cols in the outputNDArray
where all elements are equal to zero, except for the k-th diagonal,
whose values are equal to onedefault NDArray eye(int rows, int k)
rows
- the number of rows and cols in the outputk
- the index of the diagonal: a positive value refers to an upper diagonal, and a
negative value to a lower diagonalNDArray
where all elements are equal to zero, except for the k-th diagonal,
whose values are equal to onedefault NDArray eye(int rows, int cols, int k)
rows
- the number of rows in the outputcols
- the number of columns in the outputk
- the index of the diagonal: a positive value refers to an upper diagonal, and a
negative value to a lower diagonalNDArray
where all elements are equal to zero, except for the k-th diagonal,
whose values are equal to oneNDArray eye(int rows, int cols, int k, DataType dataType)
rows
- the number of rows int the outputcols
- the number of columns in the outputk
- the index of the diagonal: a positive value refers to an upper diagonal, and a
negative value to a lower diagonaldataType
- the DataType
of the NDArray
NDArray
where all elements are equal to zero, except for the k-th diagonal,
whose values are equal to onedefault NDArray eye(int rows, int cols, int k, DataType dataType, Device device)
rows
- the number of rows int the outputcols
- the number of columns in the outputk
- the index of the diagonal: a positive value refers to an upper diagonal, and a
negative value to a lower diagonaldataType
- the DataType
of the NDArray
device
- the Device
of the NDArray
NDArray
where all elements are equal to zero, except for the k-th diagonal,
whose values are equal to onedefault NDArray linspace(int start, int stop, int num)
Returns num evenly spaced samples, calculated over the interval [start, stop].
start
- the starting value of the sequencestop
- the end value of the sequencenum
- the number of samples to generateNDArray
default NDArray linspace(float start, float stop, int num)
Returns num evenly spaced samples, calculated over the interval [start, stop].
start
- the starting value of the sequencestop
- the end value of the sequencenum
- the number of samples to generateNDArray
default NDArray linspace(int start, int stop, int num, boolean endpoint)
Returns num evenly spaced samples, calculated over the interval [start, stop].The endpoint of the interval can optionally be excluded.
start
- the starting value of the sequencestop
- the end value of the sequencenum
- the number of samples to generateendpoint
- if true
, stop is the last sample, otherwise, it is not includedNDArray
NDArray linspace(float start, float stop, int num, boolean endpoint)
Returns num evenly spaced samples, calculated over the interval [start, stop].The endpoint of the interval can optionally be excluded.
start
- the starting value of the sequencestop
- the end value of the sequencenum
- the number of samples to generateendpoint
- if true
, stop is the last sample, otherwise, it is not includedNDArray
default NDArray linspace(float start, float stop, int num, boolean endpoint, Device device)
Returns num evenly spaced samples, calculated over the interval [start, stop].The endpoint of the interval can optionally be excluded.
default NDArray randomUniform(float low, float high, Shape shape)
Samples are uniformly distributed over the half-open interval [low, high) (includes low, but excludes high). In other words, any value within the given interval is equally likely to be drawn by uniform.
NDArray randomUniform(float low, float high, Shape shape, DataType dataType)
Samples are uniformly distributed over the half-open interval [low, high) (includes low, but excludes high). In other words, any value within the given interval is equally likely to be drawn by uniform.
low
- the lower boundary of the output interval. All values generated will be greater
than or equal to low.high
- the upper boundary of the output interval. All values generated will be less than
high.shape
- the Shape
of the NDArray
dataType
- the DataType
of the NDArray
NDArray
default NDArray randomUniform(float low, float high, Shape shape, DataType dataType, Device device)
Samples are uniformly distributed over the half-open interval [low, high) (includes low, but excludes high). In other words, any value within the given interval is equally likely to be drawn by uniform.
low
- the lower boundary of the output interval. All values generated will be greater
than or equal to low.high
- the upper boundary of the output interval. All values generated will be less than
high.shape
- the Shape
of the NDArray
dataType
- the DataType
of the NDArray
device
- the Device
of the NDArray
NDArray
default NDArray randomNormal(Shape shape)
Samples are distributed according to a normal distribution parametrized by mean = 0 and standard deviation = 1.
default NDArray randomNormal(Shape shape, DataType dataType)
NDArray randomNormal(float loc, float scale, Shape shape, DataType dataType)
default NDArray randomNormal(float loc, float scale, Shape shape, DataType dataType, Device device)
NDArray randomMultinomial(int n, NDArray pValues)
The multinomial distribution is a multivariate generalization of the binomial distribution. Take an experiment with one of p possible outcomes. An example of such an experiment is throwing a dice, where the outcome can be 1 through 6. Each sample drawn from the distribution represents n such experiments. Its values, X_i = [X_0, X_1, ..., X_p], represent the number of times the outcome was i.
n
- the number of experimentspValues
- the probabilities of each of the p different outcomes. These should sum to 1
The last element is always assumed to account for the remaining probability, as long as
pValues.sum().getFloat() <= 1)NDArray
NDArray randomMultinomial(int n, NDArray pValues, Shape shape)
The multinomial distribution is a multivariate generalization of the binomial distribution. Take an experiment with one of p possible outcomes. An example of such an experiment is throwing a dice, where the outcome can be 1 through 6. Each sample drawn from the distribution represents n such experiments. Its values, X_i = [X_0, X_1, ..., X_p], represent the number of times the outcome was i.
n
- the number of experimentspValues
- the probabilities of each of the p different outcomes. These should sum to 1
The last element is always assumed to account for the remaining probability, as long as
pValues.sum().getFloat() <= 1)shape
- the output Shape
NDArray
boolean isOpen()
NDManager getParentManager()
NDManager
.NDManager
NDManager newSubManager()
NDManager
.
Child NDManager
will inherit default Device
from this NDManager
.
NDManager
NDManager newSubManager(Device device)
NDManager
with specified default Device
.device
- the default Device
NDManager
Device getDevice()
Device
of this NDManager
.Device
of this NDManager
void attach(java.lang.String resourceId, java.lang.AutoCloseable resource)
NDArray
or NDManager
to this NDManager
.
Attached resource will be closed when this NDManager
is closed.
resourceId
- the unique resourceIdresource
- the AutoCloseable
resource to be attachedvoid detach(java.lang.String resourceId)
NDArray
from this NDManager
's lifecycle.
The detached NDArray
become un-managed, it's user's responsibility to close the
resource. Failed to close the resource has to wait on GC to be freed, and might cause out of
native memory.
resourceId
- the resourceId to be removed from this NDManager
's lifecyclevoid invoke(java.lang.String operation, NDArray[] src, NDArray[] dest, ai.djl.util.PairList<java.lang.String,?> params)
You should avoid using this function if possible. Since this function is engine specific, using this API may cause a portability issue. Native operation may not be compatible between each version.
operation
- the native operation to performsrc
- the NDList
of source NDArray
dest
- the NDList
to save output toparams
- the parameters to be passed to the native operationjava.lang.IllegalArgumentException
- if operation is not supported by EngineEngineException
- if operation failed in native engineNDList invoke(java.lang.String operation, NDList src, ai.djl.util.PairList<java.lang.String,?> params)
You should avoid using this function if possible. Since this function is engine specific, using this API may cause a portability issue. Native operation may not compatible between each version.
operation
- the native operation to performsrc
- the NDList
of source NDArray
params
- the parameters to be passed to the native operationNDArray
java.lang.IllegalArgumentException
- if operation is not supported by EngineEngineException
- if operation failed in native engineEngine getEngine()
Engine
associated with this manager.Engine
associated with this managervoid close()
close
in interface java.lang.AutoCloseable