org.nd4j.serde.binary

Class BinarySerde

java.lang.Object

org.nd4j.serde.binary.BinarySerde

public class BinarySerde
extends Object

Created by agibsonccc on 7/1/17.

Constructor Summary

Constructors

Constructor and Description
BinarySerde()

Method Summary

Modifier and Type	Method and Description
static int	byteBufferSizeFor(INDArray arr) Returns the byte buffer size for the given ndarray.
static void	doByteBufferPutCompressed(INDArray arr, ByteBuffer allocated, boolean rewind) Setup the given byte buffer for serialization (note that this is for compressed INDArrays) 4 bytes for rank 4 bytes for data opType shape information codec information data opType
static void	doByteBufferPutUnCompressed(INDArray arr, ByteBuffer allocated, boolean rewind) Setup the given byte buffer for serialization (note that this is for uncompressed INDArrays) 4 bytes int for rank 4 bytes for data opType shape buffer data buffer
static INDArray	readFromDisk(File readFrom) Read an ndarray from disk
static DataBuffer	readShapeFromDisk(File readFrom) This method returns shape databuffer from saved earlier file
static INDArray	toArray(ByteBuffer buffer) Create an ndarray from the unsafe buffer
static INDArray	toArray(ByteBuffer buffer, int offset) Create an ndarray from the unsafe buffer
protected static Pair<INDArray,ByteBuffer>	toArrayAndByteBuffer(ByteBuffer buffer, int offset) Create an ndarray and existing bytebuffer
static ByteBuffer	toByteBuffer(INDArray arr) Convert an ndarray to an unsafe buffer for use by aeron
static void	writeArrayToDisk(INDArray arr, File toWrite) Write an ndarray to disk in binary format
static void	writeArrayToOutputStream(INDArray arr, OutputStream outputStream) Write an array to an output stream.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BinarySerde

public BinarySerde()

Method Detail

toArray

public static INDArray toArray(ByteBuffer buffer,
                               int offset)

Create an ndarray from the unsafe buffer

Parameters:

buffer - the buffer to create the array from

Returns:

the ndarray derived from this buffer

toArray

public static INDArray toArray(ByteBuffer buffer)

Create an ndarray from the unsafe buffer

Parameters:

buffer - the buffer to create the array from

Returns:

the ndarray derived from this buffer

toArrayAndByteBuffer

protected static Pair<INDArray,ByteBuffer> toArrayAndByteBuffer(ByteBuffer buffer,
                                                                int offset)

Create an ndarray and existing bytebuffer

Parameters:

buffer - the buffer to create the arrays from

offset - position in buffer to create the arrays from.

Returns:

the created INDArray and Bytebuffer pair.

toByteBuffer

public static ByteBuffer toByteBuffer(INDArray arr)

Convert an ndarray to an unsafe buffer for use by aeron

Parameters:

arr - the array to convert

Returns:

the unsafebuffer representation of this array

byteBufferSizeFor

public static int byteBufferSizeFor(INDArray arr)

Returns the byte buffer size for the given ndarray. This is an auxillary method for determining the size of the buffer size to allocate for sending an ndarray via the aeron media driver. The math break down for uncompressed is: 2 ints for rank of the array and an ordinal representing the data opType of the data buffer The rest is in order: shape information data buffer The math break down for compressed is: 2 ints for rank and an ordinal representing the data opType for the data buffer The rest is in order: shape information codec information data buffer

Parameters:

arr - the array to compute the size for

Returns:

the size of the byte buffer that was allocated

doByteBufferPutUnCompressed

public static void doByteBufferPutUnCompressed(INDArray arr,
                                               ByteBuffer allocated,
                                               boolean rewind)

Setup the given byte buffer for serialization (note that this is for uncompressed INDArrays) 4 bytes int for rank 4 bytes for data opType shape buffer data buffer

Parameters:

arr - the array to setup

allocated - the byte buffer to setup

rewind - whether to rewind the byte buffer or nt

doByteBufferPutCompressed

public static void doByteBufferPutCompressed(INDArray arr,
                                             ByteBuffer allocated,
                                             boolean rewind)

Setup the given byte buffer for serialization (note that this is for compressed INDArrays) 4 bytes for rank 4 bytes for data opType shape information codec information data opType

Parameters:

arr - the array to setup

allocated - the byte buffer to setup

rewind - whether to rewind the byte buffer or not

writeArrayToOutputStream

public static void writeArrayToOutputStream(INDArray arr,
                                            OutputStream outputStream)

Write an array to an output stream.

Parameters:

arr - the array to write

outputStream - the output stream to write to

writeArrayToDisk

public static void writeArrayToDisk(INDArray arr,
                                    File toWrite)
                             throws IOException

Write an ndarray to disk in binary format

Parameters:

arr - the array to write

toWrite - the file tow rite to

Throws:

IOException - on an I/O exception.

readFromDisk

public static INDArray readFromDisk(File readFrom)
                             throws IOException

Read an ndarray from disk

Parameters:

readFrom - file to read

Returns:

the created INDArray.

Throws:

IOException - on an I/O exception.

readShapeFromDisk

public static DataBuffer readShapeFromDisk(File readFrom)
                                    throws IOException

This method returns shape databuffer from saved earlier file

Parameters:

readFrom - file to read

Returns:

the created databuffer,

Throws:

IOException - on an I/O exception.