ai.djl.nn.convolutional

Class Convolution

java.lang.Object

ai.djl.nn.AbstractBlock

ai.djl.nn.convolutional.Convolution

All Implemented Interfaces:

Block

Direct Known Subclasses:

Conv1d, Conv2d, Conv3d

public abstract class Convolution
extends AbstractBlock

A convolution layer does a dot product calculation on each channel of \(k\)-channel input data by specified number of filters, each containing \(k\) kernels for calculating each channel in the input data and then summed per filter, hence the number of filters denote the number of output channels of a convolution layer. Some modifications may be set on a convolution layer, namely stride which shows the distance between each convolved input data in a channel, and padding which shows the preservation of input size (width and/or height and/or depth) by adding specified padding to the sides of the output. A convolution layer extracts features of input data with different representations where each representation lies per channel in the output, often known as feature map or feature vector.

While convolution process itself has been around for quite some time in mathematics, in 1998 LeCun et al. implemented the very first convolution layers forming a network called LeNet-5 for character recognition task; details of the network's implementation can be find in LeNet-5's paper. When other approaches at that time used handcrafted features with external stage of feature extraction, convolution layer performed feature extraction on its own with no human interference. This marks a new era of machine-extracted features, but it was not until 2012 that the published paper of AlexNet marked the beginning of convolutional neural networks, which by the name itself heavily relies on convolution layer.

Convolution layer is usually used in image-related tasks due to its well-renowned performance as shown by existing works and currently, other non-image-related fields of study are beginning to incorporate convolution layer as an addition or replacement of previous approaches, with one example being time series processing with 1-dimensional convolution layer. Due to the nature of convolution that processes all points in the input data, it is computationally expensive, hence the use of GPU is strongly recommended for faster performance as opposed to using CPU. Note that it is also common to stack convolution layers with different output channels for more representations of the input data.

Current implementations of Convolution are Conv1d with input dimension of LayoutType.WIDTH, Conv2d with input dimension of LayoutType.WIDTH and LayoutType.HEIGHT, and lastly Conv3d with input dimension of LayoutType.WIDTH, LayoutType.HEIGHT, and LayoutType.DEPTH. These implementations share the same core principal as a Convolution layer does, with the difference being the number of input dimension each operates on as denoted by ConvXD for X dimension(s).

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	Convolution.ConvolutionBuilder<T extends Convolution.ConvolutionBuilder> A builder that can build any Convolution block.

Field Summary

Fields

Modifier and Type	Field and Description
protected Parameter	bias
protected Shape	dilation
protected int	filters
protected int	groups
protected boolean	includeBias
protected Shape	kernelShape
protected Shape	padding
protected Shape	stride
protected Parameter	weight

Fields inherited from class ai.djl.nn.AbstractBlock

children, inputNames, inputShapes, parameters, parameterShapeCallbacks, version

Constructor Summary

Constructors

Constructor and Description
Convolution(Convolution.ConvolutionBuilder<?> builder) Creates a Convolution object.

Method Summary

Modifier and Type	Method and Description
protected void	beforeInitialize(Shape[] inputs) Performs any action necessary before initialization.
NDList	forward(ParameterStore parameterStore, NDList inputs, boolean training, ai.djl.util.PairList<java.lang.String,java.lang.Object> params) Applies the operating function of the block once.
protected abstract LayoutType[]	getExpectedLayout() Returns the expected layout of the input.
Shape[]	getOutputShapes(NDManager manager, Shape[] inputs) Returns the expected output shapes of the block for the specified input shapes.
protected abstract java.lang.String	getStringLayout() Returns the string representing the layout of the input.
void	loadMetadata(byte version, java.io.DataInputStream is) Overwrite this to load additional metadata with the parameter values.
protected abstract int	numDimensions() Returns the number of dimensions of the input.

Methods inherited from class ai.djl.nn.AbstractBlock

addChildBlock, addParameter, addParameter, addParameter, cast, clear, describeInput, getChildren, getDirectParameters, getParameters, getParameterShape, initialize, initializeChildBlocks, isInitialized, loadParameters, readInputShapes, saveInputShapes, saveMetadata, saveParameters, setInitializer, setInitializer, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface ai.djl.nn.Block

forward, forward, validateLayout

Field Detail

kernelShape

protected Shape kernelShape

stride

protected Shape stride

padding

protected Shape padding

dilation

protected Shape dilation

filters

protected int filters

groups

protected int groups

includeBias

protected boolean includeBias

weight

protected Parameter weight

bias

protected Parameter bias

Constructor Detail

Convolution

public Convolution(Convolution.ConvolutionBuilder<?> builder)

Creates a Convolution object.

Parameters:

builder - the Builder that has the necessary configurations

Method Detail

getExpectedLayout

protected abstract LayoutType[] getExpectedLayout()

Returns the expected layout of the input.

Returns:

the expected layout of the input

getStringLayout

protected abstract java.lang.String getStringLayout()

Returns the string representing the layout of the input.

Returns:

the string representing the layout of the input

numDimensions

protected abstract int numDimensions()

Returns the number of dimensions of the input.

Returns:

the number of dimensions of the input

forward

public NDList forward(ParameterStore parameterStore,
                      NDList inputs,
                      boolean training,
                      ai.djl.util.PairList<java.lang.String,java.lang.Object> params)

Applies the operating function of the block once. This method should be called only on blocks that are initialized.

Parameters:

parameterStore - the parameter store

inputs - the input NDList

training - true for a training forward pass

params - optional parameters

Returns:

the output of the forward pass

beforeInitialize

protected void beforeInitialize(Shape[] inputs)

Performs any action necessary before initialization.

Overrides:

beforeInitialize in class AbstractBlock

Parameters:

inputs - the expected shapes of the input

getOutputShapes

public Shape[] getOutputShapes(NDManager manager,
                               Shape[] inputs)

Returns the expected output shapes of the block for the specified input shapes.

Parameters:

manager - an NDManager

inputs - the shapes of the inputs

Returns:

the expected output shapes of the block

loadMetadata

public void loadMetadata(byte version,
                         java.io.DataInputStream is)
                  throws java.io.IOException,
                         MalformedModelException

Overwrite this to load additional metadata with the parameter values.

If you overwrite AbstractBlock.saveMetadata(DataOutputStream) or need to provide backward compatibility to older binary formats, you prabably need to overwrite this. This default implementation checks if the version number fits, if not it throws an MalformedModelException. After that it restores the input shapes.

Overrides:

loadMetadata in class AbstractBlock

Parameters:

version - the version used for loading this metadata.

is - the input stream we are loading from

Throws:

java.io.IOException - loading failed

MalformedModelException - data can be loaded but has wrong format