org.nd4j.linalg.api.ops.impl.transforms.custom

Class LayerNormBp

java.lang.Object

org.nd4j.autodiff.functions.DifferentialFunction

org.nd4j.linalg.api.ops.DynamicCustomOp

org.nd4j.linalg.api.ops.impl.transforms.custom.LayerNormBp

All Implemented Interfaces:

CustomOp

public class LayerNormBp
extends DynamicCustomOp

Composed op: g*standarize(x) + b Bias is optional, and can be set as null

Author:

Paul Dubs

Nested Class Summary

Nested classes/interfaces inherited from class org.nd4j.linalg.api.ops.DynamicCustomOp

DynamicCustomOp.DynamicCustomOpsBuilder

Field Summary

Fields inherited from class org.nd4j.linalg.api.ops.DynamicCustomOp

axis, bArguments, iArguments, inplaceCall, inputArguments, outputArguments, outputVariables, tArguments

Fields inherited from class org.nd4j.autodiff.functions.DifferentialFunction

dimensions, extraArgs, inPlace, sameDiff, scalarValue

Constructor Summary

Constructors

Constructor and Description
LayerNormBp(INDArray input, INDArray gain, INDArray grad, INDArray dLdx, INDArray dLdg, boolean channelsFirst, int... dimensions)
LayerNormBp(INDArray input, INDArray gain, INDArray bias, INDArray grad, INDArray dLdx, INDArray dLdg, INDArray dLdb, boolean channelsFirst, int... dimensions)
LayerNormBp(SameDiff sameDiff, SDVariable input, SDVariable gain, SDVariable gradient, boolean channelsFirst, int... dimensions)
LayerNormBp(SameDiff sameDiff, SDVariable input, SDVariable gain, SDVariable bias, SDVariable gradient, boolean channelsFirst, int... dimensions)

Method Summary

Modifier and Type	Method and Description
List<DataType>	calculateOutputDataTypes(List<DataType> dataTypes) Calculate the data types for the output arrays.
List<SDVariable>	doDiff(List<SDVariable> grad) The actual implementation for automatic differentiation.
int	getNumOutputs()
String	onnxName() The opName of this function in onnx
String	opName() This method returns op opName as string
void	setDimensions(int[] dimensions)
String	tensorflowName() The opName of this function tensorflow

Methods inherited from class org.nd4j.linalg.api.ops.DynamicCustomOp

addBArgument, addIArgument, addIArgument, addInputArgument, addOutputArgument, addTArgument, assertValidForExecution, bArgs, builder, calculateOutputShape, getBArgument, getDescriptor, getIArgument, getInputArgument, getOutputArgument, getTArgument, iArgs, initFromOnnx, initFromTensorFlow, inputArguments, numBArguments, numIArguments, numInputArguments, numOutputArguments, numTArguments, opHash, opNum, opType, outputArguments, outputVariables, outputVariables, removeIArgument, removeInputArgument, removeOutputArgument, removeTArgument, setInputArgument, setInputArguments, setOutputArgument, tArgs, toString, wrapFilterNull, wrapOrNull

Methods inherited from class org.nd4j.autodiff.functions.DifferentialFunction

arg, arg, argNames, args, attributeAdaptersForFunction, configFieldName, diff, dup, equals, f, getValue, hashCode, isConfigProperties, larg, mappingsForFunction, onnxNames, outputVariable, outputVariablesNames, propertiesForFunction, rarg, replaceArg, resolvePropertiesFromSameDiffBeforeExecution, setInstanceId, setPropertiesForFunction, setValueFor, tensorflowNames

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.nd4j.linalg.api.ops.CustomOp

isInplaceCall

Constructor Detail

LayerNormBp

public LayerNormBp(@NonNull
                   SameDiff sameDiff,
                   @NonNull
                   SDVariable input,
                   @NonNull
                   SDVariable gain,
                   SDVariable bias,
                   @NonNull
                   SDVariable gradient,
                   boolean channelsFirst,
                   int... dimensions)

LayerNormBp

public LayerNormBp(@NonNull
                   INDArray input,
                   @NonNull
                   INDArray gain,
                   INDArray bias,
                   @NonNull
                   INDArray grad,
                   @NonNull
                   INDArray dLdx,
                   @NonNull
                   INDArray dLdg,
                   INDArray dLdb,
                   boolean channelsFirst,
                   int... dimensions)

LayerNormBp

public LayerNormBp(SameDiff sameDiff,
                   SDVariable input,
                   SDVariable gain,
                   SDVariable gradient,
                   boolean channelsFirst,
                   int... dimensions)

LayerNormBp

public LayerNormBp(INDArray input,
                   INDArray gain,
                   INDArray grad,
                   INDArray dLdx,
                   INDArray dLdg,
                   boolean channelsFirst,
                   int... dimensions)

Method Detail

setDimensions

public void setDimensions(int[] dimensions)

opName

public String opName()

Description copied from class: DynamicCustomOp

This method returns op opName as string

Specified by:

opName in interface CustomOp

Overrides:

opName in class DynamicCustomOp

Returns:

tensorflowName

public String tensorflowName()

Description copied from class: DifferentialFunction

The opName of this function tensorflow

Overrides:

tensorflowName in class DynamicCustomOp

Returns:

onnxName

public String onnxName()

Description copied from class: DifferentialFunction

The opName of this function in onnx

Overrides:

onnxName in class DynamicCustomOp

Returns:

doDiff

public List<SDVariable> doDiff(List<SDVariable> grad)

Description copied from class: DifferentialFunction

The actual implementation for automatic differentiation.

Overrides:

doDiff in class DynamicCustomOp

Returns:

calculateOutputDataTypes

public List<DataType> calculateOutputDataTypes(List<DataType> dataTypes)

Description copied from class: DifferentialFunction

Calculate the data types for the output arrays. Though datatypes can also be inferred from DifferentialFunction.calculateOutputShape(), this method differs in that it does not require the input arrays to be populated. This is important as it allows us to do greedy datatype inference for the entire net - even if arrays are not available.

Overrides:

calculateOutputDataTypes in class DifferentialFunction

Parameters:

dataTypes - The data types of the inputs

Returns:

The data types of the outputs

getNumOutputs

public int getNumOutputs()

Overrides:

getNumOutputs in class DifferentialFunction