---

8.1.0.7.3. Basic Neural Net Modules

(packages/gblearn2/gb-modules-nn.lsh)

Author(s): Yann LeCun

a bunch of standard compiled modules for building neural net architectures: sigmoids, RBF, fully connected, convolutions, subsampling layers. Each of these classes has methods for fprop, bprop, bbprop, load, save, forget, and setting various parameters.

---

8.1.0.7.3.0. f-layer

(packages/gblearn2/gb-modules-nn.lsh)

full connection between replicable 3D layers the full connection is only across the first dimension of the input and output layer. the other two dimensions are spatial dimensions accross which the weight matrix is shared. This is much more efficient than using a c-layer with a 1x1 convolution and a full-table.

---

8.1.0.7.3.0.0. (new f-layer tin tout si sj sqsh)

[CLASS] (packages/gblearn2/gb-modules-nn.lsh)

makes a new replicable fully-connected layer. tin and tout are the number of units in the input and output. si and sj are the vertical and horizontal sizes used for pre-allocating the internal state variables (they, as well as the output. are automatically resized, but the resizing is more efficient of it is a downsizing).

---

8.1.0.7.3.0.1. (==> f-layer save file)

[MSG] (packages/gblearn2/gb-modules-nn.lsh)

save weights and biases

---

8.1.0.7.3.0.2. (==> f-layer load file)

[MSG] (packages/gblearn2/gb-modules-nn.lsh)

load weights and biases

---

8.1.0.7.3.0.3. (==> f-layer set-squash squash)

[MSG] (packages/gblearn2/gb-modules-nn.lsh)

sets the squashing function to squash . squash must be an instance of the the class idx3-module .

---

8.1.0.7.3.0.4. (==> f-layer forget v p)

[MSG] (packages/gblearn2/gb-modules-nn.lsh)

initialize weights to random values uniformly distributed between -c and c, where c is v/(fanin^(1/p)). reasonnable values for v and p are 2.5 and 2.

---

8.1.0.7.3.0.5. (==> f-layer fprop in out)

[MSG] (packages/gblearn2/gb-modules-nn.lsh)

forward prop with in as input and out as output. in and out must both be idx3-state .

---

8.1.0.7.3.0.6. (==> f-layer bprop in out)

[MSG] (packages/gblearn2/gb-modules-nn.lsh)

backward prop with in as input and out as output. in and out must both be idx3-state .

---

8.1.0.7.3.0.7. (==> f-layer bbprop in out)

[MSG] (packages/gblearn2/gb-modules-nn.lsh)

backward prop of second derivatives with in as input and out as output. in and out must both be idx3-state .

---

8.1.0.7.3.1. e-layer

(packages/gblearn2/gb-modules-nn.lsh)

full connection with Euclidean distance between replicable 3D layers. This is like a layer of RBF WITHOUT NON-LINEAR FUNCTIONS. the output is the square Euclidean distance between the input and the weight the full connection is only across the first dimension of the input and output layer. the other two dimensions are spatial dimensions accross which the weight matrix is shared.

---

8.1.0.7.3.1.0. (new e-layer tin tout prm)

[CLASS] (packages/gblearn2/gb-modules-nn.lsh)

new e-layer (Eucliden distance RBF). tin is the thickness of the input layer, tout is the thickness of the output layer, prm is the parameter.

---

8.1.0.7.3.2. e-layer-sparse

(packages/gblearn2/gb-modules-nn.lsh)

like e-layer, but each RBF is only connected to a subset of the inputs this should be used for "multiple bitmap" outputs.

---

8.1.0.7.3.2.0. (new e-layer-sparse tin off prm)

[CLASS] (packages/gblearn2/gb-modules-nn.lsh)

new e-layer (Eucliden distance RBF). tin is the thickness of the input layer (dimension of prototypes), off is an idx1 with as many elements as there are output. each element contains the offset in the input for the input vector of each output. prm is the parameter.

---

8.1.0.7.3.3. c-layer

(packages/gblearn2/gb-modules-nn.lsh)

convolutional layer module. Performs multiple convolutions between an idx3-state input and an idx3-state output. includes bias and sigmoid.

---

8.1.0.7.3.3.0. (new c-layer ki kj ri rj tbl thick si sj sqsh)

[CLASS] (packages/gblearn2/gb-modules-nn.lsh)

Creates a new convolution layer. ki (int) vertical kernel size. kj (int) horizontal kernel size. ri int) vertical stride (number of pixels by which the kernels are stepped) rj <int) horizontal stride tbl (idx2) N by 2 matrix containing source and destination feature maps for coresponding kernel thick (int) thickness of output layer (number of feature maps) si (int) vertical size for preallocation of internal state sj (int) horizontal size for preallocation of internal state sqsh (idx3-module) a squashing function module that operates on idx3-state. prm and idx1-ddparam from which the parameters will be allocated

---

8.1.0.7.3.4. s-layer

(packages/gblearn2/gb-modules-nn.lsh)

subsampling layer class

---

8.1.0.7.3.4.0. (new s-layer ki kj thick si sj sqsh prm)

[CLASS] (packages/gblearn2/gb-modules-nn.lsh)

Creates a new subsampling layer for convolutional nets ki (int) vertical subsampling ratio. kj (int) horizontal subsampling ratio. thick (int) thickness of output layer (number of feature maps) si (int) vertical size for preallocation of internal state sj (int) horizontal size for preallocation of internal state sqsh (idx3-module) a squashing function module that operates on idx3-state. prm and idx1-ddparam from which the parameters will be allocated