Safe Haskell	None
Language	Haskell2010

ML.NN

Contents

Data types
Network initialization
Running networks
Training networks
Cost function derivatives
Internal (exposed for testing)

Description

Simple neural network implementation, for learning purposes only.

Synopsis

Data types

type ActivationFunction = R -> R Source

An activation function for a neuron.

type ActivationFunctionDerivative = R -> R Source

The derivative of a neuron activation function.

data Layer Source

A layer in a neural network is a bias vector and weight matrix.

Let n be the number of neurons in the layer and m be the number of inputs to the layer. Then layerBiases is a vector in Rⁿ and layerWeights is an nxm real matrix.

Constructors

Layer
Fields layerBiases :: !(Vector R) A vector in Rⁿ representing the neuron biases. layerWeights :: !(Matrix R) An nxm real matrix of the neuron weights.

Instances

Read Layer Source
Show Layer Source

data Network Source

A network is a list of layers.

Constructors

Network
Fields networkLayers :: [Layer]

Instances

Read Network Source
Show Network Source

Network initialization

randLayer Source

Arguments

:: RandomGen g
=> Int	Number of inputs into the layer.
-> Int	Number of neurons in the layer.
-> Rand g Layer	Randomly generated layer.

Generate a randomly initialized layer in a neural network.

randNetwork :: RandomGen g => [Int] -> Rand g Network Source

Generate a random neural network given the size of each layer.

For example, [3,2,4] will generate a neural network with 3 input neurons, 1 hidden layer with 2 neurons and 4 output neurons.

Running networks

feedForward Source

Arguments

:: ActivationFunction	Neuron activation function.
-> Vector R	Input to the layer.
-> Layer	Layer to run.
-> Vector R	Output from the layer.

Feed the output from a previous layer to the next layer.

runNetwork Source

Arguments

:: Network	Network to run.
-> ActivationFunction	Neuron activation function.
-> Vector R	Network input.
-> Vector R	Network output.

Run a neural network.

Training networks

data TrainingConfig Source

Configuration for training a network.

Constructors

TrainingConfig
Fields trainingEta :: R η - Learning rate. trainingActivation :: ActivationFunction trainingActivationDerivative :: ActivationFunctionDerivative trainingCostDerivative :: CostDerivative

data Sample Source

A training sample.

Constructors

Sample
Fields sampleInput :: !(Vector R) sampleExpectedOutput :: !(Vector R)

Instances

Read Sample Source
Show Sample Source

data Gradient Source

The gradient of the network.

Constructors

Gradient
Fields gradientNablaB :: !(Vector (Vector R)) gradientNablaW :: !(Vector (Matrix R))

Instances

Read Gradient Source
Show Gradient Source

type CostDerivative = Vector R -> Vector R -> Vector R Source

A vectorized function which returns ∂Cₓ/∂a.

The first parameter is the output activation, the second parameter is the expected output.

sgd Source

Arguments

:: TrainingConfig
-> Int	epochs
-> Int	mini-batch size
-> Vector Sample
-> Network
-> IO Network

Train the neural network using stochastic gradient descent.

gradientDescentCore :: TrainingConfig -> Vector Sample -> Network -> Network Source

Update the network's weights and biases by applying gradient descent for the given sample input.

Cost function derivatives

mse' :: CostDerivative Source

The derivative of the mean squared error cost function.

Internal (exposed for testing)

computeZsAndAs :: ActivationFunction -> Network -> Vector R -> ([Vector R], [Vector R]) Source

Return z and activation values for each layer in the network.

The values are returned in reverse order for use by the backpropogation algorithm. We the State monad so it's more explicit that the output activation of one layer is the input to the next.