Without even a mention of the ‘M’ word!

One of the things that attracted me to SPA 2013 was the Neural Net workshop on the Sunday. It turned out to be even better than I hoped; although we didn’t get quite as far as we could have done.

The first thing we wanted to do was implement a Neuron. In this particular case, a neuron took two inputs (let’s say they are doubles, for the time being) and then performs a computation on those inputs to determine a single output.

In this case, our neuron’s computation was as follows:

```
output = inputOne * weightOne + inputTwo * weightTwo > threshold ? 1.0 : 0.0
```

My pair and I quickly put together an implementation of this in Java that looked a bit like this:

```
package net.digihippo;
public class NeuronImpl implements Neuron {
private final double threshold;
private final double[] weights;
public NeuronImpl(double threshold, double[] weights) {
this.threshold = threshold;
this.weights = weights;
}
@Override
public double run(double[] inputs) {
double sum = 0.0;
for (int i = 0; i < weights.length; ++i) {
sum += (weights[i] * inputs[i]);
}
return sum > threshold ? 1.0 : 0.0;
}
}
```

You could obviously implement this as a static function, but suitable hints were given that the threshold and weights were properties of a given neuron, and so fields it was.

## Many connected neurons – a neural network

To cut a long story short; we eventually want to plumb several neurons together to form a network. A single neuron is limited in the functions it can express, for reasons I won’t explore in this post; networks of them do not suffer in the same way.

Handily, once the weight and threshold of a given neuron are fields, the call to a neuron matches that to a network of neurons. But we needed a way to plumb a tree of neurons together, like so:

```
inputOne ->| |
| neuron 1 |_
inputTwo ->| | _| |
| neuron 3 |---> output
inputOne ->| | _| |
| neuron 2 |_/
inputTwo ->| |
```

**N.B** Each neuron on the far left of the network is passed the same
pair of inputs.

We can see that each Neuron either has precisely two ancestors or none
at all (proof by terrible ASCII picture + some induction). A light,
well, actually it was more a series of small, excited explosions, went
off in my head. Now I just needed to remember that
`Node a Tree a Tree a`

magic and I would be laughing. My pair was
working in Java, however, and somewhere between the magic and the .java
files being edited in Eclipse, something wasn’t working. I needed to get
it onto the screen before I could translate it across. Ten minutes or so
later, I had this Haskell version of Neuron (which incorporates the
concept of a network):

```
data Neuron = Neuron Double Double Double
| NeuralNetwork Neuron Neuron Neuron
deriving (Show)
runNet :: Double -> Double -> Neuron -> Double
runNet i1 i2 (Neuron t w1 w2) = bool2doub $ (i1 * w1) + (i2 * w2) > t
runNet i1 i2 (NeuralNetwork n n1 n2) = runNet n (runNet i1 i2 n1) (runNet i1 i2 n2)
bool2doub :: Bool -> Double
bool2doub True = 1.0
bool2doub False = 0.0
```

Side note: Feedback on any Haskell naïveté within this post would be greatly appreciated!

As you can see we have a neat, recursively defined, data type, and we
defer any actual neuron `run`

until we reach a “leaf” (which might have
been a better name for the first constructor) at the far left of the
ASCII diagram from before. It should be possible to translate this to
Java without too much difficulty; we can map the *data type* `Neuron`

to
an *interface*, and the two *constructors* to two *implementations* of
that interface.

## Compiling Haskell to Java via human

The sharper eyed readers will have noticed that we were on the way to
doing this already in the Java source; `NeuronImpl implements Neuron`

.
An accidental bit of foreshadowing there, apologies. So, our
`NeuronImpl`

doesn’t have to change, but what does `NeuralNetwork`

look
like?

**N.B** Apologies for the use of `above`

and `below`

– they are
extremely poor names for the two parent neurons.

```
package net.digihippo;
public class NeuralNetwork implements Neuron {
private final Neuron neuron;
private final Neuron above;
private final Neuron below;
public NeuralNetwork(Neuron neuron, Neuron above, Neuron below) {
this.neuron = neuron;
this.above = above;
this.below = below;
}
@Override
public double run(double[] inputs) {
return neuron.run(new double[]{above.run(inputs), below.run(inputs)});
}
}
```

Remarkably similar, it turns out! The Haskell version is somewhat terser, but the idea appears to have survived translation. It’s quite pleasing to know that lessons from Haskell can (albeit in this simple example) be applied in less fun languages. One wonders if there are places where we cannot translate Haskell data types and their constructors into interfaces and their implementations.

## A new requirement arrives

Two inputs? Bah! We laugh in the face of two inputs. We want to be able
to deal with `n`

inputs. N.B This means that each level of the network
has `n`

times more neurons than the next. Perhaps this might be where
our Haskell -> Java translation starts to creak? Let’s add the idea of
multiple inputs to our Haskell implementation.

```
data Neuron = Neuron Double [Double]
| NeuralNetwork Neuron [Neuron]
deriving (Show)
runNet :: [Double] -> Neuron -> Double
runNet inputs (Neuron t weights) = bool2doub $ (sum (zipWith (*) inputs weights)) > t
runNet inputs (NeuralNetwork n ns) = runNet (map (runNet inputs) ns) n
```

Nice and simple – if anything, the variable input version is actually
simpler than the fixed, two input version. The use of `sum (zipWith ...`

is particularly pleasing, to my eye. Let’s run it through the mangler
and see what we get out in Java…

```
package net.digihippo;
public class NeuralNetwork implements Neuron {
private final Neuron neuron;
private final Neuron[] parents;
public NeuralNetwork(Neuron neuron, Neuron[] parents) {
this.neuron = neuron;
this.parents = parents;
}
@Override
public double run(double[] inputs) {
final double[] results = new double[inputs.length];
for (int i = 0; i < inputs.length; ++i) results[i] = parents[i].run(inputs);
return neuron.run(results);
}
}
```

## Conclusion

Translating a well meaning Haskell implementation into Java:

- can work
- might even be a good idea.
- involves writing some Haskell, and is therefore good for you.

## A horrible warning

There is recursion here. Mind how you go.