Haskell’s benevolent influence

2013-06-24

6 min read

Without even a mention of the ‘M’ word!

One of the things that attracted me to SPA 2013 was the Neural Net workshop on the Sunday. It turned out to be even better than I hoped; although we didn’t get quite as far as we could have done.

The first thing we wanted to do was implement a Neuron. In this particular case, a neuron took two inputs (let’s say they are doubles, for the time being) and then performs a computation on those inputs to determine a single output.

In this case, our neuron’s computation was as follows:

output = inputOne * weightOne + inputTwo * weightTwo > threshold ? 1.0 : 0.0

My pair and I quickly put together an implementation of this in Java that looked a bit like this:

package net.digihippo;

public class NeuronImpl implements Neuron {
    private final double threshold;
    private final double[] weights;

    public NeuronImpl(double threshold, double[] weights) {
        this.threshold = threshold;
        this.weights = weights;
    }

    @Override
    public double run(double[] inputs) {
        double sum = 0.0;
        for (int i = 0; i < weights.length; ++i) {
            sum += (weights[i] * inputs[i]);
        }
        return sum > threshold ? 1.0 : 0.0;
    }
}

You could obviously implement this as a static function, but suitable hints were given that the threshold and weights were properties of a given neuron, and so fields it was.

Many connected neurons – a neural network

To cut a long story short; we eventually want to plumb several neurons together to form a network. A single neuron is limited in the functions it can express, for reasons I won’t explore in this post; networks of them do not suffer in the same way.

Handily, once the weight and threshold of a given neuron are fields, the call to a neuron matches that to a network of neurons. But we needed a way to plumb a tree of neurons together, like so:

inputOne ->|          |
           | neuron 1 |_  
inputTwo ->|          | _|          |
                          | neuron 3 |---> output
inputOne ->|          |  _|          |
           | neuron 2 |_/ 
inputTwo ->|          |

N.B Each neuron on the far left of the network is passed the same pair of inputs.

We can see that each Neuron either has precisely two ancestors or none at all (proof by terrible ASCII picture + some induction). A light, well, actually it was more a series of small, excited explosions, went off in my head. Now I just needed to remember that Node a Tree a Tree a magic and I would be laughing. My pair was working in Java, however, and somewhere between the magic and the .java files being edited in Eclipse, something wasn’t working. I needed to get it onto the screen before I could translate it across. Ten minutes or so later, I had this Haskell version of Neuron (which incorporates the concept of a network):

data Neuron = Neuron Double Double Double
            | NeuralNetwork Neuron Neuron Neuron
              deriving (Show)

runNet :: Double -> Double -> Neuron -> Double
runNet i1 i2 (Neuron t w1 w2) = bool2doub $ (i1 * w1) + (i2 * w2) > t
runNet i1 i2 (NeuralNetwork n n1 n2) = runNet n (runNet i1 i2 n1) (runNet i1 i2 n2)

bool2doub :: Bool -> Double
bool2doub True = 1.0
bool2doub False = 0.0

Side note: Feedback on any Haskell naïveté within this post would be greatly appreciated!

As you can see we have a neat, recursively defined, data type, and we defer any actual neuron run until we reach a “leaf” (which might have been a better name for the first constructor) at the far left of the ASCII diagram from before. It should be possible to translate this to Java without too much difficulty; we can map the data type Neuron to an interface, and the two constructors to two implementations of that interface.

Compiling Haskell to Java via human

The sharper eyed readers will have noticed that we were on the way to doing this already in the Java source; NeuronImpl implements Neuron. An accidental bit of foreshadowing there, apologies. So, our NeuronImpl doesn’t have to change, but what does NeuralNetwork look like?

N.B Apologies for the use of above and below – they are extremely poor names for the two parent neurons.

package net.digihippo;

public class NeuralNetwork implements Neuron {
    private final Neuron neuron;
    private final Neuron above;
    private final Neuron below;

    public NeuralNetwork(Neuron neuron, Neuron above, Neuron below) {
        this.neuron = neuron;
        this.above = above;
        this.below = below;
    }

    @Override
    public double run(double[] inputs) {
        return neuron.run(new double[]{above.run(inputs), below.run(inputs)});
    }
}

Remarkably similar, it turns out! The Haskell version is somewhat terser, but the idea appears to have survived translation. It’s quite pleasing to know that lessons from Haskell can (albeit in this simple example) be applied in less fun languages. One wonders if there are places where we cannot translate Haskell data types and their constructors into interfaces and their implementations.

A new requirement arrives

Two inputs? Bah! We laugh in the face of two inputs. We want to be able to deal with n inputs. N.B This means that each level of the network has n times more neurons than the next. Perhaps this might be where our Haskell -> Java translation starts to creak? Let’s add the idea of multiple inputs to our Haskell implementation.

data Neuron = Neuron Double [Double]
            | NeuralNetwork Neuron [Neuron]
              deriving (Show)

runNet :: [Double] -> Neuron -> Double
runNet inputs (Neuron t weights) = bool2doub $ (sum (zipWith (*) inputs weights)) > t
runNet inputs (NeuralNetwork n ns) = runNet (map (runNet inputs) ns) n

Nice and simple – if anything, the variable input version is actually simpler than the fixed, two input version. The use of sum (zipWith ... is particularly pleasing, to my eye. Let’s run it through the mangler and see what we get out in Java…

package net.digihippo;

public class NeuralNetwork implements Neuron {
    private final Neuron neuron;
    private final Neuron[] parents;

    public NeuralNetwork(Neuron neuron, Neuron[] parents) {
        this.neuron = neuron;
        this.parents = parents;
    }

    @Override
    public double run(double[] inputs) {
        final double[] results = new double[inputs.length];
        for (int i = 0; i < inputs.length; ++i) results[i] = parents[i].run(inputs);
        return neuron.run(results);
    }
}

Notbad