Tinyml

Github Source

I wanted to determine what a dependency free embedded scripting language would need to mirror (a fraction) of what pytorch and python do in a machine learning context. Implemented is a tiny ~1000 line recursive descent compiler with basic matrix operators, matrices as types, and unary operators as functions. This yields some interesting concepts, mainly right to left syntaxing that feels like English in nature, eg. assert all okay:

{
    let a = [2][2] { 1 2 3 4 };
    let b = transpose a;
    let okay = b == [2][2] { 1 3 2 4 };
    assert all okay;
}

Matrices can be initialized from file too, or randomly with the ? operator:

let data = [1593][256] { "mlp/train/data.txt"   };
let lbls = [1593][ 10] { "mlp/train/labels.txt" };

...

let w1 = [256][256] { ? -0.1 0.1 };

And naturally, zipping was designed to be a little more ergonomic than python:

for x t : data labels
{
    ...
}

Included in the git link above is a tinyml.c, the interpretor, and in tinyml/mpl/digits.t, an MLP of recognizing hand digits.

Such a framework might benefit embedded contexts where pytorch or python are too-large-a dependency to pull in. Extending the interpretor to include support for CNNs and RNNs is also possible. A machine learning expert would just need to implement unary operators for convolutions (eg. conv) and friends. Value semantics are in tact with the usual lvalue/rvalue approach. Touching on something like xent:

static value_t xent(value_t y, value_t t)
{
    float loss = 0.0f;
    for(int i = 0; i < size(y); i++)
    {
        float p = y.num[i];
        float minimum = 1e-12f;
        if(p < minimum)
        {
            p = minimum;
        }
        loss += -t.num[i] * logf(p);
    }
    value_t out = {};
    out.rows = 1;
    out.cols = 1;
    out.num = new(float, size(out));
    *out.num = loss;
    return out;
}

It is value(s) in and value out, and destruction of lvalue/rvalue semantics filter through a simple check:

static void kill(value_t value)
{
    free(value.num);
}

static void destroy(value_t value)
{
    if(is_rvalue(value))
    {
        kill(value);
    }
}

static value_t parse_xent()
{
    lex_alnum();
    value_t x = parse_p0();
    value_t y = parse_p0();
    value_t out = xent(x, y);
    destroy(x);
    destroy(y);
    return out;
}