Skip to main content

Unit Testing in Lua

I'm fairly new to unit tests. I'm also fairly new to Lua. I've been working on a C library for embedded realtime audio programming with patchable DSP objects. (Think something like an embeddable Max/~MSP engine). Every DSP object in the graph exposes programmable functionality with the mighty Lua at the core. I really can't give high enough  praise for Lua. At its heart, Lua is a very fast and very small VM implemented in pure ANSI C which makes it trivially portable, and offers very powerful features for building DSLs due to it's powerful metamethods, coroutines, closures, first-class functions and tail calls.  As such, it's very commonly integrated in video games, for game scripting, AI, entity generation etc. etc. On of the big challenges to me is testing the sanity of the Lua engine in this fledgling DSP system.  As most testable programs aren't running embedded in another application, and don't have hard realtime limits, testing this beast was going to be tricky. Or so I thought... I like procrastinating, and don't particularly relish writing unit tests (don't drink the TDD folks' kool-aid, brothers - they're making it up to hide their shame), I decided to go testing-inception, and write a unit testing suite, that's as trivially embeddable as Lua itself. Enter TestingUnit. The source is on Github. Try it out.

Usage

Here's a basic test, mytest.lua:

    mytest = TestingUnit{
        test_am_i_bad_at_maths = function(self)
            self:assert_equal(1, 2) --how many apples?
        end
    }

Most of the expected assertions are provided, (assert_raises, assert_false, assert_calls, assert_in, assert_match) but as it's so simple and compact, extending the TestingUnit table with your own assertions is trivial. Fixtures and expected failures are also fully supported. To simplify matters, the design relies on naming conventions.

    mytest = TestingUnit{
       fixtures = {
           test_am_i_bad_at_maths_expected_failure = {{1,2}, {3,4}, {5, 6}}
       },
        test_am_i_bad_at_maths_expected_failure = function(self, a, b)
            self:assert_equal(a, b) --how many apples?
        end
    }

Member functions with names matching the pattern, 'test*expected*failure' will be marked by the test runner as an expected failure. Additionally, if your TestingUnit['fixtures'][test_method_name] ~= nil, then your test method will be called once for each item in the fixtures array with those values as arguments. Given the test table above, TestingUnit will do something like the following internally:

    mytest:test_am_i_bad_at_maths_expected_failure(1, 2)
    mytest:test_am_i_bad_at_maths_expected_failure(3, 4)
    mytest:test_am_i_bad_at_maths_expected_failure(5, 6)

This test will have 3 expected failures, so the test suite will be considered a success.

Test Discovery

TestingUnit does something a little bastardised (in the Lua world at least), which I'm going to attempt to rationalize. It discovers unit tests in two steps *

  1. First, search the working directory for files that look like tests (hint, they are Lua files with the word 'test' in the filename)
  2. Iterate over all matching files, load()ing each one in turn.
  3. Search the global namespace for tables that match the TestingUnit spec.

The issue here is that there is no per-file scope in Lua as with some languages:

In Lua there are local and global scopes. Variables default to global scope unless they have the local qualifier.

Because of Lua's scoping rules, most Lua programmers are very careful to limit the scope of their variables to be local which prevents pollution of the global namespace. This might feel a bit weird to programmers of other languages (particularly Python folks who are spoiled by the everything is an object mentality), but Lua is different, and in this case different means goodest of heart. So why break convention? In this case it makes writing discoverable tests as simple as possible.  When the unit test is in charge (i.e. it is main())then the onus of test discovery falls on the unit test which means you must defer the test discovery to some other program, which probably won't integrate well with your embedded system. Defining the TestingUnit table globally means test discovery is much simpler and writing a unit test requires almost no boilerplate. Additionally, it becomes trivial to extend testing to your embedded system.

    #include <lauxlib.h>
    #include <lualib.h>

    int 
    run_lua_tests(lua_State *L, char *test_file, const char *test_string)
    {
        int err = 0;
        if(test_file){
            err = luaL_dofile(L, test_file);
            if(err){
                fprintf(stderr, "could not run tests:%s\n", lua_tostring(L, -1));
                return -1;
            }
        }
        if(test_string){
            err = luaL_loadstring(L, test_string);
            if(err || lua_pcall(L, 0, 1, 0)){
                fprintf(stderr, "could not run tests:%s\n", lua_tostring(L, -1));
                return -1;
            }
            err = luaL_checkinteger(L, -1);
            return err;
        }


    int main(void)
    {
        lua_State * L = luaL_newstate();
        luaL_openlibs(L);
        //... Do setup and junk for embedded lua.

        return run_lua_tests(
            L, 
            "testingunit.lua", 
            "return runtests(find_test_files({'mytests/'}, 1, \"*.lua\"))"
        );
    }

Writing unit tests for my embedded Lua programs has now become lot simpler, and I'll now be more likely to actually do it; that's the premise at least. I hope someone else gets some use from this too. Please fork at will.

Edits Sat 14 Feb 2015

  • Some minor syntax improvements since conversion from WordPress (with language detection via pygments
  • "Gihub" "GitHub"