Extensions: adding a language without touching core

stupidtools is deliberately not a one-trick C compiler. The core binary knows only about a generic extension ABI (src/ext/abi.h); the C/C++ languages are themselves just the first modules over it (src/ext/lang_c.c, src/ext/lang_cpp.c). Any other language — Fortran, Rust, D, a toy DSL — is added by dropping a Lua module into an extension directory. No core changes, no recompiling.

The worked example this document walks through is examples/ext_fortran.lua, a complete third-party Fortran stub. Read that file first; it is heavily commented.

1. Where extensions live (discovery)

Extension discovery (src/ext/discovery.h) looks in this order:

--ext-dir DIR — repeatable command-line flag (stupidtools --ext-dir DIR [--ext-dir DIR2 ...] <buildfile>).
STUPIDTOOLS_EXT — colon-separated directory list (STUPIDTOOLS_EXT=/a:/b stupidtools <buildfile>); empty entries are skipped.
User directory — $XDG_DATA_HOME/stupidtools/ext, falling back to $HOME/.local/share/stupidtools/ext when XDG_DATA_HOME is unset or empty. Skipped entirely when neither variable yields a path.
Builtin directory — builtin-ext/ relative to the working directory (compile-time configurable via -DSTUPIDTOOLS_BUILTIN_EXT_DIR for installed layouts).

Scanning rules: only regular files ending in .lua are loaded (dotfiles excluded); load order within a directory is bytewise- lexicographic; a missing or empty directory is a silent no-op; an empty .lua file is a clean no-op. Discovery fail-fasts: the first module that fails to load (syntax error, runtime error, sandbox violation, unreadable file) aborts with extension load failed: <path>: <message>, and later modules are not attempted.

Each loaded module logs one line built from its actual registrations:

loaded extension: /tmp/extdemo/ext_fortran.lua — checks: 1 (fortran_compiler_flag) languages: 1 (fortran)

2. The sandbox contract

Extension code runs in a stripped Lua 5.4 runtime (src/ext/lua.h). The module is declarative — it registers capabilities; it cannot execute or touch the outside world:

Capability	Status
`st.register_language`, `st.register_check`	the only interface
base / table / string / curated `math` / hand-built `os`	available
`os.execute`, `os.exit`, `os.remove`, `os.rename`	present but blocked — calling them raises `sandbox blocked: ...` and fails discovery
`dofile`, `loadfile`, `require`, `package`, `io`, `debug`, `coroutine`	nil — their C libraries are not even linked

A module that violates the sandbox is rejected with a clear load error; the binary never runs untrusted program execution.

3. The registration API

Two functions on the st table. Call shapes (exact — see src/ext/api.h and src/ext/lua.c):

st.register_language(name, [spec_table])
st.register_check(name, { kind = "compile"|"link"|"run", source = "<C snippet>", link = { "-lfoo", ... } })

register_language(name [, table]) — registers a language name. The optional second argument, if present, must be a table; v1 records the name only (the table's contents are reserved for future language detection).
register_check(name, spec) — registers a custom check. kind is the probe mode (compile / link / run), source is the C snippet the probe compiles, and link is an optional list of extra argv elements for the link step. The check's name becomes a new check kind a build file can reference, exactly like the builtin header kind.
Names must be non-empty strings with no NUL bytes and must not collide with an already-registered name (builtin or earlier module).

The Lua tables are materialized into a struct st_lua_probe_spec (src/ext/api.h): all fields strdup'd, owned by the runtime's registry, freed with the runtime.

4. v1 limitations (documented, deliberate)

Custom checks compile C snippets against the C toolchain. The source in a probe spec is a C snippet driven by the detected C compiler (st_lua_probe_run in src/ext/api.h). There is no generic per-language probe runner yet; the Fortran example's check is a C probe whose name says "fortran". (This is why the example keeps its source libm-free and trivially true.)
Lua language detection is a stub. register_language records the name only; nothing yet detects a fortran compiler on the host, and the DSL has no language-scoped feature resolution wired to it. The table argument exists so a later release can add register_language("fortran", { detect = ... }) without breaking existing modules.
The bridge maps every Lua check to the "c" language internally (ST_LUA_CHECK_LANGUAGE in src/ext/api.c) — again, v1 custom checks are C snippets, by design.

5. Step-by-step: adding a new language

Copy this recipe for any language, not just Fortran.

Write a .lua module. Model it on examples/ext_fortran.lua:

-- mylang.lua
st.register_language("mylang", { kind = "compile" })
st.register_check("mylang_compiler_flag", {
    kind   = "compile",
    source = "int main(void) { return 0; }",
    link   = { "-lmylang" },
})

Only st.* calls are allowed; no os.execute, no io, no require — the sandbox enforces it.

Drop it in a discovery directory. Either:

mkdir -p /tmp/extdemo
cp mylang.lua /tmp/extdemo/
./src/stupidtools --ext-dir /tmp/extdemo

or:

STUPIDTOOLS_EXT=/tmp/extdemo ./src/stupidtools

or install it to $XDG_DATA_HOME/stupidtools/ext/ (default user dir).

Check the load line. You should see your module named with its actual registrations:
```
loaded extension: /tmp/extdemo/mylang.lua — checks: 1 (mylang_compiler_flag) languages: 1 (mylang)
```
A malformed module aborts discovery with extension load failed: <path>: <file>:<line>: <message> — fix and rerun.
Reference the new check kind from a build file. Once a module has registered mylang_compiler_flag, a feature block in the KDL build file can name that kind like any builtin check kind. (Full fixture- driven resolution lands with the buildfile pipeline; discovery + registration is what this version proves.)

That is the whole contract: discover → sandbox → register → reference. Languages are data to core, and the builtin C/C++ modules are just the first extensions.

6.5 KiB Raw Blame History