stupidtools/docs/extensions.md

# Extensions: adding a language without touching core

stupidtools is deliberately not a one-trick C compiler. The core binary
knows only about a generic extension ABI (`src/ext/abi.h`); the C/C++
languages are themselves just the first modules over it (`src/ext/lang_c.c`,
`src/ext/lang_cpp.c`). Any other language — Fortran, Rust, D, a toy DSL —
is added by dropping a Lua module into an extension directory. No core
changes, no recompiling.

The worked example this document walks through is
[`examples/ext_fortran.lua`](../examples/ext_fortran.lua), a complete
third-party Fortran stub. Read that file first; it is heavily commented.

---

## 1. Where extensions live (discovery)

Extension discovery (`src/ext/discovery.h`) looks in this order:

1. **`--ext-dir DIR`** — repeatable command-line flag (`stupidtools
   --ext-dir DIR [--ext-dir DIR2 ...] <buildfile>`).
2. **`STUPIDTOOLS_EXT`** — colon-separated directory list
   (`STUPIDTOOLS_EXT=/a:/b stupidtools <buildfile>`); empty entries are
   skipped.
3. **User directory** — `$XDG_DATA_HOME/stupidtools/ext`, falling back to
   `$HOME/.local/share/stupidtools/ext` when `XDG_DATA_HOME` is unset or
   empty. Skipped entirely when neither variable yields a path.
4. **Builtin directory** — `builtin-ext/` relative to the working
   directory (compile-time configurable via
   `-DSTUPIDTOOLS_BUILTIN_EXT_DIR` for installed layouts).

Scanning rules: only **regular files ending in `.lua`** are loaded
(dotfiles excluded); load order within a directory is bytewise-
lexicographic; a missing or empty directory is a silent no-op; an empty
`.lua` file is a clean no-op. Discovery **fail-fasts**: the first module
that fails to load (syntax error, runtime error, sandbox violation,
unreadable file) aborts with `extension load failed: <path>: <message>`,
and later modules are not attempted.

Each loaded module logs one line built from its **actual** registrations:

```
loaded extension: /tmp/extdemo/ext_fortran.lua — checks: 1 (fortran_compiler_flag) languages: 1 (fortran)
```

## 2. The sandbox contract

Extension code runs in a stripped Lua 5.4 runtime (`src/ext/lua.h`). The
module is **declarative** — it registers capabilities; it cannot execute
or touch the outside world:

| Capability | Status |
| --- | --- |
| `st.register_language`, `st.register_check` | the **only** interface |
| base / table / string / curated `math` / hand-built `os` | available |
| `os.execute`, `os.exit`, `os.remove`, `os.rename` | **present but blocked** — calling them raises `sandbox blocked: ...` and fails discovery |
| `dofile`, `loadfile`, `require`, `package`, `io`, `debug`, `coroutine` | **nil** — their C libraries are not even linked |

A module that violates the sandbox is rejected with a clear load error;
the binary never runs untrusted program execution.

## 3. The registration API

Two functions on the `st` table. Call shapes (exact — see
`src/ext/api.h` and `src/ext/lua.c`):

```lua
st.register_language(name, [spec_table])
st.register_check(name, { kind = "compile"|"link"|"run", source = "<C snippet>", link = { "-lfoo", ... } })
```

- **`register_language(name [, table])`** — registers a language *name*.
  The optional second argument, if present, must be a table; v1 records
  the name only (the table's contents are reserved for future language
  detection).
- **`register_check(name, spec)`** — registers a custom check. `kind` is
  the probe **mode** (compile / link / run), `source` is the C snippet
  the probe compiles, and `link` is an optional list of extra argv
  elements for the link step. The check's *name* becomes a new check kind
  a build file can reference, exactly like the builtin `header` kind.
- Names must be non-empty strings with no NUL bytes and must not collide
  with an already-registered name (builtin or earlier module).

The Lua tables are materialized into a `struct st_lua_probe_spec`
(`src/ext/api.h`): all fields strdup'd, owned by the runtime's registry,
freed with the runtime.

## 4. v1 limitations (documented, deliberate)

- **Custom checks compile C snippets against the C toolchain.** The
  `source` in a probe spec is a *C* snippet driven by the detected C
  compiler (`st_lua_probe_run` in `src/ext/api.h`). There is no generic
  per-language probe runner yet; the Fortran example's check is a C
  probe whose name says "fortran". (This is why the example keeps its
  source libm-free and trivially true.)
- **Lua language detection is a stub.** `register_language` records the
  name only; nothing yet detects a `fortran` compiler on the host, and
  the DSL has no language-scoped feature resolution wired to it. The
  table argument exists so a later release can add
  `register_language("fortran", { detect = ... })` without breaking
  existing modules.
- **The bridge maps every Lua check to the `"c"` language** internally
  (`ST_LUA_CHECK_LANGUAGE` in `src/ext/api.c`) — again, v1 custom checks
  are C snippets, by design.

## 5. Step-by-step: adding a new language

Copy this recipe for any language, not just Fortran.

1. **Write a `.lua` module.** Model it on `examples/ext_fortran.lua`:

   ```lua
   -- mylang.lua
   st.register_language("mylang", { kind = "compile" })
   st.register_check("mylang_compiler_flag", {
       kind   = "compile",
       source = "int main(void) { return 0; }",
       link   = { "-lmylang" },
   })
   ```

   Only `st.*` calls are allowed; no `os.execute`, no `io`, no
   `require` — the sandbox enforces it.

2. **Drop it in a discovery directory.** Either:

   ```sh
   mkdir -p /tmp/extdemo
   cp mylang.lua /tmp/extdemo/
   ./src/stupidtools --ext-dir /tmp/extdemo
   ```

   or:

   ```sh
   STUPIDTOOLS_EXT=/tmp/extdemo ./src/stupidtools
   ```

   or install it to `$XDG_DATA_HOME/stupidtools/ext/` (default user dir).

3. **Check the load line.** You should see your module named with its
   actual registrations:

   ```
   loaded extension: /tmp/extdemo/mylang.lua — checks: 1 (mylang_compiler_flag) languages: 1 (mylang)
   ```

   A malformed module aborts discovery with `extension load failed:
   <path>: <file>:<line>: <message>` — fix and rerun.

4. **Reference the new check kind from a build file.** Once a module has
   registered `mylang_compiler_flag`, a `feature` block in the KDL build
   file can name that kind like any builtin check kind. (Full fixture-
   driven resolution lands with the buildfile pipeline; discovery +
   registration is what this version proves.)

That is the whole contract: **discover → sandbox → register → reference.**
Languages are data to core, and the builtin C/C++ modules are just the
first extensions.