docs: add example extension + extensions guide

docs/extensions.md

@@ -0,0 +1,159 @@
+# Extensions: adding a language without touching core
+
+stupidtools is deliberately not a one-trick C compiler. The core binary
+knows only about a generic extension ABI (`src/ext/abi.h`); the C/C++
+languages are themselves just the first modules over it (`src/ext/lang_c.c`,
+`src/ext/lang_cpp.c`). Any other language — Fortran, Rust, D, a toy DSL —
+is added by dropping a Lua module into an extension directory. No core
+changes, no recompiling.
+
+The worked example this document walks through is
+[`examples/ext_fortran.lua`](../examples/ext_fortran.lua), a complete
+third-party Fortran stub. Read that file first; it is heavily commented.
+
+---
+
+## 1. Where extensions live (discovery)
+
+Extension discovery (`src/ext/discovery.h`) looks in this order:
+
+1. **`--ext-dir DIR`** — repeatable command-line flag (`stupidtools
+   --ext-dir DIR [--ext-dir DIR2 ...] <buildfile>`).
+2. **`STUPIDTOOLS_EXT`** — colon-separated directory list
+   (`STUPIDTOOLS_EXT=/a:/b stupidtools <buildfile>`); empty entries are
+   skipped.
+3. **User directory** — `$XDG_DATA_HOME/stupidtools/ext`, falling back to
+   `$HOME/.local/share/stupidtools/ext` when `XDG_DATA_HOME` is unset or
+   empty. Skipped entirely when neither variable yields a path.
+4. **Builtin directory** — `builtin-ext/` relative to the working
+   directory (compile-time configurable via
+   `-DSTUPIDTOOLS_BUILTIN_EXT_DIR` for installed layouts).
+
+Scanning rules: only **regular files ending in `.lua`** are loaded
+(dotfiles excluded); load order within a directory is bytewise-
+lexicographic; a missing or empty directory is a silent no-op; an empty
+`.lua` file is a clean no-op. Discovery **fail-fasts**: the first module
+that fails to load (syntax error, runtime error, sandbox violation,
+unreadable file) aborts with `extension load failed: <path>: <message>`,
+and later modules are not attempted.
+
+Each loaded module logs one line built from its **actual** registrations:
+
+```
+loaded extension: /tmp/extdemo/ext_fortran.lua — checks: 1 (fortran_compiler_flag) languages: 1 (fortran)
+```
+
+## 2. The sandbox contract
+
+Extension code runs in a stripped Lua 5.4 runtime (`src/ext/lua.h`). The
+module is **declarative** — it registers capabilities; it cannot execute
+or touch the outside world:
+
+| Capability | Status |
+| --- | --- |
+| `st.register_language`, `st.register_check` | the **only** interface |
+| base / table / string / curated `math` / hand-built `os` | available |
+| `os.execute`, `os.exit`, `os.remove`, `os.rename` | **present but blocked** — calling them raises `sandbox blocked: ...` and fails discovery |
+| `dofile`, `loadfile`, `require`, `package`, `io`, `debug`, `coroutine` | **nil** — their C libraries are not even linked |
+
+A module that violates the sandbox is rejected with a clear load error;
+the binary never runs untrusted program execution.
+
+## 3. The registration API
+
+Two functions on the `st` table. Call shapes (exact — see
+`src/ext/api.h` and `src/ext/lua.c`):
+
+```lua
+st.register_language(name, [spec_table])
+st.register_check(name, { kind = "compile"|"link"|"run", source = "<C snippet>", link = { "-lfoo", ... } })
+```
+
+- **`register_language(name [, table])`** — registers a language *name*.
+  The optional second argument, if present, must be a table; v1 records
+  the name only (the table's contents are reserved for future language
+  detection).
+- **`register_check(name, spec)`** — registers a custom check. `kind` is
+  the probe **mode** (compile / link / run), `source` is the C snippet
+  the probe compiles, and `link` is an optional list of extra argv
+  elements for the link step. The check's *name* becomes a new check kind
+  a build file can reference, exactly like the builtin `header` kind.
+- Names must be non-empty strings with no NUL bytes and must not collide
+  with an already-registered name (builtin or earlier module).
+
+The Lua tables are materialized into a `struct st_lua_probe_spec`
+(`src/ext/api.h`): all fields strdup'd, owned by the runtime's registry,
+freed with the runtime.
+
+## 4. v1 limitations (documented, deliberate)
+
+- **Custom checks compile C snippets against the C toolchain.** The
+  `source` in a probe spec is a *C* snippet driven by the detected C
+  compiler (`st_lua_probe_run` in `src/ext/api.h`). There is no generic
+  per-language probe runner yet; the Fortran example's check is a C
+  probe whose name says "fortran". (This is why the example keeps its
+  source libm-free and trivially true.)
+- **Lua language detection is a stub.** `register_language` records the
+  name only; nothing yet detects a `fortran` compiler on the host, and
+  the DSL has no language-scoped feature resolution wired to it. The
+  table argument exists so a later release can add
+  `register_language("fortran", { detect = ... })` without breaking
+  existing modules.
+- **The bridge maps every Lua check to the `"c"` language** internally
+  (`ST_LUA_CHECK_LANGUAGE` in `src/ext/api.c`) — again, v1 custom checks
+  are C snippets, by design.
+
+## 5. Step-by-step: adding a new language
+
+Copy this recipe for any language, not just Fortran.
+
+1. **Write a `.lua` module.** Model it on `examples/ext_fortran.lua`:
+
+   ```lua
+   -- mylang.lua
+   st.register_language("mylang", { kind = "compile" })
+   st.register_check("mylang_compiler_flag", {
+       kind   = "compile",
+       source = "int main(void) { return 0; }",
+       link   = { "-lmylang" },
+   })
+   ```
+
+   Only `st.*` calls are allowed; no `os.execute`, no `io`, no
+   `require` — the sandbox enforces it.
+
+2. **Drop it in a discovery directory.** Either:
+
+   ```sh
+   mkdir -p /tmp/extdemo
+   cp mylang.lua /tmp/extdemo/
+   ./src/stupidtools --ext-dir /tmp/extdemo
+   ```
+
+   or:
+
+   ```sh
+   STUPIDTOOLS_EXT=/tmp/extdemo ./src/stupidtools
+   ```
+
+   or install it to `$XDG_DATA_HOME/stupidtools/ext/` (default user dir).
+
+3. **Check the load line.** You should see your module named with its
+   actual registrations:
+
+   ```
+   loaded extension: /tmp/extdemo/mylang.lua — checks: 1 (mylang_compiler_flag) languages: 1 (mylang)
+   ```
+
+   A malformed module aborts discovery with `extension load failed:
+   <path>: <file>:<line>: <message>` — fix and rerun.
+
+4. **Reference the new check kind from a build file.** Once a module has
+   registered `mylang_compiler_flag`, a `feature` block in the KDL build
+   file can name that kind like any builtin check kind. (Full fixture-
+   driven resolution lands with the buildfile pipeline; discovery +
+   registration is what this version proves.)
+
+That is the whole contract: **discover → sandbox → register → reference.**
+Languages are data to core, and the builtin C/C++ modules are just the
+first extensions.

examples/ext_fortran.lua

@@ -0,0 +1,103 @@
+-- ext_fortran.lua -- an example third-party language extension.
+--
+-- WHAT THIS MODULE DOES
+--
+-- Fortran is NOT built into stupidtools: the builtin modules are only the
+-- C/C++ language pair (src/ext/lang_c.c / src/ext/lang_cpp.c). This file
+-- shows how a third party adds another language -- and a language-specific
+-- compiler-flag check -- without touching core or recompiling the binary.
+--
+-- HOW DISCOVERY LOADS IT
+--
+-- Any regular file whose name ends in ".lua" is discovered (src/ext/
+-- discovery.h), from any of these sources, in this order:
+--
+--   1. each --ext-dir DIR command-line flag (repeatable);
+--   2. each directory in the colon-separated STUPIDTOOLS_EXT env var;
+--   3. the per-user directory: $XDG_DATA_HOME/stupidtools/ext, falling
+--      back to $HOME/.local/share/stupidtools/ext;
+--   4. the builtin directory ("builtin-ext", compile-time configured).
+--
+-- Each module is read into memory and run as ONE sandboxed Lua chunk with
+-- the full file path as its chunkname (so diagnostics name file AND line).
+-- Load order within a directory is bytewise-lexicographic, and a module
+-- that fails to load aborts discovery (fail-fast). Try it:
+--
+--     cp examples/ext_fortran.lua /tmp/extdemo/
+--     ./src/stupidtools --ext-dir /tmp/extdemo
+--
+--   or, equivalently:
+--
+--     STUPIDTOOLS_EXT=/tmp/extdemo ./src/stupidtools
+--
+-- Discovery prints one line per loaded module, built from the ACTUAL
+-- registrations the module performed:
+--
+--     loaded extension: /tmp/extdemo/ext_fortran.lua - checks: 1
+--     (fortran_compiler_flag) languages: 1 (fortran)
+--
+-- THE SANDBOX CONTRACT (what a module CANNOT do)
+--
+-- The runtime is deliberately stripped (src/ext/lua.h). Extension code
+-- runs with:
+--   * os.execute / os.exit / os.remove / os.rename PRESENT but raising
+--     a "sandbox blocked:" error when called;
+--   * dofile / loadfile / require / package / io / debug / coroutine
+--     REMOVED (the globals are nil; their C libraries are not even
+--     linked);
+--   * only the base, table, string, a curated libm-free `math`, and the
+--     hand-built `os` tables available, plus the `st` registration table.
+--
+-- So a module can declare capabilities but cannot execute programs, touch
+-- the filesystem, or load other code. Violating the sandbox fails
+-- discovery with a clear error. Note the probe SOURCE strings below are
+-- inert text compiled by the host C toolchain, not executed here.
+--
+-- THE EXTENSION POINTS (the only API)
+--
+--   st.register_language(name [, spec-table])
+--       Registers a language. v1 registers the NAME only; the optional
+--       table is accepted (and must BE a table) for forward compatibility
+--       -- no language detection or toolchain is wired to it yet (see
+--       "v1 limitations" below).
+--
+--   st.register_check(name, { kind=..., source=..., link={...} })
+--       Registers a custom check. `kind` is the probe MODE
+--       ("compile" | "link" | "run"), `source` is the C snippet the probe
+--       compiles, and `link` (optional) is a list of extra argv elements
+--       for the link step. v1 custom checks are C snippets compiled with
+--       the detected C toolchain (see "v1 limitations" below).
+--
+-- Both names must be non-empty strings free of NUL bytes and must not
+-- collide with an already-registered name (builtin or earlier module).
+
+-- ---------------------------------------------------------------------------
+-- 1. Register the language itself. "fortran" is arbitrary from core's
+--    point of view -- any name works. The table argument is validated
+--    (must be a table) but its contents are NOT consumed in v1.
+-- ---------------------------------------------------------------------------
+st.register_language("fortran", { kind = "compile" })
+
+-- ---------------------------------------------------------------------------
+-- 2. Register a language-specific check. The name becomes a new check kind
+--    a build file can reference (like the builtin "header" kind is). The
+--    probe spec says: compile `source` with the C toolchain and require
+--    the compile (and, with `link`, the link) to succeed.
+--
+--    The source below is deliberately trivially-true C and libm-free: it
+--    links nothing, calls nothing, includes nothing -- a probe that must
+--    pass even on the smallest toolchain. A realistic flag probe would
+--    compile a C snippet with the flag in question and FAIL when the
+--    compiler rejects the flag. The `-lfortran` link argument is inert
+--    while the probe compiles (not links) but shows how extra link argv
+--    is carried through for "link"/"run" mode probes.
+-- ---------------------------------------------------------------------------
+st.register_check("fortran_compiler_flag", {
+    kind   = "compile",
+    source = "int main(void) { return 0; }",
+    link   = { "-lfortran" },
+})
+
+-- That is the whole module. Everything else -- detection, probing, the
+-- generated ./configure -- is core's job; an extension only declares
+-- what languages and checks exist.