Template
docs: add example extension + extensions guide
This commit is contained in:
@@ -0,0 +1,159 @@
|
||||
# Extensions: adding a language without touching core
|
||||
|
||||
stupidtools is deliberately not a one-trick C compiler. The core binary
|
||||
knows only about a generic extension ABI (`src/ext/abi.h`); the C/C++
|
||||
languages are themselves just the first modules over it (`src/ext/lang_c.c`,
|
||||
`src/ext/lang_cpp.c`). Any other language — Fortran, Rust, D, a toy DSL —
|
||||
is added by dropping a Lua module into an extension directory. No core
|
||||
changes, no recompiling.
|
||||
|
||||
The worked example this document walks through is
|
||||
[`examples/ext_fortran.lua`](../examples/ext_fortran.lua), a complete
|
||||
third-party Fortran stub. Read that file first; it is heavily commented.
|
||||
|
||||
---
|
||||
|
||||
## 1. Where extensions live (discovery)
|
||||
|
||||
Extension discovery (`src/ext/discovery.h`) looks in this order:
|
||||
|
||||
1. **`--ext-dir DIR`** — repeatable command-line flag (`stupidtools
|
||||
--ext-dir DIR [--ext-dir DIR2 ...] <buildfile>`).
|
||||
2. **`STUPIDTOOLS_EXT`** — colon-separated directory list
|
||||
(`STUPIDTOOLS_EXT=/a:/b stupidtools <buildfile>`); empty entries are
|
||||
skipped.
|
||||
3. **User directory** — `$XDG_DATA_HOME/stupidtools/ext`, falling back to
|
||||
`$HOME/.local/share/stupidtools/ext` when `XDG_DATA_HOME` is unset or
|
||||
empty. Skipped entirely when neither variable yields a path.
|
||||
4. **Builtin directory** — `builtin-ext/` relative to the working
|
||||
directory (compile-time configurable via
|
||||
`-DSTUPIDTOOLS_BUILTIN_EXT_DIR` for installed layouts).
|
||||
|
||||
Scanning rules: only **regular files ending in `.lua`** are loaded
|
||||
(dotfiles excluded); load order within a directory is bytewise-
|
||||
lexicographic; a missing or empty directory is a silent no-op; an empty
|
||||
`.lua` file is a clean no-op. Discovery **fail-fasts**: the first module
|
||||
that fails to load (syntax error, runtime error, sandbox violation,
|
||||
unreadable file) aborts with `extension load failed: <path>: <message>`,
|
||||
and later modules are not attempted.
|
||||
|
||||
Each loaded module logs one line built from its **actual** registrations:
|
||||
|
||||
```
|
||||
loaded extension: /tmp/extdemo/ext_fortran.lua — checks: 1 (fortran_compiler_flag) languages: 1 (fortran)
|
||||
```
|
||||
|
||||
## 2. The sandbox contract
|
||||
|
||||
Extension code runs in a stripped Lua 5.4 runtime (`src/ext/lua.h`). The
|
||||
module is **declarative** — it registers capabilities; it cannot execute
|
||||
or touch the outside world:
|
||||
|
||||
| Capability | Status |
|
||||
| --- | --- |
|
||||
| `st.register_language`, `st.register_check` | the **only** interface |
|
||||
| base / table / string / curated `math` / hand-built `os` | available |
|
||||
| `os.execute`, `os.exit`, `os.remove`, `os.rename` | **present but blocked** — calling them raises `sandbox blocked: ...` and fails discovery |
|
||||
| `dofile`, `loadfile`, `require`, `package`, `io`, `debug`, `coroutine` | **nil** — their C libraries are not even linked |
|
||||
|
||||
A module that violates the sandbox is rejected with a clear load error;
|
||||
the binary never runs untrusted program execution.
|
||||
|
||||
## 3. The registration API
|
||||
|
||||
Two functions on the `st` table. Call shapes (exact — see
|
||||
`src/ext/api.h` and `src/ext/lua.c`):
|
||||
|
||||
```lua
|
||||
st.register_language(name, [spec_table])
|
||||
st.register_check(name, { kind = "compile"|"link"|"run", source = "<C snippet>", link = { "-lfoo", ... } })
|
||||
```
|
||||
|
||||
- **`register_language(name [, table])`** — registers a language *name*.
|
||||
The optional second argument, if present, must be a table; v1 records
|
||||
the name only (the table's contents are reserved for future language
|
||||
detection).
|
||||
- **`register_check(name, spec)`** — registers a custom check. `kind` is
|
||||
the probe **mode** (compile / link / run), `source` is the C snippet
|
||||
the probe compiles, and `link` is an optional list of extra argv
|
||||
elements for the link step. The check's *name* becomes a new check kind
|
||||
a build file can reference, exactly like the builtin `header` kind.
|
||||
- Names must be non-empty strings with no NUL bytes and must not collide
|
||||
with an already-registered name (builtin or earlier module).
|
||||
|
||||
The Lua tables are materialized into a `struct st_lua_probe_spec`
|
||||
(`src/ext/api.h`): all fields strdup'd, owned by the runtime's registry,
|
||||
freed with the runtime.
|
||||
|
||||
## 4. v1 limitations (documented, deliberate)
|
||||
|
||||
- **Custom checks compile C snippets against the C toolchain.** The
|
||||
`source` in a probe spec is a *C* snippet driven by the detected C
|
||||
compiler (`st_lua_probe_run` in `src/ext/api.h`). There is no generic
|
||||
per-language probe runner yet; the Fortran example's check is a C
|
||||
probe whose name says "fortran". (This is why the example keeps its
|
||||
source libm-free and trivially true.)
|
||||
- **Lua language detection is a stub.** `register_language` records the
|
||||
name only; nothing yet detects a `fortran` compiler on the host, and
|
||||
the DSL has no language-scoped feature resolution wired to it. The
|
||||
table argument exists so a later release can add
|
||||
`register_language("fortran", { detect = ... })` without breaking
|
||||
existing modules.
|
||||
- **The bridge maps every Lua check to the `"c"` language** internally
|
||||
(`ST_LUA_CHECK_LANGUAGE` in `src/ext/api.c`) — again, v1 custom checks
|
||||
are C snippets, by design.
|
||||
|
||||
## 5. Step-by-step: adding a new language
|
||||
|
||||
Copy this recipe for any language, not just Fortran.
|
||||
|
||||
1. **Write a `.lua` module.** Model it on `examples/ext_fortran.lua`:
|
||||
|
||||
```lua
|
||||
-- mylang.lua
|
||||
st.register_language("mylang", { kind = "compile" })
|
||||
st.register_check("mylang_compiler_flag", {
|
||||
kind = "compile",
|
||||
source = "int main(void) { return 0; }",
|
||||
link = { "-lmylang" },
|
||||
})
|
||||
```
|
||||
|
||||
Only `st.*` calls are allowed; no `os.execute`, no `io`, no
|
||||
`require` — the sandbox enforces it.
|
||||
|
||||
2. **Drop it in a discovery directory.** Either:
|
||||
|
||||
```sh
|
||||
mkdir -p /tmp/extdemo
|
||||
cp mylang.lua /tmp/extdemo/
|
||||
./src/stupidtools --ext-dir /tmp/extdemo
|
||||
```
|
||||
|
||||
or:
|
||||
|
||||
```sh
|
||||
STUPIDTOOLS_EXT=/tmp/extdemo ./src/stupidtools
|
||||
```
|
||||
|
||||
or install it to `$XDG_DATA_HOME/stupidtools/ext/` (default user dir).
|
||||
|
||||
3. **Check the load line.** You should see your module named with its
|
||||
actual registrations:
|
||||
|
||||
```
|
||||
loaded extension: /tmp/extdemo/mylang.lua — checks: 1 (mylang_compiler_flag) languages: 1 (mylang)
|
||||
```
|
||||
|
||||
A malformed module aborts discovery with `extension load failed:
|
||||
<path>: <file>:<line>: <message>` — fix and rerun.
|
||||
|
||||
4. **Reference the new check kind from a build file.** Once a module has
|
||||
registered `mylang_compiler_flag`, a `feature` block in the KDL build
|
||||
file can name that kind like any builtin check kind. (Full fixture-
|
||||
driven resolution lands with the buildfile pipeline; discovery +
|
||||
registration is what this version proves.)
|
||||
|
||||
That is the whole contract: **discover → sandbox → register → reference.**
|
||||
Languages are data to core, and the builtin C/C++ modules are just the
|
||||
first extensions.
|
||||
Reference in New Issue
Block a user