Standard Library

Want ships a standard library of Spore packages. Planning files can import these packages. An import names a package (a directory of .sp files) or a single file. Standard-library packages are namespaced: the want standard library is want: and the spore standard library is core:. An import with no colon resolves in the same source as the importing file — a module-relative path in your module, or a sibling package within a standard library:

ns    = (#import "core:ns")
build = (#import "want:build")
linux = (#import "want:linux")
paths = (#import "want:build/paths")

Callers are responsible for spawning and awaiting Jobs through their job port; the standard library functions take that port (build.JobCtx) as their first argument, by convention. A tool package may set up a package-specific context instead and take it first: the zig and go commands take a zig.Ctx or go.Ctx (built with new from the job port, limits, platform, and version).

build

The core of the build library (stdlib/build).

Types

  • FS: a wantfs tree. This is the value a do proc returns and the type almost everything operates on.
  • Ctx: the build context handed to a do proc:
    Ctx = (#Struct { .jobs = JobCtx .input = FS .secrets = (#List (#Port Unit Unit)) })
    
    secrets is empty unless the build publishes; see SinkCtx below.
  • SinkCtx: the context handed to a unit file's .sink proc by want build --publish: (#Struct { .jobs = JobCtx .input = FS .internet = (#Port Unit Unit) .secrets = (#List (#Port Unit Unit)) }). input is the target's do result; secrets is the file's granted secret ports, in the order the WANT's .secrets names them (supplied with --secret NAME:PATH or --secrets NAME:PATH). Sinks run only for targets the build query names directly, and a publish build fails if a directly queried target (or its running sink) is granted a name that was not supplied.
  • JobCtx: the jobs port used to spawn and await child Jobs.
  • Task, JobID, JobState, JobResult: the job system's wire types.
  • Limits: the resource ceiling a spawn requests for its child ((#Struct { .memory = B64 .cpu = B64 .timeout = B64 })). memory is bytes, cpu is microseconds per second, and timeout is seconds (0 means no ceiling); a child running longer than timeout is cancelled.

Jobs

  • spawn jc limits op input -> JobID: start a child job for a Task under limits.
  • limits jc -> Limits: the current job's spawn-time ceiling. Passing it to spawn propagates the ceiling down the tree unchanged.
  • await jc id -> AnyValue: wait for a child; a failed child faults the parent.
  • inspect jc id -> JobState: check a child without blocking.
  • awaitAny jc ids -> JobID: wait for any of several children.
  • collectJobLogs jc id name -> String: read a child's named log stream ("stdout", "stderr", ...) as its bytes. If the child's result came from the task cache, the bytes come from the job the cache recorded as the producer, so a cached task still exposes the logs that produced it; a stream that was never written reads as the empty string.

Filesystem

  • pick jc root path -> FS: re-root the tree at path; picking a file yields a flat single-file FS.
  • pickRoot jc root path -> FS: the same operation, named for re-rooting a directory.
  • fileAt jc root path -> FS: pick a single file (a synonym for pick).
  • place jc segment path -> FS: mount segment at path, prefixing every entry's path; the inverse of pick.
  • fsAssemble jc ents -> FS: mount named single-file FSes at their names:
    (build.fsAssemble jc [
        (build.ent "bin/hello" exe)
    ])
    
  • ls jc root path -> (#List String): the immediate children of a directory.
  • filterPaths jc root set -> FS: keep only the entries of root whose path the PathSet set contains; entry paths are unchanged, so the filtered tree is not re-rooted.
  • emptyDir jc -> FS: a tree with no entries.

Downloads

  • downloadFile jc task -> FS: fetch a file from one of task.urls, verified against task.hash under task.algo (SHA1, SHA256, or BLAKE3); a non-NONE task.compression decompresses the verified payload into the returned file.
  • downloadTree jc task -> FS: fetch an archive (tar, zip, or deb) and expand it into a tree. task.algo names the archive's hash algorithm and task.compression the archive payload's compression (GZIP, XZ, ZSTD, or NONE); decompression is explicit, never sniffed.

OCI

  • downloadOciImage jc task -> FS: pull an OCI/Docker image into an OCI layout.
  • pushOciImage jc task -> PushedOCIImage: push an OCI layout to a registry repository and attach the task's tags. The push is idempotent: blobs, manifests, and tags the registry already holds are skipped. The registry credentials come from task.secrets, a --secrets map whose exports include .OCI_USERNAME and .OCI_PASSWORD.
  • OCIImageTask (url, repo, algo, hash), PushOCIImageTask (url, repo, tags, image, secrets), and PushedOCIImage (digest, tags): the OCI wire types.

Importing build also gives you its download, job, and oci modules' exported functions, since a package is compiled from every immediate child .sp file.

build/paths

The PathSet constructors used in .input and .output:

  • Set: the PathSet tagged union itself ((#TaggedUnion {.unit = String .prefix = String .suffix = String})), the parameter type of build.filterPaths.
  • unit p -> Set: the single path p.
  • prefix p -> Set: every path beginning with p.
  • suffix s -> Set: every path ending in s.

linux

The Linux container executor (stdlib/linux).

  • run jc cspec -> CSpawnResult: run a container described by cspec, and return its exit code plus the tree captured from its output mount.
  • CSpec: the container description: init, args, env, wd, mounts, network, stdin, stderr, stdout.
  • initMemfd fs: run an executable from a flat single-file FS (copied into an anonymous memfd).
  • initPath p: run the executable already at path p inside the container.
  • Mount sources: snapshot fs (read-only), output, emptyDir, tmpfs n, drive n, overlay rlayer wlayer (lower may be a snapshot/drive; upper must be a writable drive/tmpfs), devNull, procfs, sysfs, cgroup2fs, devpts.
  • mount dst src -> Mount: pin a source at a container path.
  • net name backend, loopback, private, internet cap: network interfaces.
  • stdio: stdinNull, stdin ctx, stdout ctx, stderr ctx, outputLog, outputClosed, stdinClosed.
  • fileAtPath jc root path -> FS: pull a file out of a container tree.

linux/dist/alpine

A pinned Alpine minirootfs: a small musl userland with the busybox shell.

  • rootfs jc -> FS: expand the pinned minirootfs tarball into a tree.
  • fileAt jc path -> FS: pull a single file out of it.
  • libgcc jc -> FS: expand the pinned Alpine libgcc package (usr/lib/libgcc_s.so.1), for tools whose dynamic binaries need the gcc runtime; the minirootfs ships none.
  • version, arch: the pinned release.

linux/dist/debian

A pinned rblaine95/debian base image (a GHCR mirror of Docker Hub's official image). Like Alpine, but a full glibc userland.

  • image jc -> FS: pull the pinned image as an OCI layout.
  • rootfs jc -> FS: unpack the image's root filesystem.
  • fileAt jc path -> FS: pull a single file out of it.

linux/dist/arch

Arch Linux packaging on top of a pinned official Arch container image (with the bash, pacman, and bsdtar makepkg itself uses). package takes already-built artifacts as a build.FS tree and produces the .pkg.tar.zst pacman installs; nothing is compiled.

  • rootfs jc -> FS: the pinned image's root filesystem.
  • fileAt jc path -> FS: pull a single file out of it.
  • version: the pinned image build.
  • Spec: the required metadata (.name, .version, .arch, .desc, .builddate, .files). version is the full pkgver including the pkgrel (and optional epoch), and builddate is a decimal Unix timestamp that fixes every file's mtime, so equal inputs produce equal packages.
  • Meta: the full metadata record; meta spec -> Meta fills the optional fields with defaults.
  • withUrl, withPackager, withLicense, withDepends, withOptdepends, withProvides, withConflicts, withReplaces, withGroups, withBackup, withInstall: rebind one optional field.
  • filename meta -> String: the output file's name, name-version-arch.pkg.tar.zst.
  • pkginfo meta -> String: the rendered .PKGINFO.
  • package jc meta -> FS: assemble the package and return the tree captured from the output mount (one file, filename meta); use build.fileAt for the package as a flat single-file tree. The target faults if the packager fails.
  • checkExit code -> Unit: fault unless a container exited zero.

The payload is a tree rooted at /, built with build.fsAssemble or picked from a build output:

files = (build.fsAssemble jc [
    (build.ent "usr/bin/hello" exe),
])
m = (arch.meta {.name = "hello" .version = "1.0-1" .arch = "x86_64"
                .desc = "..." .builddate = "1700000000" .files = files})
m = (arch.withLicense m ["MIT"])
pkg = (arch.package jc m)

docker

  • pull jc task -> FS: pull an OCI image.
  • umoci jc -> FS: the pinned umoci binary.
  • rootfs jc img -> FS: unpack a pulled image's root filesystem.

git

A pinned static git build (darkvertex/static-git), used to give a container a git binary with no package manager or network.

  • tool jc -> FS: the expanded static git tree (git at its root). Mount it on PATH (e.g. at /git) and set GIT_TEMPLATE_DIR to an existing directory.

github

Upload release assets to an existing GitHub release (stdlib/com/github). Credentials come from a --secrets map whose only required export is .GITHUB_TOKEN, a token with write access to the repository's releases (contents: write). The preferred setup is a spore file:

.GITHUB_TOKEN = "..."
want build --publish --secrets github:~/.config/want/github.sp
  • Ctx: the GitHub context ((#Struct {.jobs = JobCtx .secrets = (#Port Unit Unit)})); build it once with new.
  • new jc secrets -> Ctx: bind the job port and the secrets port granted by the WANT's .secrets.
  • UploadOpts: (#Struct {.owner = String .repo = String .tag = String}), naming the existing release to attach assets to.
  • Asset: one uploaded asset (.id, .name, .size, .content_type, .browser_download_url).
  • ReleaseUpload: the op's result: the release and the assets it now carries.
  • uploadRelease ctx opts assets -> ReleaseUpload: upload every top-level file in assets as an asset of the named release; an existing asset with the same name is replaced.

go

The pinned Go toolchain (stdlib/lang/go).

  • Ctx: the go environment ((#Struct {.jc = JobCtx .limits = Limits .platform = Platform .version = Version .modcache = FS .env = (#List String)})). Every command takes it as its first argument.
  • new jc limits platform version -> Ctx: set up the environment once, then hand it to every command. platform/version select the toolchain distribution to fetch; jc and limits are the job resources commands run under. The module cache starts empty.
  • withModcache ctx modcache -> Ctx: rebind the module cache to the tree returned by pkg.
  • withEnv ctx env -> Ctx: rebind the command-wide environment. Entries are prepended to the fixed environment and the first entry for a variable wins, so they override it (for example GOFLAGS, GOPROXY, CGO_ENABLED).
  • withTarget ctx platform -> Ctx: prepend GOOS/GOARCH for platform, cross-compiling pure-Go builds with the host toolchain.
  • LATEST, v1_27_1: the pinned Version values; linux_amd64 is the pinned Platform; GOOS_*/GOARCH_* name every target.
  • dist ctx -> FS: download the pinned distribution and re-root it as a container root.
  • pkg ctx src hash -> FS: run go mod download for the module tree src (under the context's environment) in a network-enabled container and return the resulting module cache, audited against hash.
  • cmd ctx cwd args -> CSpawnResult: run the pinned go tool over the module tree cwd (mounted at /src) in a container with no network. args is the full argv, e.g. ["go", "build", "-o", "/out/app", "./cmd/foo"].
  • build ctx modfs entrypoint -> FS: compile the main package at entrypoint into a static executable.
  • test ctx modfs -> CSpawnResult: run go test ./...; its stdout and stderr are captured as /stdout and /stderr in the output tree.
  • testBinaries ctx modfs -> CSpawnResult: compile every package's test binaries into the output tree without running them.
  • checkExit code -> Unit: fault the build unless code is zero.

go/dist

The Go release pins and the mirror URLs used to fetch them (stdlib/lang/go/dist.sp, part of the go package): Version, LATEST, v1_27_1, Platform, linux_amd64, tarballName, topDir, dist, and mkurls.

python

The pinned Python toolchains: CPython (the interpreter, from python-build-standalone) and uv (the package manager), each fetched from its own release artifacts (stdlib/lang/python).

  • Ctx: the python environment ((#Struct {.jc = JobCtx .limits = Limits .platform = Platform .uv = UvVersion .python = PythonVersion .venv = FS .env = (#List String)})). Every command takes it as its first argument.
  • new jc limits platform uv python -> Ctx: set up the environment once, then hand it to every command. platform selects both distributions; uv and python pin the versions to fetch (see below). jc and limits are the job resources commands run under. The project environment starts empty.
  • withVenv ctx venv -> Ctx: rebind the project environment to the tree returned by pkg; commands mount it at /venv and prefer its bin/ on PATH.
  • withEnv ctx env -> Ctx: rebind the command-wide environment. Entries are prepended to the fixed environment and the first entry for a variable wins, so they override it (for example UV_NO_DEV).
  • UV_LATEST, PYTHON_LATEST, PYTHON_3_13: the pinned UvVersion and PythonVersion values; x86_64_linux is the pinned Platform.
  • uvDist ctx -> FS: the pinned uv distribution (uv and uvx), to mount at /uv.
  • pythonDist ctx -> FS: the pinned CPython distribution, to mount at /python.
  • pkg ctx src hash -> FS: run uv sync --frozen for the project tree src (which must contain pyproject.toml and uv.lock) in a network-enabled container with the pinned toolchains, and return the resulting venv, audited against hash. Mount the result on the context with withVenv before running commands.
  • cmd ctx cwd argv -> CSpawnResult: run any command over the tree cwd in the hermetic command container; argv is resolved on PATH through the venv, the pinned interpreter, and uv, and each element is passed verbatim.
  • python ctx cwd args, uv ctx cwd args, sh ctx cwd script: cmd with the pinned interpreter, uv, or the Debian shell (for pipelines and redirects) as the program.
  • checkExit code -> Unit: fault the build unless code is zero.

python/dist

The uv and CPython release pins and the asset URLs used to fetch them (stdlib/lang/python/dist.sp, part of the python package): UvVersion, PythonVersion, UV_LATEST, PYTHON_LATEST, PYTHON_3_13, Platform, x86_64_linux, uvUrl, pythonUrl, uvDist, and pythonDist.

rust

The pinned Rust toolchain (stdlib/lang/rust). The distribution is musl-hosted: the containers run the pinned Alpine minirootfs, and rustc links static musl binaries with its bundled rust-lld, so no C toolchain is needed.

  • Ctx: the rust environment ((#Struct {.jc = JobCtx .limits = Limits .platform = Platform .version = Version .vendor = FS .env = (#List String)})). Every command takes it as its first argument.
  • new jc limits platform version -> Ctx: set up the environment once, then hand it to every command. platform and version select the toolchain distribution to fetch; any Version value works, so a caller can force a newer rust than LATEST tracks without editing the stdlib. jc and limits are the job resources commands run under. The vendor directory starts empty.
  • withVendor ctx vendor -> Ctx: rebind the vendor directory to the tree returned by pkg; commands mount it at /vendor and use it as CARGO_HOME, so cargo reads the captured source-replacement config there.
  • withEnv ctx env -> Ctx: rebind the command-wide environment. Entries are prepended to the fixed environment and the first entry for a variable wins, so they override it (for example RUSTFLAGS or CARGO_*).
  • LATEST, v1_99_1: the pinned Version values; x86_64_linux is the pinned Platform.
  • rustcDist ctx -> FS: the compiler component (rustc, rustdoc, and the bundled rust-lld), to mount at /rust.
  • cargoDist ctx -> FS: cargo, to mount at /cargo.
  • stdDist ctx -> FS: the precompiled standard library plus the self-contained musl libraries, to mount inside the compiler tree at /rust/lib/rustlib/<triple>/lib.
  • pkg ctx src hash -> FS: run cargo vendor --locked for the project tree src (which must contain Cargo.toml and a committed Cargo.lock) in a network-enabled container and return the vendored crates plus cargo's source-replacement config, audited against hash. Mount the result on the context with withVendor before running commands.
  • cmd ctx cwd argv -> CSpawnResult: run the pinned cargo over the tree cwd in the hermetic container; argv is the full argv, e.g. ["cargo", "build", "--release"]. Every dependency resolves from the vendor tree and the network is off.
  • build ctx src bin -> FS: compile the binary target bin in release mode and return it as "app" at the root of a wantfs tree.
  • test ctx src -> CSpawnResult: run cargo test --locked; its stdout and stderr are captured as /stdout and /stderr in the output tree.
  • sh ctx src script -> CSpawnResult: run a shell script in the same container, for pipelines and redirects.
  • checkExit code -> Unit: fault the build unless code is zero.

rust/dist

The Rust release pins and the component picks used to fetch them (stdlib/lang/rust/dist.sp and stdlib/lang/rust/versions.sp, part of the rust package): Version, LATEST, v1_99_1, Platform, x86_64_linux, tarballName, topDir, url, rustcDist, cargoDist, and stdDist.

vm

Boots hardware virtual machines through the vm.amd64 executor (see doc/23_Executors.md). Runs need a KVM host and a crosvm (WANT_CROSVM or PATH; build/crosvm.want builds a pinned one).

  • run jc task -> VmResult: boot an AMD64Task and return crosvm's status and the run's output. A non-zero status faults the build instead of returning a result, so a failed run is never cached.
  • AMD64Task: kernel and initrd (flat single-file trees; some fs wraps an initrd), kargs (extra kernel command line arguments), block devices (.raw / .qcow2 / .output), and fs shares (.snapshot / .output). The guest sees disks as vda, vdb, ... and shares tagged fs0, fs1, ... in list order; a read-write input device is a fresh copy per run.
  • A task may declare at most one output, created empty for the run: outputDisk size attaches a fresh disk, outputShare size a fresh directory. When the guest stops, run captures it as Output{ block = FS | virtiofs = FS | none }: a disk image or a tree, content-addressed like any other FS.
  • Every run attaches a vhost-user-console device whose ports are named stdout and stderr. A guest init that opens the named ports (the vmzig init does) gets the process's two streams in the job's stdout and stderr logs; the kernel serial console and crosvm's diagnostics go to console and vmm.
  • snapshotShare fs shares a stored tree read-only.
  • stdlib/vm/smoke.want boots a downloaded kernel end to end (see DEV.md).

vm/crosvm

Builds the pinned crosvm, the VMM the vm executor runs (stdlib/vm/crosvm). crosvm is not pure Rust: minijail's C library (the third_party/minijail submodule cargo patches in), libclang for bindgen, and python3 for the seccomp policy build are all part of it. The build containers therefore run a dedicated Alpine dev environment (the minirootfs plus a pinned closure of build-base, linux headers, libcap, libclang, python3, and pkgconf, installed offline) and link through gcc rather than the toolchain's rust-lld.

  • build jc -> FS: the whole pipeline: fetch the pinned crosvm and minijail trees, install the dev environment, vendor the crate graph with cargo vendor --locked (audited), then cargo build --release --locked --offline --no-default-features --features qcow with no network. The result holds crosvm at its root; build/crosvm.want is the target that materializes it.
  • source jc -> FS, minijailTree jc -> FS: the pinned source trees, fetched with download.git_commit_tree.
  • vendorPkgs jc src minijail devenv hash -> FS: the audited cargo vendor result, to mount at /vendor as CARGO_HOME.
  • package jc src minijail devenv vendor -> FS: the offline compile, which captures the crosvm binary.
  • versions.sp pins the crosvm and minijail revisions, the Alpine branch, the name, version, and sha256 of every .apk in the dev environment's package closure, and the audited dev-environment and vendor-tree checksums. The packages are fetched by their versioned URLs and installed offline, so the dev environment does not move when Alpine updates the branch. When a fetch changes, the build faults with the actual checksum, which should be pasted back.

vm/qemu

Pins and fetches a QEMU distribution (stdlib/vm/qemu), mirroring the lang/zig and lang/go toolchain packages for versions and fetching. QEMU publishes no prebuilt Linux binaries (qemu.org serves source tarballs and points Linux users at distribution packages), so the pinned release is the official source tarball and dist re-roots the expanded source tree. A build that compiles it into a runnable VMM is future work.

  • Ctx: the qemu environment ((#Struct {.jc = JobCtx .limits = Limits .platform = Platform .version = Version})).
  • new jc limits platform version -> Ctx: set up the environment once, then hand it to dist; platform/version select the distribution, jc and limits are the resources a build runs under.
  • dist ctx -> FS: download the source distribution pinned by ctx.version (checked against its sha256), expand the xz-compressed tar, and re-root it so its top-level directory (e.g. qemu-11.1.2) is the tree root; mount it at /src.
  • versions.sp pins the QEMU release tag and the sha256 of its source tarball; LATEST names the tracked release. QEMU ships only a detached GPG signature, so the hash is computed here and a mismatch faults with the actual checksum to paste back.

zig

The pinned Zig toolchain.

  • Ctx: the zig environment ((#Struct {.jc = JobCtx .limits = Limits .platform = Platform .version = Version .pkgfs = FS .env = (#List String)})). Commands take it as their first argument.
  • new jc limits platform version -> Ctx: set up the environment once, then hand it to every command. platform/version select the toolchain; jc and limits are the job resources commands run under. The package directory starts empty.
  • withPkgfs ctx pkgfs -> Ctx: rebind the package directory to the tree returned by pkg; commands mount it at /src/zig-pkg.
  • withEnv ctx env -> Ctx: rebind the command-wide environment. Entries are prepended to the fixed environment and the first entry for a variable wins, so they override it.
  • LATEST, STABLE, v0_16_0: the pinned Version values; x86_64_linux is the pinned Platform.
  • dist ctx -> FS: download the pinned distribution and re-root it as a container root.
  • pkg ctx src hash -> FS: fetch the build.zig.zon dependencies of src through zig build --fetch, audited against hash (the zig-pkg dirhash/BLAKE3 checksum).
  • cmd ctx cwd args -> CSpawnResult: run the pinned zig tool over the source tree cwd (mounted at /src); args is the full argv, e.g. ["/zig/zig", "build", "test"].
  • buildArgs ctx src args -> CSpawnResult: cmd plus the fixed --build-file, --prefix, and --cache-dir flags, for arbitrary zig build steps; returns the exit code plus the captured output tree.
  • build ctx src target -> FS: run the binaries step for the Zig target triple target and return its output tree.
  • checkExit code -> Unit: fault the build unless code is zero.
  • run ctx args -> AnyValue: run the compiler with stdio wired to the host.
  • watch ctx stdin stdout stderr -> B8: run zig build --watch on the build context's working directory, with stdio wired to the given ports; returns the container's exit code.

zig/dist

The Zig release pins and the mirror URLs used to fetch them (stdlib/lang/zig/dist.sp, part of the zig package): Version, LATEST, STABLE, v0_16_0, Platform, x86_64_linux, tarballName, topDir, dist, and mkurls.

mdbook

Renders a markdown book to static HTML, using the pinned static-musl mdbook release inside an Alpine container. The environment (Alpine rootfs and mdbook) is mounted from constant snapshots, so only the book source changes between builds.

  • tool jc -> FS: the pinned mdbook distribution, re-rooted.
  • build jc src book_toml summary -> FS: render the markdown in src, with book_toml as its book.toml and summary as src/SUMMARY.md, and return the generated site. The build faults if mdbook fails or produces no index.html.

sys

Types for the surrounding system context used by entrypoints (not by build planning files): Ctx, ctxPort, and the NetBackend choices.

ns

Helpers for building namespaces (NS, get, mkEntry), used when a Spore entrypoint is handed its context as a namespace.

bits

Bit-width aliases (B8…B256) and the bitwise primitives: AND, OR, and XOR on Bit, plus b32AND and b32OR on B32.

strings

Byte-level string helpers mirroring a subset of Go's strings (a String is a List of bytes):

  • len s -> B32.
  • hasPrefix s p -> Bit, hasSuffix s p -> Bit, contains s p -> Bit.
  • index s p -> (Maybe B32): the first index of p in s, or Nothing.
  • count s p -> B32: non-overlapping instances of p; an empty p counts len(s) + 1 insertion points.
  • split s sep -> (#List String): an empty sep splits into single bytes.
  • join parts sep -> String, repeat s count -> String.
  • trimPrefix s p -> String, trimSuffix s p -> String, trimSpace s -> String (ASCII whitespace).