Introduction

What is Want?

Want is a build system (WBS).

Software projects use build systems to reliably and reproducibly derive artifacts from source code. WBS manages setting up and tearing down isolated environments to build artifacts like static sites, assets, and executables for release or testing.

Why would I use Want?

If you've ever tried to get up and running with a new software project, you will have dealt with the problem of downloading all the tools that the project uses.

Even for simple tasks like building the docs, or producing shippable executables, you may have to install several tools. This requires developers to know about and understand tools which they will never work with themselves, to merely benefit from the output of a computation that others have set up.

If you or someone else put in the time to get something to work, it should also just work for you going forward. You shouldn't have to replay previous steps like: picking the right version, learning command line arguments, order of operations, etc.

With Want, every build target you add will be computed in exactly the same way on your laptop tomorrow, on your co-worker's development machine, and in CI. Want forces build steps to be fully and exactly specified, so they can be reproduced by anyone at anytime, in any environment.

Why is it called Want?

You write down what you want, and Want builds it.

More seriously: .want and .wants were good extensions for the files where you specify build targets, and naming the entire build system after them came later.

Using Want

This section will get you up and running with Want.

Most of the time you should run want build. This command is suitable to use as a pre-merge check. It will build all the targets in the current module. It does not and cannot perform publishing or other side effects.

Creating a Want Module

Want operates on Modules. A Module is just a filesystem tree with a file named WANT at its root. The WANT file holds module level configuration, and its presence is what tells Want that a directory is meant to be built.

To start a new module, run want init in your project's root. It writes a WANT file with the default module config — the want standard library as a dependency — and a default .wantignore:

.deps = [
	{"want", \(_: Ctx) FS LATEST},
]

See Module Files for what the WANT file can contain, and how the sibling .wantignore file controls which paths are imported.

Creating a Build Target

Want calls the things that you, the user, want, but don't yet have, and need to build, Targets.

A target is declared by a file ending in .want. Create a file named hello.txt.want at the root of your module, and fill it with the following:

build = (#import "want:build")
paths = (#import "want:build/paths")

.input = [
    (paths.unit "./input.txt")
]
.do = \(ctx: build.Ctx) build.FS {
    (build.pick ctx.jobs ctx.input "input.txt")
}

The file declares two things: the module paths the build reads (.input), and the computation that produces the output (.do). Here do simply picks the input file input.txt out of the target's input tree and returns it.

Create input.txt next to it containing Hello World!, then run

$ want cat hello.txt
Hello World!

The unit file hello.txt.want produces the output path hello.txt (its file name minus the .want extension), and want cat reads that file out of the build output.

Inspecting the Build Output

Normally cat hello.txt would print the source file, but want cat reads from the build output, while the regular cat reads from the regular filesystem, which is the build input.

Want organizes build targets into an output filesystem. This filesystem only exists virtually, within Want, but it can be inspected using UNIX-like tools (ls and cat).

want ls <path> will list the contents of a tree in the build output. Compare this to the output of regular ls.

want ls and want cat build only what the path they print could come from: want cat hello.txt builds the target that produces hello.txt, and want ls fetch the targets that can add entries under fetch/. A source file needs no targets at all, so want cat input.txt just prints it. (want ls with no path lists the whole output, so it builds the whole workspace.)

want serve-http <addr> builds the output and serves it as a read-only HTTP file tree (for example want serve-http 127.0.0.1:8080): directories render as an index of links, and files are written with a content type guessed from their extension. Stop it with Ctrl-C.

This simple translation of inputs to outputs is what makes Want so easy to use and reason about. You will never find yourself in a situation where you don't know when/how/where a Target is being computed.

Armed with just want ls, want cat, and want build, you can get pretty far. To learn more about what you can build with Want, read the Build Configuration section.

Generating Files

want generate <path...> builds the local module over the query (exactly as want build does) and then writes every derived file the query matches into your working tree. Generated files remain part of the source tree on the next build, so they must be ignored before they can be regenerated in place: add the generated paths (for example *.pb.go) to .wantignore.

# write every derived file ending in .pb.go into the worktree
$ want generate '*.pb.go'
generate: wrote api.pb.go
generate: wrote messages.pb.go

want generate refuses to overwrite a file that is part of the source snapshot (the worktree with .wantignore applied): it reports every conflict and writes nothing at all if any match conflicts. A query that matches no derived files is an error, and a missing query or an unknown flag is a usage error.

Publishing and Secrets

A unit file may export an optional .sink proc: the side effect to run after the build is green. Sinks do not run in a normal build; want build --publish runs them. A sink belongs to its target, consumes the target's output as ctx.input, and reaches the outside world through the job port and the capabilities in build.SinkCtx.

Secrets reach a sink through ctx.secrets, a list of ports in the order the WANT's .secrets grants name them. Supply them on the command line:

# one raw secret
want build --publish --secret mySecret:/path/to/my_secret.txt
# or one spore map carrying several named values
want build --publish --secrets cloudflare:~/.config/want/cloudflare.sp
want build --publish --secrets github:~/.config/want/github.sp

Only the targets the build query names directly are published; a target pulled in to satisfy a dependency is built but its sink does not run. A publish build fails if a directly queried target (or the sink that will run for it) is granted a secret that was not supplied. See Module Files for the .secrets grant syntax.

Concepts

This section provides a conceptual overview of WBS. If something doesn't make sense or you want to learn more about WBS's architecture, this is a good section to read.

TLDR

Want represents all build inputs and outputs as values in a content-addressed data model called Mycelium. Every value has a cryptographic hash, and equal values have equal hashes, so Want can easily tell whether two trees, files, or computations are identical.

A build is a transformation from an input filesystem (a Module, plus the outputs of its dependencies) to an output filesystem. Everything a computation reads is named explicitly, and the only way to reach outside the module is through the small set of executors, so a computation can always be cached and replayed.

The pieces you will read about next:

  • Data Model: how Mycelium values and the Want filesystem (wantfs) are shaped.
  • Tasks & Jobs: how units of work are named, cached, and run.
  • Executors: responsible for executing Tasks, in the context of a Job.
  • Spore: the language that want files are written in.

Data Model

Mycelium

Want stores all data in the Mycelium format. Mycelium is a serialization format like JSON or Protocol Buffers. In addition to the functionality of those two technologies, Mycelium allows functions to be sent over the wire and stored.

Mycelium uses cryptographic hashes as references. So all data-structures stored in Mycelium are cryptographic commitments to the full transitive closure of a node in the data structure.

Mycelium allows Want to cache anything in a meaningful representation. This covers Bits, Bytes, Strings, Programs, and Procedures, as well as Lists, Arrays, Products, and Sums of all those. Instead of converting those to another less accurate format like JSON, which also throws away any concept of reference, Mycelium allows a faithful representation.

The Want Filesystem (WantFS)

WBS stores all file data in the WantFS format. WantFS is built on top of Mycelium, and inherits the property of being content-addressed, and copy-on-write.

WantFS is used to represent file data whenever it has to be cached, or presented as the input to a build step. The output of want files is also stored in WantFS.

The first step in a build is to import the module from the local filesystem into WantFS.

Tasks & Jobs

Tasks

A Task is a well-defined unit of work. Well-defined means that all the information that could be required to complete the Task is explicitly contained in the Task.

A Task has 2 parts: an operation name and an input (an AnyValue):

Task = (#Struct {
  .op_name = String
  .input   = AnyValue
})

The operation names one of a small number of built-in operators, for example:

  • want.pick
  • want.place
  • want.assemblefs
  • download.tree
  • linux.cspawn
  • wasm.run
  • mvm.apply

The input is the operator's argument value. Both parts are part of the hash which becomes the Task's identity, so a Task with the same operation and input is the same piece of work.

Want caches the output of Tasks when they complete, and checks this cache before computing a Task. An immediate rerun of want build without any changes therefore produces all the same Tasks, and every one of them can be skipped.

Jobs

Jobs are used to track running computations in Want.

Jobs are created to compute Tasks. A Job carries active state, like whether it has completed and when it started and finished: it is spawned when created, running once its task is started, and done (with an ok value or an error) once it has an outcome. While a running Job waits for its queue resources it is reported as running with a non-null queuing_since, and the time it has spent waiting accumulates in queued_acc, so queuing is a subset of running rather than a state of its own.

Jobs are organized into a hierarchical tree. All the major operations in Want create a new root Job; that Job may spawn additional child Jobs.

Child Jobs give a computation access to additional compute, since children run concurrently, and they are each an opportunity for caching.

The Job System Port

A build computation is not given access to the host; instead it is handed a jobs port, a capability it uses to spawn and await child Jobs. In Spore this port has the type build.JobCtx, and a Task's do proc receives it on ctx.jobs.

Through the port a computation can:

  • spawn jc limits op input: start a child Job for a Task and get its JobID immediately, under a resource ceiling. A Job that runs past its timeout ceiling is cancelled automatically.
  • limits jc: the current Job's resource ceiling. Spawning with it passes the ceiling down unchanged.
  • await jc id: block until that child finishes and return its result; a child that failed faults the parent.
  • inspect jc id: check a child's state without blocking.
  • awaitAny jc ids: block until any one of a set of children finishes.
  • collectJobLogs jc id name: read a child's named log stream (stdout, stderr, ...). On a cached task the bytes come from the job that computed it, so the log of a cached child is still readable.

A Job cannot spawn root or sibling Jobs; it can only spawn its immediate children. This keeps runaway computations from escaping their parent, and gives every Job an audit trail back to the user who asked for the result.

Executors

WBS dispatches Tasks to Executors. An Executor knows how to do particular types of Tasks, and can provide any additional context.

Executors are the small set of built-in operations that can touch the outside world or run programs. They are the only place where a computation can do something non-deterministic (like use the network), and each is designed so that its successful outputs are fully determined by its inputs.

The persistent task cache and single-flight coordination live in src/sys/exec.zig: its Executor wraps the concrete executor behind a Backend interface and owns the cache. The executors under src/exec are backend implementations and never touch LMDB or the task cache.

Want Executor

Tasks directly related to planning, building, or manipulating files are executed by the Want Executor.

Here are a few examples:

  • want.pick extracts a subpath from a WantFS instance and returns a new instance containing just the subpath shifted to the root.
  • want.place is the inverse of pick: it mounts a segment under a path, prefixing every entry's path.
  • want.assemblefs assembles named single-file segments into one tree.
  • want.build builds a want module provided as a WantFS instance.
  • want.plan is the planning portion of the build, separated so that it can be cached separately.

I/O Executor

This executor performs all input and output tasks. If something is incoming or outgoing to/from the build system, then it goes through this executor.

Linux Executor

This executor spawns Linux containers to execute tasks. Think of this as "Linux as a Service".

VM Executor

This executor runs hardware virtual machines with crosvm (https://crosvm.dev). vm.amd64 boots an amd64 guest from a kernel, an optional initrd, any block devices the task provides, and any virtio-fs shares it exposes (read-only snapshots of stored trees, fresh host-disk scratch, or a named cache persisted across runs), all as WantFS segments. A task may declare at most one output — a fresh disk or an empty share — which is captured back into WantFS when the guest stops. A task may also declare network interfaces. Each becomes a virtio-net device whose queues crosvm exports to a vhost-user-net backend the executor runs; the backend moves the frames to the userspace IPv4 stack (usnet), so a guest reaches the host network with no TAP and no host networking privileges. The internet and log capability ports resolve exactly as for linux.cspawn, and a task with a network interface is never cached. Every run also attaches a vhost-user-console device with two ports named stdout and stderr. A guest's init can open them (the vmzig init does) so the process's two streams land in the job's stdout and stderr logs rather than being merged into the serial console; the kernel's serial console and crosvm's own diagnostics go to console and vmm. Think of this as "Linux as a Service", with the guest kernel coming from the task instead of the host.

Wasm Executor

This executor runs WebAssembly programs.

MVM Executor

This executor performs computations defined in the Mycelium format. Mycelium Procs can be applied to data with mvm.apply. Mycelium Lazys can be evaluated with mvm.eval.

All of the want files are evaluated using this executor.

Spore Executor

This executor calls into the Spore compiler. It is used to compile want files, module configuration files, and Spore libraries into Mycelium programs, which can be executed by the MVM executor.

Build Configuration

Want configuration is done using 3 kinds of files, all written in Spore:

When want build runs, it:

  1. Checks that the current directory has a WANT file which is a valid module config; if not, it exits with a "not a module" error.
  2. Finds every file ending in .want and .wants, and evaluates each into one or more Targets.
  3. Validates that the Targets together are a valid build plan.
  4. Runs the build according to the plan.

Each .want/.wants file is evaluated independently as a single-file Spore body, and must export the fields described for its kind.

The build departs from the module's source files only through the paths a target declares in its .input. Which paths are imported into the module in the first place is controlled by the root .wantignore file, described in Module Files.

Module Files

Want operates on units called Modules. A Module is a filesystem tree with a file named WANT at its root.

The WANT file provides module-level configuration and doubles as a sanity check that a directory is supposed to be built by Want. It is a Spore body that evaluates to a module config:

.deps = [
    {"want", \(_: Ctx) FS LATEST},
]

.deps

.deps lists the module's dependencies. Each dep is a tuple of a name and a proc that produces the dependency's filesystem from a build context ((#Product String (#Proc build.Ctx build.FS))). At plan time every dep's proc runs and its returned filesystem is registered as a Spore source under <name>:; planning files import its packages as <name>:pkg.

The WANT file is compiled with a preamble binding Ctx (build.Ctx), FS (build.FS, a wantfs.Segment) and LATEST (the compiled-in want standard library as an FS), so the default dep above gives the module the standard library under want: without any imports. The spore standard library (core:) is always implicit; a module that does not declare a want dep has no want stdlib. want init writes the default config above.

.budget

.budget optionally sets the total resource ceiling for a build of the module, including everything its child jobs claim. It has the same shape as build.Limits: memory in bytes, cpu in microseconds per second (one core is 1000000), and a wall-clock timeout in seconds.

.budget = {
    .memory  = 1073741824  // bytes
    .cpu     = 2000000     // microseconds per second (2 cores)
    .timeout = 600         // seconds
}

The export is optional: a WANT with no .budget has no module-level ceiling, and the root job's own limit (the config directory's .budget, or 75% of the host) applies. When .budget is present its fields are literal ceilings: the build job inspects its own spawn-time ceiling and claims the difference between it and the budget, so the module's child jobs can only draw the budget in total, and each child is spawned with the budget as its ceiling. A zero memory or cpu is therefore a zero ceiling, which leaves the build with no resources for that dimension (it will block or fail); it is not "unset". If the build job's own ceiling is already below the budget, nothing is claimed and the child ceiling is unchanged. A zero .timeout means no module-level timeout (the job's own timeout is kept). .timeout is a per-job wall-clock ceiling, so it bounds each child job, not the build job itself. want init does not write a .budget (a WANT with none builds unconstrained at the module level).

.secrets

.secrets grants named secrets to the module's planning files. It is a list of {paths, names} tuples, where paths is a paths.Set matched against a planning file's module-root-relative path, and names is the list of secret names that file may access:

paths = (#import "want:build/paths")

.secrets = [
    {{.all = {}}, ["cloudflare"]},                       // the whole module
    {(paths.prefix "site/"), ["S3_KEY"]},                // files under site/
    {(paths.unit "build/doc.want"), ["DOCS_TOKEN"]},     // one file
]

Only want build --publish uses secrets. Each matching target's build and sink contexts receive the granted names as a (#List (#Port Unit Unit)) in grant order (first occurrence wins), so a sink reads the first granted secret with (#listslot ctx.secrets 0). The names must be supplied on the command line:

  • --secret NAME:PATH — a single secret: the file's bytes.

  • --secrets NAME:PATH — a spore module whose exports are the map entries:

    .CLOUDFLARE_API_TOKEN = "..."
    .CLOUDFLARE_ACCOUNT_ID = "..."
    

--secrets cloudflare:~/.config/want/cloudflare.sp supplies the map under the name cloudflare, matching the grant above. A GitHub release upload takes a file exporting .GITHUB_TOKEN in the same way: --secrets github:~/.config/want/github.sp. A name supplied twice is an error; names supplied but unused are ignored. A publish build fails before running any target if a file it will publish is granted a name that was not supplied. Secret values are loaded into memory (never the content store), and only port tokens travel through tasks. Keep secret files outside the module tree (or dot-ignored, as .* in the default .wantignore) so the workspace import does not capture them as ordinary sources.

.wantignore

Which paths are imported into the module is configured separately, by a root .wantignore file. It uses gitignore-style patterns, one per line: any path matching a pattern is left out of the imported module tree.

By default a project that uses Git should ignore its VCS metadata and build outputs. For example:

.*
zig-out/
zig-cache/

.* ignores every dotfile (including .git), which keeps the import fast; the other entries ignore build output directories.

.want is always ignored, and files starting with .want cannot be imported.

Unit Files

Unit files end in .want. Each unit file declares a single Target.

A unit file is a Spore body with two exports:

build = (#import "want:build")
paths = (#import "want:build/paths")

.input = [
    (paths.unit "./input.txt")
]
.do = \(ctx: build.Ctx) build.FS {
    (build.pick ctx.jobs ctx.input "input.txt")
}
  • .input is a list of PathSets the target reads; the list is implicitly OR'd together. An empty list means the target reads nothing.
  • .do is a proc that is called to produce the build. It is handed a build.Ctx (its job port on ctx.jobs, and its filtered input tree on ctx.input) and returns an FS.

The output path

A unit file does not declare its output; it is derived from the file's name. A file hello.txt.want produces the output path hello.txt, and a file build/app.want produces build/app.

The FS returned by do is shifted into place under that path. A flat single-file FS (the result of picking a file) lands exactly at the output path; a tree becomes the subtree beneath it.

The build fails during planning if the derived output path already exists in the source tree, so a target can never silently overwrite a source file.

Path rules

Paths in .input are module-root-relative, with two exceptions:

  • A path beginning with ./ is relative to the directory of the unit file. In a/build_my_output.want, the input ./input.txt names a/input.txt, while input.txt names the file at the module root.
  • Paths may not begin with / (absolute paths are rejected), may not be empty, and may not contain a .. segment.

Inputs implicitly exclude the target's own output path.

Example: download a rootfs

fetch.want downloads a pinned Alpine minirootfs and returns it:

build   = (#import "want:build")
alpine  = (#import "want:linux/dist/alpine")

.input = []
.do = \(ctx: build.Ctx) build.FS {
    (alpine.rootfs ctx.jobs)
}

The result lands at fetch/ in the build output. want ls fetch lists it, and want build computes the whole module.

Multi Files

Multi files end in .wants. Unlike a unit file, a multi file declares a list of Targets, and each Target must declare the paths it is allowed to write.

A multi file is a Spore body with a single export, .targets:

build = (#import "want:build")
paths = (#import "want:build/paths")

.targets = [
  {
    .input = [
      (paths.unit "./input.txt")
    ]
    .output = [
      (paths.suffix ".out.txt")
    ]
    .do = \(ctx: build.Ctx) build.FS {
        // generate a bunch of files ending in ".out.txt"
    }
  }
]
  • .input is a list of PathSets the target reads, exactly as for a unit file.
  • .output is a list of PathSets the target may write to. The FS returned from do is filtered through this set: writing to an undeclared path fails the build.
  • .do is the proc that produces the build, the same as for a unit file.

Rooting

Every target in a multi file is rooted at the parent directory of the .wants file. The FS returned by do is positioned there, and .output is interpreted there too.

A protobuf.wants file at the root of a module produces files at the module root. If it were at a/b/c/protobuf.wants, and its do returned a filesystem containing messages.pb.go and api.pb.go, the resulting output would be:

a/b/c/messages.pb.go
a/b/c/api.pb.go

The .output declaration does not need to repeat the prefix. Either of these would accept the files above:

.output = [
  (paths.unit "messages.pb.go")
  (paths.unit "api.pb.go")
]

or, more flexibly, allowing the names to change as long as they keep the suffix:

.output = [
  (paths.suffix ".pb.go")
]

Why declare outputs?

A unit file's output path is fixed by its name, so conflicts with other targets are impossible. A multi file can write anywhere under its directory, so declaring .output lets Want check, before anything runs, that no two targets claim overlapping outputs, and that a target only writes where it said it would.

Standard Library

Want ships a standard library of Spore packages. Planning files can import these packages. An import names a package (a directory of .sp files) or a single file. Standard-library packages are namespaced: the want standard library is want: and the spore standard library is core:. An import with no colon resolves in the same source as the importing file — a module-relative path in your module, or a sibling package within a standard library:

ns    = (#import "core:ns")
build = (#import "want:build")
linux = (#import "want:linux")
paths = (#import "want:build/paths")

Callers are responsible for spawning and awaiting Jobs through their job port; the standard library functions take that port (build.JobCtx) as their first argument, by convention. A tool package may set up a package-specific context instead and take it first: the zig and go commands take a zig.Ctx or go.Ctx (built with new from the job port, limits, platform, and version).

build

The core of the build library (stdlib/build).

Types

  • FS: a wantfs tree. This is the value a do proc returns and the type almost everything operates on.
  • Ctx: the build context handed to a do proc:
    Ctx = (#Struct { .jobs = JobCtx .input = FS .secrets = (#List (#Port Unit Unit)) })
    
    secrets is empty unless the build publishes; see SinkCtx below.
  • SinkCtx: the context handed to a unit file's .sink proc by want build --publish: (#Struct { .jobs = JobCtx .input = FS .internet = (#Port Unit Unit) .secrets = (#List (#Port Unit Unit)) }). input is the target's do result; secrets is the file's granted secret ports, in the order the WANT's .secrets names them (supplied with --secret NAME:PATH or --secrets NAME:PATH). Sinks run only for targets the build query names directly, and a publish build fails if a directly queried target (or its running sink) is granted a name that was not supplied.
  • JobCtx: the jobs port used to spawn and await child Jobs.
  • Task, JobID, JobState, JobResult: the job system's wire types.
  • Limits: the resource ceiling a spawn requests for its child ((#Struct { .memory = B64 .cpu = B64 .timeout = B64 })). memory is bytes, cpu is microseconds per second, and timeout is seconds (0 means no ceiling); a child running longer than timeout is cancelled.

Jobs

  • spawn jc limits op input -> JobID: start a child job for a Task under limits.
  • limits jc -> Limits: the current job's spawn-time ceiling. Passing it to spawn propagates the ceiling down the tree unchanged.
  • await jc id -> AnyValue: wait for a child; a failed child faults the parent.
  • inspect jc id -> JobState: check a child without blocking.
  • awaitAny jc ids -> JobID: wait for any of several children.
  • collectJobLogs jc id name -> String: read a child's named log stream ("stdout", "stderr", ...) as its bytes. If the child's result came from the task cache, the bytes come from the job the cache recorded as the producer, so a cached task still exposes the logs that produced it; a stream that was never written reads as the empty string.

Filesystem

  • pick jc root path -> FS: re-root the tree at path; picking a file yields a flat single-file FS.
  • pickRoot jc root path -> FS: the same operation, named for re-rooting a directory.
  • fileAt jc root path -> FS: pick a single file (a synonym for pick).
  • place jc segment path -> FS: mount segment at path, prefixing every entry's path; the inverse of pick.
  • fsAssemble jc ents -> FS: mount named single-file FSes at their names:
    (build.fsAssemble jc [
        (build.ent "bin/hello" exe)
    ])
    
  • ls jc root path -> (#List String): the immediate children of a directory.
  • filterPaths jc root set -> FS: keep only the entries of root whose path the PathSet set contains; entry paths are unchanged, so the filtered tree is not re-rooted.
  • emptyDir jc -> FS: a tree with no entries.

Downloads

  • downloadFile jc task -> FS: fetch a file from one of task.urls, verified against task.hash under task.algo (SHA1, SHA256, or BLAKE3); a non-NONE task.compression decompresses the verified payload into the returned file.
  • downloadTree jc task -> FS: fetch an archive (tar, zip, or deb) and expand it into a tree. task.algo names the archive's hash algorithm and task.compression the archive payload's compression (GZIP, XZ, ZSTD, or NONE); decompression is explicit, never sniffed.

OCI

  • downloadOciImage jc task -> FS: pull an OCI/Docker image into an OCI layout.
  • pushOciImage jc task -> PushedOCIImage: push an OCI layout to a registry repository and attach the task's tags. The push is idempotent: blobs, manifests, and tags the registry already holds are skipped. The registry credentials come from task.secrets, a --secrets map whose exports include .OCI_USERNAME and .OCI_PASSWORD.
  • OCIImageTask (url, repo, algo, hash), PushOCIImageTask (url, repo, tags, image, secrets), and PushedOCIImage (digest, tags): the OCI wire types.

Importing build also gives you its download, job, and oci modules' exported functions, since a package is compiled from every immediate child .sp file.

build/paths

The PathSet constructors used in .input and .output:

  • Set: the PathSet tagged union itself ((#TaggedUnion {.unit = String .prefix = String .suffix = String})), the parameter type of build.filterPaths.
  • unit p -> Set: the single path p.
  • prefix p -> Set: every path beginning with p.
  • suffix s -> Set: every path ending in s.

linux

The Linux container executor (stdlib/linux).

  • run jc cspec -> CSpawnResult: run a container described by cspec, and return its exit code plus the tree captured from its output mount.
  • CSpec: the container description: init, args, env, wd, mounts, network, stdin, stderr, stdout.
  • initMemfd fs: run an executable from a flat single-file FS (copied into an anonymous memfd).
  • initPath p: run the executable already at path p inside the container.
  • Mount sources: snapshot fs (read-only), output, emptyDir, tmpfs n, drive n, overlay rlayer wlayer (lower may be a snapshot/drive; upper must be a writable drive/tmpfs), devNull, procfs, sysfs, cgroup2fs, devpts.
  • mount dst src -> Mount: pin a source at a container path.
  • net name backend, loopback, private, internet cap: network interfaces.
  • stdio: stdinNull, stdin ctx, stdout ctx, stderr ctx, outputLog, outputClosed, stdinClosed.
  • fileAtPath jc root path -> FS: pull a file out of a container tree.

linux/dist/alpine

A pinned Alpine minirootfs: a small musl userland with the busybox shell.

  • rootfs jc -> FS: expand the pinned minirootfs tarball into a tree.
  • fileAt jc path -> FS: pull a single file out of it.
  • libgcc jc -> FS: expand the pinned Alpine libgcc package (usr/lib/libgcc_s.so.1), for tools whose dynamic binaries need the gcc runtime; the minirootfs ships none.
  • version, arch: the pinned release.

linux/dist/debian

A pinned rblaine95/debian base image (a GHCR mirror of Docker Hub's official image). Like Alpine, but a full glibc userland.

  • image jc -> FS: pull the pinned image as an OCI layout.
  • rootfs jc -> FS: unpack the image's root filesystem.
  • fileAt jc path -> FS: pull a single file out of it.

linux/dist/arch

Arch Linux packaging on top of a pinned official Arch container image (with the bash, pacman, and bsdtar makepkg itself uses). package takes already-built artifacts as a build.FS tree and produces the .pkg.tar.zst pacman installs; nothing is compiled.

  • rootfs jc -> FS: the pinned image's root filesystem.
  • fileAt jc path -> FS: pull a single file out of it.
  • version: the pinned image build.
  • Spec: the required metadata (.name, .version, .arch, .desc, .builddate, .files). version is the full pkgver including the pkgrel (and optional epoch), and builddate is a decimal Unix timestamp that fixes every file's mtime, so equal inputs produce equal packages.
  • Meta: the full metadata record; meta spec -> Meta fills the optional fields with defaults.
  • withUrl, withPackager, withLicense, withDepends, withOptdepends, withProvides, withConflicts, withReplaces, withGroups, withBackup, withInstall: rebind one optional field.
  • filename meta -> String: the output file's name, name-version-arch.pkg.tar.zst.
  • pkginfo meta -> String: the rendered .PKGINFO.
  • package jc meta -> FS: assemble the package and return the tree captured from the output mount (one file, filename meta); use build.fileAt for the package as a flat single-file tree. The target faults if the packager fails.
  • checkExit code -> Unit: fault unless a container exited zero.

The payload is a tree rooted at /, built with build.fsAssemble or picked from a build output:

files = (build.fsAssemble jc [
    (build.ent "usr/bin/hello" exe),
])
m = (arch.meta {.name = "hello" .version = "1.0-1" .arch = "x86_64"
                .desc = "..." .builddate = "1700000000" .files = files})
m = (arch.withLicense m ["MIT"])
pkg = (arch.package jc m)

docker

  • pull jc task -> FS: pull an OCI image.
  • umoci jc -> FS: the pinned umoci binary.
  • rootfs jc img -> FS: unpack a pulled image's root filesystem.

git

A pinned static git build (darkvertex/static-git), used to give a container a git binary with no package manager or network.

  • tool jc -> FS: the expanded static git tree (git at its root). Mount it on PATH (e.g. at /git) and set GIT_TEMPLATE_DIR to an existing directory.

github

Upload release assets to an existing GitHub release (stdlib/com/github). Credentials come from a --secrets map whose only required export is .GITHUB_TOKEN, a token with write access to the repository's releases (contents: write). The preferred setup is a spore file:

.GITHUB_TOKEN = "..."
want build --publish --secrets github:~/.config/want/github.sp
  • Ctx: the GitHub context ((#Struct {.jobs = JobCtx .secrets = (#Port Unit Unit)})); build it once with new.
  • new jc secrets -> Ctx: bind the job port and the secrets port granted by the WANT's .secrets.
  • UploadOpts: (#Struct {.owner = String .repo = String .tag = String}), naming the existing release to attach assets to.
  • Asset: one uploaded asset (.id, .name, .size, .content_type, .browser_download_url).
  • ReleaseUpload: the op's result: the release and the assets it now carries.
  • uploadRelease ctx opts assets -> ReleaseUpload: upload every top-level file in assets as an asset of the named release; an existing asset with the same name is replaced.

go

The pinned Go toolchain (stdlib/lang/go).

  • Ctx: the go environment ((#Struct {.jc = JobCtx .limits = Limits .platform = Platform .version = Version .modcache = FS .env = (#List String)})). Every command takes it as its first argument.
  • new jc limits platform version -> Ctx: set up the environment once, then hand it to every command. platform/version select the toolchain distribution to fetch; jc and limits are the job resources commands run under. The module cache starts empty.
  • withModcache ctx modcache -> Ctx: rebind the module cache to the tree returned by pkg.
  • withEnv ctx env -> Ctx: rebind the command-wide environment. Entries are prepended to the fixed environment and the first entry for a variable wins, so they override it (for example GOFLAGS, GOPROXY, CGO_ENABLED).
  • withTarget ctx platform -> Ctx: prepend GOOS/GOARCH for platform, cross-compiling pure-Go builds with the host toolchain.
  • LATEST, v1_27_1: the pinned Version values; linux_amd64 is the pinned Platform; GOOS_*/GOARCH_* name every target.
  • dist ctx -> FS: download the pinned distribution and re-root it as a container root.
  • pkg ctx src hash -> FS: run go mod download for the module tree src (under the context's environment) in a network-enabled container and return the resulting module cache, audited against hash.
  • cmd ctx cwd args -> CSpawnResult: run the pinned go tool over the module tree cwd (mounted at /src) in a container with no network. args is the full argv, e.g. ["go", "build", "-o", "/out/app", "./cmd/foo"].
  • build ctx modfs entrypoint -> FS: compile the main package at entrypoint into a static executable.
  • test ctx modfs -> CSpawnResult: run go test ./...; its stdout and stderr are captured as /stdout and /stderr in the output tree.
  • testBinaries ctx modfs -> CSpawnResult: compile every package's test binaries into the output tree without running them.
  • checkExit code -> Unit: fault the build unless code is zero.

go/dist

The Go release pins and the mirror URLs used to fetch them (stdlib/lang/go/dist.sp, part of the go package): Version, LATEST, v1_27_1, Platform, linux_amd64, tarballName, topDir, dist, and mkurls.

python

The pinned Python toolchains: CPython (the interpreter, from python-build-standalone) and uv (the package manager), each fetched from its own release artifacts (stdlib/lang/python).

  • Ctx: the python environment ((#Struct {.jc = JobCtx .limits = Limits .platform = Platform .uv = UvVersion .python = PythonVersion .venv = FS .env = (#List String)})). Every command takes it as its first argument.
  • new jc limits platform uv python -> Ctx: set up the environment once, then hand it to every command. platform selects both distributions; uv and python pin the versions to fetch (see below). jc and limits are the job resources commands run under. The project environment starts empty.
  • withVenv ctx venv -> Ctx: rebind the project environment to the tree returned by pkg; commands mount it at /venv and prefer its bin/ on PATH.
  • withEnv ctx env -> Ctx: rebind the command-wide environment. Entries are prepended to the fixed environment and the first entry for a variable wins, so they override it (for example UV_NO_DEV).
  • UV_LATEST, PYTHON_LATEST, PYTHON_3_13: the pinned UvVersion and PythonVersion values; x86_64_linux is the pinned Platform.
  • uvDist ctx -> FS: the pinned uv distribution (uv and uvx), to mount at /uv.
  • pythonDist ctx -> FS: the pinned CPython distribution, to mount at /python.
  • pkg ctx src hash -> FS: run uv sync --frozen for the project tree src (which must contain pyproject.toml and uv.lock) in a network-enabled container with the pinned toolchains, and return the resulting venv, audited against hash. Mount the result on the context with withVenv before running commands.
  • cmd ctx cwd argv -> CSpawnResult: run any command over the tree cwd in the hermetic command container; argv is resolved on PATH through the venv, the pinned interpreter, and uv, and each element is passed verbatim.
  • python ctx cwd args, uv ctx cwd args, sh ctx cwd script: cmd with the pinned interpreter, uv, or the Debian shell (for pipelines and redirects) as the program.
  • checkExit code -> Unit: fault the build unless code is zero.

python/dist

The uv and CPython release pins and the asset URLs used to fetch them (stdlib/lang/python/dist.sp, part of the python package): UvVersion, PythonVersion, UV_LATEST, PYTHON_LATEST, PYTHON_3_13, Platform, x86_64_linux, uvUrl, pythonUrl, uvDist, and pythonDist.

rust

The pinned Rust toolchain (stdlib/lang/rust). The distribution is musl-hosted: the containers run the pinned Alpine minirootfs, and rustc links static musl binaries with its bundled rust-lld, so no C toolchain is needed.

  • Ctx: the rust environment ((#Struct {.jc = JobCtx .limits = Limits .platform = Platform .version = Version .vendor = FS .env = (#List String)})). Every command takes it as its first argument.
  • new jc limits platform version -> Ctx: set up the environment once, then hand it to every command. platform and version select the toolchain distribution to fetch; any Version value works, so a caller can force a newer rust than LATEST tracks without editing the stdlib. jc and limits are the job resources commands run under. The vendor directory starts empty.
  • withVendor ctx vendor -> Ctx: rebind the vendor directory to the tree returned by pkg; commands mount it at /vendor and use it as CARGO_HOME, so cargo reads the captured source-replacement config there.
  • withEnv ctx env -> Ctx: rebind the command-wide environment. Entries are prepended to the fixed environment and the first entry for a variable wins, so they override it (for example RUSTFLAGS or CARGO_*).
  • LATEST, v1_99_1: the pinned Version values; x86_64_linux is the pinned Platform.
  • rustcDist ctx -> FS: the compiler component (rustc, rustdoc, and the bundled rust-lld), to mount at /rust.
  • cargoDist ctx -> FS: cargo, to mount at /cargo.
  • stdDist ctx -> FS: the precompiled standard library plus the self-contained musl libraries, to mount inside the compiler tree at /rust/lib/rustlib/<triple>/lib.
  • pkg ctx src hash -> FS: run cargo vendor --locked for the project tree src (which must contain Cargo.toml and a committed Cargo.lock) in a network-enabled container and return the vendored crates plus cargo's source-replacement config, audited against hash. Mount the result on the context with withVendor before running commands.
  • cmd ctx cwd argv -> CSpawnResult: run the pinned cargo over the tree cwd in the hermetic container; argv is the full argv, e.g. ["cargo", "build", "--release"]. Every dependency resolves from the vendor tree and the network is off.
  • build ctx src bin -> FS: compile the binary target bin in release mode and return it as "app" at the root of a wantfs tree.
  • test ctx src -> CSpawnResult: run cargo test --locked; its stdout and stderr are captured as /stdout and /stderr in the output tree.
  • sh ctx src script -> CSpawnResult: run a shell script in the same container, for pipelines and redirects.
  • checkExit code -> Unit: fault the build unless code is zero.

rust/dist

The Rust release pins and the component picks used to fetch them (stdlib/lang/rust/dist.sp and stdlib/lang/rust/versions.sp, part of the rust package): Version, LATEST, v1_99_1, Platform, x86_64_linux, tarballName, topDir, url, rustcDist, cargoDist, and stdDist.

vm

Boots hardware virtual machines through the vm.amd64 executor (see doc/23_Executors.md). Runs need a KVM host and a crosvm (WANT_CROSVM or PATH; build/crosvm.want builds a pinned one).

  • run jc task -> VmResult: boot an AMD64Task and return crosvm's status and the run's output. A non-zero status faults the build instead of returning a result, so a failed run is never cached.
  • AMD64Task: kernel and initrd (flat single-file trees; some fs wraps an initrd), kargs (extra kernel command line arguments), block devices (.raw / .qcow2 / .output), and fs shares (.snapshot / .output). The guest sees disks as vda, vdb, ... and shares tagged fs0, fs1, ... in list order; a read-write input device is a fresh copy per run.
  • A task may declare at most one output, created empty for the run: outputDisk size attaches a fresh disk, outputShare size a fresh directory. When the guest stops, run captures it as Output{ block = FS | virtiofs = FS | none }: a disk image or a tree, content-addressed like any other FS.
  • Every run attaches a vhost-user-console device whose ports are named stdout and stderr. A guest init that opens the named ports (the vmzig init does) gets the process's two streams in the job's stdout and stderr logs; the kernel serial console and crosvm's diagnostics go to console and vmm.
  • snapshotShare fs shares a stored tree read-only.
  • stdlib/vm/smoke.want boots a downloaded kernel end to end (see DEV.md).

vm/crosvm

Builds the pinned crosvm, the VMM the vm executor runs (stdlib/vm/crosvm). crosvm is not pure Rust: minijail's C library (the third_party/minijail submodule cargo patches in), libclang for bindgen, and python3 for the seccomp policy build are all part of it. The build containers therefore run a dedicated Alpine dev environment (the minirootfs plus a pinned closure of build-base, linux headers, libcap, libclang, python3, and pkgconf, installed offline) and link through gcc rather than the toolchain's rust-lld.

  • build jc -> FS: the whole pipeline: fetch the pinned crosvm and minijail trees, install the dev environment, vendor the crate graph with cargo vendor --locked (audited), then cargo build --release --locked --offline --no-default-features --features qcow with no network. The result holds crosvm at its root; build/crosvm.want is the target that materializes it.
  • source jc -> FS, minijailTree jc -> FS: the pinned source trees, fetched with download.git_commit_tree.
  • vendorPkgs jc src minijail devenv hash -> FS: the audited cargo vendor result, to mount at /vendor as CARGO_HOME.
  • package jc src minijail devenv vendor -> FS: the offline compile, which captures the crosvm binary.
  • versions.sp pins the crosvm and minijail revisions, the Alpine branch, the name, version, and sha256 of every .apk in the dev environment's package closure, and the audited dev-environment and vendor-tree checksums. The packages are fetched by their versioned URLs and installed offline, so the dev environment does not move when Alpine updates the branch. When a fetch changes, the build faults with the actual checksum, which should be pasted back.

vm/qemu

Pins and fetches a QEMU distribution (stdlib/vm/qemu), mirroring the lang/zig and lang/go toolchain packages for versions and fetching. QEMU publishes no prebuilt Linux binaries (qemu.org serves source tarballs and points Linux users at distribution packages), so the pinned release is the official source tarball and dist re-roots the expanded source tree. A build that compiles it into a runnable VMM is future work.

  • Ctx: the qemu environment ((#Struct {.jc = JobCtx .limits = Limits .platform = Platform .version = Version})).
  • new jc limits platform version -> Ctx: set up the environment once, then hand it to dist; platform/version select the distribution, jc and limits are the resources a build runs under.
  • dist ctx -> FS: download the source distribution pinned by ctx.version (checked against its sha256), expand the xz-compressed tar, and re-root it so its top-level directory (e.g. qemu-11.1.2) is the tree root; mount it at /src.
  • versions.sp pins the QEMU release tag and the sha256 of its source tarball; LATEST names the tracked release. QEMU ships only a detached GPG signature, so the hash is computed here and a mismatch faults with the actual checksum to paste back.

zig

The pinned Zig toolchain.

  • Ctx: the zig environment ((#Struct {.jc = JobCtx .limits = Limits .platform = Platform .version = Version .pkgfs = FS .env = (#List String)})). Commands take it as their first argument.
  • new jc limits platform version -> Ctx: set up the environment once, then hand it to every command. platform/version select the toolchain; jc and limits are the job resources commands run under. The package directory starts empty.
  • withPkgfs ctx pkgfs -> Ctx: rebind the package directory to the tree returned by pkg; commands mount it at /src/zig-pkg.
  • withEnv ctx env -> Ctx: rebind the command-wide environment. Entries are prepended to the fixed environment and the first entry for a variable wins, so they override it.
  • LATEST, STABLE, v0_16_0: the pinned Version values; x86_64_linux is the pinned Platform.
  • dist ctx -> FS: download the pinned distribution and re-root it as a container root.
  • pkg ctx src hash -> FS: fetch the build.zig.zon dependencies of src through zig build --fetch, audited against hash (the zig-pkg dirhash/BLAKE3 checksum).
  • cmd ctx cwd args -> CSpawnResult: run the pinned zig tool over the source tree cwd (mounted at /src); args is the full argv, e.g. ["/zig/zig", "build", "test"].
  • buildArgs ctx src args -> CSpawnResult: cmd plus the fixed --build-file, --prefix, and --cache-dir flags, for arbitrary zig build steps; returns the exit code plus the captured output tree.
  • build ctx src target -> FS: run the binaries step for the Zig target triple target and return its output tree.
  • checkExit code -> Unit: fault the build unless code is zero.
  • run ctx args -> AnyValue: run the compiler with stdio wired to the host.
  • watch ctx stdin stdout stderr -> B8: run zig build --watch on the build context's working directory, with stdio wired to the given ports; returns the container's exit code.

zig/dist

The Zig release pins and the mirror URLs used to fetch them (stdlib/lang/zig/dist.sp, part of the zig package): Version, LATEST, STABLE, v0_16_0, Platform, x86_64_linux, tarballName, topDir, dist, and mkurls.

mdbook

Renders a markdown book to static HTML, using the pinned static-musl mdbook release inside an Alpine container. The environment (Alpine rootfs and mdbook) is mounted from constant snapshots, so only the book source changes between builds.

  • tool jc -> FS: the pinned mdbook distribution, re-rooted.
  • build jc src book_toml summary -> FS: render the markdown in src, with book_toml as its book.toml and summary as src/SUMMARY.md, and return the generated site. The build faults if mdbook fails or produces no index.html.

sys

Types for the surrounding system context used by entrypoints (not by build planning files): Ctx, ctxPort, and the NetBackend choices.

ns

Helpers for building namespaces (NS, get, mkEntry), used when a Spore entrypoint is handed its context as a namespace.

bits

Bit-width aliases (B8…B256) and the bitwise primitives: AND, OR, and XOR on Bit, plus b32AND and b32OR on B32.

strings

Byte-level string helpers mirroring a subset of Go's strings (a String is a List of bytes):

  • len s -> B32.
  • hasPrefix s p -> Bit, hasSuffix s p -> Bit, contains s p -> Bit.
  • index s p -> (Maybe B32): the first index of p in s, or Nothing.
  • count s p -> B32: non-overlapping instances of p; an empty p counts len(s) + 1 insertion points.
  • split s sep -> (#List String): an empty sep splits into single bytes.
  • join parts sep -> String, repeat s count -> String.
  • trimPrefix s p -> String, trimSuffix s p -> String, trimSpace s -> String (ASCII whitespace).

The Spore Programming Language

Spore is a small programming language which compiles to Mycelium. WBS uses Spore as the configuration language.

Spore is designed for simplicity; it is a smaller language than Python or even Go. This article gives a complete tour of Spore. It resembles what an engineer would write on a whiteboard when outlining a program.

Spore is a compiled, typed language with immutable values.

Symbols & Variables

Spore has immutable variables, which you can bind with =. Variables must be bound to a symbol. Symbols must start with a letter and can contain alphanumerics and _.

Assign a number to myVar.

myVar = 1234

Literals

Bits

A bit is the unit of information. There are two possible values 0 and 1.

firstBit = 0
secondBit = 1

The natural type of the numerals 0 and 1 is Bit, but they can be coerced to other types as well.

Numbers

We have already seen numbers used in the previous examples. Spore's numbers are coerced to (#Array Bit _) where _ is some value of type (#Array Bit 32) aka B32. Numbers must start with - or a numeral.

Underscores are allowed for readability

aMillion = 1_000_000

Hexadecimal can be specified with 0x

sixteen = 0x10

Octal can be specified with 0o

mode644 = 0o100644

Binary can be specified with 0b

eight = 0b1000

The preamble to the compiler (included before all code sent for compilation) includes a few basic integer types, to which these numeral literals can be coerced.

B8
B16
B32
B64
B128
B256

You can define an integer type of any bitwidth

(#Array Bit 4096)

The natural type of a numeral is the smallest bit array that can represent it. For example 31 would have type (#Array #Bit 5)

Strings

Strings are literal values of type List[Byte].

Double-quoted strings interpret escape sequences:

EscapeByte
\nnewline
\ttab
\rcarriage return
\0zero byte
\\backslash
\"double quote
\xHHthe byte with hex value HH
aString = "this is a string"
aTabbed = "col1\tcol2"

Backtick strings take everything between the backticks literally, newlines, backslashes, and double quotes included. They cannot contain a backtick.

aRawString = `C:\path\to\file`
aMultiline = `set -e
echo hello
`

Arrays

Spore Arrays are ordered, finite sequences of elements, which are all the same type. They can be coerced to Mycelium Arrays or Lists.

Lists are delimited with [ and ]. The elements are separated with a comma. A trailing comma is tolerated and not required.

aListOfStrings = ["a", "b", "c"]

Tuples

A Tuple is an ordered, finite sequence of elements, each of an independent and not necessarily distinct type. Tuples compile to Mycelium Products.

Tuples are delimited with { and }. The elements are separated with a comma. A trailing comma is tolerated and not required.

aTuple = {"a", 1, "b", 2}

Lookup Table

Lookup Tables (LUT) are the primitive unit of computation in Spore. They look like match/switch statements in other languages. A LUT is a mapping from one set of values to another.

LUTs are delimited by { and }. Inside are comma separated cases. A case is a pair joined by =>.

{
  0 => "zero",
  1 => "one",
}

The natural type of a LUT is a Proc type, which takes the type of all the things to the left of the =>, and returns a value of the same type as the things to the right. If a common type cannot be inferred, then a type must be specified.

((#Proc B2 String) {
  0 => "a",
  1 => "b",
  2 => "c",
  3 => "d",
})

All LUT cases must be mutually exclusive.

_ can be used once per LUT as a catch-all.

myLookTableProc = ((#Proc String B2) {
  "a" => 1,
  "b" => 2,
  "c" => 3,
  
  _ => 0,
})

Lambdas

Lambdas represent procedures. Lambdas start with \, which is followed by a list of arguments, a return type, and then an expression.

This lambda takes a String and returns a String which repeats it 3 times.

repeatThrice = \(a: String) String {
  (#flatten [a, a, a])
}

Lazies

Lazy values are delayed computations which require no additional input. They are specified like Lambdas, but without the arguments.

thisWillBeA4 = \B32 {
  two = (B32 2)
  (#add two two)
}

Content-ID (CID)

A CID is the hash of some data. CIDs are often long, and copy-pasted rather than written by humans. They start with an @ and contain hex characters after that. CIDs take up 256 bits.

ref = ((#Ref (#Run Byte)) @deadbeef)

As a special case, 160-bit values are padded with zeros to 256 bits. This is to accommodate SHA1 hashes.

S-Expressions

In Spore, computation is expressed by enclosing a function and an input to that function together in parenthesis. The whole thing, from ( to the corresponding ), is called an S-Expression, or SExpr for short.

Apply a Procedure

twice = \(x: B32) B32 (#add x x)
(twice 50)

evaluates to 100 :: B32

The first element of the SExpr is evaluated first. If it is a procedure, then that procedure is applied to a product created from the rest of the elements of the SExpr.

Eval a Lazy

An SExpr with a single element will evaluate a Lazy value.

lazy = \B32 (#add 1 2)
(lazy)

evaluates to 3 :: B32

Type an Expression

You can force any expression to be a certain type by having a type in the first position of the SExpr.

tenAsB4Naturally = 10
tenAsB32 = ((#Array Bit 32) 10)

Both of these are fine

3array = ((#Array String 3) ["a", "b", "c"])
list = ((#List String) ["a", "b", "c"])

Access a Field

A Tuple is like a function from indices to values.

aTuple = {"a", 2, \B32 4}

(#check (aTuple 0) "a")
(#check (aTuple 1) 2)
(#check ((aTuple 2)) 4)

Primitives

Primitives are operations exposed by the compiler. They are defined in the compiler's source, not anywhere in the Spore source. A Primitive always starts with #, followed by alphanumerics and symbols.

Importing

  • mypkg = (#import "a/b/c") imports a directory of Spore files at a/b/c.
  • mypkg = (#import "dir/file.sp") imports a single Spore file at dir/file.sp.

Embedding

  • data = (#embedFile "assets/logo.bin") evaluates to a String (a (#List Byte)) holding the referenced file's bytes, like a string literal read from a file. The path is resolved like an import (relative to the containing source, or prefixed such as "core:..."), the file is read at compile time, and a missing or unreadable path is a compile error. It may appear in any value position.

Math

  • (#add 1000 1000) performs modular arithmetic on integers of the same type. 1000 has type (#Array Bit 10) naturally.

Types

  • #Bit the bit type.
  • (#Array ElementType len) evaluates to a new Array type.
  • (#List ElementType) evaluates to a new List type.
  • (#Product TypeA TypeB TypeC)
  • (#Sum TypeA TypeB TypeC)
  • (#Proc Input Output)
  • (#Lazy Output)
  • (#Ref ElementType)
  • (#Distinct BaseType anyValue)
  • #AnyType a type capable of holding instances of any type.
  • #AnyValue a type capable of holding instances of any value (that's everything).
  • #AnyProg a type capable of holding any program.

Errors

  • (#panic value) faults the MVM.
  • result = (#brace thingThatCouldPanic arg1 arg2). Returns a (#Sum Output #AnyValue) if thingThatCouldPanic has return type Output.
  • (#check expr) panics if expr is not equal to 1.
  • (#check expr1 expr2) panics if expr1 and expr2 are not equal.

Store

  • ref = (#post value) turns any T into a (#Ref T).
  • value = (#load ref) turns any (#Ref T) into a T.

Miscellaneous

  • (#=> data proc) is the same as (proc data)
  • 'expr quotes the next value: it evaluates to an #AST value instead of compiling the expression.
  • #AST is the type of a quoted AST node.
  • (#compileAST ast) compiles a compile-time #AST at the call site, so it resolves names in the call site's scope.
  • (#TypeOfEval ast) returns the AnyType that (#compileAST ast) would produce.
  • (#TypeOf 123) returns the type of 123, which is B7, without evaluating the expression.
  • (#TypeOf (#post "abc")) => (#Ref String) and does not evaluate the expression. "abc" will not be posted to the store.
  • (#anyValue "a string") creates an AnyValue wrapper containing "a string"
  • (#anyType #Bit) creates an AnyType wrapper containing (#Bit)
  • (#unwrap Type x) unboxes an AnyValue or AnyType and evaluates to the content it holds. Against an AnyValue it panics unless the wrapped value is exactly Type, and evaluates to that value. Against an AnyType it panics unless the wrapped descriptor's kind matches Type's, and evaluates to that type descriptor.
  • (#unwrap List anyValue) recovers a List value; (#unwrap List anyType) recovers the List type descriptor.

Macros

  • X = (#fnr (params) body) defines a find-and-replace macro: (X a b) substitutes the argument forms into body.
  • X = (#macro fn) defines a function macro: fn takes #AST arguments and returns #AST, and (X a b) evaluates it at compile time and compiles the AST it returns.

Namespaces

Symbols in Spore are used to identify values in a namespace. The current local namespace's symbols can always be used directly, as we have seen previously. To access symbols in other namespaces, like an imported namespace, use the . operator.

Symbol within Import

bits = (#import "abc")

(#check (bits.xor 0 1) 1)
(#check (bits.xor 1 1) 0)

Exporting Symbols

Symbols can be exported from the current namespace, for import into another namespace, by prefixing them with ..

localOnly = "abc123"
.exported = "exported"

Namespaced Types

The #Struct and #TaggedUnion types:

myStructType = (#Struct {
  .fieldA = B32
  .fieldB = String
})

myStructValue = (myStructType {
  .fieldA = 1234

  tmp1 = "foo"
  tmp2 = "bar"
  .fieldB = (#flatten [tmp1 tmp2])
})
myTaggedUnionType = (#TaggedUnion {
  .fieldA = B32
  .fieldB = String
})
myTaggedUnionValue1 = (myTaggedUnionType {
  .fieldB = "foobar"
  // also providing .fieldA is not allowed
})
myTaggedUnionValue2 = (myTaggedUnionType {
  .fieldA = 12345
  // also providing .fieldB is not allowed
})

Core Library

Spore has a core library of packages. It is importable using the core: prefix with #import.

This is not the same as Want's own standard library, accessible with the want: prefix.

The bits package provides functions for bitwise operations.

bits = (#import "core:bits")

The ns package provides helpers for building namespaces (NS, get, mkEntry), used when a Spore entrypoint is handed its context as a namespace.

ns = (#import "core:ns")

Mycelium

Mycelium is a typed data serialization format. Mycelium values are made up of fundamental values: Bits, Refs, and Ports, and all other types composed from this fundamental set.

Mycelium values are immutable; any change to a Value is necessarily a new Value.

0. Opaque

Opaque is a type that gives no information; it cannot be instantiated. It exists mostly to make the 0 value for TypeCodes an obvious error.

1. Bit

The Bit type requires no additional information; it has two Values 0 and 1.

2. Array

An Array is a finite sequence of elements, all the same type, where the length of the sequence and the type of an element are known at compile time. Arrays are parameterized by 1 type (the element type) and 1 value (the length).

3. Run

A Run is a contiguous segment of elements, all the same type, where the length is unknown. Runs cannot be used directly; they must always exist within a Ref.

4. Ref

A Ref is a reference to another value, kept in the store. Refs are all the same size, and are redeemable for a Value. A Ref can be converted to a Value by reading from the store. Refs are parameterized by 1 type and 0 values.

5. Sum

A Sum has several possible integer-identified fields. Each field has its own independent type. Only one field may be set at a time.

6. Product

A Product is a collection of fields, each of which has an independent Value of an independent type. All fields must be set. Products are parameterized by N types and 0 values.

7. List

A List is an ordered collection of elements, all the same type, with a length known at runtime. A List is represented as a Ref[Run] of known length, where the length is tracked in the list. Lists are parameterized by 1 type and 0 values.

8. Lazy

Lazy is a program which requires no additional input, producing an output of a known type. Lazys are parameterized by 1 type and 0 values.

9. Procedure (Proc)

A Procedure is a program which requires additional input, producing an output of a known type. Procs are parameterized by 2 types (input and output) and 0 values.

10. Port

A Port is a token for communication outside the Mycelium virtual machine.

11. Distinct

A Distinct type is marked with a value. This causes the type to become distinct from its base type. A Distinct type is parameterized by 1 type (the base) and 1 value (the mark).

12. Any

Any is a family of formats that can hold a wide variety of values with different types. These values must be converted to another type in order to be used.

AnyValue

AnyValue is a type capable of holding any value.

AnyType

AnyType is a type capable of holding the type of any value.

AnyProg

AnyProg is a type capable of holding any program. Programs are Runs of B32 instructions.

255. Type2

The type of a Type is a Type2. Type2s always take up 32 bits in Mycelium.