Introduction
What is Want?
Want is a build system (WBS).
Software projects use build systems to reliably and reproducibly derive artifacts from source code. WBS manages setting up and tearing down isolated environments to build artifacts like static sites, assets, and executables for release or testing.
Why would I use Want?
If you've ever tried to get up and running with a new software project, you will have dealt with the problem of downloading all the tools that the project uses.
Even for simple tasks like building the docs, or producing shippable executables, you may have to install several tools. This requires developers to know about and understand tools which they will never work with themselves, to merely benefit from the output of a computation that others have set up.
If you or someone else put in the time to get something to work, it should also just work for you going forward. You shouldn't have to replay previous steps like: picking the right version, learning command line arguments, order of operations, etc.
With Want, every build target you add will be computed in exactly the same way on your laptop tomorrow, on your co-worker's development machine, and in CI. Want forces build steps to be fully and exactly specified, so they can be reproduced by anyone at anytime, in any environment.
Why is it called Want?
You write down what you want, and Want builds it.
More seriously: .want and .wants were good extensions for the files where you specify build targets, and naming the entire build system after them came later.
Using Want
This section will get you up and running with Want.
Most of the time you should run
want build. This command is suitable to use as a pre-merge check. It will build all the targets in the current module. It does not and cannot perform publishing or other side effects.
Creating a Want Module
Want operates on Modules. A Module is just a filesystem tree with a file named
WANT at its root. The WANT file holds module level configuration, and its
presence is what tells Want that a directory is meant to be built.
To start a new module, run want init in your project's root. It writes a
WANT file with the default module config — the want standard library as a
dependency — and a default .wantignore:
.deps = [
{"want", \(_: Ctx) FS LATEST},
]
See Module Files for what the WANT file can contain,
and how the sibling .wantignore file controls which paths are imported.
Creating a Build Target
Want calls the things that you, the user, want, but don't yet have, and need to build, Targets.
A target is declared by a file ending in .want. Create a file named
hello.txt.want at the root of your module, and fill it with the following:
build = (#import "want:build")
paths = (#import "want:build/paths")
.input = [
(paths.unit "./input.txt")
]
.do = \(ctx: build.Ctx) build.FS {
(build.pick ctx.jobs ctx.input "input.txt")
}
The file declares two things: the module paths the build reads (.input), and
the computation that produces the output (.do). Here do simply picks the
input file input.txt out of the target's input tree and returns it.
Create input.txt next to it containing Hello World!, then run
$ want cat hello.txt
Hello World!
The unit file hello.txt.want produces the output path hello.txt (its file
name minus the .want extension), and want cat reads that file out of the
build output.
Inspecting the Build Output
Normally cat hello.txt would print the source file, but want cat reads from
the build output, while the regular cat reads from the regular filesystem,
which is the build input.
Want organizes build targets into an output filesystem.
This filesystem only exists virtually, within Want, but it can be inspected using UNIX-like tools (ls and cat).
want ls <path> will list the contents of a tree in the build output. Compare this to the output of regular ls.
want ls and want cat build only what the path they print could come from:
want cat hello.txt builds the target that produces hello.txt, and want ls fetch the targets that can add entries under fetch/. A source file needs
no targets at all, so want cat input.txt just prints it. (want ls with no
path lists the whole output, so it builds the whole workspace.)
want serve-http <addr> builds the output and serves it as a read-only HTTP
file tree (for example want serve-http 127.0.0.1:8080): directories render
as an index of links, and files are written with a content type guessed from
their extension. Stop it with Ctrl-C.
This simple translation of inputs to outputs is what makes Want so easy to use and reason about. You will never find yourself in a situation where you don't know when/how/where a Target is being computed.
Armed with just want ls, want cat, and want build, you can get pretty far. To learn more about what you can build with Want, read the Build Configuration section.
Generating Files
want generate <path...> builds the local module over the query (exactly as
want build does) and then writes every derived file the query matches into
your working tree. Generated files remain part of the source tree on the next
build, so they must be ignored before they can be regenerated in place: add
the generated paths (for example *.pb.go) to .wantignore.
# write every derived file ending in .pb.go into the worktree
$ want generate '*.pb.go'
generate: wrote api.pb.go
generate: wrote messages.pb.go
want generate refuses to overwrite a file that is part of the source
snapshot (the worktree with .wantignore applied): it reports every conflict
and writes nothing at all if any match conflicts. A query that matches no
derived files is an error, and a missing query or an unknown flag is a usage
error.
Publishing and Secrets
A unit file may export an optional .sink proc: the side effect to run after
the build is green. Sinks do not run in a normal build; want build --publish runs them. A sink belongs to its target, consumes the target's
output as ctx.input, and reaches the outside world through the job port and
the capabilities in build.SinkCtx.
Secrets reach a sink through ctx.secrets, a list of ports in the order the
WANT's .secrets grants name them. Supply them on the command line:
# one raw secret
want build --publish --secret mySecret:/path/to/my_secret.txt
# or one spore map carrying several named values
want build --publish --secrets cloudflare:~/.config/want/cloudflare.sp
want build --publish --secrets github:~/.config/want/github.sp
Only the targets the build query names directly are published; a target pulled
in to satisfy a dependency is built but its sink does not run. A publish build
fails if a directly queried target (or the sink that will run for it) is
granted a secret that was not supplied. See
Module Files for the .secrets grant syntax.
Concepts
This section provides a conceptual overview of WBS. If something doesn't make sense or you want to learn more about WBS's architecture, this is a good section to read.
TLDR
Want represents all build inputs and outputs as values in a content-addressed data model called Mycelium. Every value has a cryptographic hash, and equal values have equal hashes, so Want can easily tell whether two trees, files, or computations are identical.
A build is a transformation from an input filesystem (a Module, plus the outputs of its dependencies) to an output filesystem. Everything a computation reads is named explicitly, and the only way to reach outside the module is through the small set of executors, so a computation can always be cached and replayed.
The pieces you will read about next:
- Data Model: how Mycelium values and the Want filesystem (
wantfs) are shaped. - Tasks & Jobs: how units of work are named, cached, and run.
- Executors: responsible for executing Tasks, in the context of a Job.
- Spore: the language that want files are written in.
Data Model
Mycelium
Want stores all data in the Mycelium format. Mycelium is a serialization format like JSON or Protocol Buffers. In addition to the functionality of those two technologies, Mycelium allows functions to be sent over the wire and stored.
Mycelium uses cryptographic hashes as references. So all data-structures stored in Mycelium are cryptographic commitments to the full transitive closure of a node in the data structure.
Mycelium allows Want to cache anything in a meaningful representation. This covers Bits, Bytes, Strings, Programs, and Procedures, as well as Lists, Arrays, Products, and Sums of all those. Instead of converting those to another less accurate format like JSON, which also throws away any concept of reference, Mycelium allows a faithful representation.
The Want Filesystem (WantFS)
WBS stores all file data in the WantFS format. WantFS is built on top of Mycelium, and inherits the property of being content-addressed, and copy-on-write.
WantFS is used to represent file data whenever it has to be cached, or presented as the input to a build step. The output of want files is also stored in WantFS.
The first step in a build is to import the module from the local filesystem into WantFS.
Tasks & Jobs
Tasks
A Task is a well-defined unit of work. Well-defined means that all the information that could be required to complete the Task is explicitly contained in the Task.
A Task has 2 parts: an operation name and an input (an AnyValue):
Task = (#Struct {
.op_name = String
.input = AnyValue
})
The operation names one of a small number of built-in operators, for example:
want.pickwant.placewant.assemblefsdownload.treelinux.cspawnwasm.runmvm.apply
The input is the operator's argument value. Both parts are part of the hash which becomes the Task's identity, so a Task with the same operation and input is the same piece of work.
Want caches the output of Tasks when they complete, and checks this cache
before computing a Task. An immediate rerun of want build without any
changes therefore produces all the same Tasks, and every one of them can be
skipped.
Jobs
Jobs are used to track running computations in Want.
Jobs are created to compute Tasks. A Job carries active state, like whether
it has completed and when it started and finished: it is spawned when created,
running once its task is started, and done (with an ok value or an error) once it
has an outcome. While a running Job waits for its queue resources it is
reported as running with a non-null queuing_since, and the time it has spent
waiting accumulates in queued_acc, so queuing is a subset of running rather
than a state of its own.
Jobs are organized into a hierarchical tree. All the major operations in Want create a new root Job; that Job may spawn additional child Jobs.
Child Jobs give a computation access to additional compute, since children run concurrently, and they are each an opportunity for caching.
The Job System Port
A build computation is not given access to the host; instead it is handed a jobs port, a capability it uses to spawn and await child Jobs. In Spore this
port has the type build.JobCtx, and a Task's do proc receives it on
ctx.jobs.
Through the port a computation can:
spawn jc limits op input: start a child Job for a Task and get itsJobIDimmediately, under a resource ceiling. A Job that runs past itstimeoutceiling is cancelled automatically.limits jc: the current Job's resource ceiling. Spawning with it passes the ceiling down unchanged.await jc id: block until that child finishes and return its result; a child that failed faults the parent.inspect jc id: check a child's state without blocking.awaitAny jc ids: block until any one of a set of children finishes.collectJobLogs jc id name: read a child's named log stream (stdout,stderr, ...). On a cached task the bytes come from the job that computed it, so the log of a cached child is still readable.
A Job cannot spawn root or sibling Jobs; it can only spawn its immediate children. This keeps runaway computations from escaping their parent, and gives every Job an audit trail back to the user who asked for the result.
Executors
WBS dispatches Tasks to Executors. An Executor knows how to do particular types of Tasks, and can provide any additional context.
Executors are the small set of built-in operations that can touch the outside world or run programs. They are the only place where a computation can do something non-deterministic (like use the network), and each is designed so that its successful outputs are fully determined by its inputs.
The persistent task cache and single-flight coordination live in
src/sys/exec.zig: its Executor wraps the concrete executor behind a
Backend interface and owns the cache. The executors under src/exec are
backend implementations and never touch LMDB or the task cache.
Want Executor
Tasks directly related to planning, building, or manipulating files are executed by the Want Executor.
Here are a few examples:
want.pickextracts a subpath from a WantFS instance and returns a new instance containing just the subpath shifted to the root.want.placeis the inverse of pick: it mounts a segment under a path, prefixing every entry's path.want.assemblefsassembles named single-file segments into one tree.want.buildbuilds a want module provided as a WantFS instance.want.planis the planning portion of the build, separated so that it can be cached separately.
I/O Executor
This executor performs all input and output tasks. If something is incoming or outgoing to/from the build system, then it goes through this executor.
Linux Executor
This executor spawns Linux containers to execute tasks. Think of this as "Linux as a Service".
VM Executor
This executor runs hardware virtual machines with crosvm (https://crosvm.dev).
vm.amd64 boots an amd64 guest from a kernel, an optional initrd, any block
devices the task provides, and any virtio-fs shares it exposes (read-only
snapshots of stored trees, fresh host-disk scratch, or a named cache
persisted across runs), all as WantFS segments.
A task may declare at most one output — a fresh disk or an empty share — which
is captured back into WantFS when the guest stops.
A task may also declare network interfaces. Each becomes a virtio-net device
whose queues crosvm exports to a vhost-user-net backend the executor runs; the
backend moves the frames to the userspace IPv4 stack (usnet), so a guest
reaches the host network with no TAP and no host networking privileges. The
internet and log capability ports resolve exactly as for linux.cspawn,
and a task with a network interface is never cached.
Every run also attaches a vhost-user-console device with two ports named
stdout and stderr. A guest's init can open them (the vmzig init does) so
the process's two streams land in the job's stdout and stderr logs rather
than being merged into the serial console; the kernel's serial console and
crosvm's own diagnostics go to console and vmm.
Think of this as "Linux as a Service", with the guest kernel coming from the
task instead of the host.
Wasm Executor
This executor runs WebAssembly programs.
MVM Executor
This executor performs computations defined in the Mycelium format.
Mycelium Procs can be applied to data with mvm.apply.
Mycelium Lazys can be evaluated with mvm.eval.
All of the want files are evaluated using this executor.
Spore Executor
This executor calls into the Spore compiler. It is used to compile want files, module configuration files, and Spore libraries into Mycelium programs, which can be executed by the MVM executor.
Build Configuration
Want configuration is done using 3 kinds of files, all written in Spore:
- the
WANTfile at the root of every module (Module Files), - unit files ending in
.want(Unit Files), - multi files ending in
.wants(Multi Files).
When want build runs, it:
- Checks that the current directory has a
WANTfile which is a valid module config; if not, it exits with a "not a module" error. - Finds every file ending in
.wantand.wants, and evaluates each into one or more Targets. - Validates that the Targets together are a valid build plan.
- Runs the build according to the plan.
Each .want/.wants file is evaluated independently as a single-file Spore
body, and must export the fields described for its kind.
The build departs from the module's source files only through the paths a
target declares in its .input. Which paths are imported into the module in
the first place is controlled by the root .wantignore file, described in Module Files.
Module Files
Want operates on units called Modules. A Module is a filesystem tree with a file named WANT at its root.
The WANT file provides module-level configuration and doubles as a sanity
check that a directory is supposed to be built by Want. It is a Spore body that evaluates to a module config:
.deps = [
{"want", \(_: Ctx) FS LATEST},
]
.deps
.deps lists the module's dependencies. Each dep is a tuple of a name and a
proc that produces the dependency's filesystem from a build context
((#Product String (#Proc build.Ctx build.FS))). At plan time every dep's
proc runs and its returned filesystem is registered as a Spore source under
<name>:; planning files import its packages as <name>:pkg.
The WANT file is compiled with a preamble binding Ctx (build.Ctx), FS
(build.FS, a wantfs.Segment) and LATEST (the compiled-in want standard
library as an FS), so the default dep above gives the module the standard
library under want: without any imports. The spore standard library
(core:) is always implicit; a module that does not declare a want dep has
no want stdlib. want init writes the default config above.
.budget
.budget optionally sets the total resource ceiling for a build of the
module, including everything its child jobs claim. It has the same shape as
build.Limits: memory in bytes, cpu in microseconds per second (one core is
1000000), and a wall-clock timeout in seconds.
.budget = {
.memory = 1073741824 // bytes
.cpu = 2000000 // microseconds per second (2 cores)
.timeout = 600 // seconds
}
The export is optional: a WANT with no .budget has no module-level ceiling,
and the root job's own limit (the config directory's .budget, or 75% of the
host) applies. When .budget is present its fields are literal ceilings: the
build job inspects its own spawn-time ceiling and claims the difference
between it and the budget, so the module's child jobs can only draw the budget
in total, and each child is spawned with the budget as its ceiling. A zero
memory or cpu is therefore a zero ceiling, which leaves the build with no
resources for that dimension (it will block or fail); it is not "unset". If
the build job's own ceiling is already below the budget, nothing is claimed
and the child ceiling is unchanged. A zero .timeout means no module-level
timeout (the job's own timeout is kept). .timeout is a per-job wall-clock
ceiling, so it bounds each child job, not the build job itself. want init
does not write a .budget (a WANT with none builds unconstrained at the
module level).
.secrets
.secrets grants named secrets to the module's planning files. It is a list
of {paths, names} tuples, where paths is a paths.Set matched against a
planning file's module-root-relative path, and names is the list of secret
names that file may access:
paths = (#import "want:build/paths")
.secrets = [
{{.all = {}}, ["cloudflare"]}, // the whole module
{(paths.prefix "site/"), ["S3_KEY"]}, // files under site/
{(paths.unit "build/doc.want"), ["DOCS_TOKEN"]}, // one file
]
Only want build --publish uses secrets. Each matching target's build and
sink contexts receive the granted names as a (#List (#Port Unit Unit)) in
grant order (first occurrence wins), so a sink reads the first granted secret
with (#listslot ctx.secrets 0). The names must be supplied on the command
line:
-
--secret NAME:PATH— a single secret: the file's bytes. -
--secrets NAME:PATH— a spore module whose exports are the map entries:.CLOUDFLARE_API_TOKEN = "..." .CLOUDFLARE_ACCOUNT_ID = "..."
--secrets cloudflare:~/.config/want/cloudflare.sp supplies the map under the
name cloudflare, matching the grant above. A GitHub release upload takes a
file exporting .GITHUB_TOKEN in the same way:
--secrets github:~/.config/want/github.sp. A name supplied twice is an
error; names supplied but unused are ignored. A publish build fails before
running any target if a file it will publish is granted a name that was not
supplied. Secret values are loaded into memory (never the content store), and
only port tokens travel through tasks. Keep secret files outside the module
tree (or dot-ignored, as .* in the default .wantignore) so the workspace
import does not capture them as ordinary sources.
.wantignore
Which paths are imported into the module is configured separately, by a root
.wantignore file. It uses gitignore-style patterns, one per line: any path
matching a pattern is left out of the imported module tree.
By default a project that uses Git should ignore its VCS metadata and build outputs. For example:
.*
zig-out/
zig-cache/
.* ignores every dotfile (including .git), which keeps the import fast; the
other entries ignore build output directories.
.want is always ignored, and files starting with .want cannot be imported.
Unit Files
Unit files end in .want. Each unit file declares a single Target.
A unit file is a Spore body with two exports:
build = (#import "want:build")
paths = (#import "want:build/paths")
.input = [
(paths.unit "./input.txt")
]
.do = \(ctx: build.Ctx) build.FS {
(build.pick ctx.jobs ctx.input "input.txt")
}
.inputis a list of PathSets the target reads; the list is implicitly OR'd together. An empty list means the target reads nothing..dois a proc that is called to produce the build. It is handed abuild.Ctx(its job port onctx.jobs, and its filtered input tree onctx.input) and returns anFS.
The output path
A unit file does not declare its output; it is derived from the file's name.
A file hello.txt.want produces the output path hello.txt, and a file
build/app.want produces build/app.
The FS returned by do is shifted into place under that path. A flat
single-file FS (the result of picking a file) lands exactly at the output
path; a tree becomes the subtree beneath it.
The build fails during planning if the derived output path already exists in the source tree, so a target can never silently overwrite a source file.
Path rules
Paths in .input are module-root-relative, with two exceptions:
- A path beginning with
./is relative to the directory of the unit file. Ina/build_my_output.want, the input./input.txtnamesa/input.txt, whileinput.txtnames the file at the module root. - Paths may not begin with
/(absolute paths are rejected), may not be empty, and may not contain a..segment.
Inputs implicitly exclude the target's own output path.
Example: download a rootfs
fetch.want downloads a pinned Alpine minirootfs and returns it:
build = (#import "want:build")
alpine = (#import "want:linux/dist/alpine")
.input = []
.do = \(ctx: build.Ctx) build.FS {
(alpine.rootfs ctx.jobs)
}
The result lands at fetch/ in the build output. want ls fetch lists it,
and want build computes the whole module.
Multi Files
Multi files end in .wants. Unlike a unit file, a multi file declares a list
of Targets, and each Target must declare the paths it is allowed to write.
A multi file is a Spore body with a single export, .targets:
build = (#import "want:build")
paths = (#import "want:build/paths")
.targets = [
{
.input = [
(paths.unit "./input.txt")
]
.output = [
(paths.suffix ".out.txt")
]
.do = \(ctx: build.Ctx) build.FS {
// generate a bunch of files ending in ".out.txt"
}
}
]
.inputis a list of PathSets the target reads, exactly as for a unit file..outputis a list of PathSets the target may write to. TheFSreturned fromdois filtered through this set: writing to an undeclared path fails the build..dois the proc that produces the build, the same as for a unit file.
Rooting
Every target in a multi file is rooted at the parent directory of the .wants
file. The FS returned by do is positioned there, and .output is
interpreted there too.
A protobuf.wants file at the root of a module produces files at the module
root. If it were at a/b/c/protobuf.wants, and its do returned a filesystem
containing messages.pb.go and api.pb.go, the resulting output would be:
a/b/c/messages.pb.go
a/b/c/api.pb.go
The .output declaration does not need to repeat the prefix. Either of these
would accept the files above:
.output = [
(paths.unit "messages.pb.go")
(paths.unit "api.pb.go")
]
or, more flexibly, allowing the names to change as long as they keep the suffix:
.output = [
(paths.suffix ".pb.go")
]
Why declare outputs?
A unit file's output path is fixed by its name, so conflicts with other targets
are impossible. A multi file can write anywhere under its directory, so
declaring .output lets Want check, before anything runs, that no two targets
claim overlapping outputs, and that a target only writes where it said it
would.
Standard Library
Want ships a standard library of Spore packages.
Planning files can import these packages.
An import names a package (a directory of .sp files) or a single
file.
Standard-library packages are namespaced: the want standard library is want: and the spore standard library is core:.
An import with no colon resolves in the same source as the importing file — a module-relative path in your module, or a sibling package within a standard library:
ns = (#import "core:ns")
build = (#import "want:build")
linux = (#import "want:linux")
paths = (#import "want:build/paths")
Callers are responsible for spawning and awaiting Jobs through their job port; the standard library functions take that port (build.JobCtx) as their first argument, by convention.
A tool package may set up a package-specific context
instead and take it first: the zig and go commands take a zig.Ctx or
go.Ctx (built with new from the job port, limits, platform, and version).
build
The core of the build library (stdlib/build).
Types
FS: a wantfs tree. This is the value adoproc returns and the type almost everything operates on.Ctx: the build context handed to adoproc:Ctx = (#Struct { .jobs = JobCtx .input = FS .secrets = (#List (#Port Unit Unit)) })secretsis empty unless the build publishes; seeSinkCtxbelow.SinkCtx: the context handed to a unit file's.sinkproc bywant build --publish:(#Struct { .jobs = JobCtx .input = FS .internet = (#Port Unit Unit) .secrets = (#List (#Port Unit Unit)) }).inputis the target'sdoresult;secretsis the file's granted secret ports, in the order the WANT's.secretsnames them (supplied with--secret NAME:PATHor--secrets NAME:PATH). Sinks run only for targets the build query names directly, and a publish build fails if a directly queried target (or its running sink) is granted a name that was not supplied.JobCtx: the jobs port used to spawn and await child Jobs.Task,JobID,JobState,JobResult: the job system's wire types.Limits: the resource ceiling a spawn requests for its child ((#Struct { .memory = B64 .cpu = B64 .timeout = B64 })).memoryis bytes,cpuis microseconds per second, andtimeoutis seconds (0 means no ceiling); a child running longer thantimeoutis cancelled.
Jobs
spawn jc limits op input -> JobID: start a child job for a Task underlimits.limits jc -> Limits: the current job's spawn-time ceiling. Passing it tospawnpropagates the ceiling down the tree unchanged.await jc id -> AnyValue: wait for a child; a failed child faults the parent.inspect jc id -> JobState: check a child without blocking.awaitAny jc ids -> JobID: wait for any of several children.collectJobLogs jc id name -> String: read a child's named log stream ("stdout","stderr", ...) as its bytes. If the child's result came from the task cache, the bytes come from the job the cache recorded as the producer, so a cached task still exposes the logs that produced it; a stream that was never written reads as the empty string.
Filesystem
pick jc root path -> FS: re-root the tree atpath; picking a file yields a flat single-file FS.pickRoot jc root path -> FS: the same operation, named for re-rooting a directory.fileAt jc root path -> FS: pick a single file (a synonym forpick).place jc segment path -> FS: mountsegmentatpath, prefixing every entry's path; the inverse ofpick.fsAssemble jc ents -> FS: mount named single-file FSes at their names:(build.fsAssemble jc [ (build.ent "bin/hello" exe) ])ls jc root path -> (#List String): the immediate children of a directory.filterPaths jc root set -> FS: keep only the entries ofrootwhose path the PathSetsetcontains; entry paths are unchanged, so the filtered tree is not re-rooted.emptyDir jc -> FS: a tree with no entries.
Downloads
downloadFile jc task -> FS: fetch a file from one oftask.urls, verified againsttask.hashundertask.algo(SHA1,SHA256, orBLAKE3); a non-NONEtask.compressiondecompresses the verified payload into the returned file.downloadTree jc task -> FS: fetch an archive (tar, zip, or deb) and expand it into a tree.task.algonames the archive's hash algorithm andtask.compressionthe archive payload's compression (GZIP, XZ, ZSTD, or NONE); decompression is explicit, never sniffed.
OCI
downloadOciImage jc task -> FS: pull an OCI/Docker image into an OCI layout.pushOciImage jc task -> PushedOCIImage: push an OCI layout to a registry repository and attach the task's tags. The push is idempotent: blobs, manifests, and tags the registry already holds are skipped. The registry credentials come fromtask.secrets, a--secretsmap whose exports include.OCI_USERNAMEand.OCI_PASSWORD.OCIImageTask(url,repo,algo,hash),PushOCIImageTask(url,repo,tags,image,secrets), andPushedOCIImage(digest,tags): the OCI wire types.
Importing build also gives you its download, job, and oci modules'
exported functions, since a package is compiled from every immediate child
.sp file.
build/paths
The PathSet constructors used in .input and .output:
Set: the PathSet tagged union itself ((#TaggedUnion {.unit = String .prefix = String .suffix = String})), the parameter type ofbuild.filterPaths.unit p -> Set: the single pathp.prefix p -> Set: every path beginning withp.suffix s -> Set: every path ending ins.
linux
The Linux container executor (stdlib/linux).
run jc cspec -> CSpawnResult: run a container described bycspec, and return its exit code plus the tree captured from its output mount.CSpec: the container description:init,args,env,wd,mounts,network,stdin,stderr,stdout.initMemfd fs: run an executable from a flat single-file FS (copied into an anonymous memfd).initPath p: run the executable already at pathpinside the container.- Mount sources:
snapshot fs(read-only),output,emptyDir,tmpfs n,drive n,overlay rlayer wlayer(lower may be asnapshot/drive; upper must be a writabledrive/tmpfs),devNull,procfs,sysfs,cgroup2fs,devpts. mount dst src -> Mount: pin a source at a container path.net name backend,loopback,private,internet cap: network interfaces.- stdio:
stdinNull,stdin ctx,stdout ctx,stderr ctx,outputLog,outputClosed,stdinClosed. fileAtPath jc root path -> FS: pull a file out of a container tree.
linux/dist/alpine
A pinned Alpine minirootfs: a small musl userland with the busybox shell.
rootfs jc -> FS: expand the pinned minirootfs tarball into a tree.fileAt jc path -> FS: pull a single file out of it.libgcc jc -> FS: expand the pinned Alpine libgcc package (usr/lib/libgcc_s.so.1), for tools whose dynamic binaries need the gcc runtime; the minirootfs ships none.version,arch: the pinned release.
linux/dist/debian
A pinned rblaine95/debian base image (a GHCR mirror of Docker Hub's
official image). Like Alpine, but a full glibc userland.
image jc -> FS: pull the pinned image as an OCI layout.rootfs jc -> FS: unpack the image's root filesystem.fileAt jc path -> FS: pull a single file out of it.
linux/dist/arch
Arch Linux packaging on top of a pinned official Arch container image (with
the bash, pacman, and bsdtar makepkg itself uses). package takes
already-built artifacts as a build.FS tree and produces the
.pkg.tar.zst pacman installs; nothing is compiled.
rootfs jc -> FS: the pinned image's root filesystem.fileAt jc path -> FS: pull a single file out of it.version: the pinned image build.Spec: the required metadata (.name,.version,.arch,.desc,.builddate,.files).versionis the full pkgver including the pkgrel (and optional epoch), andbuilddateis a decimal Unix timestamp that fixes every file's mtime, so equal inputs produce equal packages.Meta: the full metadata record;meta spec -> Metafills the optional fields with defaults.withUrl,withPackager,withLicense,withDepends,withOptdepends,withProvides,withConflicts,withReplaces,withGroups,withBackup,withInstall: rebind one optional field.filename meta -> String: the output file's name,name-version-arch.pkg.tar.zst.pkginfo meta -> String: the rendered.PKGINFO.package jc meta -> FS: assemble the package and return the tree captured from the output mount (one file,filename meta); usebuild.fileAtfor the package as a flat single-file tree. The target faults if the packager fails.checkExit code -> Unit: fault unless a container exited zero.
The payload is a tree rooted at /, built with build.fsAssemble or picked from
a build output:
files = (build.fsAssemble jc [
(build.ent "usr/bin/hello" exe),
])
m = (arch.meta {.name = "hello" .version = "1.0-1" .arch = "x86_64"
.desc = "..." .builddate = "1700000000" .files = files})
m = (arch.withLicense m ["MIT"])
pkg = (arch.package jc m)
docker
pull jc task -> FS: pull an OCI image.umoci jc -> FS: the pinned umoci binary.rootfs jc img -> FS: unpack a pulled image's root filesystem.
git
A pinned static git build (darkvertex/static-git), used to give a container a
git binary with no package manager or network.
tool jc -> FS: the expanded static git tree (gitat its root). Mount it onPATH(e.g. at/git) and setGIT_TEMPLATE_DIRto an existing directory.
github
Upload release assets to an existing GitHub release (stdlib/com/github).
Credentials come from a --secrets map whose only required export is
.GITHUB_TOKEN, a token with write access to the repository's releases
(contents: write). The preferred setup is a spore file:
.GITHUB_TOKEN = "..."
want build --publish --secrets github:~/.config/want/github.sp
Ctx: the GitHub context ((#Struct {.jobs = JobCtx .secrets = (#Port Unit Unit)})); build it once withnew.new jc secrets -> Ctx: bind the job port and the secrets port granted by the WANT's.secrets.UploadOpts:(#Struct {.owner = String .repo = String .tag = String}), naming the existing release to attach assets to.Asset: one uploaded asset (.id,.name,.size,.content_type,.browser_download_url).ReleaseUpload: the op's result: the release and the assets it now carries.uploadRelease ctx opts assets -> ReleaseUpload: upload every top-level file inassetsas an asset of the named release; an existing asset with the same name is replaced.
go
The pinned Go toolchain (stdlib/lang/go).
Ctx: the go environment ((#Struct {.jc = JobCtx .limits = Limits .platform = Platform .version = Version .modcache = FS .env = (#List String)})). Every command takes it as its first argument.new jc limits platform version -> Ctx: set up the environment once, then hand it to every command.platform/versionselect the toolchain distribution to fetch;jcandlimitsare the job resources commands run under. The module cache starts empty.withModcache ctx modcache -> Ctx: rebind the module cache to the tree returned bypkg.withEnv ctx env -> Ctx: rebind the command-wide environment. Entries are prepended to the fixed environment and the first entry for a variable wins, so they override it (for exampleGOFLAGS,GOPROXY,CGO_ENABLED).withTarget ctx platform -> Ctx: prependGOOS/GOARCHforplatform, cross-compiling pure-Go builds with the host toolchain.LATEST,v1_27_1: the pinnedVersionvalues;linux_amd64is the pinnedPlatform;GOOS_*/GOARCH_*name every target.dist ctx -> FS: download the pinned distribution and re-root it as a container root.pkg ctx src hash -> FS: rungo mod downloadfor the module treesrc(under the context's environment) in a network-enabled container and return the resulting module cache, audited againsthash.cmd ctx cwd args -> CSpawnResult: run the pinned go tool over the module treecwd(mounted at /src) in a container with no network.argsis the full argv, e.g.["go", "build", "-o", "/out/app", "./cmd/foo"].build ctx modfs entrypoint -> FS: compile the main package atentrypointinto a static executable.test ctx modfs -> CSpawnResult: rungo test ./...; its stdout and stderr are captured as/stdoutand/stderrin the output tree.testBinaries ctx modfs -> CSpawnResult: compile every package's test binaries into the output tree without running them.checkExit code -> Unit: fault the build unlesscodeis zero.
go/dist
The Go release pins and the mirror URLs used to fetch them (stdlib/lang/go/dist.sp,
part of the go package): Version, LATEST, v1_27_1, Platform,
linux_amd64, tarballName, topDir, dist, and mkurls.
python
The pinned Python toolchains: CPython (the interpreter, from
python-build-standalone) and uv (the package manager), each fetched from its
own release artifacts (stdlib/lang/python).
Ctx: the python environment ((#Struct {.jc = JobCtx .limits = Limits .platform = Platform .uv = UvVersion .python = PythonVersion .venv = FS .env = (#List String)})). Every command takes it as its first argument.new jc limits platform uv python -> Ctx: set up the environment once, then hand it to every command.platformselects both distributions;uvandpythonpin the versions to fetch (see below).jcandlimitsare the job resources commands run under. The project environment starts empty.withVenv ctx venv -> Ctx: rebind the project environment to the tree returned bypkg; commands mount it at /venv and prefer itsbin/on PATH.withEnv ctx env -> Ctx: rebind the command-wide environment. Entries are prepended to the fixed environment and the first entry for a variable wins, so they override it (for exampleUV_NO_DEV).UV_LATEST,PYTHON_LATEST,PYTHON_3_13: the pinnedUvVersionandPythonVersionvalues;x86_64_linuxis the pinnedPlatform.uvDist ctx -> FS: the pinned uv distribution (uvanduvx), to mount at /uv.pythonDist ctx -> FS: the pinned CPython distribution, to mount at /python.pkg ctx src hash -> FS: runuv sync --frozenfor the project treesrc(which must contain pyproject.toml and uv.lock) in a network-enabled container with the pinned toolchains, and return the resulting venv, audited againsthash. Mount the result on the context with withVenv before running commands.cmd ctx cwd argv -> CSpawnResult: run any command over the treecwdin the hermetic command container;argvis resolved on PATH through the venv, the pinned interpreter, and uv, and each element is passed verbatim.python ctx cwd args,uv ctx cwd args,sh ctx cwd script:cmdwith the pinned interpreter, uv, or the Debian shell (for pipelines and redirects) as the program.checkExit code -> Unit: fault the build unlesscodeis zero.
python/dist
The uv and CPython release pins and the asset URLs used to fetch them
(stdlib/lang/python/dist.sp, part of the python package): UvVersion,
PythonVersion, UV_LATEST, PYTHON_LATEST, PYTHON_3_13, Platform,
x86_64_linux, uvUrl, pythonUrl, uvDist, and pythonDist.
rust
The pinned Rust toolchain (stdlib/lang/rust). The distribution is musl-hosted:
the containers run the pinned Alpine minirootfs, and rustc links static musl
binaries with its bundled rust-lld, so no C toolchain is needed.
Ctx: the rust environment ((#Struct {.jc = JobCtx .limits = Limits .platform = Platform .version = Version .vendor = FS .env = (#List String)})). Every command takes it as its first argument.new jc limits platform version -> Ctx: set up the environment once, then hand it to every command.platformandversionselect the toolchain distribution to fetch; anyVersionvalue works, so a caller can force a newer rust thanLATESTtracks without editing the stdlib.jcandlimitsare the job resources commands run under. The vendor directory starts empty.withVendor ctx vendor -> Ctx: rebind the vendor directory to the tree returned bypkg; commands mount it at /vendor and use it as CARGO_HOME, so cargo reads the captured source-replacement config there.withEnv ctx env -> Ctx: rebind the command-wide environment. Entries are prepended to the fixed environment and the first entry for a variable wins, so they override it (for exampleRUSTFLAGSorCARGO_*).LATEST,v1_99_1: the pinnedVersionvalues;x86_64_linuxis the pinnedPlatform.rustcDist ctx -> FS: the compiler component (rustc, rustdoc, and the bundled rust-lld), to mount at /rust.cargoDist ctx -> FS: cargo, to mount at /cargo.stdDist ctx -> FS: the precompiled standard library plus the self-contained musl libraries, to mount inside the compiler tree at/rust/lib/rustlib/<triple>/lib.pkg ctx src hash -> FS: runcargo vendor --lockedfor the project treesrc(which must contain Cargo.toml and a committed Cargo.lock) in a network-enabled container and return the vendored crates plus cargo's source-replacement config, audited againsthash. Mount the result on the context with withVendor before running commands.cmd ctx cwd argv -> CSpawnResult: run the pinned cargo over the treecwdin the hermetic container;argvis the full argv, e.g.["cargo", "build", "--release"]. Every dependency resolves from the vendor tree and the network is off.build ctx src bin -> FS: compile the binary targetbinin release mode and return it as "app" at the root of a wantfs tree.test ctx src -> CSpawnResult: runcargo test --locked; its stdout and stderr are captured as/stdoutand/stderrin the output tree.sh ctx src script -> CSpawnResult: run a shell script in the same container, for pipelines and redirects.checkExit code -> Unit: fault the build unlesscodeis zero.
rust/dist
The Rust release pins and the component picks used to fetch them
(stdlib/lang/rust/dist.sp and stdlib/lang/rust/versions.sp, part of the rust
package): Version, LATEST, v1_99_1, Platform, x86_64_linux,
tarballName, topDir, url, rustcDist, cargoDist, and stdDist.
vm
Boots hardware virtual machines through the vm.amd64 executor
(see doc/23_Executors.md). Runs need a KVM host and a crosvm
(WANT_CROSVM or PATH; build/crosvm.want builds a pinned one).
run jc task -> VmResult: boot anAMD64Taskand return crosvm's status and the run's output. A non-zero status faults the build instead of returning a result, so a failed run is never cached.AMD64Task:kernelandinitrd(flat single-file trees;some fswraps an initrd),kargs(extra kernel command line arguments),blockdevices (.raw/.qcow2/.output), andfsshares (.snapshot/.output). The guest sees disks asvda,vdb, ... and shares taggedfs0,fs1, ... in list order; a read-write input device is a fresh copy per run.- A task may declare at most one output, created empty for the run:
outputDisk sizeattaches a fresh disk,outputShare sizea fresh directory. When the guest stops,runcaptures it asOutput{ block = FS | virtiofs = FS | none }: a disk image or a tree, content-addressed like any other FS. - Every run attaches a vhost-user-console device whose ports are named
stdoutandstderr. A guest init that opens the named ports (thevmziginit does) gets the process's two streams in the job'sstdoutandstderrlogs; the kernel serial console and crosvm's diagnostics go toconsoleandvmm. snapshotShare fsshares a stored tree read-only.stdlib/vm/smoke.wantboots a downloaded kernel end to end (see DEV.md).
vm/crosvm
Builds the pinned crosvm, the VMM the vm executor runs (stdlib/vm/crosvm).
crosvm is not pure Rust: minijail's C library (the third_party/minijail
submodule cargo patches in), libclang for bindgen, and python3 for the seccomp
policy build are all part of it. The build containers therefore run a
dedicated Alpine dev environment (the minirootfs plus a pinned closure of
build-base, linux headers, libcap, libclang, python3, and pkgconf, installed
offline) and link through gcc rather than the toolchain's rust-lld.
build jc -> FS: the whole pipeline: fetch the pinned crosvm and minijail trees, install the dev environment, vendor the crate graph withcargo vendor --locked(audited), thencargo build --release --locked --offline --no-default-features --features qcowwith no network. The result holdscrosvmat its root;build/crosvm.wantis the target that materializes it.source jc -> FS,minijailTree jc -> FS: the pinned source trees, fetched withdownload.git_commit_tree.vendorPkgs jc src minijail devenv hash -> FS: the auditedcargo vendorresult, to mount at /vendor as CARGO_HOME.package jc src minijail devenv vendor -> FS: the offline compile, which captures thecrosvmbinary.versions.sppins the crosvm and minijail revisions, the Alpine branch, the name, version, and sha256 of every .apk in the dev environment's package closure, and the audited dev-environment and vendor-tree checksums. The packages are fetched by their versioned URLs and installed offline, so the dev environment does not move when Alpine updates the branch. When a fetch changes, the build faults with the actual checksum, which should be pasted back.
vm/qemu
Pins and fetches a QEMU distribution (stdlib/vm/qemu), mirroring the
lang/zig and lang/go toolchain packages for versions and fetching. QEMU
publishes no prebuilt Linux binaries (qemu.org serves source tarballs and
points Linux users at distribution packages), so the pinned release is the
official source tarball and dist re-roots the expanded source tree. A build
that compiles it into a runnable VMM is future work.
Ctx: the qemu environment ((#Struct {.jc = JobCtx .limits = Limits .platform = Platform .version = Version})).new jc limits platform version -> Ctx: set up the environment once, then hand it todist;platform/versionselect the distribution,jcandlimitsare the resources a build runs under.dist ctx -> FS: download the source distribution pinned byctx.version(checked against its sha256), expand the xz-compressed tar, and re-root it so its top-level directory (e.g.qemu-11.1.2) is the tree root; mount it at /src.versions.sppins the QEMU release tag and the sha256 of its source tarball;LATESTnames the tracked release. QEMU ships only a detached GPG signature, so the hash is computed here and a mismatch faults with the actual checksum to paste back.
zig
The pinned Zig toolchain.
Ctx: the zig environment ((#Struct {.jc = JobCtx .limits = Limits .platform = Platform .version = Version .pkgfs = FS .env = (#List String)})). Commands take it as their first argument.new jc limits platform version -> Ctx: set up the environment once, then hand it to every command.platform/versionselect the toolchain;jcandlimitsare the job resources commands run under. The package directory starts empty.withPkgfs ctx pkgfs -> Ctx: rebind the package directory to the tree returned bypkg; commands mount it at /src/zig-pkg.withEnv ctx env -> Ctx: rebind the command-wide environment. Entries are prepended to the fixed environment and the first entry for a variable wins, so they override it.LATEST,STABLE,v0_16_0: the pinnedVersionvalues;x86_64_linuxis the pinnedPlatform.dist ctx -> FS: download the pinned distribution and re-root it as a container root.pkg ctx src hash -> FS: fetch thebuild.zig.zondependencies ofsrcthroughzig build --fetch, audited againsthash(thezig-pkgdirhash/BLAKE3 checksum).cmd ctx cwd args -> CSpawnResult: run the pinned zig tool over the source treecwd(mounted at /src);argsis the full argv, e.g.["/zig/zig", "build", "test"].buildArgs ctx src args -> CSpawnResult:cmdplus the fixed--build-file,--prefix, and--cache-dirflags, for arbitraryzig buildsteps; returns the exit code plus the captured output tree.build ctx src target -> FS: run thebinariesstep for the Zig target tripletargetand return its output tree.checkExit code -> Unit: fault the build unlesscodeis zero.run ctx args -> AnyValue: run the compiler with stdio wired to the host.watch ctx stdin stdout stderr -> B8: runzig build --watchon the build context's working directory, with stdio wired to the given ports; returns the container's exit code.
zig/dist
The Zig release pins and the mirror URLs used to fetch them (stdlib/lang/zig/dist.sp,
part of the zig package): Version, LATEST, STABLE, v0_16_0, Platform,
x86_64_linux, tarballName, topDir, dist, and mkurls.
mdbook
Renders a markdown book to static HTML, using the pinned static-musl mdbook release inside an Alpine container. The environment (Alpine rootfs and mdbook) is mounted from constant snapshots, so only the book source changes between builds.
tool jc -> FS: the pinned mdbook distribution, re-rooted.build jc src book_toml summary -> FS: render the markdown insrc, withbook_tomlas itsbook.tomlandsummaryassrc/SUMMARY.md, and return the generated site. The build faults if mdbook fails or produces noindex.html.
sys
Types for the surrounding system context used by entrypoints (not by build
planning files): Ctx, ctxPort, and the NetBackend choices.
ns
Helpers for building namespaces (NS, get, mkEntry), used when a Spore
entrypoint is handed its context as a namespace.
bits
Bit-width aliases (B8…B256) and the bitwise primitives: AND, OR, and
XOR on Bit, plus b32AND and b32OR on B32.
strings
Byte-level string helpers mirroring a subset of Go's strings (a String is
a List of bytes):
len s -> B32.hasPrefix s p -> Bit,hasSuffix s p -> Bit,contains s p -> Bit.index s p -> (Maybe B32): the first index ofpins, or Nothing.count s p -> B32: non-overlapping instances ofp; an emptypcountslen(s) + 1insertion points.split s sep -> (#List String): an emptysepsplits into single bytes.join parts sep -> String,repeat s count -> String.trimPrefix s p -> String,trimSuffix s p -> String,trimSpace s -> String(ASCII whitespace).
The Spore Programming Language
Spore is a small programming language which compiles to Mycelium. WBS uses Spore as the configuration language.
Spore is designed for simplicity; it is a smaller language than Python or even Go. This article gives a complete tour of Spore. It resembles what an engineer would write on a whiteboard when outlining a program.
Spore is a compiled, typed language with immutable values.
Symbols & Variables
Spore has immutable variables, which you can bind with =.
Variables must be bound to a symbol.
Symbols must start with a letter and can contain alphanumerics and _.
Assign a number to myVar.
myVar = 1234
Literals
Bits
A bit is the unit of information.
There are two possible values 0 and 1.
firstBit = 0
secondBit = 1
The natural type of the numerals 0 and 1 is Bit, but they can be coerced to other types as well.
Numbers
We have already seen numbers used in the previous examples.
Spore's numbers are coerced to (#Array Bit _) where _ is some value of type (#Array Bit 32) aka B32.
Numbers must start with - or a numeral.
Underscores are allowed for readability
aMillion = 1_000_000
Hexadecimal can be specified with 0x
sixteen = 0x10
Octal can be specified with 0o
mode644 = 0o100644
Binary can be specified with 0b
eight = 0b1000
The preamble to the compiler (included before all code sent for compilation) includes a few basic integer types, to which these numeral literals can be coerced.
B8
B16
B32
B64
B128
B256
You can define an integer type of any bitwidth
(#Array Bit 4096)
The natural type of a numeral is the smallest bit array that can represent it.
For example 31 would have type (#Array #Bit 5)
Strings
Strings are literal values of type List[Byte].
Double-quoted strings interpret escape sequences:
| Escape | Byte |
|---|---|
\n | newline |
\t | tab |
\r | carriage return |
\0 | zero byte |
\\ | backslash |
\" | double quote |
\xHH | the byte with hex value HH |
aString = "this is a string"
aTabbed = "col1\tcol2"
Backtick strings take everything between the backticks literally, newlines, backslashes, and double quotes included. They cannot contain a backtick.
aRawString = `C:\path\to\file`
aMultiline = `set -e
echo hello
`
Arrays
Spore Arrays are ordered, finite sequences of elements, which are all the same type. They can be coerced to Mycelium Arrays or Lists.
Lists are delimited with [ and ].
The elements are separated with a comma.
A trailing comma is tolerated and not required.
aListOfStrings = ["a", "b", "c"]
Tuples
A Tuple is an ordered, finite sequence of elements, each of an independent and not necessarily distinct type. Tuples compile to Mycelium Products.
Tuples are delimited with { and }.
The elements are separated with a comma.
A trailing comma is tolerated and not required.
aTuple = {"a", 1, "b", 2}
Lookup Table
Lookup Tables (LUT) are the primitive unit of computation in Spore. They look like match/switch statements in other languages. A LUT is a mapping from one set of values to another.
LUTs are delimited by { and }.
Inside are comma separated cases.
A case is a pair joined by =>.
{
0 => "zero",
1 => "one",
}
The natural type of a LUT is a Proc type, which takes the type of all the things to the left of the =>, and returns a value of the same type as the things to the right.
If a common type cannot be inferred, then a type must be specified.
((#Proc B2 String) {
0 => "a",
1 => "b",
2 => "c",
3 => "d",
})
All LUT cases must be mutually exclusive.
_ can be used once per LUT as a catch-all.
myLookTableProc = ((#Proc String B2) {
"a" => 1,
"b" => 2,
"c" => 3,
_ => 0,
})
Lambdas
Lambdas represent procedures.
Lambdas start with \, which is followed by a list of arguments, a return type, and then an expression.
This lambda takes a String and returns a String which repeats it 3 times.
repeatThrice = \(a: String) String {
(#flatten [a, a, a])
}
Lazies
Lazy values are delayed computations which require no additional input. They are specified like Lambdas, but without the arguments.
thisWillBeA4 = \B32 {
two = (B32 2)
(#add two two)
}
Content-ID (CID)
A CID is the hash of some data.
CIDs are often long, and copy-pasted rather than written by humans.
They start with an @ and contain hex characters after that.
CIDs take up 256 bits.
ref = ((#Ref (#Run Byte)) @deadbeef)
As a special case, 160-bit values are padded with zeros to 256 bits. This is to accommodate SHA1 hashes.
S-Expressions
In Spore, computation is expressed by enclosing a function and an input to that function together in parenthesis.
The whole thing, from ( to the corresponding ), is called an S-Expression, or SExpr for short.
Apply a Procedure
twice = \(x: B32) B32 (#add x x)
(twice 50)
evaluates to 100 :: B32
The first element of the SExpr is evaluated first. If it is a procedure, then that procedure is applied to a product created from the rest of the elements of the SExpr.
Eval a Lazy
An SExpr with a single element will evaluate a Lazy value.
lazy = \B32 (#add 1 2)
(lazy)
evaluates to 3 :: B32
Type an Expression
You can force any expression to be a certain type by having a type in the first position of the SExpr.
tenAsB4Naturally = 10
tenAsB32 = ((#Array Bit 32) 10)
Both of these are fine
3array = ((#Array String 3) ["a", "b", "c"])
list = ((#List String) ["a", "b", "c"])
Access a Field
A Tuple is like a function from indices to values.
aTuple = {"a", 2, \B32 4}
(#check (aTuple 0) "a")
(#check (aTuple 1) 2)
(#check ((aTuple 2)) 4)
Primitives
Primitives are operations exposed by the compiler.
They are defined in the compiler's source, not anywhere in the Spore source.
A Primitive always starts with #, followed by alphanumerics and symbols.
Importing
mypkg = (#import "a/b/c")imports a directory of Spore files ata/b/c.mypkg = (#import "dir/file.sp")imports a single Spore file atdir/file.sp.
Embedding
data = (#embedFile "assets/logo.bin")evaluates to aString(a(#List Byte)) holding the referenced file's bytes, like a string literal read from a file. The path is resolved like an import (relative to the containing source, or prefixed such as"core:..."), the file is read at compile time, and a missing or unreadable path is a compile error. It may appear in any value position.
Math
(#add 1000 1000)performs modular arithmetic on integers of the same type.1000has type(#Array Bit 10)naturally.
Types
#Bitthe bit type.(#Array ElementType len)evaluates to a new Array type.(#List ElementType)evaluates to a new List type.(#Product TypeA TypeB TypeC)(#Sum TypeA TypeB TypeC)(#Proc Input Output)(#Lazy Output)(#Ref ElementType)(#Distinct BaseType anyValue)#AnyTypea type capable of holding instances of any type.#AnyValuea type capable of holding instances of any value (that's everything).#AnyProga type capable of holding any program.
Errors
(#panic value)faults the MVM.result = (#brace thingThatCouldPanic arg1 arg2). Returns a(#Sum Output #AnyValue)ifthingThatCouldPanichas return typeOutput.(#check expr)panics if expr is not equal to 1.(#check expr1 expr2)panics ifexpr1andexpr2are not equal.
Store
ref = (#post value)turns anyTinto a(#Ref T).value = (#load ref)turns any(#Ref T)into aT.
Miscellaneous
(#=> data proc)is the same as(proc data)'exprquotes the next value: it evaluates to an#ASTvalue instead of compiling the expression.#ASTis the type of a quoted AST node.(#compileAST ast)compiles a compile-time#ASTat the call site, so it resolves names in the call site's scope.(#TypeOfEval ast)returns theAnyTypethat(#compileAST ast)would produce.(#TypeOf 123)returns the type of123, which isB7, without evaluating the expression.(#TypeOf (#post "abc"))=>(#Ref String)and does not evaluate the expression."abc"will not be posted to the store.(#anyValue "a string")creates an AnyValue wrapper containing "a string"(#anyType #Bit)creates an AnyType wrapper containing(#Bit)(#unwrap Type x)unboxes anAnyValueorAnyTypeand evaluates to the content it holds. Against anAnyValueit panics unless the wrapped value is exactlyType, and evaluates to that value. Against anAnyTypeit panics unless the wrapped descriptor's kind matchesType's, and evaluates to that type descriptor.(#unwrap List anyValue)recovers aListvalue;(#unwrap List anyType)recovers theListtype descriptor.
Macros
X = (#fnr (params) body)defines a find-and-replace macro:(X a b)substitutes the argument forms intobody.X = (#macro fn)defines a function macro:fntakes#ASTarguments and returns#AST, and(X a b)evaluates it at compile time and compiles the AST it returns.
Namespaces
Symbols in Spore are used to identify values in a namespace.
The current local namespace's symbols can always be used directly, as we have seen previously.
To access symbols in other namespaces, like an imported namespace, use the . operator.
Symbol within Import
bits = (#import "abc")
(#check (bits.xor 0 1) 1)
(#check (bits.xor 1 1) 0)
Exporting Symbols
Symbols can be exported from the current namespace, for import into another namespace, by prefixing them with ..
localOnly = "abc123"
.exported = "exported"
Namespaced Types
The #Struct and #TaggedUnion types:
myStructType = (#Struct {
.fieldA = B32
.fieldB = String
})
myStructValue = (myStructType {
.fieldA = 1234
tmp1 = "foo"
tmp2 = "bar"
.fieldB = (#flatten [tmp1 tmp2])
})
myTaggedUnionType = (#TaggedUnion {
.fieldA = B32
.fieldB = String
})
myTaggedUnionValue1 = (myTaggedUnionType {
.fieldB = "foobar"
// also providing .fieldA is not allowed
})
myTaggedUnionValue2 = (myTaggedUnionType {
.fieldA = 12345
// also providing .fieldB is not allowed
})
Core Library
Spore has a core library of packages.
It is importable using the core: prefix with #import.
This is not the same as Want's own standard library, accessible with the
want:prefix.
The bits package provides functions for bitwise operations.
bits = (#import "core:bits")
The ns package provides helpers for building namespaces (NS, get, mkEntry), used when a Spore entrypoint is handed its context as a namespace.
ns = (#import "core:ns")
Mycelium
Mycelium is a typed data serialization format. Mycelium values are made up of fundamental values: Bits, Refs, and Ports, and all other types composed from this fundamental set.
Mycelium values are immutable; any change to a Value is necessarily a new Value.
0. Opaque
Opaque is a type that gives no information; it cannot be instantiated. It exists mostly to make the 0 value for TypeCodes an obvious error.
1. Bit
The Bit type requires no additional information; it has two Values 0 and 1.
2. Array
An Array is a finite sequence of elements, all the same type, where the length of the sequence and the type of an element are known at compile time. Arrays are parameterized by 1 type (the element type) and 1 value (the length).
3. Run
A Run is a contiguous segment of elements, all the same type, where the length is unknown. Runs cannot be used directly; they must always exist within a Ref.
4. Ref
A Ref is a reference to another value, kept in the store. Refs are all the same size, and are redeemable for a Value. A Ref can be converted to a Value by reading from the store. Refs are parameterized by 1 type and 0 values.
5. Sum
A Sum has several possible integer-identified fields. Each field has its own independent type. Only one field may be set at a time.
6. Product
A Product is a collection of fields, each of which has an independent Value of an independent type. All fields must be set. Products are parameterized by N types and 0 values.
7. List
A List is an ordered collection of elements, all the same type, with a length known at runtime.
A List is represented as a Ref[Run] of known length, where the length is tracked in the list.
Lists are parameterized by 1 type and 0 values.
8. Lazy
Lazy is a program which requires no additional input, producing an output of a known type. Lazys are parameterized by 1 type and 0 values.
9. Procedure (Proc)
A Procedure is a program which requires additional input, producing an output of a known type. Procs are parameterized by 2 types (input and output) and 0 values.
10. Port
A Port is a token for communication outside the Mycelium virtual machine.
11. Distinct
A Distinct type is marked with a value. This causes the type to become distinct from its base type. A Distinct type is parameterized by 1 type (the base) and 1 value (the mark).
12. Any
Any is a family of formats that can hold a wide variety of values with different types. These values must be converted to another type in order to be used.
AnyValue
AnyValue is a type capable of holding any value.
AnyType
AnyType is a type capable of holding the type of any value.
AnyProg
AnyProg is a type capable of holding any program. Programs are Runs of B32 instructions.
255. Type2
The type of a Type is a Type2.
Type2s always take up 32 bits in Mycelium.