Tasks & Jobs
Tasks
A Task is a well-defined unit of work. Well-defined means that all the information that could be required to complete the Task is explicitly contained in the Task.
A Task has 2 parts: an operation name and an input (an AnyValue):
Task = (#Struct {
.op_name = String
.input = AnyValue
})
The operation names one of a small number of built-in operators, for example:
want.pickwant.placewant.assemblefsdownload.treelinux.cspawnwasm.runmvm.apply
The input is the operator's argument value. Both parts are part of the hash which becomes the Task's identity, so a Task with the same operation and input is the same piece of work.
Want caches the output of Tasks when they complete, and checks this cache
before computing a Task. An immediate rerun of want build without any
changes therefore produces all the same Tasks, and every one of them can be
skipped.
Jobs
Jobs are used to track running computations in Want.
Jobs are created to compute Tasks. A Job carries active state, like whether
it has completed and when it started and finished: it is spawned when created,
running once its task is started, and done (with an ok value or an error) once it
has an outcome. While a running Job waits for its queue resources it is
reported as running with a non-null queuing_since, and the time it has spent
waiting accumulates in queued_acc, so queuing is a subset of running rather
than a state of its own.
Jobs are organized into a hierarchical tree. All the major operations in Want create a new root Job; that Job may spawn additional child Jobs.
Child Jobs give a computation access to additional compute, since children run concurrently, and they are each an opportunity for caching.
The Job System Port
A build computation is not given access to the host; instead it is handed a jobs port, a capability it uses to spawn and await child Jobs. In Spore this
port has the type build.JobCtx, and a Task's do proc receives it on
ctx.jobs.
Through the port a computation can:
spawn jc limits op input: start a child Job for a Task and get itsJobIDimmediately, under a resource ceiling. A Job that runs past itstimeoutceiling is cancelled automatically.limits jc: the current Job's resource ceiling. Spawning with it passes the ceiling down unchanged.await jc id: block until that child finishes and return its result; a child that failed faults the parent.inspect jc id: check a child's state without blocking.awaitAny jc ids: block until any one of a set of children finishes.collectJobLogs jc id name: read a child's named log stream (stdout,stderr, ...). On a cached task the bytes come from the job that computed it, so the log of a cached child is still readable.
A Job cannot spawn root or sibling Jobs; it can only spawn its immediate children. This keeps runaway computations from escaping their parent, and gives every Job an audit trail back to the user who asked for the result.