Jobs

Chains

A chain is how a job describes what to run. It binds repo images, or other jobs, to named steps and wires them into a pipeline where one step's output feeds the next.

AI assisted, human approved — novem uses AI to review and keep our documentation up to date.

A job's config/chains file defines its pipeline. It ranges from a single image to a directed graph of steps that pipe data through one another. You write it to the job's config/chains path:

POST/v1/code/jobs/daily_report/config/chains
report_builder

A step points at a repo image. There are two reference forms:

  • Short form — a bare repo name resolves to your latest image: report_builder → @<your_username>/report_builder:latest.
  • Canonical form — name the owner and label explicitly: @novem_demo/report_builder:latest. Use this to pin a label or reference an image shared with you by another user.

A step can run another novem job instead of a repo image. Reference it by path: /j/<job> for one of your own jobs, or /u/<user>/j/<job> for a job shared with you by someone else.

fetch   <- data_fetcher
rollup  <- /j/nightly_rollup

fetch -> rollup

At run time novem expands the referenced job into the steps it is built from and splices them into the chain, so rollup runs whatever nightly_rollup runs, fed by fetch's output. A referenced job may itself reference further jobs; novem expands the whole tree, rejecting reference cycles and nesting that runs too deep. Each referenced job records its own run, nested under this run — see runs.

Each referenced job runs with its own environment variables; a referenced job's steps do not inherit the environment of the job that composes them.

Permissions. You must have execute access to every job your chain references directly — your own jobs, or another user's job shared with you with execute permission. Whatever those jobs reference internally is their own concern and is not checked against you. Conversely, someone you grant execute on this job does not need their own access to the jobs it references.

Note: A leading / carrying a /j/ segment always means a job; @name and a bare name always mean a repo. Jobs have no bare or @ form, so a repo reference can never be read as a job, and the reference style alone tells you which kind a step runs.

The simplest chain is one reference. This runs that image and is exactly what the quick start uses.

report_builder

To pipe steps together, use the chain grammar. It has two kinds of line:

  • Bindings assign a short label to a repo image with <-.
  • Edges wire labels together with ->, left to right.
fetch  <- data_fetcher
render <- chart_builder

fetch -> render

This runs fetch first, then pipes its output files into render as that step's input. The labels keep the pipeline readable and let you reference the same image more than once.

Edges can be chained in a single line for longer pipelines:

fetch     <- data_fetcher
transform <- transformer
render    <- chart_builder

fetch -> transform -> render

Each step receives the previous step's output as its input; the final step's output becomes the job's result.

Prefix a step with a dot to fan out over the previous step's output: the step runs once per output file, in parallel. A fan-out step sits between a producer and a barrier — the step before it supplies the files to spread over, and the step after it waits for every parallel instance and collects their output — so a fan-out is never the first or last step in a chain.

split   <- splitter
render  <- chart_builder
collect <- collector

split -> .render -> collect

Here render runs once for each file split produced, and collect receives all of their outputs together once the instances finish. Fan-out currently supports the per-file mode shown above.

Note: A few constructs are reserved and not yet available: AND dependencies (,), OR dependencies (|), error handlers (:) and cross-joins (.X.). Using them is a configuration error rather than a silent no-op.

novem normalises whatever you write into a canonical form — bindings first, then the edge line — so reading config/chains back may look slightly different from what you wrote:

fetch <- @you/data_fetcher:latest
render <- @you/chart_builder:latest

fetch -> render

  • Jobs overview — env vars, schedules and running.
  • Repos — build the images your chain references.
Next
Schedule