Guides

Jobs

A novem job runs your code. It strings one or more repo-built images into a pipeline — a chain — and executes them, on demand or on a schedule.

AI assisted, human approved — novem uses AI to review and keep our documentation up to date.

A job is novem's unit of code execution. Where a repo holds your code and builds it into an image, a job decides how and when that image runs — by itself, or as a multi-step pipeline that pipes one step's output into the next.

Impatient? Jump to the quick start.

Jobs default to the chains runtime. A chains job points at one or more repo images through its config/chains definition and runs them via novem's worker fleet. A chain can be a single image or a directed pipeline of several — see chains for the full grammar.

Like other novem resources, a job is a hierarchical folder structure.

daily_report              => Job name
├── config                => Configuration options
│   ├── type              => Runtime type (chains)
│   ├── chains            => Pipeline definition (repo images + DAG)
│   ├── env               => Environment variables (encrypted, lower-case)
│   │   └── api_token     => A single variable
│   ├── schedule          => Cron schedule (optional)
│   └── enabled           => Kill switch (true / false)
├── data                  => POST here to trigger a run
├── runs                  => Run history
│   └── <run>
│       └── log           => Per-run log output
├── log                   => Job log
├── description           => Description (meta)
├── name                  => Name (meta)
├── shortname             => Auto-generated short id
└── shared                => Who can view the job
    ├── +org~group        => Shared with an org group
    ├── @username~group   => Shared with a user group
    └── public            => Shared with everyone

A job is created with an HTTP PUT to the https://api.novem.io/v1/code/jobs/JOB_NAME endpoint, then pointed at the repo image(s) it should run via config/chains.

# create the job (defaults to the chains runtime)
novem -j daily_report -C

# point it at a repo image — the chain is a single step here
echo "report_builder" | novem -j daily_report -w config/chains

report_builder is the name of one of your repos; novem resolves it to your built image @<your_username>/report_builder:latest. See chains for multi-step pipelines.

Jobs inject environment variables into the running container. Set them under config/env — values are encrypted at rest and never returned in plaintext. Create each variable with a lower-case name.

# value can be piped on stdin to keep it out of your shell history
printf '%s' "$SOME_SECRET" | novem -j daily_report -w config/env/api_token

# list the variables that are set (names only)
novem -j daily_report --tree config/env

Inside the running job both the lower-case name and an upper-cased variant are injected, so api_token and API_TOKEN are both visible to your code.

Note: Environment variables are encrypted for the job's runtime when you set them. If you change a job's config/type after setting variables, the next run is rejected (409) with a message naming the affected variables — re-set them so they are re-encrypted for the new runtime.

When a job runs, novem starts your repo's image and appends two arguments to its entrypoint: the paths /input and /output. Those are also where novem mounts the run's input and output directories — the arguments are a convenience, the mount points are always the same, so code can simply read from /input and write to /output.

import sys, os

input_dir = sys.argv[1]   # "/input"  (or just hardcode "/input")
output_dir = sys.argv[2]  # "/output"

for name in os.listdir(input_dir):
    ...                    # process each input file
# write your results into output_dir

This is a convention, not an enforced contract — novem runs whatever ENTRYPOINT/CMD your image defines. The convention is just: read /input, write /output.

A run's input depends on how it was triggered:

  • A JSON payload (novem -j <job> -R, or POST .../data) arrives as /input/input.json.
  • Uploaded files (-R @data.csv, or a browser attachment) arrive as /input/<filename>, original names preserved.

Both can be present at once, so when you expect an upload, prefer real files over input.json rather than assuming a single input.

Write one or more result files into /output. novem packages whatever it finds:

  • One file → returned as-is, with its filename and a content type guessed from the extension.
  • Multiple files (or subdirectories) → bundled into a single .zip.
  • Nothing → the run produces no result.

There is no special filename you must use — result.txt, report.pdf, out.csv all work. (The 100 MB result cap below still applies.)

Trigger a run with the CLI's -R flag, or by posting to the job's data endpoint directly. Anything you send becomes the job's input, available to the first step of the chain. Runs are recorded under runs, each with its own log. To pull the files a run produces back to your machine, point -o/ --output at a directory.

# trigger a run
novem -j daily_report -R

# upload one or more @file.ext as input for the run
novem -j daily_report -R @data.csv @config.json

# save the run's output files to disk with -o/--output
novem -j daily_report -R -o ./out

# read the latest run's log
novem -j daily_report -r runs/latest/log

Note: A run's result is capped at 100 MB. Mail clients and mail servers enforce strict size limits, so the cap keeps a job's result deliverable to the inboxes and downstream consumers it tends to feed. A larger result fails in finalisation and nothing is stored — the run log shows a 413 error like Job result is too large: 142.0 MB exceeds the 100 MB limit.. Trim or compress the output, or split the work across chain steps, to stay under the cap.

Jobs can also run automatically on a cron config/schedule, and be paused with the config/enabled kill switch.

Next
Themes