Skip to content
Nodus
Blog

Sandboxes

View Markdown

A Sandbox is an isolated, long-running container for agents and untrusted code. You drive it with commands and file requests. CPU checkpoints select /nodus/state before process shutdown. Flush the files you need to keep before stopping, and verify them after starting; /workspace and the rest of the image filesystem are not captured. GPU Sandboxes default to ephemeral storage; opt into local state-directory snapshots (Beta) below, or keep durable data in your external filesystem.

Sandboxes run on CPU machines or dedicated on-demand GPU machines, each in its own isolated runtime with no network access unless you open it. Sandbox isolation explains what keeps one Sandbox from another.

Quick start

Section titled “Quick start”

This manifest is the whole Sandbox. It asks for 1 vCPU and 2 GiB of memory, stops after 5 idle minutes and caps its spending at 1 USD:

sandbox.yaml
apiVersion: nodus.dev/v1
kind: Sandbox
metadata:
  name: hello
spec:
  image: nodus/agent-tools # required: Python 3.12, Node 22 and git
  resources:
    cpu: "1"
    memory: 2Gi
  network:
    egress:
      policy: Deny # the default: no outbound traffic
  lifecycle:
    idleTimeout: 5m # stop after 5 minutes with no exec, file request or running process
    onIdle: Stop # keep /workspace and stop paying for compute; Delete removes the Sandbox instead
    maxLifetime: 2h # delete it 2 hours after creation whatever it is doing
  maxCostUSD: "1.00"

Create it, run a command, copy a file in and read it back:

Terminal window
$ nodus apply -f sandbox.yaml
sandbox/hello created
$ nodus exec sb/hello -- echo hello from a sandbox
hello from a sandbox
$ nodus cp ./notes.txt sb/hello:/nodus/state/notes.txt
$ nodus exec sb/hello -- cat /nodus/state/notes.txt
written from my laptop
$ nodus stop sb/hello
$ nodus exec sb/hello -- cat /nodus/state/notes.txt    # starts it again; verify the saved state file
written from my laptop
$ nodus delete sb/hello

From Python, the same steps are in the Python Sandboxes guide.

Create a Sandbox

Section titled “Create a Sandbox”

nodus apply -f sandbox.yaml, nodus create sandbox <name> and the SDK all create the same object. image is required on the API; the CLI and the SDK fill in nodus/agent-tools (Python 3.12, Node 22 and git) when you leave it out.

Field Default Accepted values
spec.image — (required) Any image reference, or a Nodus catalog image such as nodus/agent-tools
spec.resources.cpu 1 0.25 to 64 vCPU
spec.resources.memory 1Gi 128 MiB to 256 GiB
spec.resources.disk 10Gi 1 GiB to 200 GiB
spec.resources.gpu none Supported accelerators on a dedicated, on-demand VM; never Spot/preemptible
spec.placement.regions any Optional Nodus region classes such as us; a nonempty list is enforced for quote and placement
spec.fuseMount false true only with a GPU request; enables guest FUSE, but does not provision JuiceFS
spec.virtualization.profile Unset gpu-child-v1 requests a qualified single-GPU parent for child VMs (Alpha); availability is checked by the API
spec.network.egress.policy Deny Deny (no outbound traffic), AllowList (named HTTP/HTTPS hosts), or Open (public internet)
spec.lifecycle.idleTimeout 5m 0s (never idle) or 1 minute to 24 hours
spec.lifecycle.onIdle Stop Stop or Delete
spec.lifecycle.maxLifetime 24h 1 minute to 720 hours
spec.continuity.mode Snapshotted for CPU; Ephemeral for GPU GPU Snapshotted requires an explicit stateDir (Beta)
spec.continuity.stateDir Unset GPU Snapshotted only: an immutable local directory, such as /nodus/state
spec.workingDir /workspace An absolute path
spec.maxCostUSD none A USD amount such as "5.00"; the Sandbox stops when it has cost this much
spec.state Running Running, Stopped, or Paused (CPU memory pause, Beta)

nodus get sb/hello shows PHASE (Pending, Starting, Running, Pausing, Paused, Resuming, Stopping, Stopped, Recovering, Terminating, Failed), ACTIVITY (Busy or Idle), the CPU, MEMORY and GPU you asked for, and COST so far.

Before creating a Sandbox through the REST API, submit the same request with ?dryRun=All to inspect status.estimate. It includes the hourly price, initial funding hold, minimum charge and selected offering when capacity is available. Send the returned ETag as If-Match when creating that unchanged request. A quote does not reserve a machine; check the Sandbox’s readiness before sending work.

Pause process memory (Beta)

Section titled “Pause process memory (Beta)”

For a short break, open the Sandbox menu and choose Pause memory. On a compatible exclusive CPU machine, Nodus freezes the running processes and retains their memory for up to 30 minutes. Wait for Paused before leaving it. The overview shows the retention deadline. Choose Resume to continue the same processes.

Compute billing stops when the freeze is acknowledged. Retained storage remains billable against the Sandbox’s budget, including the backing machine’s disk. Resume requires available credits and budget; compute billing restarts when the machine resumes. Minimum compute increments apply to total active time across pauses. The reservation alone is not a compute charge.

GPU Sandboxes and shared or incompatible CPU machines cannot use memory pause. A refused request leaves the Sandbox running, with its normal compute charges and idle policy. Ordinary stop/start preserves only the configured file state, not RAM. Retained memory expires after 30 minutes and is also lost on machine or guest restart failure; Nodus reports the loss and requires a new start rather than silently replacing the process. Stopping or deleting a paused Sandbox discards its retained memory without taking a new file checkpoint. After memory loss or expiry, choose Start explicitly. Terminal, file, and reconnect requests cannot start a replacement process automatically.

The API accepts a merge patch on an existing running Sandbox:

pause.json
{"spec":{"state":"Paused"}}

Patch spec.state back to Running to request resume. Check status.phase and status.pauseUntil; a submitted request is not proof that the machine has paused or resumed. Creating a Sandbox directly in Paused is rejected.

Create by name reconnects

Section titled “Create by name reconnects”

Creating a Sandbox whose name already exists in the project does not make a second one:

  • Same spec. You get the existing Sandbox back, and if it is stopped it starts again. This is how an agent reconnects to its Sandbox after a restart: it creates the same manifest every time.
  • A maxCostUSD that is added or raised. The new budget is applied; nothing else changes. A budget can only be raised: a lower one, or none when the Sandbox has one, counts as a different spec.
  • Any other change. The request fails with 409 AlreadyExists and lists the fields that differ in details.diff and details.causes. Delete the Sandbox and create it again to change its image, shape or lifecycle.

GPU Sandboxes use a dedicated rented GPU VM and are non-interruptible. They default to Ephemeral continuity; a stopped GPU Sandbox starts empty. fuseMount: true is an opt-in guest capability for FUSE clients such as JuiceFS. Nodus does not create the filesystem or mount it for you, and unsupported GPU/runtime combinations fail closed.

GPU child virtual machines (Alpha)

Section titled “GPU child virtual machines (Alpha)”

This capability is under qualification. A GPU listing by itself does not promise child-VM support. Check spec.supports.sandboxVirtualizationProfiles on an Offering; an absent or empty list means no qualified profile is advertised for it. The console reads that same field. The API validates the full request and returns the selected profile’s quote; the ordinary offering price and stock are not a child-VM availability guarantee.

The shared request field is spec.virtualization.profile: gpu-child-v1. The CLI convenience flag is --virtualization-profile gpu-child-v1, and Python accepts virtualization_profile="gpu-child-v1" on Sandbox.create. MCP accepts the identical Sandbox manifest through estimate and apply. An unsupported profile fails before allocation instead of silently creating an ordinary Sandbox.

This enables a parent for VMs you build and manage. Keep your own parent container image and run your VMM inside it; Nodus does not replace your entrypoint with a VM manager. The parent exposes /dev/kvm, /dev/fuse, /dev/net/tun, /dev/vhost-net and its assigned NVIDIA devices within a reserved user namespace. Your VMM must work within those permissions and the parent’s resource and network limits. It receives no host networking, VFIO device passthrough or /dev/vhost-vsock access. Forward a child service to a parent loopback port to use the existing authenticated Sandbox HTTP or port-forward route.

GPU access inside a child requires a compatible GPU transport and guest driver stack; KVM alone does not provide it. The reference path uses software forwarding with pinned components. The installed, verified companion includes an optional launcher and example guest; you do not have to use that launcher or guest to manage your own VMs. Its --rootfs option accepts a guest image with its required init and compatible userspace, while retaining the reference kernel and GPU stack. That helper’s requirements do not apply to a separately managed VMM, whose kernel and GPU integration need their own compatibility validation. Arbitrary VMM/kernel combinations have not been verified.

The profile requires one GPU and cannot change after creation. Child VMs share that GPU within the exclusive parent; they do not receive hardware-isolated GPU partitions. Stop/start can preserve the parent’s declared local state directory, but does not preserve child RAM or GPU memory. Keep remote filesystem mounts outside the saved directory. Existing Sandboxes continue to use their current runtime when this field is omitted.

Stop and start a GPU Sandbox

Section titled “Stop and start a GPU Sandbox”

GPU Sandboxes support stop/start with Ephemeral continuity. Stop releases the GPU allocation and keeps the Sandbox definition, image reference and Secret references. Customer compute occupancy ends when the execution controller accepts the stop; Nodus pays subsequent allocation cleanup. The minimum billing increment still applies to usage before the stop, and API-to-controller queue delay remains billable. Start launches fresh processes from the image; it does not restore process memory, local files or a previous FUSE mount. The Sandbox still expires at its original maxLifetime.

These commands are supported by CLI 0.1.72. Replace gpu-dev with your existing Sandbox name:

Terminal window
nodus stop sb/gpu-dev
nodus wait sb/gpu-dev --for=jsonpath='{.status.phase}'=Stopped --timeout 20m
nodus start sb/gpu-dev
nodus wait sb/gpu-dev --for=condition=Ready --timeout 20m

Data successfully saved to your external JuiceFS filesystem remains there. Put its mount and readiness checks in the image’s startup command so each start remounts it using the existing Secret. A mount made only by an interactive exec is not replayed. If your image instead contains a manual setup script, run it after readiness:

Terminal window
# This script must exist in your image and use your configured Secret environment.
nodus exec --no-wake sb/gpu-dev -- sh /opt/app/mount-juicefs.sh

Verify a known remote file before restarting application work. Ready alone does not prove that your filesystem is mounted. A fresh GPU VM may need to download the image again; stop/start does not promise an image cache or an instant start. Use the bounded prepared Sandbox workflow below when you need compute ready ahead of work.

Preserve a GPU Sandbox’s local state directory (Beta)

Section titled “Preserve a GPU Sandbox’s local state directory (Beta)”

Choose the directory when you create a new Sandbox. Existing Sandboxes cannot change continuity in place:

spec:
  # Keep your existing image, GPU, secrets, FUSE, lifetime and budget settings.
  continuity:
    mode: Snapshotted
    stateDir: /nodus/state

The Python equivalent is nodus.Sandbox.create(..., gpu="A100", state_dir="/nodus/state"). The directory starts empty and hides any image files already at that path. Write the files you need to retain there (NODUS_STATE_DIR names the same path). System directories and credential mounts are rejected. Only this local directory is retained; installed packages elsewhere, /workspace outside that directory, process memory and GPU memory are not saved. Do not mount JuiceFS inside it: keep the remote mount separate and remount it in startup. Nodus snapshots the local host directory, not the remote FUSE contents.

Flush application state on shutdown. Nodus saves the directory after the process exits during stop and periodically while running, then restores it before launching a fresh process on start. Stop remains bounded: if saving fails, the previous useful snapshot is retained and StateSaved=False reports the failure. Check nodus describe sb/<name> before relying on a final save. An empty directory cannot replace a saved checkpoint: it reports StateSaved=False, and a later start restores the previous snapshot rather than treating the deletion as saved. Snapshot size is bounded by the requested disk size. Startup and restore take time, and storage remains billed while the saved data is retained. This is disk recovery, not GPU pause or an instant warm restart.

A complete bounded manifest and verification commands are in examples/sandboxes/gpu-state/ in the repository.

Prepare one Sandbox for one job

Section titled “Prepare one Sandbox for one job”

The tested prepared-queue example creates at most one Sandbox, waits for the server’s Ready condition, runs an image-specific setup command, and only then admits it to a local queue. By default, a consumer takes that Sandbox once; an empty queue fails without creating a cold replacement. The manifest sets idleTimeout: 0s so the prepared Sandbox stays Running and maxLifetime: 20m so the server deletes it even if the client exits. maxCostUSD is a second per-Sandbox limit.

Run the example with CPU in compose, or pass --gpu <type> and a GPU-capable image on an account with that offering. Pass --regions us to keep the request in the US; if no US offer is available, it waits rather than falling back to another region. GPU defaults to 24 vCPU and CPU can be changed with --cpu. Pass --fuse-mount to enable the guest FUSE path for a supported GPU runtime. Egress defaults to Deny; configure an explicit allow list with --egress AllowList --allow-host <host>, or opt in to unrestricted egress with --egress Open when the image’s setup needs it. For a customer-owned remote filesystem, pass an existing Nodus Secret with --secret <secret-name>; the image must include the client and mount configuration. Use --prepare-command to mount and initialize the filesystem, then --ready-command to verify a data read and any required CUDA check before queue admission. The example does not supply credentials or mount configuration. Use --prepare-only to prepare ahead of traffic, then --run-prepared in a later invocation to claim the saved entry once. Both invocations use the same --state-file; its file lock coordinates callers on one host. Checkout rechecks ownership, readiness and expiration and submits nodus exec --no-wake. A stopped or expired Sandbox fails immediately without starting replacement compute. Use --cleanup for an unused entry. Prepared GPUs continue billing while idle; keep the lifetime and spending cap bounded.

This is a customer-owned queue example on one host: it is not a managed autoscaler, a shared host pool, a provider image cache, or a latency guarantee. Region choice is opt-in and --regions us is scoped to this request. It also does not retain model memory after the setup command exits.

Reuse a prepared Sandbox within one trust boundary

Section titled “Reuse a prepared Sandbox within one trust boundary”

For repeated jobs from the same trusted application, opt in with --reuse-max-jobs (2–100), a --reset-command or --reset-script, and a --ready-command or --ready-script. Supply the same reset and readiness checks when preparing and consuming the saved entry. The example checks the configured commands against the saved entry, runs reset and readiness checks before returning it to the queue, and retains the original lifetime and cost caps. A failed job or check triggers deletion. An interrupted dispatcher leaves a claimed entry that requires --cleanup before further use; it is not returned for reuse. The final permitted job also deletes it. Single-use behavior remains the default.

Reuse keeps the same running Sandbox and its already-pulled image and mounted filesystems. Idle time remains billable. Your reset command must clean application state; this does not isolate mutually untrusted users. It does not provide a fresh-host image cache, GPU memory snapshots, or stop/start continuity. Store durable data in your external filesystem. Use --image-pull-secret for registry credentials and --secret for guest credentials; they have different purposes.

Customer-owned JuiceFS filesystem (Beta)

Section titled “Customer-owned JuiceFS filesystem (Beta)”

This self-service recipe mounts a customer-owned, already-formatted JuiceFS filesystem from a Nodus Secret, checks a customer-selected sentinel file, and runs a small CUDA smoke check before the one-item queue admits the Sandbox. It does not create or format a filesystem. GPU Sandboxes use dedicated on-demand capacity with Never interruption; choose the GPU, vCPU count, image and spend limits for your Job. Start with --regions us; if Canada is acceptable after US placement is unavailable, retry with --regions ca.

Create a protected local env file with the required KEY=VALUE entries from this table (plus backend-specific variables if needed), then create the Secret from the file. Keep the file out of source control; see the Secrets guide for how Nodus delivers Secret values. JuiceFS documents its credential environment variables and FUSE flags in the FUSE mount options reference.

Secret key Required Purpose
JUICEFS_META_URL Yes Metadata URL for the existing filesystem; keep passwords out of the URL.
JUICEFS_READY_PATH Yes Relative path to an existing readable sentinel file.
JUICEFS_READY_SHA256 Yes Expected SHA-256 digest of the sentinel file.
META_PASSWORD If needed JuiceFS metadata password.
ACCESS_KEY, SECRET_KEY If needed Credentials for the configured object-storage backend.
JUICEFS_FUSE_OPTIONS Optional Comma-separated FUSE mount options, such as allow_other.
Terminal window
touch .juicefs.env && chmod 600 .juicefs.env
$EDITOR .juicefs.env
nodus secret create customer-juicefs --from-env-file .juicefs.env && rm .juicefs.env

Save the two source blocks below as juicefs_prepare.sh and ready_queue.py in the same local directory. The command below prepares one GPU Sandbox; adjust the GPU, CPU, image, region and budget as needed. Replace the Job command with your application’s command and use an image containing that application; the sample /workspace/train.py path must exist in your chosen image.

Terminal window
python3 ready_queue.py --prepare-only \
  --image 'ghcr.io/nodus-compute/catalog/workspace-pytorch-cuda@sha256:5ce9e93b5e347b64000d12dc728b82743718c3be0593aa402d4d5f4ae0a2f8ba' \
  --gpu A100 --cpu 30 --fuse-mount --regions us --egress Open \
  --secret customer-juicefs --prepare-script juicefs_prepare.sh \
  --max-lifetime 20m --max-cost 5.00


python3 ready_queue.py --run-prepared \
  --job-command 'python /workspace/train.py --data-dir /mnt/juicefs'

The prepared GPU remains billable while idle until its bounded lifetime or spend cap ends. This is a local, one-Sandbox queue for one later job, not an autoscaling service or proof that a particular customer filesystem or GPU offering is qualified. For the general FUSE API, see the Python Sandbox guide.

juicefs_prepare.sh
#!/usr/bin/env bash
# Mount an already formatted JuiceFS filesystem and admit this Sandbox only after its data and CUDA checks pass.
set +x
set -eu


exec 3>&1
exec >/dev/null 2>&1


mountpoint_path=${JUICEFS_MOUNTPOINT:-/mnt/juicefs}
ready_path=${JUICEFS_READY_PATH:-}
ready_sha256=${JUICEFS_READY_SHA256:-}
phase=configuration
mounted=0
mount_owned=0
tmpdir=


cleanup() {
  status=$?
  trap - EXIT
  if [ "$status" -ne 0 ]; then
    if [ "$mount_owned" -eq 1 ] && command -v juicefs >/dev/null 2>&1 && command -v mountpoint >/dev/null 2>&1 && mountpoint -q "$mountpoint_path"; then
      juicefs umount "$mountpoint_path" >/dev/null 2>&1 || true
    fi
    if [ -n "$tmpdir" ]; then rm -rf -- "$tmpdir" >/dev/null 2>&1 || true; fi
    printf 'JuiceFS Sandbox preparation failed at %s\n' "$phase" >&3
  fi
  exit "$status"
}
trap cleanup EXIT


if command -v mountpoint >/dev/null 2>&1 && mountpoint -q "$mountpoint_path"; then
  phase=mountpoint_already_mounted
  exit 2
fi


if [ -z "${JUICEFS_META_URL:-}" ] || [ -z "$ready_path" ] || [ -z "$ready_sha256" ]; then
  phase=required_secret_missing
  exit 2
fi
case "$ready_path" in
  /*|*..*) phase=invalid_ready_path; exit 2 ;;
esac
if ! [[ "$ready_sha256" =~ ^[0-9a-fA-F]{64}$ ]]; then
  phase=invalid_ready_digest
  exit 2
fi


phase=dependencies
if ! command -v juicefs >/dev/null 2>&1; then
  command -v apt-get >/dev/null 2>&1
  command -v curl >/dev/null 2>&1
  command -v sha256sum >/dev/null 2>&1
  tmpdir=$(mktemp -d)
  apt-get update -qq
  apt-get install -qq -y --no-install-recommends ca-certificates curl fuse3
  curl --fail --silent --show-error --location --retry 3 \
    --output "$tmpdir/juicefs.tar.gz" \
    https://github.com/juicedata/juicefs/releases/download/v1.3.4/juicefs-1.3.4-linux-amd64.tar.gz
  printf '%s  %s\n' 98111cc7a67660eee9dd35915c0d33529e519282c9a5adf44c85a9e6dabfa676 \
    "$tmpdir/juicefs.tar.gz" | sha256sum --check --status
  tar -xzf "$tmpdir/juicefs.tar.gz" -C "$tmpdir" juicefs
  install -m 0755 "$tmpdir/juicefs" /usr/local/bin/juicefs
fi
phase=juicefs_version
juicefs version | grep -Eq '(^|[[:space:]])v?1\.3\.4([+[:space:]]|$)'
phase=fuse_dependencies
command -v fusermount3 >/dev/null 2>&1 || command -v fusermount >/dev/null 2>&1
command -v mountpoint >/dev/null 2>&1
command -v python3 >/dev/null 2>&1


if mountpoint -q "$mountpoint_path"; then
  phase=mountpoint_already_mounted
  exit 2
fi


phase=mount
mkdir -p "$mountpoint_path"
# The background mount must not retain the preparation Process output pipe.
if [ -n "${JUICEFS_FUSE_OPTIONS:-}" ]; then
  juicefs mount -d -o "$JUICEFS_FUSE_OPTIONS" "$JUICEFS_META_URL" "$mountpoint_path" >/dev/null 2>&1 3>&-
else
  juicefs mount -d "$JUICEFS_META_URL" "$mountpoint_path" >/dev/null 2>&1 3>&-
fi
mount_owned=1
for _ in $(seq 1 30); do
  if mountpoint -q "$mountpoint_path"; then mounted=1; break; fi
  sleep 1
done
if [ "$mounted" -ne 1 ]; then phase=mountpoint_check; exit 1; fi


phase=filesystem_data_check
JUICEFS_MOUNTPOINT="$mountpoint_path" python3 - "$ready_path" "$ready_sha256" <<'PY'
import hashlib
import hmac
import os
import pathlib
import sys


root = pathlib.Path(os.environ["JUICEFS_MOUNTPOINT"]).resolve()
candidate = (root / sys.argv[1]).resolve()
if root not in candidate.parents or not candidate.is_file():
    raise SystemExit(1)
digest = hashlib.sha256(candidate.read_bytes()).hexdigest()
if not hmac.compare_digest(digest, sys.argv[2].lower()):
    raise SystemExit(1)
PY


phase=cuda_compute_check
python3 - <<'PY'
import torch


if not torch.cuda.is_available():
    raise SystemExit(1)
matrix = torch.arange(64 * 64, dtype=torch.float32, device="cuda").reshape(64, 64)
result = matrix @ matrix.T
torch.cuda.synchronize()
if result.shape != (64, 64) or not torch.isfinite(result).all().item():
    raise SystemExit(1)
PY


if [ -n "$tmpdir" ]; then rm -rf -- "$tmpdir"; tmpdir=; fi
trap - EXIT
printf 'JuiceFS data and CUDA smoke checks passed; Sandbox remains mounted\n' >&3
ready_queue.py
"""Prepare one bounded Sandbox ahead of a request, then claim it once without a cold fallback."""


from __future__ import annotations


import argparse
import fcntl
import hashlib
import json
import os
import re
import shlex
import subprocess
import sys
import tempfile
import uuid
from collections.abc import Iterator
from contextlib import contextmanager
from datetime import datetime, timezone
from decimal import Decimal
from pathlib import Path


_DURATION_PART = re.compile(r"(\d+(?:\.\d+)?)(ns|us|µs|μs|ms|s|m|h)")
_DURATION_SECONDS = {
    "ns": Decimal("0.000000001"),
    "us": Decimal("0.000001"),
    "µs": Decimal("0.000001"),
    "μs": Decimal("0.000001"),
    "ms": Decimal("0.001"),
    "s": Decimal(1),
    "m": Decimal(60),
    "h": Decimal(3600),
}




def duration_seconds(value: str) -> Decimal | None:
    """Parse the compound Go duration strings emitted by metav1.Duration JSON."""
    total = Decimal(0)
    offset = 0
    while offset < len(value):
        part = _DURATION_PART.match(value, offset)
        if part is None:
            return None
        total += Decimal(part.group(1)) * _DURATION_SECONDS[part.group(2)]
        offset = part.end()
    return total if offset else None




def lifecycle_matches(lifecycle: dict[str, object], requested: str) -> bool:
    actual_lifetime = duration_seconds(str(lifecycle.get("maxLifetime", "")))
    requested_lifetime = duration_seconds(requested)
    idle_timeout = duration_seconds(str(lifecycle.get("idleTimeout", "")))
    return (
        actual_lifetime is not None
        and requested_lifetime is not None
        and requested_lifetime > 0
        and actual_lifetime == requested_lifetime
        and idle_timeout == 0
    )




def guest_command(command: str | None, script: str | None, label: str) -> tuple[list[str], bytes | None]:
    if script is not None:
        if command is not None:
            raise ValueError(f"choose either --{label}-command or --{label}-script")
        return ["bash", "-s"], Path(script).read_bytes()
    argv = shlex.split(command) if command is not None else None
    if argv is not None and not argv:
        raise ValueError(f"--{label}-command must not be empty")
    return argv or [], None




def prepare_commands(
    args: argparse.Namespace,
) -> tuple[tuple[list[str], bytes | None], tuple[list[str], bytes | None] | None]:
    prepare = guest_command(args.prepare_command, args.prepare_script, "prepare")
    if not prepare[0]:
        prepare = (["python", "-c", "print('prepared-image-ready')"], None)
    ready = None
    if args.ready_command is not None or args.ready_script is not None:
        ready = guest_command(args.ready_command, args.ready_script, "ready")
    return prepare, ready




def exec_guest(name: str, command: list[str], stdin: bytes | None = None) -> None:
    args = ["nodus", "exec", "--no-wake"]
    if stdin is not None:
        args.append("--stdin")
    args.extend([f"sb/{name}", "--", *command])
    subprocess.run(args, check=True, input=stdin)




class ReadyQueue:
    """A bounded queue operation that admits only prepared Sandbox names."""


    def __init__(self, desired_ready: int, hard_max: int) -> None:
        if hard_max != 1 or desired_ready != 1:
            raise ValueError("this example supports desired-ready=1 and hard-max=1")
        self.desired_ready = desired_ready
        self.hard_max = hard_max
        self.created = 0
        self.ready: list[str] = []


    def prepare(self, args: argparse.Namespace) -> None:
        while len(self.ready) < self.desired_ready:
            if self.created >= self.hard_max:
                raise RuntimeError("ready queue reached its hard maximum")
            self.ready.append(self._create_and_prepare(args))
            self.created += 1
        print(f"ready queue: {len(self.ready)}/{self.hard_max}")


    def take(self) -> str:
        if not self.ready:
            raise RuntimeError("no prepared Sandbox is ready; cold fallback is disabled")
        return self.ready.pop(0)


    @staticmethod
    def _create_and_prepare(args: argparse.Namespace) -> str:
        run_id = uuid.uuid4().hex
        name = "ready-" + run_id[:12]
        if args.cpu is not None and not 1 <= args.cpu <= 64:
            raise ValueError("--cpu must be between 1 and 64")
        if not args.gpu and args.fuse_mount:
            raise ValueError("--fuse-mount requires --gpu")
        if args.egress == "AllowList" and not args.allow_host:
            raise ValueError("--egress AllowList requires at least one --allow-host")
        if args.egress != "AllowList" and args.allow_host:
            raise ValueError("--allow-host requires --egress AllowList")
        prepare, ready = prepare_commands(args)
        if not 1 <= args.reuse_max_jobs <= 100:
            raise ValueError("--reuse-max-jobs must be between 1 and 100")
        reset = guest_command(args.reset_command, args.reset_script, "reset")
        if args.reuse_max_jobs > 1 and (not reset[0] or ready is None):
            raise ValueError("reuse requires explicit reset and readiness commands/scripts")
        state_file = Path(args.state_file)
        # Record ownership before POST so cleanup can resolve a lost response without adopting a foreign object.
        state: dict[str, object] = {
            "sandbox": name,
            "run": run_id,
            "phase": "creating",
            "maxLifetime": args.max_lifetime,
            "maxCostUSD": args.max_cost,
            "maxJobs": args.reuse_max_jobs,
            "completedJobs": 0,
            "returnCheck": return_check_digest(reset, ready) if args.reuse_max_jobs > 1 else None,
        }
        write_state(state_file, state)
        resources: dict[str, object] = {
            "cpu": str(args.cpu if args.cpu is not None else 24 if args.gpu else 1),
            "memory": "2Gi",
            "disk": "10Gi",
        }
        if args.gpu:
            resources["gpu"] = {"type": [args.gpu], "count": 1}
        egress: dict[str, object] = {"policy": args.egress}
        if args.allow_host:
            egress["allow"] = args.allow_host
        secret_mounts = [{"name": name} for name in args.secret]
        manifest = {
            "apiVersion": "nodus.dev/v1",
            "kind": "Sandbox",
            "metadata": {
                "name": name,
                "labels": {"example.nodus.dev/role": "prepared-queue", "example.nodus.dev/run": run_id},
            },
            "spec": {
                "image": args.image,
                **({"imagePullSecrets": args.image_pull_secret} if args.image_pull_secret else {}),
                "resources": resources,
                "fuseMount": args.fuse_mount,
                "network": {"egress": egress},
                **({"placement": {"regions": args.regions}} if args.regions else {}),
                "lifecycle": {
                    "idleTimeout": "0s",
                    "onIdle": "Delete",
                    "maxLifetime": args.max_lifetime,
                },
                "maxCostUSD": args.max_cost,
                **({"secrets": secret_mounts} if secret_mounts else {}),
            },
        }
        with tempfile.NamedTemporaryFile(mode="w", suffix=".json", encoding="utf-8") as f:
            json.dump(manifest, f)
            f.flush()
            subprocess.run(["nodus", "create", "-f", f.name], check=True)


        # Ready is the server's Running condition; the setup command then verifies this image's own prerequisites.
        subprocess.run(
            ["nodus", "wait", f"sandbox/{name}", "--for=condition=Ready", "--timeout=10m"],
            check=True,
        )
        obj = get_owned(name, state)
        status = obj.get("status") or {}
        conditions = status.get("conditions") or []
        server_ready = any(c.get("type") == "Ready" and c.get("status") == "True" for c in conditions)
        lifecycle = (obj.get("spec") or {}).get("lifecycle") or {}
        if (
            status.get("phase") != "Running"
            or not server_ready
            or not status.get("expirationTime")
            or not lifecycle_matches(lifecycle, args.max_lifetime)
        ):
            raise RuntimeError("Sandbox did not report Running, Ready and the requested server lifetime")
        exec_guest(name, *prepare)
        if ready is not None:
            # Storage mounts and device checks must pass before this Sandbox enters the ready queue.
            exec_guest(name, *ready)
        obj = get_owned(name, state)
        status = obj.get("status") or {}
        if status.get("phase") != "Running":
            raise RuntimeError("Sandbox stopped during setup; it was not admitted to the ready queue")
        state["phase"] = "ready"
        state["uid"] = (obj.get("metadata") or {}).get("uid")
        state["expirationTime"] = status.get("expirationTime")
        if not state["uid"] or not state["expirationTime"]:
            raise RuntimeError("Sandbox did not report its UID and expiration time")
        validate_prepared(obj, state)
        write_state(state_file, state)
        print(f"prepared Sandbox {name}")
        return name




def get_owned(name: str, state: dict[str, object]) -> dict[str, object]:
    current = subprocess.run(
        ["nodus", "get", f"sandbox/{name}", "-o", "json"],
        check=True,
        capture_output=True,
        text=True,
    )
    obj = json.loads(current.stdout)
    meta = obj.get("metadata") or {}
    labels = meta.get("labels") or {}
    if (
        meta.get("name") != name
        or labels.get("example.nodus.dev/run") != state.get("run")
        or (state.get("uid") is not None and meta.get("uid") != state.get("uid"))
    ):
        raise RuntimeError("Sandbox operation refused: name or unique run label did not match")
    return obj




def delete_exact(name: str, state: dict[str, object]) -> None:
    # Check the unique run label before deleting by name; a selector could delete multiple objects.
    try:
        get_owned(name, state)
    except subprocess.CalledProcessError as exc:
        if state.get("phase") in {"ready", "claimed"} and state.get("uid") and wait_until_deleted(name):
            return
        raise RuntimeError(
            f"cleanup refused: could not verify ownership of Sandbox {name}; no delete was sent and "
            "the state file was retained for inspection"
        ) from exc
    except RuntimeError as exc:
        raise RuntimeError(
            f"cleanup refused: could not verify ownership of Sandbox {name}; no delete was sent and "
            "the state file was retained for inspection"
        ) from exc
    subprocess.run(
        [
            "nodus",
            "delete",
            f"sandbox/{name}",
            "--wait",
            "--timeout=5m",
            "--ignore-not-found",
        ],
        check=True,
    )




def wait_until_deleted(name: str) -> bool:
    """Use the CLI's structured delete-wait result to confirm absence after a failed get."""
    result = subprocess.run(
        ["nodus", "wait", f"sandbox/{name}", "--for=delete", "--timeout=1s"],
        check=False,
        stdout=subprocess.DEVNULL,
        stderr=subprocess.DEVNULL,
    )
    return result.returncode == 0




def read_state(state_file: Path) -> dict[str, object]:
    if not state_file.exists():
        raise RuntimeError(f"no prepared Sandbox state exists at {state_file}")
    state = json.loads(state_file.read_text(encoding="utf-8"))
    name = state.get("sandbox")
    run_id = state.get("run")
    if (
        not isinstance(name, str)
        or not name.startswith("ready-")
        or not isinstance(run_id, str)
        or len(run_id) != 32
        or any(char not in "0123456789abcdef" for char in run_id)
        or name != "ready-" + run_id[:12]
    ):
        raise ValueError("state does not contain a generated Sandbox name")
    return state




def write_state(state_file: Path, state: dict[str, object]) -> None:
    state_file.parent.mkdir(parents=True, exist_ok=True)
    with tempfile.NamedTemporaryFile(
        mode="w", suffix=".tmp", prefix=state_file.name + ".", dir=state_file.parent, delete=False, encoding="utf-8"
    ) as f:
        temp_path = Path(f.name)
        json.dump(state, f)
        f.flush()
        os.fsync(f.fileno())
    os.replace(temp_path, state_file)
    directory = os.open(state_file.parent, os.O_RDONLY)
    try:
        os.fsync(directory)
    finally:
        os.close(directory)




@contextmanager
def state_lock(state_file: Path) -> Iterator[None]:
    """Serialize operations sharing one local prepared-entry file."""
    state_file.parent.mkdir(parents=True, exist_ok=True)
    lock_path = state_file.with_name(state_file.name + ".lock")
    fd = os.open(lock_path, os.O_CREAT | os.O_RDWR, 0o600)
    try:
        try:
            fcntl.flock(fd, fcntl.LOCK_EX | fcntl.LOCK_NB)
        except BlockingIOError as exc:
            raise RuntimeError("prepared Sandbox state is already being used") from exc
        yield
    finally:
        try:
            fcntl.flock(fd, fcntl.LOCK_UN)
        finally:
            os.close(fd)




def cleanup_state(state_file: Path) -> None:
    if not state_file.exists():
        return
    state = read_state(state_file)
    delete_exact(str(state["sandbox"]), state)
    state_file.unlink(missing_ok=True)




def cleanup(args: argparse.Namespace) -> None:
    state_file = Path(args.state_file)
    with state_lock(state_file):
        cleanup_state(state_file)




def prepare_only(args: argparse.Namespace) -> None:
    state_file = Path(args.state_file)
    with state_lock(state_file):
        if state_file.exists():
            raise RuntimeError("an earlier prepared Sandbox still needs a job or cleanup")
        queue = ReadyQueue(args.desired_ready, args.hard_max)
        try:
            queue.prepare(args)
        except Exception:
            # Delete when ownership can be proved; retain state if creation or deletion is ambiguous.
            cleanup_state(state_file)
            raise
        print(
            "prepared Sandbox is available for one later invocation"
            if args.reuse_max_jobs == 1
            else f"prepared Sandbox supports at most {args.reuse_max_jobs} same-trust jobs"
        )




def validate_prepared(obj: dict[str, object], state: dict[str, object]) -> None:
    status = obj.get("status") or {}
    conditions = status.get("conditions") or []
    ready = any(c.get("type") == "Ready" and c.get("status") == "True" for c in conditions)
    spec = obj.get("spec") or {}
    lifecycle = spec.get("lifecycle") or {}
    expiration = status.get("expirationTime")
    recorded_expiration = state.get("expirationTime")
    if (
        state.get("phase") != "ready"
        or not isinstance(state.get("uid"), str)
        or not state.get("uid")
        or status.get("phase") != "Running"
        or not ready
        or not isinstance(expiration, str)
        or expiration != recorded_expiration
        or not lifecycle_matches(lifecycle, str(state.get("maxLifetime", "")))
        or spec.get("maxCostUSD") != state.get("maxCostUSD")
    ):
        raise RuntimeError("prepared Sandbox is no longer Running, Ready, or within its recorded lifetime")
    try:
        expires_at = datetime.fromisoformat(expiration.replace("Z", "+00:00"))
    except ValueError as exc:
        raise RuntimeError("prepared Sandbox expiration time is invalid") from exc
    if expires_at.tzinfo is None or expires_at <= datetime.now(timezone.utc):
        raise RuntimeError("prepared Sandbox has expired")




def return_check_digest(reset: tuple[list[str], bytes | None], ready: tuple[list[str], bytes | None] | None) -> str:
    """Bind return checks without persisting command text, script contents or credentials."""
    checks = [reset, ready]
    digest = hashlib.sha256()
    for check in checks:
        if check is None:
            raise ValueError("reuse requires a readiness check")
        argv, stdin = check
        digest.update(json.dumps(argv).encode())
        digest.update(b"\x00")
        digest.update(stdin or b"")
        digest.update(b"\x00")
    return digest.hexdigest()




def run_prepared(args: argparse.Namespace) -> None:
    state_file = Path(args.state_file)
    with state_lock(state_file):
        state = read_state(state_file)
        name = str(state["sandbox"])
        obj = get_owned(name, state)
        validate_prepared(obj, state)
        job_command = shlex.split(args.job_command)
        if not job_command:
            raise ValueError("--job-command must not be empty")
        max_jobs = state.get("maxJobs", 1)
        completed = state.get("completedJobs", 0)
        if (
            type(max_jobs) is not int
            or not 1 <= max_jobs <= 100
            or type(completed) is not int
            or not 0 <= completed < max_jobs
        ):
            raise RuntimeError("prepared Sandbox job limit is invalid or exhausted")
        reset = guest_command(args.reset_command, args.reset_script, "reset")
        ready = guest_command(args.ready_command, args.ready_script, "ready")
        if max_jobs > 1 and (
            not reset[0] or not ready[0] or return_check_digest(reset, ready) != state.get("returnCheck")
        ):
            raise RuntimeError("reuse requires the original reset and readiness checks")
        state["phase"] = "claimed"
        write_state(state_file, state)
        print(f"claimed prepared Sandbox once: {name}")
        returned = False
        try:
            exec_guest(name, job_command)
            state["completedJobs"] = completed + 1
            if completed + 1 < max_jobs:
                exec_guest(name, *reset)
                exec_guest(name, *ready)
                state["phase"] = "ready"
                validate_prepared(get_owned(name, state), state)
                write_state(state_file, state)
                returned = True
                print(f"returned prepared Sandbox: {completed + 1}/{max_jobs} jobs completed")
        finally:
            # Failed/uncertain execution or return is never eligible for reuse. A killed dispatcher leaves
            # durable phase=claimed, which requires explicit cleanup; it cannot execute another job.
            if not returned:
                cleanup_state(state_file)




def demo(args: argparse.Namespace) -> None:
    state_file = Path(args.state_file)
    with state_lock(state_file):
        queue = ReadyQueue(args.desired_ready, args.hard_max)
        if state_file.exists():
            raise RuntimeError("an earlier queue item still needs cleanup; run with --cleanup first")
        try:
            queue.prepare(args)
            name = queue.take()
            print(f"handed out once: {name}")
            subprocess.run(
                ["nodus", "exec", "--no-wake", f"sb/{name}", "--", *shlex.split(args.job_command)], check=True
            )
            try:
                queue.take()
            except RuntimeError as exc:
                print(f"queue empty; {exc}")
            else:
                raise AssertionError("the prepared Sandbox was handed out more than once")
        finally:
            cleanup_state(state_file)




def main() -> None:
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument("--desired-ready", type=int, default=1)
    parser.add_argument("--hard-max", type=int, default=1)
    parser.add_argument("--image", default="nodus/agent-tools")
    parser.add_argument(
        "--image-pull-secret",
        action="append",
        default=[],
        metavar="NAME",
        help="Registry Secret for image pulls only; never mounted in the guest",
    )
    parser.add_argument(
        "--reuse-max-jobs", type=int, default=1, help="opt-in same-trust job limit, 1..100; idle compute stays billed"
    )
    parser.add_argument("--reset-command", help="same-trust guest cleanup required before returning a reusable Sandbox")
    parser.add_argument(
        "--reset-script", help="local cleanup script streamed to bash; required check is pinned by digest"
    )
    parser.add_argument("--gpu", help="optional accelerator family; omit to run the compose example on CPU")
    parser.add_argument("--regions", nargs="+", help="optional Nodus region classes, for example us")
    parser.add_argument("--cpu", type=int, help="vCPU count (defaults to 1 for CPU or 24 for GPU)")
    parser.add_argument("--fuse-mount", action="store_true", help="enable guest FUSE support; requires --gpu")
    parser.add_argument(
        "--egress",
        choices=("Deny", "AllowList", "Open"),
        default="Deny",
        help="container outbound policy; Open is an explicit opt-in",
    )
    parser.add_argument(
        "--allow-host", action="append", default=[], help="allowed DNS host (requires --egress AllowList)"
    )
    parser.add_argument("--max-lifetime", default="20m")
    parser.add_argument("--max-cost", default="1.00")
    parser.add_argument("--state-file", default=".ready-queue-state.json")
    parser.add_argument(
        "--prepare-command",
        help="quoted command string parsed as argv without a shell; can mount/init customer storage before admission",
    )
    parser.add_argument(
        "--prepare-script",
        help="local script streamed to bash -s inside the Sandbox before it is admitted",
    )
    parser.add_argument(
        "--ready-command",
        help="optional quoted data-read/CUDA readiness check, parsed as argv and run before queue admission",
    )
    parser.add_argument(
        "--ready-script",
        help="optional local script streamed to bash -s inside the Sandbox before it is admitted",
    )
    parser.add_argument(
        "--secret",
        action="append",
        default=[],
        metavar="NAME",
        help="existing Nodus Secret to expose to the Sandbox (repeatable; secret values are not copied)",
    )
    parser.add_argument(
        "--job-command",
        default="python -c \"print('prepared-queue-job-ok')\"",
        help="quoted command string parsed as argv without a shell",
    )
    modes = parser.add_mutually_exclusive_group()
    modes.add_argument("--cleanup", action="store_true", help="delete the exact Sandbox recorded for cleanup")
    modes.add_argument(
        "--prepare-only", action="store_true", help="prepare one Sandbox and leave it ready for one later invocation"
    )
    modes.add_argument(
        "--run-prepared", action="store_true", help="claim and run one prepared Sandbox without creating or waking it"
    )
    args = parser.parse_args()
    try:
        if args.cleanup:
            cleanup(args)
        elif args.prepare_only:
            prepare_only(args)
        elif args.run_prepared:
            run_prepared(args)
        else:
            demo(args)
    except subprocess.CalledProcessError as exc:
        raise SystemExit(exc.returncode) from exc
    except (RuntimeError, ValueError) as exc:
        print(str(exc), file=sys.stderr)
        raise SystemExit(2) from exc




if __name__ == "__main__":
    main()

Run commands

Section titled “Run commands”

nodus exec runs one command in the running Sandbox and streams its standard output and standard error back as the command writes them. The exit code of nodus exec is the command’s. Every command is recorded as a Process of the Sandbox, whatever started it (the CLI, an agent or the MCP tools): its name, such as sb-hello-12, comes back with the result, and nodus get processes lists it with how it ended.

Terminal window
$ nodus exec sb/hello -- python -c 'print(6 * 7)'
42
$ nodus exec sb/hello -- sh -c 'pip list 2>/dev/null | head -3; exit 3'; echo "exit $?"
Package    Version
---------- -------
pip        24.2
exit 3

A command runs in spec.workingDir (/workspace unless you set it) with the Sandbox’s environment, as a non-root user. Pass a single string to sh -c to use pipes and redirection. Every command counts as activity, so a Sandbox never stops as idle while a command is running.

Pass --no-wake when a command must use an already running Sandbox. It refuses a stopped, starting or expired Sandbox without waking compute or waiting for startup. Without that flag, the usual implicit wake remains.

Open a shell

Section titled “Open a shell”

nodus shell is nodus create sandbox and nodus exec -it in one step: it opens an interactive terminal in a Sandbox and creates the Sandbox first when the name is new. It takes the same flags as nodus create sandbox, and the image is nodus/agent-tools unless you pass --image.

Terminal window
$ nodus shell sandbox/dev --cpu 2 --memory 4Gi
sandbox/dev created
sandboxes/dev is starting; waiting for it

You are now in a terminal in the Sandbox, in /workspace. When you exit, nodus shell sandbox/dev opens it again on the current running attempt. After a stop, use /nodus/state for CPU checkpoint files; /workspace is not saved.

The Sandbox stays after you exit, and stops by itself when idle. Pass --rm to delete a Sandbox this command created when the shell exits; --rm refuses a Sandbox that already exists, and so does a flag such as --cpu, because neither changes a Sandbox that is there. nodus shell with no name gives the shell a Sandbox of its own and deletes it on exit unless you pass --keep. Put a command after -- to run it instead of bash: nodus shell sandbox/dev -- zsh. With input that is not a terminal, the shell runs without one, so echo 'make test' | nodus shell sandbox/dev works in a script. The exit code is the shell’s.

Timeouts

Section titled “Timeouts”

Each command has its own timeout: 10 minutes unless you set one, and at most 24 hours. When the timeout passes, the process is killed with SIGKILL and the command ends with the reason DeadlineExceeded. The Sandbox keeps running. Set the timeout per command, for example sb.exec("make", "test", timeout="2m") in Python or "timeout": "2m" in the API request.

How a command ended is in its result:

Reason Meaning
none Exit code 0
NonZeroExit The command exited with another code
Signaled A signal ended it; the exit code is 128 plus the signal number
DeadlineExceeded Its timeout passed and it was killed
StartFailed It could not start, for example because the program does not exist
NodeLost The machine running the Sandbox was lost while the command ran
OutputLimitExceeded It wrote more output than a Process may keep
ParentStopped The Sandbox stopped or was deleted while the command ran
Cancelled You cancelled it

A command line holds at most 1,024 arguments and 64 KiB in total, plus up to 256 extra environment variables. A Sandbox runs at most 1,024 commands at once; the next one fails with 429 TooManyRequests until one ends.

Files

Section titled “Files”

In the console, open Files on a Sandbox, or Browse files inside an agent run. Upload files or a folder, open a text file to edit it, or download it. The editor supports UTF-8 text up to 1 MiB; console uploads support up to 32 MiB per file. Uploads refuse to replace existing content. If an agent changes a file while you edit, your save is refused and your draft stays available to copy or merge. Load latest and replace draft replaces your local draft with the current file. A stopped or restarted Sandbox cannot be edited through a stale panel. Choose Wake and browse explicitly to start stopped compute.

nodus cp copies files in both directions, and works on every image because the file service is part of the Sandbox runtime, not of the image:

Terminal window
$ nodus cp ./notes.txt sb/hello:/nodus/state/notes.txt     # laptop to Sandbox
$ nodus cp sb/hello:/workspace/result.json ./result.json  # Sandbox to laptop

The same operations are one request each on /apis/nodus.dev/v1/namespaces/{project}/sandboxes/{name}/files:

Request Does
GET …/files?path=/workspace/a.txt Returns the file’s bytes
PUT …/files?path=/workspace/a.txt Replaces the file with the request body, creating parent directories
DELETE …/files?path=/workspace/a.txt Deletes a file or an empty directory

A write replaces the file atomically: a reader sees the old content or the new content, never half of it. A file is at most 64 MiB per request; a larger one fails with 413 FileTooLarge. Paths are absolute, and a path that does not exist answers 404 NotFound.

For an already running Sandbox, add wake=false to refuse an implicit start. Add both expectedUID and expectedAttempt from the Sandbox’s current metadata and status to refuse a request if the Sandbox was replaced or restarted. A stale target returns 409 Conflict; no new compute starts. Without wake=false, the existing automatic wake behavior applies.

Writing without overwriting someone else’s change

Section titled “Writing without overwriting someone else’s change”

When two writers share a file, send the digest of the content you started from as expectedSHA256:

Terminal window
$ curl -X PUT --data-binary @plan.md "$API/sandboxes/hello/files?path=/workspace/plan.md&expectedSHA256=$(sha256sum old-plan.md | cut -d' ' -f1)"

The write happens only if the file still has that digest. If it changed, nothing is written and the request fails with 412 SHA256Mismatch: read the file again, merge, and retry. The digest is 64 lowercase hex digits. The digest of empty content (e3b0c442…b855) matches a file that does not exist yet, so it means “create only”.

Stop, start and wake

Section titled “Stop, start and wake”

A Sandbox is Running while it holds compute and Stopped while it does not. CPU Snapshotted continuity selects /nodus/state for a checkpoint before sending the process its stop signal. Flush state before requesting stop; shutdown-handler writes, /workspace, installed packages and the rest of the root filesystem are not captured. CPU status does not currently expose StateSaved or status.snapshot; verify the expected files after starting. A stopped phase alone is not proof that state was saved. GPU explicit state directories use the separate post-exit save behavior described above.

Terminal window
$ nodus stop sb/hello        # checkpoints /nodus/state, then releases compute
$ nodus start sb/hello       # starts a fresh process; verify the state files
$ nodus get sb/hello
NAME    PHASE     ACTIVITY   CPU   MEMORY   GPU   COST      AGE
hello   Stopped   Idle       1     1Gi      -     $0.0042   12m

nodus stop sets spec.state: Stopped and the Sandbox stays stopped until you start it. Every other stop is made by the platform, shows its reason in status.stopReason, and ends the next time something needs the Sandbox:

stopReason Why it stopped
User You ran nodus stop
Idle Nothing used it for idleTimeout
InsufficientCredits Your balance could not fund the next period
BudgetExceeded A Budget on the project reached its limit
MaxCostReached The Sandbox cost as much as spec.maxCostUSD

A stopped Sandbox starts again on any of these wake triggers:

  • a command (nodus exec, or an exec request),
  • a file request,
  • nodus start,
  • creating a Sandbox with the same name and an identical spec.

Credits and budgets still apply to a wake. A Sandbox stopped for InsufficientCredits, BudgetExceeded or MaxCostReached stays stopped until the balance, the Budget or maxCostUSD allows the next period of compute. Until then a command, a file request and nodus start answer 402 instead of SandboxStarting: InsufficientCredits states the hold the start needs and what you have, and BudgetExceeded the Budget or maxCostUSD that is full. Add credits, raise the Budget or raise maxCostUSD, then send a trigger again. A create by name records its wake too, but returns the Sandbox as it is: still stopped, with Funded false in its conditions.

SandboxStarting and Retry-After

Section titled “SandboxStarting and Retry-After”

A command or file request that finds the Sandbox stopped starts it and answers 503 SandboxStarting with a Retry-After header (2 seconds), because the container is not ready yet. The CLI and the SDK wait and retry for you, so nodus exec on a stopped Sandbox just takes a few seconds longer. If you call the API directly, retry after the time in Retry-After; each retry is also a wake trigger, so it does no harm to send them early.

Other reasons a request can fail while the Sandbox is not usable:

Status and reason Meaning
409 SandboxNotRunning The Sandbox is being deleted
409 SandboxFailed The Sandbox failed to start or lost its machine; status.reason says why
402 InsufficientCredits The Sandbox is stopped for lack of credits and cannot start yet
402 BudgetExceeded The Sandbox is stopped because a Budget or its maxCostUSD is full and cannot start yet
404 NotFound There is no Sandbox with that name

Each code has a page in the error reference.

Idle, onIdle and maxLifetime

Section titled “Idle, onIdle and maxLifetime”

A Sandbox is idle when no command is running, no file request arrived for idleTimeout and no terminal had input in that time. A running process, including one that prints nothing for hours, counts as busy; a terminal that is open but silent does not count as activity.

When the idle time passes, onIdle decides what happens:

  • Stop (the default) requests a state-directory checkpoint, releases compute and sets stopReason: Idle. The next wake trigger starts it again.
  • Delete removes the Sandbox with its files. Use it for one-off work that leaves nothing worth keeping.

idleTimeout: 0s turns idle detection off, so the Sandbox runs until you stop it, until maxLifetime or until a budget ends it.

maxLifetime is a hard deadline counted from creation, 24 hours by default. When it passes, the Sandbox is deleted whatever it is doing, running or stopped. It is the backstop that keeps a forgotten Sandbox from living forever. Create a new Sandbox, or one with a longer maxLifetime, to carry on.

Network access

Section titled “Network access”

A Sandbox has no outbound network access unless you open it. spec.network.egress.policy is one of:

  • Deny (the default): nothing leaves the Sandbox, so code in it cannot download or upload anything.
  • AllowList: set spec.network.egress.allow to explicit DNS names, such as github.com, or *.domain patterns. HTTP and HTTPS tools must use the injected HTTP_PROXY and HTTPS_PROXY settings. Other destinations and private addresses stay blocked. Presets are not accepted for Sandboxes.
  • Open: the Sandbox can reach the public internet. It still cannot reach other Sandboxes, the machine it runs on or Nodus’s own services.

A Sandbox accepts no public inbound connections. Use commands, file requests, or nodus port-forward to reach its applications through your authenticated Nodus session.

Your application can listen on 127.0.0.1 inside the Sandbox. HTTP requests and port forwarding connect to that Sandbox’s loopback, including GPU Sandboxes with fuseMount: true.

For a backend making short HTTP calls, the Beta API provides a stable authenticated endpoint: POST /apis/nodus.dev/v1/namespaces/PROJECT/sandboxes/NAME/http. It returns the application’s status, headers and base64 body inside JSON. The Sandbox must already be running; the call never starts compute.

import nodus


box = nodus.Sandbox.from_name("dev", wake=False)
response = box.http_request(8000, "GET", "/health")
response.raise_for_status()
print(response.text)

For asynchronous callers, use await box.http_request.aio(...). Existing Nodus login or API-key configuration authenticates the request. API keys need processes:write in the Sandbox’s project. The MCP sandbox_http_request tool uses this same API; its request fields are port, method, path, optional headers (arrays of values), and optional base64 body. API and SDK responses retain application errors and redirects; they do not follow redirects or retry. A connection failure can occur after the application acted, so check its state before retrying.

Calls last at most 30 seconds, with 1 MiB request and 2 MiB response bodies. Use a custom header such as X-App-Token for application authentication. Nodus credentials, cookies and routing headers are never forwarded. Use port-forward for streaming, WebSockets, browser access, or larger payloads. See the backend HTTP example.

Browser previews

Section titled “Browser previews”

In an Agent run, open Sandbox → Previews, add your app’s port, and choose Open preview. The same controls are in a Sandbox’s Ports tab. Ports are private by default: project execution access is required. The app opens in a separate tab on an isolated user-content domain. Adding a port, viewing its address, and minting a sign-in link do not start compute.

You can declare ports through the API or CLI:

Terminal window
nodus patch sandbox/dev --type merge -p '{"spec":{"ingress":{"ports":[{"port":3000}]}}}'
nodus open sandbox/dev --port 3000

Or use Python:

box = nodus.Sandbox.from_name("dev")
preview = box.tunnels.open(3000)
# Open preview.url in your browser. Do not log or share private sign-in URLs.

Private sign-in links expire after 60 seconds; browser sessions last up to one hour and remain subject to current project permissions and credential revocation. API keys need processes:write. The preview endpoint requires {port, expectedUID} so an old panel cannot mint access to a replacement Sandbox. status.endpoints contains stable addresses without credentials, not an app readiness check.

Choose Make public only for an app you intend anyone with its URL to access. Public links normally leave idle-stopped compute asleep. Enabling Allow visitors to wake permits those requests to restart compute and incur charges. Authenticated preview requests can wake idle-stopped compute; an explicit Stop, memory pause, expired Sandbox, or lost retained memory requires explicit Start. Ordinary funding and budget admission still apply. Removing a port revokes new requests; an already-open stream may remain until it closes or the runtime stops.

Declare up to 16 ports, from 1024–65535 excluding reserved ports 7777, 18080 and 49983. If preview routing is unavailable, or you prefer a local tunnel to port 8000, run:

Terminal window
nodus port-forward sandbox/dev 8000:8000

Keep the command running and connect to http://127.0.0.1:8000. The connection uses your Nodus credentials; the local listener defaults to loopback. This provides a local tunnel, not a public or persistent preview URL.

Spending limits

Section titled “Spending limits”

Review the estimate before starting a Sandbox.

nodus get sb/hello shows the running total in the COST column. Set spec.maxCostUSD to cap one Sandbox: it stops when it has cost that much, and you can raise the cap with another create by name. Without a cap, your credit balance and any Budget on the project are the limits.

Next steps

Section titled “Next steps”