Portaldocs
Run and watch

Status, logs and explain

Follow a job's state, output and Portal's decisions about it, from the CLI or a coding tool.

Warnings and limits

  • A job whose program has ended stays running until its provider confirms the machine's end (minutes for an EC2 GPU instance); the console reads it as Finishing with what it waits for. Nothing after the program's exit is charged, and a stop sent then changes nothing.
  • Portal keeps the first 256 MiB of a job's output. Past that, one line says the rest was not copied.
  • A log is read 256 KiB at a time, and a read that waits for new output waits at most 10 s.
  • portal ps shows active jobs and the 5 that ended last; --all shows the rest. Listings and name lookups reach the newest 500 jobs.
  • Portal keeps a job's newest 20,000 events and 20,000 samples (one sample every 10 s, about 55 hours).
  • A job name refers to the newest job with that name; a start of an id must match one job only.
  • When a move or a checkpoint waits for the program to reach a point where it can pause (a long native call, a running thread, an open file, a child process), the job's events say why, as pause, once per reason in 5 minutes; portal explain JOB shows them.
  • The job list and the overview answer from a cache kept 20 s: a job's creation, status change or deletion shows at once, other changes (cost, progress, a machine's state) within 20 s.
  • A node that stops answering leaves the job running with the time it was last heard (unreachable_since; the console says its node has not answered since then). Portal asks the provider after 10 minutes of silence (2 minutes past a spot notice's deadline) and declares the state lost only when the provider says the machine is gone twice, a check apart, with the node still silent; each event says what it waits for. A suspend or move asked meanwhile is refused with that, not with "the job goes on".

Jobs

portal ps
portal ps --all

JOB in every command is a job id (job-1030-...), a unique start of one, or a job's name (the newest job with that name).

StateWhat it means
queuedaccepted, waiting for its turn
provisioning, startinga machine is being made, then the job's environment installed
runningthe script runs
movingthe program is paused and travelling to another machine
suspending, suspendedits state is being saved, then kept in storage with no machine running
resumingit is being brought back on a machine
finishedthe script exited with code 0
failedthe script exited with an error, or the job could not go on
stoppedsomeone stopped it, or a policy did
no_capacityno offer could start a machine

Logs

portal logs JOB
portal logs JOB --tail 50
portal logs JOB --follow

--follow prints until the job ends; Ctrl-C stops following, not the job.

Why Portal did something

portal explain JOB

lists every policy decision about the job, oldest first: when, which policy (spot, return, oom, cheaper, lifetime, rightsize, budget, idle or ttl), what it did and why. A look for a cheaper offer that found nothing worth moving to is listed only when its reason changes.

Exit codes

portal run (while it follows the job) and portal logs --follow exit with the script's own exit code when the job finishes. Otherwise:

CodeMeaning
0done
1an error, or the job failed or was stopped
2wrong usage
3the job is suspended
4no capacity
130you detached with Ctrl-C

From a coding tool

The MCP tools give the same views: status (one job, or the overview), jobs (the list), logs (the last lines, or exactly what came after an earlier read), explain, events and samples (CPU, memory, GPU and progress points). See MCP tools.

On this page