Reference
Settings
Every setting of a workspace and every policy, with its default, and where each one is changed.
Warnings and limits
- Only an Owner changes settings, in the console's Settings, with a Clusy sign-in from the last 5 minutes. On hosted Portal,
portal settings KEY VALUEand agents cannot change them. - A change of policy applies to every job from its next check;
encrypt_capsulesapplies to jobs created after the change. - Settings that run Portal's storage and node images are its operators' and are not listed here.
- A job's own flags and its project's
portal.yamltake precedence over these defaults for that job.
Read and change
portal settings
portal settings spot.onportal settings shows the workspace's settings, and portal settings KEY one of them. To change one, an Owner opens Settings in the console: Policies holds the policies, Controller the rest.
For one job, use portal run flags (--no-cheaper, --policy idle.after_min=60) or a portal.yaml in the project, which holds the same policy object at its top level or under policy::
cheaper:
on: false
idle:
after_min: 60All settings
The tables below are generated from Portal's code.
Policies
| Setting | Kind | Default | What it does |
|---|---|---|---|
spot.on | on or off | true | Checkpoint and suspend a job when its spot or preemptible machine gets a notice. |
spot.exclude_min | minutes | 30 | How long a pool that reclaimed a machine is left out of new placements. |
spot.move_when_over | on or off | true | Move a job running on spot to an on-demand machine once its measured state is above what its machine's notice can save. A job pinned to an exact spec is warned instead. |
spot.rebalance | on or off | true | Move a job whose measured state is above what its notice can save off its machine as soon as EC2 recommends rebalancing, usually minutes before the notice. A job whose state fits waits for the notice. Every recommendation is recorded with what followed it. |
landing | one of serverless, same, cheapest | serverless | How Portal ranks destinations when it finishes an interrupted move or resume itself: serverless (offers that boot in under 30 s first), same (the machine type the job ran on) or cheapest. It never chooses where a suspended job resumes: you do. |
return.on | on or off | true | Look for spot offers while a job runs on a machine that is not spot, and move there when one qualifies. |
return.min_saving_pct | percent | 20 | The smallest saving, in percent of the current price, that makes a return worth a move. |
return.min_saving_x_move | number | 3.0 | The saving must be at least this many times the move's own cost. |
return.every_min | minutes | 10 | How often a job looks for a spot offer to return to. |
return.cooldown_min | minutes | 30 | How long after a move a job waits before it looks again. |
oom.on | on or off | true | Checkpoint and suspend a job whose process runs out of memory, when the failure is recoverable. |
oom.gpu_memory_x | factor | 1.5 | Kept with the policy; Portal does not use it to pick a machine (you choose the larger machine when you resume). |
cheaper.on | on or off | true | Look for cheaper offers while a job runs, and move there when one qualifies. |
cheaper.min_saving_pct | percent | 20 | The smallest saving, in percent of the current price, that makes a move worth it. |
cheaper.min_saving_x_move | number | 3.0 | The saving must be at least this many times the move's own cost. |
cheaper.every_min | minutes | 10 | How often a job looks for a cheaper offer. |
cheaper.cooldown_min | minutes | 30 | How long after a move a job waits before it looks again. |
lifetime.on | on or off | true | Checkpoint and suspend a job before a provider's fixed time limit ends its machine. |
lifetime.margin_min | minutes | 30 | How long before the limit the checkpoint starts. |
rightsize.mode | one of on_move, active, off | on_move | Size CPU and memory from the job's observed peaks: on_move (at a move), active (also moves to resize) or off. |
rightsize.headroom_x | factor | 1.3 | Headroom over the observed peak when sizing. |
rightsize.min_saving_pct | percent | 20 | The smallest saving that makes an active resize worth a move. |
rightsize.every_min | minutes | 10 | How often an active resize looks. |
budget.on | on or off | true | Enforce a job's budget when it has one. |
budget.usd | US dollars | none | A default budget in US dollars for jobs that set none (none by default). |
budget.action | one of suspend, stop | suspend | What a spent budget does: suspend or stop. |
budget.margin_min | minutes | 2 | How early the budget acts: this long at the machine's price before the budget is reached. |
idle.on | on or off | true | Act on a job that has been idle. |
idle.after_min | minutes | 30 | How long without progress or CPU use counts as idle. |
idle.action | one of suspend, stop | suspend | What idleness does: suspend or stop. |
idle.cpu_percent | number | 5.0 | CPU use, in percent of one core, below which a job counts as idle. |
ttl.on | on or off | true | Stop a job after a total running time. |
ttl.hours | number | 72.0 | The running time after which a running job is stopped; time suspended does not count toward it. A suspended job ends this many hours after its creation on the clock, its checkpoint kept 7 days for a resume to continue it. |
Workspace settings
| Setting | Default | What it does |
|---|---|---|
r2_regional | true | Use the regional storage locations for checkpoints when a closer one exists. |
modal_plan | starter | Your Modal plan (starter, team or enterprise), which sets the egress allowance Portal's cost model applies. |
encrypt_capsules | true | Encrypt each job's checkpoints with a key of its own. Decided when a job is created. |
baseline | aws | The provider whose on-demand offer a plan shows beside the quotes, as a reference (when it is connected). |
gcp_regions | none | The Google Cloud regions Portal may place jobs in (none set: Portal's default list). |
gcp_max_run_h | 24.0 | The hard limit, in hours, on a Google Cloud machine's life. |
gcp_preemption_notice_s | 120 | The termination notice Portal asks Google Cloud for on spot machines, in seconds. |
capture_probe | on | The checkpoint check at a job's first pause point and then every capture_probe_every_min: on checks whether a checkpoint could carry the job's state without saving anything (5 s in all; a pause asked for meanwhile is taken at once), full also rehearses a real checkpoint on the machine's disk (deleted after) and reports its size and time, on on-demand machines only (on spot the check runs without it), off skips it. A job's env PORTAL_CAPTURE_PROBE overrides it for that job. |
capture_probe_every_min | 10 | How often, in minutes, the checkpoint check repeats while a job runs. |