Quotas and capacity
What your provider's quotas and capacity mean for a launch, what Portal checks, and what happens when a machine is not available.
Warnings and limits
- Portal never requests a quota increase, and never changes your account's limits. You or your coding tool ask the provider.
- Connecting a provider does not check its quotas. Quotes do, for AWS EC2 (the regional vCPU quotas, read at most every 10 minutes), Azure VMs (vCPU quotas with their usage) and Google Cloud VMs (CPU, GPU and preemptible GPU quotas): an offer your quota cannot fit is listed as unavailable with the quota named, and a quota Portal cannot read leaves the offer unchecked. Other providers' limits show only at launch, as a start that fails and the next offer tried.
- A quote does not reserve capacity. The machine you choose can be gone by the time Portal launches it.
- When no offer has capacity, Portal tries up to 4 offers in turn, then ends the job as
no_capacity(exit code 4) with choices in the job's error and events: the same machine on-demand, spot in up to two pools it did not try, or, for a GPU type Modal could not schedule, Modal's other GPU types, each with its price and an exacton=spec. Portal never switches on its own: you or your coding tool choose. - A pool that just had no capacity ranks last for 5 minutes.
- Asking providers for quotes takes up to 45 s per provider and 120 s in all.
- On Modal, a job pinned to one GPU type ends
no_capacityafter 5 minutes without capacity (3 minutes when other offers follow). Modal's L4 pool is often short; T4, A10 and A10G start at once as a rule. - Known issue: a job launched with an exact
--onGPU machine and no--gpucan be offered a machine without a GPU among those choices. Give--gpuas well, or choose a GPU offer. See Known issues.
Quotas are yours to check
Each provider limits what your account can run: vCPUs per region and family on AWS, CPUs and GPUs per region on Google Cloud, cores per subscription and region on Azure, and concurrency on Modal and Lambda. Spot and GPU limits are often separate and often start at zero.
When you connect a provider, Portal says that quotas have not been checked, and offers a prompt your coding tool can use to compare your limits with what your workload needs and to prepare the minimum increase request. Requests go through your provider, with your own login.
What Portal checks
| Provider | Checked at launch |
|---|---|
| AWS EC2 | the regional vCPU quota the instance counts against (standard, G and VT, or P; spot and on-demand apart), against the vCPUs your account already runs |
| Others | nothing: a launch the provider refuses for quota fails like one with no capacity |
On AWS, an instance type the quota has no room for is skipped. When the quota is smaller than a single instance of that type, the type is refused until the quota is raised.
When a machine is not available
A launch goes through the offers that fit your job, best first:
- Portal asks the provider for the machine. A provider that is short of capacity is asked again: EC2 for up to 5 minutes, with waits from 5 s growing to 60 s; Lambda after a cooldown that starts at 60 s.
- If the machine still cannot start, Portal tries the next offer, up to 4 in all.
- If none starts, the job ends with "No capacity is available for the requested machine right now. Try another machine or region, check your quota, or retry later."
portal runexits with code 4.
A job you started with an exact --on spec tries only that machine.
What to do
- Run
portal priceswith the same requirements and pick another offer, region or provider. - Ask for fewer or smaller GPUs, or allow on-demand with
--no-spotoff (the default allows both). - Check the provider's quota page for the region and family, and request what the job needs.
- Retry later: capacity, especially for GPUs, changes through the day.