Five real workloads, run in one night across AWS, Modal and Lambda: a 728 GB climate analysis, a simulation of a billion particles, every ship in US waters, and the human proteome folded in eight GPU shards. Each one landed on the cheapest machine that fit it, and one of them changed clouds while it was running.
workloads in one night
5
clouds: AWS, Modal and Lambda
3
spent in all
$203
saved on the spot workloads
60%
Twenty years of hourly global temperature, all in memory
ERA5's hourly 2 m air temperature for 2005 to 2024, 240 monthly files and 267 GB on disk, loaded into one 728 GB array on a single spot machine and analysed cell by cell. The result: 79% of the planet's cells had more heatwave days in the last decade than in the one before.
state in memory
728 GB
warming per decade, 2005 to 2024
0.31 °C
heatwave days per year, last decade against the first
14.9 vs 7.8
hottest hour in the record
53.8 °C
42 to load, 5 to analyse
47 min
saved against the same machine on demand
71%
Warming trend of the annual mean, least squares over the 20 years.The 99th percentile of daily maximum temperature.
One r6in.32xlarge spot machine at $3.13/h. Paid $2.52; the same machine on demand would have cost $8.79.
A billion-particle universe
A particle-mesh N-body simulation of cosmic structure formation: 1024 cubed particles, 1.07 billion of them, on a 2048 cubed gravity mesh in a 500 Mpc/h box, from z = 50 to today in 120 leapfrog steps, numpy and numba on 64 vCPUs.
particles
1,073,741,824
gravity mesh
2048³
memory
89 GB
64 seconds each
120 steps
knots above 200 times the mean density
18,374
saved against the same machine on demand
52%
The densest knot at z = 0, a 30 Mpc/h cube.The same slab at z = 5, before the web formed.
One r7i.16xlarge spot machine at $2.02/h. Paid $4.61; the same machine on demand would have cost $9.58.
Every ship in US waters, July to December 2024
NOAA's Marine Cadastre AIS feed for the half of 2024 it has published: 178 daily files, 66 GB downloaded at 38 MB/s, parsed into compact arrays and drawn at a hundredth of a degree. 1.57 billion positions from 80,593 vessels, from the Houston Ship Channel to the lanes into New York.
positions
1.57 billion
vessels
80,593
arrays in memory
28 GB
busiest port by vessels
New York
busiest port by positions
Houston
the longest single track
87,000 km
The Gulf: the Houston Ship Channel, the Mississippi below New Orleans, the supply lanes to the platforms.The Northeast.
One r7i.8xlarge spot machine at $1.04/h. Paid $0.95; the same machine on demand would have cost $1.90.
The human proteome, folded overnight
ESMFold, a 3 billion parameter structure model, over the 16,650 reviewed human proteins of 30 to 800 residues, in eight GPU shards spread across Modal H100s, an A100 and L40Ss, and a Lambda A100. A structure for every protein by morning, with a confidence score for each and a gallery of the most confident per shard.
proteins folded, none failed
16,650
mean confidence, pLDDT
71.4
residues per second on an H100
82
the most confident structure, RPE
97.0
shards on H100, A100 and L40S GPUs
8
under AWS list price for the same GPU hours
27%
The second shard's 16 most confident structures, C-alpha traces coloured by confidence.
Modal H100, A100 and L40S GPUs and a Lambda A100. Paid $162.55; the same GPU hours at AWS list price would have cost $222.98.
A job that changed clouds while it was running
A copy of the ships job with 42 days of data was moved 100 seconds into its run, from an AWS spot machine to a Lambda GPU box and back again, with its arrays, open loops and progress intact. Portal checkpointed the running Python program, shipped it through the store and resumed it on the other side, twice.
positions held in memory
411 million
moves, both committed
2
into the run when the first move began
100 s
on Lambda before moving back
240 s
the whole run, both moves included
17 min
steps lost
0
AWS r7i.2xlarge spot to Lambda gpu_1x_a10 and back, the whole run for $0.78.
Karpathy's autoresearch: 78% cheaper than AWS
What the same run costs on eight providers, with AWS on demand as 100. On Portal it came to 22. Portal put the run on the cheapest machine that fit it, across the clouds it was connected to.
