Record Breaking Google Cloud Compute 220,000+ CPU Core Cluster!

eva2000 · Apr 22, 2017

An MIT math professor recently broke the record for the largest ever Compute Engine cluster, with 220,000 cores on Preemptible VMs, the largest known high-performance computing cluster to ever run in the public cloud Google Cloud Platform Blog: 220,000 cores and counting: MIT math professor breaks record for largest ever Compute Engine job

Sutherland began using Compute Engine in 2015. For his first-ever job, he fired up 2,250 32-core instances and completed about 60 CPU-years of computation in a single afternoon.

Before settling on Compute Engine, Sutherland ran jobs on his own 64-core machine, which could take months, or wrangled for compute time on one of MIT’s clusters. But getting the number of cores he needed often raised eyebrows, and he was limited by the software configurations he could use. By running on Compute Engine, Sutherland can install exactly the operating system, libraries and applications he needs, and thanks to root access, he can update his environment at will.

Sutherland considered running his jobs on AWS before choosing Google but was dissuaded by its Spot Instances model, which forces you to name your price up front, with prices that can vary significantly by region and fluctuate over time. A colleague encouraged him to try Compute Engine Preemptible VMs. These are full-featured instances that are priced up to 80% less than regular equivalents, but can be interrupted by Compute Engine.
Click to expand...

For his next run, he hopes to expand his search to non-hyperelliptic curves of genus 3, breaking his own record with a 400,000-core cluster.
Click to expand...

Log in / Register

Forums

Get the most out of your Centmin Mod LEMP stack

Record Breaking Google Cloud Compute 220,000+ CPU Core Cluster!

eva2000 Administrator Staff Member

.

Log in / Register

Useful Searches

Get the most out of your Centmin Mod LEMP stack

Record Breaking Google Cloud Compute 220,000+ CPU Core Cluster!

eva2000 Administrator Staff Member

.