We benchmarked node efficiency on a bunch of Kubernetes providers. After reading the responses, I have more questions than I started with. The hot issue is how burstable nodes should be scheduled.
What are burstable instances?
Here’s the gist of it:
- AWS, GCP, and Azure all have “burstable” node types.
- Burstable nodes guarantee 1 CPU, but let you temporarily “burst” to 2 CPUs
- There are variants with different CPU numbers
It’s important to read the fine print to understand how a burstable node actually performs.
The big controversy: How should burstable CPUs be scheduled?
Think hard before continuing: how do you schedule Kubernetes Pods to burstable CPUs?
There’s no great answer here. For simplicity, imagine a 1vCPU node that can burst to 2vCPUs. GKE takes the approach that this is fundamentally a 1vCPU node. So they schedule 1vCPU worth of requests to the node and mark the remaining 1vCPU as unavailable.
On the other hand, AWS and AKS schedule the node as if it had 2vCPUs.
Which is better? I’m honestly not sure. AKS is violating the assumption that Pods are guaranteed their requests. EKS will bill you exorbitantly if you have accurate CPU requests. GKE will be… OK? If you set requests according to peak CPU, you won’t benefit from the ability to burst.