SageMaker Inference now supports Multi-Instance GPU to maximize GPU utilization for model endpoints
Amazon SageMaker Inference now supports GPU partitioning with NVIDIA Multi-Instance GPU (MIG), which divides one GPU into hardware-isolated partitions so you can run a different model in each or take a right-sized fraction of a newer accelerator. This post shows how two optional API fields place one model per partition, and what we measured against a whole older GPU.
What happened
Amazon SageMaker Inference now supports GPU partitioning with NVIDIA Multi-Instance GPU (MIG), which divides one GPU into hardware-isolated partitions so you can run a different model in each or take a right-sized fraction of a newer accelerator. This post shows how two optional API fields place one model per partition, and what we measured against a whole older GPU.
Why it matters
The development may change operating conditions or market expectations around NVIDIA, Amazon. Further confirmation and measurable outcomes matter.
Affected entities
View evidence
1 reports · 1 original report · 1 independent
- AWS Machine Learning BlogPrimary source · Supports · EN · 100%SageMaker Inference now supports Multi-Instance GPU to maximize GPU utilization for model endpoints ↗
Claims
- SageMaker Inference now supports Multi-Instance GPU to maximize GPU utilization for model endpoints Observed
Conflicts
No material conflict detected in the available evidence.
Timeline
- First reported
Market move following event
Market reaction is not yet available for this asset and time window.