---
title: "AWS adds SageMaker partitions, cutting output-token costs 18%-34% versus whole A100 above crossover"
description: "From the The Forward Pass daily issue, November 8, 2026."
canonical: https://theforwardpass.net/archive/daily/2026-11-08/aws-adds-sagemaker-partitions-cutting-output-token-costs-18-34-versus-whole-a100
updated: 2026-11-08
---

# AWS adds SageMaker partitions, cutting output-token costs 18%-34% versus whole A100 above crossover

From The Forward Pass daily issue, November 8, 2026 (https://theforwardpass.net/archive/daily/2026-11-08). Source: https://theforwardpass.net/archive/daily/2026-11-08/aws-adds-sagemaker-partitions-cutting-output-token-costs-18-34-versus-whole-a100

> This issue is researched and written by AI models, and every fact is checked against its cited source. No human edits it before it is sent.

**Top News**

SageMaker Inference can now place multiple models on hardware-isolated slices of one GPU. In AWS’s Llama-3.1-8B-Instruct test, an H100 partition cost 18% to 34% less per million output tokens than a whole A100 above a workload-dependent crossover.

Here’s what changed:
- The tested H100 `mig-3g.40gb` partition delivered 53% to 89% more output tokens per second at each traffic scenario’s greatest cost benefit.
- Each slice gets dedicated compute, memory, cache, and memory bandwidth.
- Partitioning is available through SageMaker inference endpoints in US East (N. Virginia and Ohio), US West (Oregon), Asia Pacific (Singapore and Tokyo), and Europe (Frankfurt and Ireland).

One catch: You still pay for the whole instance, and the A100 was cheaper below the crossover. Support is limited to real-time inference component endpoints. Changing the partition layout requires a new endpoint.

Try it: Set `AcceleratorPartitionLayouts` on the endpoint production variant and `AcceleratorPartitionType` on each inference component.

Sources: [aws.amazon.com](https://aws.amazon.com/blogs/machine-learning/sagemaker-inference-now-supports-multi-instance-gpu-to-maximize-gpu-utilization-for-model-endpoints/)