TorchNeuron brings PyTorch to Trainium, Deepgram reports 60% more audio per dollar

Top News · Reported · pytorch.org

If you want to move an existing PyTorch model onto Trainium, TorchNeuron gives you a native backend. Deepgram reports about 60% more input audio seconds per dollar on Trainium than on its existing GPU baselines.

Here's what it supports:

• Public beta is scheduled for October 20, 2026, for researchers and developers, with packages from the AWS Neuron pip repository.
• Trainium2 and Trainium3 support eager execution, torch.compile, and custom kernels through NKI.
• Distributed workloads can use DDP, FSDP2, tensor parallelism, and pipeline parallelism.

One catch: The first release excludes dynamic and symbolic shapes and sparse tensors, and some operators lack native lowerings.

Try it: Install pip install torch-neuronx from the AWS Neuron pip repository. Move your model with model = MyModel().to('neuron'), then compile with torch.compile(model, backend="neuron").

Sources: pytorch.org