Nvidia unveils Dynamo open-source inference software

At GTC, Nvidia (NVDA) unveiled NVIDIA Dynamo, an open-source inference software for “accelerating and scaling AI reasoning models in AI factories at the lowest cost and with the highest efficiency.” The company stated: “NVIDIA Dynamo, the successor to NVIDIA Triton Inference Server, is new AI inference-serving software designed to maximize token revenue generation for AI factories deploying reasoning AI models. It orchestrates and accelerates inference communication across thousands of GPUs, and uses disaggregated serving to separate the processing and generation phases of large language models on different GPUs. This allows each phase to be optimized independently for its specific needs and ensures maximum GPU resource utilization.” Jensen Huang, founder and CEO, added: “Industries around the world are training AI models to think and learn in different ways, making them more sophisticated over time. To enable a future of custom reasoning AI, NVIDIA Dynamo helps serve these models at scale, driving cost savings and efficiencies across AI factories.”