IBM and Together AI Commit $240 Million to Open-Model Inference

Key Takeaways

IBM and Together AI will build a dedicated AI inference cluster on IBM Cloud under a $240 million, multi-year agreement.
The deployment will use Nvidia HGX B300 systems and Nvidia Spectrum-X Ethernet networking.
The project reflects rising enterprise demand for lower-cost inference, model choice, security controls, and scalable GPU capacity.

Reuters reported that IBM and Together AI have signed a $240 million, multi-year agreement to build a large-scale artificial intelligence inference cluster on IBM Cloud. The infrastructure will support open-source AI models and use Nvidia systems based on its newer Blackwell processors.

According to the announcement distributed through PR Newswire, the deployment combines Nvidia HGX B300 systems with Nvidia Spectrum-X Ethernet networking. Nvidia optimized the Blackwell-based HGX B300 platform specifically for AI inference workloads to increase output capacity.

Inference, the process of running trained AI models to generate responses, has become one of the largest drivers of demand for computing capacity. This demand is prompting cloud providers and chipmakers to invest heavily in expanding AI infrastructure to support recurring enterprise workloads.

Together AI gives IBM a route into this expanding workload category through a platform centered on open models. The San Francisco-based startup lets enterprise customers train and run models including DeepSeek, MiniMax, and Kimi. Together AI states that this approach can lower costs compared with closed systems while giving customers more control over deployment decisions. The startup was valued at $8.3 billion in July 2026 after raising $800 million.

Open models still require optimized serving software, high-bandwidth networking, available accelerators, and strict security controls. IBM Cloud, Together AI, and Nvidia are packaging these infrastructure layers into a dedicated environment to support the high computing requirements of large production volumes.

For IBM, the agreement expands an AI infrastructure strategy that reaches beyond a single cluster. IBM Newsroom has detailed broader work with Nvidia, including planned support for Blackwell Ultra GPUs and AI tooling across IBM Cloud and Red Hat AI Factory initiatives. That combination targets enterprises running mixed environments, particularly those seeking to connect hosted inference with Red Hat-based applications and existing IBM systems.

Security and control are also central to the commercial pitch. Businesses are actively weighing cybersecurity incidents involving models and services from Anthropic, OpenAI, and Meta. Open-model deployment can give customers more influence over where workloads run and how data is handled, though configuration, access management, model behavior, software dependencies, and monitoring remain ongoing operational priorities.

Moving forward, IBM and Together AI must attract sustained enterprise workloads and demonstrate that open-model inference delivers tangible cost and control advantages. If successful, the cluster will strengthen IBM Cloud’s position in AI computing while providing Together AI with a larger enterprise distribution channel. Nvidia, meanwhile, secures another major deployment for its Blackwell and Spectrum-X infrastructure.

Editorial oversight by Rich Tehrani — CEO of TMC and creator of ITEXPO.

Disclosure: Rich Tehrani is also CEO of RT Advisors and a Registered Representative offering securities through Four Points Capital Partners LLC (Member FINRA/SIPC). Nothing in this article is an endorsement or recommendation to buy or sell any security or sector mentioned. RT Advisors may have or have had business relationships with companies mentioned in TMC Insight articles. Such relationships do not affect editorial judgment or coverage.