WEB DESK: OpenAI has reportedly been dissatisfied with certain Nvidia chips and has been exploring other options since last year, according to eight sources familiar with the matter. This development could pose a challenge to the longstanding dominance of Nvidia in the AI hardware market, especially as the two giants are currently engaged in investment negotiations.
The shift in OpenAI’s hardware strategy, which is detailed here for the first time, revolves around a growing focus on specialized chips designed for AI inference—the stage where AI models like ChatGPT generate responses to user prompts. While Nvidia continues to lead in supplying chips for training large AI models, inference hardware has become a new battleground for competition.
This move by OpenAI and other firms to find alternatives for inference chips signals a crucial test of Nvidia’s supremacy in AI hardware. The situation is further complicated by ongoing talks for a major investment deal. Nvidia had announced intentions in September to invest up to $100 billion in OpenAI, securing a stake in the startup and providing funds for the purchase of advanced chips. The deal was expected to close shortly, but negotiations have stalled for several months.
During this period, OpenAI has entered into agreements with AMD and other chipmakers to acquire GPUs that compete with Nvidia’s offerings. Additionally, OpenAI’s evolving product roadmap has shifted its hardware needs, contributing to delays in negotiations with Nvidia, according to sources.
Nvidia’s CEO Jensen Huang dismissed reports of any tension with OpenAI, describing such claims as “nonsense,” and reaffirmed Nvidia’s commitment to investing heavily in OpenAI. “Customers continue to choose Nvidia for inference because we deliver the best performance and overall cost efficiency at scale,” Nvidia stated.
OpenAI’s spokesperson echoed this sentiment, emphasizing that Nvidia currently powers the majority of their inference workloads and offers the best value for performance. Following a Reuters report, OpenAI CEO Sam Altman publicly acknowledged Nvidia’s leadership in AI chips, stating, “We hope to remain a major customer of Nvidia for a long time.”
However, sources reveal that OpenAI is dissatisfied with the speed of Nvidia’s hardware, particularly for specific applications such as software development and AI-to-software communication. The company is seeking new hardware that could meet about 10% of its future inference processing requirements, especially for tasks demanding rapid responses.
In line with this, OpenAI has discussed partnering with startups like Cerebras and Groq to develop faster inference chips. Nevertheless, Nvidia’s recent $20 billion licensing agreement with Groq has limited OpenAI’s options, effectively halting talks with the startup. Nvidia’s acquisition of Groq’s intellectual property appears aimed at strengthening its position in a rapidly evolving AI landscape, according to industry insiders.
Alternatives to Nvidia
Nvidia’s GPUs are highly effective for training large AI models like ChatGPT, which has driven the global expansion of AI technology. However, the focus is shifting toward inference and reasoning, which may represent a larger frontier for AI development—prompting companies like OpenAI to explore alternative hardware solutions.
Since last year, OpenAI’s search for GPU alternatives has centered on chips with large on-chip memory, known as SRAM. Embedding more SRAM on each chip accelerates processing speeds critical for chatbot responsiveness and other AI services that handle millions of requests. Inference tasks require more memory than training because the chips spend more time retrieving data from memory than performing calculations.
Nvidia and AMD GPUs rely on external memory modules, which can slow response times. This issue has become particularly evident in OpenAI’s Codex product, used for generating computer code, which has faced performance bottlenecks attributed partly to Nvidia hardware.OpenAI CEO Sam Altman highlighted the importance of speed, especially for coding applications, during a January 30 press call. To address this, OpenAI has partnered with Cerebras, which specializes in SRAM-rich chips, to improve inference speed—though for less time-sensitive applications, like casual ChatGPT interactions.
Meanwhile, competitors such as Google’s Gemini and Anthropic’s Claude leverage in-house-designed tensor processing units (TPUs), which are optimized for inference tasks and provide performance advantages over general-purpose GPUs.
Nvidia’s Strategic Moves
Despite OpenAI’s reservations about Nvidia’s hardware, Nvidia has approached companies working on SRAM-intensive chips, including Cerebras and Groq, about potential acquisitions. While Cerebras declined and instead signed a commercial deal with OpenAI last month, Groq has been in talks with OpenAI for computing services and has attracted investor interest valued at approximately $14 billion.
By December, Nvidia acquired a license for Groq’s technology through a non-exclusive, cash-based deal. Although this allows other companies to license Groq’s IP, Nvidia has shifted its focus toward selling cloud-based software, having hired away Groq’s chip designers in the process.
