- Nvidia on Monday unveiled the H200, a graphics processor designed for training and deploying the types of artificial intelligence models that are driving the generative AI boom.
- The H200 has 141GB of next-generation “HBM3” memory, which helps it generate text, images or predictions using AI models.
- Interest in Nvidia’s AI GPUs has buoyed the company, with sales expected to rise 170% this quarter.
Jensen Huang, president of Nvidia, holds the Grace Hopper superchip CPU for generative AI at Supermicro’s keynote presentation during Computex 2023.
Walid Berrazeg | Light rocket | Getty Images
Nvidia on Monday unveiled the H200, a graphics processing unit designed to train and deploy the types of artificial intelligence models that are powering the generative AI boom.
The new GPU is an upgrade from the H100, the chip OpenAI uses to train its most advanced large language model, GPT-4. Large companies, startups and government agencies are vying for a limited supply of chips.
According to an estimate by Raymond James, H100 chips cost between $25,000 and $40,000, and thousands of them must work together to create the largest models in a process called “training.”
Excitement over Nvidia’s AI GPUs has boosted the company’s stock, which is up more than 230% so far in 2023. Nvidia expects revenue of around $16 billion for the third fiscal quarter, up 170% from a year ago.
The H200’s key improvement is that it has 141GB of next-generation “HBM3” memory, which helps the chip perform “inference,” or use a large model after it has been used to generate text, images, or predictions was trained.
Nvidia said the H200 will produce output almost twice as fast as the H100. This is based on a test with Metas Llama 2 LLM.
The H200, expected to ship in the second quarter of 2024, will compete with AMD’s MI300X GPU. AMD’s chip, similar to the H200, has additional memory compared to its predecessors, helping to accommodate large models on the hardware to run inference.
Nvidia H200 chips in an Nvidia HGX system with eight GPUs.
Nvidia
Nvidia said the H200 will be compatible with the H100, meaning AI companies that already train with the previous model won’t need to change their server systems or software to use the new version.
Nvidia says it will be available in server configurations with four or eight GPUs on the company’s HGX complete systems, as well as in a chip called GH200, which combines the H200 GPU with an Arm-based processor.
However, the H200 may not hold the crown as the fastest Nvidia AI chip for long.
While companies like Nvidia offer many different configurations of their chips, new semiconductors often take a big step forward every two years or so as manufacturers switch to a different architecture that allows for larger performance gains than adding memory or other minor tweaks. Both the H100 and H200 are based on Nvidia’s Hopper architecture.
In October, Nvidia told investors that it would move from a two-year architecture cadence to a one-year release cadence due to high demand for its GPUs. The company showed a slide indicating that it will announce and release its B100 chip, based on the upcoming Blackwell architecture, in 2024.
REGARD: We firmly believe in the AI trend next year
Don’t miss these stories from CNBC PRO: