Chip startup Graphcore Ltd. today introduced a new artificial intelligence processor, the Bow IPU, that uses an innovation dubbed wafer-on-wafer technology to speed up calculations.

U.K.-based Graphcore is backed by more than $680 million in venture funding. The startup has shipped tens of thousands of AI processors in recent years to customers such as cloud providers and research institutions.

Graphcore’s new Bow IPU processor can manage up to 350 trillion processing operations per second, the startup says. It’s an improved version of an existing Graphcore chip dubbed the Colossus Mk2. The Bow IPU features the same number of processing cores as the Colossus Mk2 and an identical amount of memory, but provides up to 40% more performance thanks to the fact that it’s based on a new wafer-on-wafer design.

The wafer-on-wafer design is the fruit of a collaboration between Graphcore and chipmaker TSMC Ltd., which manufactures the startup’s processors. The technology makes it possible to integrate two separate silicon wafers in a vertical, or three-dimensional, configuration to form a single chip.

In the case of the newly announced Bow IPU processor, Graphcore has combined the wafer that holds its chip’s AI circuits with a wafer containing power management components. The power management components enable electricity to reach the AI circuits more efficiently than before. This increased efficiency is what allows the Bow IPU to run faster than Graphcore’s earlier Colossus Mk2 chip, even though they feature the same number of processing cores.

The power management wafer includes a set of devices dubbed deep trench capacitors. They’re responsible for storing electricity, which the Bow IPU uses to increase the frequency of its AI circuits. The Bow IPU operates at a frequency of 1.8 gigahertz, which is significantly higher than the 1.325GHz clock rate of the startup’s previous-generation Colossus Mk2 processor. The increased clock rate is a key contributor to its increased performance.

“With Wafer-on-Wafer in the BOW IPU, two wafers are bonded together to generate a new 3D die,” Graphcore co-founders Nigel Toon and Simon Knowles detailed in a blog post. “One wafer for AI processing, which is architecturally compatible with the GC200 IPU processor with 1,472 independent IPU-Core tiles, capable of running more than 8,800 threads, with 900MB of In-Processor Memory, and a second wafer with power delivery die.”

Graphcore is selling the processors as part of a computing system dubbed the Bow-2000 IPU Machine. Each system includes four Bow IPU processors that together provide 1.4 petaflops of processing capacity. One petaflop equals a quadrillion computing operations per second.

Companies that require even more processing capacity can combine multiple Bow-2000 IPU Machines into a cluster. Graphcore offers five different cluster configurations, or Bow Pods, that feature between 16 and 1,025 processors.

According to the startup, the 16-processor Bow Pod provides more than five times the performance of a comparable DGX A100 appliance from Nvidia Corp. at half the cost. Meanwhile, the top-end 1,025-processor configuration can manage 350 petaflops of performance.

“In addition to up to 40% performance gains, Bow Pod systems are also significantly more power-efficient than their predecessors,” Toon and Knowles stated. “Tested across a range of real world applications, Bow Pods show an improved performance-per-watt of up to 16%.

Graphcore plans to introduce an even more powerful system dubbed the Good Computer in 2024. It will be capable of running AI models with 500 trillion parameters, according to the startup.

Parameters are the settings that define how an AI model goes about processing data. The more such settings there are in an AI, the more processing power is required to run it. OpenAI’s GPT-3 neural network, one of the most sophisticated AI models developed to date, features 175 billion parameters or less than 500 times the number that Graphcore’s upcoming Good Computer will support.

Several organizations have already signed up to use the startup’s latest Bow IPU processor. One early customer is the Pacific Northwest National Laboratory, which plans to use the processors to carry out research in areas such as cybersecurity and computational chemistry.

Image: Graphcore