When it comes to computer chips, size matters. Smaller has always meant better. According to Moore’s law, the number of transistors that can squeeze onto a silicon chip doubles every two years, enabling sleeker and more powerful devices.

Now, a California-based company is upending that notion. In August 2019, the Silicon Valley, Los Altos-headquartered Cerebras introduced the largest chip ever produced. The chip, named the Wafer Scale Engine (WSE), is made out of an entire silicon wafer. Roughly the size of an Apple iPad, it measures 46 225 mm² , an area about 56 times bigger than the next largest chip, NVIDIA’S GV100 Volta graphics processing unit (GPU), which spans 815 mm² (Fig. 1). While the NVIDIA GPU contains 21.1 billion transistors, the WSE holds 1.2 trillion[1–2].

“The chip is clearly an integration marvel,” said Rakesh Kumar, associate professor of electrical and computer engineering at the University of Illinois at Urbana Champaign. ‘‘It is a big deal to have shown it possible to put together a working chip this large.”

Such a mammoth chip, according to Cerebras, is needed to meet the growing demands of artificial intelligence (AI). AI algorithms learn to do a task by first training on a huge amount of data. In particular, deep learning algorithms, which use neural networks that roughly mimic how the brain works, require enormous computing power, with training runs that can take hours or even days. According to a recent analysis from OpenAI, a San Francisco-based, AIfocused company backed by Microsoft, the computing power demanded by AI training has, from 2012 to 2018, increased by a factor of 300 000, with a doubling time of 3.5 months. That is 25 000 times faster than Moore’s law at its peak [3].

Handling all that computation requires more cores than are on a single standard chip. So multiple chips must work together in concert. But that also means data must be shuttled between chips—a process tens of thousands of times slower than possible within a single chip [1].

Dozens or even hundreds of small chips can be fabricated out of a silicon wafer. But by producing a single integrated chip from one entire wafer, Cerebras has built a processor packed with cores that does not rely on the off-chip communication that can bog down conventional systems. The WSE also holds memory cores close to the compute cores, so the former can constantly feed data to the latter, reducing the idling time of the compute cores. Boasting an architecture designed for machine learning, the chip is optimized for AI training, according to Cerebras. It has 400 000 programmable cores, 18 GB of static random access memory (SRAM), and a memory bandwidth of 9 PB·s^-1 . This is 78 times more cores, 3000 times more on-chip memory, and 10 000 times more memory bandwidth than the next best chip[1–4].

The WSE is a notable achievement, said Mike Demler, a senior analyst with the Linley Group, an analyst firm in Mountain View, California, that focuses on the microprocessor industry. ‘‘This has been tried in the past and it never succeeded.” In 1980, for example, chip engineer Gene Amdahl founded Trilogy with $230 million USD in funding—the highest at the time—to build a wafer-scale chip [5]. The company, however, did not succeed and folded after only five years [6].

One of the reasons these early attempts failed, Demler said, is that the chip fabrication process produced too many defects. When you fabricate many chips from a wafer, you simply discard the defective chips. That does not work with a chip made from the entire wafer.

While fabrication today is much improved, defects are still inevitable. Cerebras gets around the problem with spare cores and an architecture that bypasses any defects. During manufacturing, any bad cores are identified, and the interconnects are rerouted around the bad cores to the spare ones [7].

In addition to dealing with the defects, building such a large chip poses other challenges, including with cooling and power delivery. Heat causes silicon to expand differently than the