Hardware for artificial intelligence
Specialized hardware accelerates AI training and inference with greater speed and efficiency.
Hardware for artificial intelligence refers to computing components and systems specifically designed or optimized to accelerate AI workloads such as machine-learning training or inference. This includes general-purpose accelerators like GPUs and domain-specific accelerators like TPUs, NPUs, and other AI ASICs. Since 2017, several consumer-grade CPUs and SoCs have included on-die NPUs, and by 2023 the market is dominated by GPUs from Nvidia and AMD, Google's TPUs, and various NPUs in consumer hardware.
- Compute increase 2012 to 2017
- 300,000-fold
- Doubling time trend
- 3.4 months
- Tokens per second etched model
- 17,000
- Hbm3 standardized year
- 2022
- Tpu introduction year
- 2016
- H100 introduction year
- 2022
Lore & Background
Lisp machines were developed in the late 1970s and early 1980s to accelerate AI programs written in Lisp. Dataflow architecture processors for AI include the polymorphic dataflow Convolution Engine by Kinara, structure-driven dataflow by Hailo, and dataflow scheduling by Cerebras. Since the 2010s, GPUs with AI-specific enhancements displaced CPUs as the dominant means to train large-scale commercial cloud AI by 2019. OpenAI estimated a 300,000-fold increase in compute used from AlexNet (2012) to AlphaZero (2017), with a doubling-time trend of 3.4 months. Modern data-center GPUs include dedicated tensor or matrix-math units. In 2022, NVIDIA introduced the Hopper-generation H100 GPU with FP8 precision support and faster interconnects. Google introduced the TPU in 2016 for inference, with later generations supporting training via dense systolic arrays and optical interconnects. Apple's Neural Engine and various on-device NPUs emphasize energy-efficient inference in mobile or edge environments. Cerebras launched a wafer-scale processor with large amounts of SRAM, and Groq designed an LPU with large SRAM for low time to first token. Taalas developed ASICs that implement a specific AI model as hardware logic gates, achieving up to 17,000 tokens per second, but the model cannot be changed.
Reader's Guide
The article describes AI hardware as a critical enabler of modern deep learning, with GPUs becoming dominant for training by 2019 due to their parallel architecture and high memory bandwidth. Domain-specific accelerators like TPUs and NPUs offer tailored efficiency for inference and training. The rapid growth in compute demand—a 300,000-fold increase from 2012 to 2017—highlights the importance of specialized hardware. Innovations such as wafer-scale processors, LPUs for low latency, and etched ASICs for extreme throughput demonstrate ongoing diversification. Memory and interconnects like HBM3 and NVLink are essential for scaling across thousands of chips. The article notes that neuromorphic and analog accelerators promise reduced power consumption, while photonic processors have been explored. Overall, the field is characterized by a shift from general-purpose CPUs to a variety of accelerators optimized for specific AI workloads, with consumer hardware increasingly incorporating NPUs.
Frequently Asked Questions
What exactly is AI hardware in the PC context?
It refers to any computing component purpose-built or heavily optimized to speed up machine-learning training and inference tasks. That covers everything from general-purpose GPUs to domain-specific chips like TPUs, NPUs, and other AI ASICs.
What are the main families of AI accelerators fans should know about?
The landscape splits into general-purpose accelerators (Nvidia and AMD GPUs), Google's TPU line (introduced in 2016), and on-die NPUs that have shipped inside consumer CPUs and SoCs since 2017. By 2023 these three categories dominate both data-center and consumer markets.
How rapidly has AI compute demand grown?
Between 2012 and 2017 the required compute for AI workloads ballooned roughly 300,000-fold, and the trend has continued with compute needs doubling roughly every 3.4 months. That pace is why dedicated silicon keeps outpacing what a generic CPU can deliver.
When did Nvidia's H100 and the HBM3 memory standard arrive?
Both landed in 2022: the H100 data-center GPU and the HBM3 high-bandwidth memory standard were introduced that same year. Together they set the performance baseline for large-scale model training and high-throughput inference.
Why does a fan need to care about AI hardware at all?
Because the speed and efficiency of your AI workloads—whether you're fine-tuning a local model or running inference—depend almost entirely on the accelerator you pair with your build. Choosing the right GPU, TPU, or NPU can be the difference between a usable system and one that crawls through a single forward pass.
More in PC Hardware 1-24
Spotted an error? Know more?
Reader corrections go straight into our review queue. Suggest an edit · How this site is sourced