Multithreading (computer architecture)
Hardware technique enabling multiple threads on a single CPU core.
Multithreading, in computer architecture, is a CPU's (or a single core's) ability to handle multiple threads of execution. This approach gained traction after the late 1990s, when further improvements in instruction-level parallelism became difficult to achieve. It revived the idea of throughput computing, which had previously been limited to specialized transaction processing. While speeding up a single thread or program is very hard, most computer systems already multitask among multiple threads or programs, so techniques that improve overall task throughput deliver real performance gains. Two major methods for throughput computing are multithreading and multiprocessing.
A key advantage of multithreading is that when one thread suffers many cache misses, other threads can use the otherwise idle computing resources, leading to faster overall execution. Similarly, if a thread cannot fully utilize the CPU because its instructions depend on each other's results, running another thread prevents those resources from going to waste.
However, multiple threads can interfere with each other when sharing hardware like caches or translation lookaside buffers (TLBs). This means a single thread's execution time is not improved and can actually get worse, even when it runs alone, due to lower frequencies or extra pipeline stages needed for thread-switching hardware. Efficiency varies widely: Intel reports up to a 30% improvement with its Hyper-Threading Technology, while a synthetic program running a loop of non-optimized dependent floating-point operations can see a 100% speed gain when parallelized. On the other hand, hand-tuned assembly programs using MMX or AltiVec extensions with data prefetches (like a good video encoder) do not suffer from cache misses or idle resources, so they gain nothing from hardware multithreading and may actually perform worse due to resource contention. From a software perspective, hardware multithreading is more visible and requires more changes to both applications and operating systems than multiprocessing. The hardware techniques often mirror software multitasking methods, and thread scheduling itself is a major challenge.
- Intel improvement claim
- up to 30% with Hyper-Threading Technology
- Synthetic program speed gain
- 100% when run in parallel
Lore & Background
Multithreading techniques improve overall throughput by allowing other threads to use computing resources when one thread experiences cache misses or cannot use all resources due to instruction dependencies. Multiple threads can interfere with each other when sharing hardware resources such as caches or translation lookaside buffers (TLBs), potentially degrading single-thread execution times due to lower frequencies or additional pipeline stages. Overall efficiency varies; Intel claims up to 30% improvement with its Hyper-Threading Technology, while a synthetic program performing a loop of non-optimized dependent floating-point operations gains a 100% speed improvement when run in parallel. Hand-tuned assembly programs using MMX or AltiVec extensions and performing data prefetches do not suffer from cache misses or idle computing resources and may see degraded performance due to contention for shared resources.
The simplest type is block (cooperative or coarse-grained) multithreading, where one thread runs until blocked by a long-latency event such as a cache miss, then switches to another thread. Fine-grained multithreading removes all data dependency stalls by issuing instructions from different threads each cycle, first called barrel processing. Simultaneous multithreading (SMT) applies to superscalar processors, issuing instructions from multiple threads every cycle to exploit parallelism across threads. Implementations include Intel Hyper-Threading Technology, IBM POWER5/POWER6/POWER7/POWER8/POWER9, IBM z13/z14/z15, Sun Microsystems UltraSPARC T2, Cray XMT, and AMD Bulldozer and Zen microarchitectures.
Reader's Guide
Multithreading has become a key technique for throughput computing, alongside multiprocessing, as it allows CPUs to better utilize resources that would otherwise remain idle. By enabling quick switching between threads—often in a single CPU cycle with replicated register sets—hardware multithreading reduces the impact of long-latency stalls and data dependency stalls. However, it introduces challenges: thread scheduling is a major problem, and shared resource contention can degrade performance for certain workloads. The technique requires more changes to both application programs and operating systems than multiprocessing, and merging data from two processes can incur significantly higher costs—potentially two or more orders of magnitude—due to inter-process communication and synchronization overhead. Despite these drawbacks, multithreading has been widely adopted in modern processors, with implementations ranging from embedded microcontrollers with multiple register banks for interrupt handling to advanced superscalar designs like SMT. The thread scheduler, whether implemented in software, hardware, or a combination, remains a major area of research, as does the choice of events that trigger thread switches.
Did You Know?
- Block multithreading switches threads only when a long-latency event like a cache miss occurs.
- Fine-grained multithreading was first called barrel processing, where pipeline stages resemble barrel staves.
- Simultaneous multithreading can issue instructions from multiple threads in the same CPU cycle.
- Hand-tuned assembly programs using MMX or AltiVec extensions may see degraded performance with multithreading.
Frequently Asked Questions
Who is Multithreading (computer architecture)?
Multithreading is a hardware capability built into a CPU core that lets it juggle more than one thread of execution at the same time. It essentially turns a single processing unit into a multitasker without requiring a whole second core.
What are Multithreading's powers?
Its core trick is partitioning a core's internal resources so two or more threads can make progress simultaneously, lifting overall task throughput. Intel's Hyper-Threading implementation claims up to a 30% performance boost, and a synthetic benchmark can show a full 100% speed gain when a program is executed in parallel.
When did Multithreading enter the scene?
The approach really took hold in the late 1990s, after engineers hit a wall trying to extract more instruction-level parallelism from single-threaded designs. It resurrected the older idea of throughput computing, which before that had been confined to niche transaction-processing machines.
Why is Multithreading important?
Because accelerating a single thread has become extremely difficult, yet virtually every modern system already juggles many threads at once. Techniques that raise overall throughput therefore deliver tangible, real-world speedups that single-thread optimization simply cannot match.
More in Microprocessors 1-24
Spotted an error? Know more?
Reader corrections go straight into our review queue. Suggest an edit · How this site is sourced