Spatial architecture

Lore & Background

Spatial architectures are classifiable as a SPMD array processor, where each processing element runs the same operations on a different subset of data, programmed through a single mapping. The architecture of an individual processing element can belong to any Flynn class. They are well suited for applications whose dataflow exhibits producer-consumer relationships or can leverage efficient data sharing among a region of PEs. Spatial architectures can typically be found as hardware accelerators in heterogeneous systems, under the broader category of manycore processor.

The core element is a multidimensional array of processing elements, each simple—a multiply-and-accumulate functional unit, a stripped-down core, or application-specific logic. Processing elements are connected with each other and the memory hierarchy through busses or a network on chip, or even asynchronous logic. The memory hierarchy is explicitly managed and may consist of multiple on-chip buffers, like register files, scratchpads, and FIFOs, backed by large off-chip DRAM and non-volatile memories.

Due to technology and bandwidth limitations, the energy and latency required to access larger memories dominate those of computation, being hundreds of times more than what's needed for storage near processing elements. The memory hierarchy is intended to localize most repeated value accesses on faster and more efficient on-chip memories, exploiting data reuse to minimize costly accesses. Data reuse mechanisms include multicast and reduction, classified as spatial and temporal.

Reader's Guide

Spatial architectures are significant for their ability to reduce latency and power consumption when running very large kernels through scalable parallelism and data reuse. Their effectiveness depends on the kernel's data reuse opportunities, which can be inferred by analyzing data dependencies. Mapping a workload involves binding computations to processing elements and scheduling data movements, using loop transformations such as tiling, parallelization, and computation ordering. Tiling enables temporal reuse, parallelization enables spatial reuse, and computation order determines which values undergo reuse. The architecture's legacy lies in its application to AI accelerators and neural network support, where matrix multiplications and convolutions are common. The amount of data reuse exploitable is a property of the kernel, and limited hardware resources require proper planning to leverage the most effective opportunities. Spatial architectures are typically found as hardware accelerators in heterogeneous systems, under the broader category of manycore processor.

Did You Know?

• The key performance metrics are consumed energy and latency when running a given workload.
• Mapping a workload involves loop transformations: tiling, parallelization, and computation ordering.