CUDA

Lore & Background

CUDA traces to the early 2000s, when Ian Buck, a computer science PhD student at Stanford University, began experimenting with using GPUs for purposes beyond rendering graphics. Buck had become interested in GPUs during his undergraduate studies at Princeton University, initially through video gaming. After graduation, he interned at Nvidia, gaining deeper exposure to GPU architecture. At Stanford, he built an 8K gaming rig using 32 GeForce graphics cards, originally to push the limits of graphics performance in games like Quake and Doom. However, his interests shifted toward exploring the potential of GPUs for general-purpose parallel computing. To that end, Buck developed Brook, a programming language designed to enable general-purpose computing on GPUs. His work attracted support from Nvidia and the Defense Advanced Research Projects Agency (DARPA). In 2004, Nvidia hired Buck and paired him with John Nickolls, then director of architecture for GPU computing. Together, they began transforming Brook into CUDA. CUDA was officially released in 2007. CUDA became central to the company's strategy of positioning GPUs as versatile hardware for scientific applications. By 2015, CUDA's development increasingly focused on accelerating machine learning and artificial neural network workloads.

Reader's Guide

CUDA is notable for making GPU resources accessible to specialists in parallel programming without requiring advanced graphics programming skills, in contrast to prior APIs like Direct3D and OpenGL. It is written in C but designed to work with C++, Fortran, Python, and Julia. CUDA-powered GPUs support programming frameworks such as OpenMP, OpenACC, and OpenCL. The platform includes compilers, libraries, and developer tools. CUDA provides both a low-level API (CUDA Driver API) and a higher-level API (CUDA Runtime API). It works with all Nvidia GPUs from the G8x series onwards, including GeForce, Quadro, and Tesla lines, and is compatible with most standard operating systems. CUDA has been used to accelerate non-graphical applications in computational biology, cryptography, and other fields by an order of magnitude or more. Advantages include scattered reads, unified virtual memory (CUDA 4.0 and above), unified memory (CUDA 6.0 and above), shared memory, faster downloads and readbacks, and full support for integer and bitwise operations. Limitations include one-way interoperability with OpenGL, potential performance hits from host-device memory copying, and the requirement that threads run in groups of at least 32 for best performance.

Did You Know?

• CUDA was created in 2004 and officially released in 2007.
• The initial CUDA SDK was made public on 15 February 2007 for Microsoft Windows and Linux.

From Brook to CUDA: The Birth of a Platform

Ian Buck first became fascinated with GPUs during his undergraduate years at Princeton, where video games sparked his curiosity. After an internship at Nvidia gave him deeper insight into GPU architecture, he enrolled at Stanford, where he assembled an ambitious 8K gaming rig equipped with 32 GeForce cards. What began as an effort to push graphics performance in titles like Quake and Doom gradually evolved into something far more ambitious: exploring whether these massively parallel processors could handle general-purpose computation. Buck developed a programming language called Brook to unlock that potential, attracting backing from both Nvidia and DARPA. In 2004, Nvidia brought Buck on board and paired him with John Nickolls, the company's director of architecture for GPU computing. Together, they set about transforming Brook into a full production platform. The result was officially released in 2007, and while the name originally stood for Compute Unified Device Architecture, Nvidia has since largely dropped the acronym expansion, letting CUDA simply be CUDA.

A Two-Layer Architecture for Parallel Programming

CUDA is not a single tool but a layered ecosystem. At its core sits a software layer that orchestrates data movement and grants developers direct access to both GPU and CPU resources as the workload demands. Wrapped around that foundation is a comprehensive library of APIs designed to enable parallel computation at scale. The platform ships with compilers, runtime kernels, and developer tooling that collectively lower the barrier to entry. Unlike earlier graphics APIs such as Direct3D or OpenGL, which demanded deep expertise in rendering pipelines, CUDA is written in C and structured so that scientists and engineers in fields like physics or bioinformatics can leverage GPU parallelism without becoming graphics specialists. The platform exposes two distinct API tiers: a lower-level CUDA Driver API for non-single-source workflows, and a higher-level CUDA Runtime API that supports single-source programming. C and C++ code is compiled to PTX through nvcc, Nvidia's LLVM-based compiler, while Fortran users have access to a dedicated CUDA Fortran compiler from The Portland Group.

A Growing Ecosystem of Libraries and Language Bridges

One of CUDA's most significant strengths is the breadth of its software ecosystem. The platform ships with a rich set of optimized libraries covering linear algebra (cuBLAS), fast Fourier transforms (cuFFT), random number generation (cuRAND), dense and sparse solvers (cuSOLVER), sparse matrix operations (cuSPARSE), graph analytics (nvGRAPH), and performance primitives (NPP). Beyond Nvidia's own stack, CUDA integrates with industry-standard parallel frameworks including OpenMP, OpenACC, and the Khronos Group's OpenCL, as well as Microsoft's DirectCompute and OpenGL Compute Shaders. Third-party wrappers extend CUDA's reach into Python, Perl, Java, Ruby, Lua, Common Lisp, Haskell, R, MATLAB, IDL, Julia, and even native Mathematica support. The ecosystem has continued to evolve: CUDA 10 introduced nvJPEG for hybrid CPU-GPU image processing, while the 11.x series added CUB for C++ parallel primitives, multi-instance GPU support, and JPEG 2000 encoding. This layered, multi-language approach means that virtually every major scientific and engineering discipline can tap into GPU acceleration through familiar tooling.

From Game Rendering to the AI Revolution

When GPUs were first designed, their purpose was narrow: real-time rendering of high-resolution 3D graphics. By 2012, however, they had evolved into highly parallel multi-core systems capable of efficiently manipulating enormous blocks of data simultaneously, a design philosophy that outperforms traditional CPUs for workloads where parallelism is key. CUDA capitalized on this shift, enabling GPUs to accelerate cryptographic hash functions, molecular dynamics simulations, physics engines, and machine learning pipelines. In the gaming industry, CUDA-powered GPUs handle not just visual rendering but also physical effects like debris, smoke, fire, and fluid dynamics through engines such as PhysX and Bullet. Beyond entertainment, CUDA has delivered order-of-magnitude speedups in computational biology, cryptography, and high-performance scientific computing. A pivotal strategic shift came around 2015, when Nvidia's CUDA development efforts increasingly centered on accelerating machine learning and artificial neural network workloads, cementing the platform's role as the computational backbone of the modern AI era.