Probability density function

Lore & Background

In probability theory, a probability density function (PDF) is a function whose value at any given point in the sample space can be interpreted as providing a 'relative probability' that the value of the random variable would be equal to that point. The probability density is the probability per unit length, and the absolute probability for a continuous random variable to take on any particular value is zero. The PDF is used to specify the probability of the random variable falling within a particular range of values, given by the integral of the PDF over that range.

The function is nonnegative everywhere, and the total area under the entire curve equals one, meaning the probability of the variable falling within its set of possible values is 100%. The terms "probability distribution function" and "probability function" can also denote the PDF, though this usage is not standard among probabilists and statisticians, as these terms may instead refer to the cumulative distribution function or probability mass function (PMF). The PMF applies to discrete random variables (taking values on a countable set), while the PDF applies to continuous random variables, and both are fundamental to statistical inference.

A PDF can take on values greater than one; for example, the continuous uniform distribution can have a density exceeding one. In formal measure-theoretic terms, a random variable's density is the Radon–Nikodym derivative of its probability distribution with respect to a reference measure, such as the Lebesgue measure for continuous cases. When a density exists, it is almost unique, meaning any two such densities coincide almost everywhere.

Defining the Discrete Landscape

A probability mass function serves as the foundational tool for characterizing how discrete random variables behave. Rather than spreading probability across a continuum, it assigns a specific probability to each individual value that a discrete random variable might take. This makes it the primary mechanism through which a discrete probability distribution is defined, whether the variable in question is a single scalar quantity or a multivariate entity whose domain consists of discrete points. The term "mass" is not merely decorative; it reflects the idea that probability behaves like a conserved quantity, much as physical mass is preserved in a closed system.

The point at which this function reaches its highest value is known as the mode, identifying the single most likely outcome. Because the function maps real numbers into the interval from zero to one, every assigned probability is inherently bounded and meaningful. The PMF is also referred to in the literature as a probability function, a frequency function, or even a discrete probability density function, though the last name can be misleading given its distinction from the continuous case.

Two Inviolable Constraints

The formal structure of a probability mass function rests on two non-negotiable mathematical requirements. First, for every possible value in the domain, the assigned probability must be non-negative; no outcome can carry a negative likelihood. Second, when you sum the probabilities across all hypothetical values the random variable could assume, the total must equal exactly one. This normalization condition mirrors the physical intuition that probability, like mass, is conserved: the full collection of outcomes accounts for all possible ways the world could turn out.

The function is formally written as p mapping the real line into the closed interval from zero to one, defined for every real number between negative and positive infinity. In practice, the subscript notation p_X(x) is often shortened to simply p(x) for brevity. The underlying object P is a probability measure, and the PMF extracts the probability of each individual outcome from that measure. These two constraints together ensure that the function is a valid probability distribution rather than an arbitrary assignment of numbers to values.

Where Discrete Meets Continuous

A critical distinction in probability theory separates the mass function from its continuous counterpart. A probability mass function applies exclusively to random variables whose possible values form a discrete set, and it directly reports the probability that the variable equals a particular value. By contrast, a continuous probability density function is attached to random variables that can take any value within an interval, and no single point carries a meaningful probability on its own. To extract a probability from a continuous PDF, one must integrate the density over a range of values.

The PMF, however, requires no such integration; the value of the function at a given point is itself the probability of that outcome. This is why the PMF is sometimes called a "discrete probability density function," though the terminology can cause confusion. Understanding which framework applies—summation over countable values versus integration over a continuum—is essential for correctly modeling and interpreting probabilistic phenomena.

The Measure-Theoretic Underpinning

At a deeper mathematical level, the probability mass function emerges as a special instance of two broader constructions in measure theory: the distribution of a random variable and its density with respect to the counting measure. Consider a probability space equipped with a measurable space whose underlying sigma-algebra is discrete, meaning it contains every singleton set. A random variable mapping from this space into that measurable space is classified as discrete precisely when its image is countable. The pushforward measure, often called the distribution of the variable, is a probability measure on the target space.

When this measure is restricted to individual singleton sets, it reproduces exactly the PMF values. Alternatively, if the target space is equipped with the counting measure, the PMF can be recovered as the Radon-Nikodym derivative of the pushforward measure with respect to that counting measure. This derivative is a function mapping the target space into the non-negative reals, and for any point b, the probability that the variable equals b is obtained by evaluating this derivative at b.

Reader's Guide

The probability density function is a cornerstone of statistical inference for continuous random variables. The terms 'probability distribution function' and 'probability function' can also denote the PDF, but this use is not standard among probabilists and statisticians.

In other sources, 'probability distribution function' may refer to the cumulative distribution function (CDF) or the probability mass function (PMF). The PMF is used for discrete random variables, while the PDF is used for continuous ones. The PDF is formally defined as the Radon–Nikodym derivative of the pushforward measure with respect to a reference measure.