Speech processing

Speech processing examines how speech signals are captured and manipulated. Because these signals are typically handled in digital form, the field is essentially a subset of digital signal processing focused on speech. Its scope covers everything from acquiring and storing speech to transferring, altering, and outputting it. Common applications include recognizing who is speaking, understanding what is said, generating synthetic speech, cleaning up noisy audio, and distinguishing between different speakers.

The earliest work in this area concentrated on recognizing basic vowel sounds. In 1952, a team at Bell Labs—Stephen Balashek, R. Biddulph, and K. H. Davis—built a system that could identify digits spoken by a single person. During the 1940s, researchers had already begun analyzing speech through its frequency spectrum. A key algorithm, linear predictive coding (LPC), was introduced in 1966 by Fumitada Itakura and Shuzo Saito. Later, in the 1970s, Bishnu S. Atal and Manfred R. Schroeder at Bell Labs refined LPC further. This technique became the foundation for voice-over-IP technology and for speech synthesizer chips, like those used in Texas Instruments’ Speak & Spell toys from 1978.

The first commercially successful speech recognition product, Dragon Dictate, hit the market in 1990. Two years later, AT&T deployed technology from Lawrence Rabiner and his Bell Labs colleagues to route calls automatically without human operators. By then, these systems had vocabularies larger than the average person’s. Around the early 2000s, the field began moving away from hidden Markov models (HMMs) toward neural networks and deep learning. In 2012, Geoffrey Hinton’s team at the University of Toronto showed that deep neural networks could beat traditional HMM-based systems on large-vocabulary continuous speech recognition tasks, sparking widespread industry adoption. By the mid-2010s, Google, Microsoft, Amazon, and Apple had built advanced speech recognition into their virtual assistants—Google Assistant, Cortana, Alexa, and Siri—using deep learning for more natural interactions. Transformer-based models like BERT and GPT later improved context-aware understanding of speech. More recently, end-to-end models have become popular, converting audio directly into text without separate steps for feature extraction or acoustic modeling, which simplifies development and boosts performance.

Dynamic time warping (DTW) measures similarity between two sequences that may differ in speed. It finds the optimal alignment between them, satisfying specific constraints, by minimizing the total cost—calculated as the sum of absolute differences between matched index values.

A hidden Markov model is a simple dynamic Bayesian network. It estimates an unobserved variable x(t) based on observed data y(t). The Markov property means that the probability of x(t) depends only on x(t−1), and each observation y(t) depends only on the hidden state x(t) at that same time.

An artificial neural network consists of connected units called artificial neurons, loosely inspired by biological brains. Each connection transmits a real-number signal between neurons. When a neuron receives a signal, it processes it and passes the result to others, typically using a nonlinear function of the summed inputs.

Phase information in speech is often treated as random but actually carries useful data. Phase wrapping occurs due to periodic jumps of 2π. Phase unwrapping can be expressed as φ(h,l) = φ_lin(h,l) + Ψ(h,l), where φ_lin(h,l) = ω₀.