Why the sampling theorem is wrong. (Part 1).

 Why the sampling theorem is wrong. (Part 1).

 (This is on-line translate of a part of an article published in Dec-1999 in largest Russian computer magazine “Computerra”)

The mathematician Jean-Baptiste-Joseph Fourier (not the utopian who is the source and integral part of Marxism, that one, Charles Fourier) invented  a simple method (based on the work of the semi-Russian mathematician Leonhard Euler) for representing periodic processes of various natures by expressing them as the sum of sines and cosines of multiple frequencies. Only for periodic processes.

For no other. It must be known in advance that the process is 100% periodic. An example is the bending vibrations of rotating helicopter blades (I took this from Nosach's book on interpolation). Later, mathematicians managed to adapt this method to non-periodic processes, or rather, to parts of non-periodic processes. In this case, the sum of the sines and cosines necessarily becomes infinite, and therefore the method has purely theoretical value.

What did they do with this essentially elegant mathematical method of radio engineering (I wouldn't call them engineers)? True, these technicians corrupted it. No, they passed it on to posterity in its original form. They didn't modify it. They simply began to apply it everywhere.

The funniest thing is that none of them asked each other: "Hey guys, maybe our signal is aperiodic?" For a mathematician, this is a no-brainer: if the initial conditions of a method aren't met, the method is inapplicable. This didn't stop the radio engineers with their "crash courses" in higher mathematics. Thus was born the famous 1933 "sampling theorem" of Comrade Vladimir Aleksandrovich Kotelnikov. He was 25 years old at the time, living in Kazan. His father was a professor of higher mathematics at the University. Apparently, his father had something to do with it, since the "sampling theorem" had been implicitly proposed much earlier as a numerical method for interpolating functions defined in tables.

The first mistake was that the "theorem" understood the signal's "spectrum" as the result of processing the signal using a Fourier transform. This implied that the signals being sampled and reconstructed were periodic and limited in amplitude. Therefore, the proof of the "theorem" was reduced to a tautology: the condition of a signal's spectrum being bounded (by Fourier) from above is equivalent to the signal being representable by a countable number of sinusoids of multiple frequencies. In other words, it sounded like this: a periodic signal, which can be represented by a finite sum of multiple frequencies using Fourier transform, can be reconstructed point by point by interpolating a finite Fourier sum. Radio engineers interpret this as follows: any signal in which high frequencies have been effectively suppressed using passive filtering can be accurately reconstructed from individual (not very precise) samples using linear passive low-pass filtering.

The second mistake was that V. A. Kotelnikov drew the spectrum graph of the reconstructed signal incorrectly. Even at 17, I could draw graphs much better, which is why I was accepted into the Radio Engineering Department at MIPT. The error in Kotelnikov's signal spectrum graph is that the spectrum terminates at zero on the left. That is, it consists of an infinite number of low-frequency spectral components. This is impossible because the signal is then unbounded in amplitude, meaning the conditions for the signal's amplitude to be bounded and the Fourier integral to converge at infinity—a necessary condition for the existence of a Fourier spectrum—are not satisfied. A signal, even a periodic one, must also be bounded at the bottom to be reconstructed according to Kotelnikov. And no worse than at the top.

You know what the joke is? You don't, so I'll tell you. The joke is that almost all modern mixing consoles for digital audio recording have very cool (not fancy, just with a steep cutoff) high-pass filters that greatly attenuate the low-frequency components of the signal. Not only that, the latest models of studio microphones also have a switchable high-cut filter. Why? Because it sounds much better. Get it? What? You'll say that the low-frequency components simply have... You know what the joke is? You don't know, so I'll tell you. The joke is that almost all modern mixing consoles for digital audio recording have very cool (not fancy, just with a steep cutoff) high-pass filters that greatly attenuate the low-frequency components of the signal. Not only that, the latest models of studio microphones also have a switchable high-cut filter. Why? Because it sounds much better. Get it? What? You're saying that the low-frequency components simply have high amplitude and overload the input analog-to-digital converter? The trick is that instruments without strong low-frequency components, as well as vocals, sound better.

The requirement I mentioned for a lower bound on the spectrum was first noticed in the USSR.

 Specifically, it immediately and implicitly follows from Ageyev's proven theorem (described in Fink's book). Ageyev's theorem was a thorn in the side of V.A. Kotelnikov's successors (at the time, Academician Kotelnikov was, I believe, practically the Chairman of the Supreme Soviet of the RSFSR), and so it was "forgotten." If not for Fink's book "Noise, Signals, Errors," I wouldn't have known about Ageyev's theorem either.

 The third mistake in the sampling "theorem" is that it is "proven" for a signal with a rectangular spectrum. The reconstruction function (interpolating digital sample points) has the form sin(x)/x. The "theorem" says nothing about what the reconstruction function should be for a signal with a non-rectangular spectrum. It should have a completely different form. I wrote about this in the audiophile magazine "Class A," and I'll repeat it here: I personally find it ironic that it was Kotelnikov, long after his sampling "theorem," who laid the foundations for the theory of optimal filtering, which posits that signals with different spectra require different restoration functions (read: different filters). In other words, as the signal spectrum changes, the output filter of the digital-to-analog converter must also change!

Guys! You're closer to this in Moscow than I am in Vinnytsia (Vinnytsia is near Zhmerynka): go up to Academician Vladimir Aleksandrovich Kotelnikov—he's 91 now, he's still alive, and he edits the journal "Radio Engineering"—and ask him what he thinks about the limitations of digital audio.

 Nevertheless, I personally don't think Kotelnikov's sampling "theorem" was a big mistake back then. After all, that was 1933. On the contrary, I'm even a little proud that I'm also Russian (in the broad sense). I'm especially proud when I see how seriously Americans take the sampling theorem, which now (for now) forms the basis of all communications theory and radio circuit analysis. They don't even whisper about its incorrectness!

 Let me explain for the uninitiated Computerra reader: Shannon invented the sampling "theorem" for America in 1949. Exactly the same as Kotelnikov's! Not a year, not two, but 16 years later! With the same errors! Americans aren't bothered by such well-known concepts as scientific priority and plagiarism!

 Guys, there's no mention of a person named Kotelnikov in American literature! Talk to me about the lack of ideological censorship in the States! It's just that everything was invented in the States!

 Where's the way out of the digital impasse? Just ask Sony!

 Wait, you don't have to ask – Sony's president told the editor of the world's most popular audiophile magazine, Stereophile, that Sony sees no need for a new digital audio format.

 How many gigabucks has Sony made on digital audio? A lot. How many megabucks did Sony spend on just the marketing of the stupid (sound-wise) mini-disc? About three hundred million dollars, according to Sony's own official figures. How much did they spend on serious scientific research in sound? Zero? Slightly more than zero.

Please note that Sony is targeting young people with mini-discs. Their hearing and tastes haven't yet developed. The older generation can still distinguish between a dry, empty digital sound and the normal, harmonious sound of a live musical instrument. Do you know how the MP3 compression algorithm differs from the ATRAC format of that stupid mini-disc from you-know-which company? Nothing! Only the size of the "window" in which the compression calculations are performed. Sony bought the entire analog archive of a good recording company, RCA, and what are they going to do with it? That's right, convert it to digital form "for better preservation." Akio Morita, the founder and former president of Sony, died in Tokyo on October 3, 1999, of pneumonia. He started this whole digital audio mess, and he could have stopped it himself. But he died, and Sony will stubbornly march toward its digital grave, glittering with chrome control knobs, platinum-plated enhanced technical specifications, and gold-plated extended service features.

The Crisis of Radio Engineering

 Now it's time to talk about the crisis in radio engineering in general. Besides the inadequacy of the mathematical methods currently used, there's a crisis in information theory and radio engineering measurements. Here's the thing. A radio engineer measures the passage of a single sinusoidal signal, say, through a computer sound card, and based on the absence of distortion in this single sinusoid, he concludes that any combination of such signals will also pass without distortion.

 Let's give an analogy: in the room (radio path) you're sitting in, there's a door (a radio device, such as a sound card). Any one normal person (a monoharmonic signal—with a single frequency component) can pass through this doorway (the device's bandwidth): a woman, a man, a child, fat, thin, tall, or short.

 But this doesn't mean that a hundred or two hundred people (signal harmonics) can simultaneously pass through this door!

 The theory of measuring the radio characteristics of devices is based (in 99% of cases) on the assumption that if one signal transmits well, any combination of them will also. But this is incorrect! This is still a vast field for research and calculations.