The sampling theorem stipulates a minimum sampling frequency that we need to use. In order not to lose information in the conversion from the analog domain to the digital domain and vice versa. There are two applications, however, in which oversampling the signal. Namely, using the sampling rate that is higher than the minimum necessary. Will result in better reconstruction when we go from the digital to the analog domain. Or better sampling when we quantize the signal into the digital domain. Oversample analog to digital conversion uses a higher sampling rate than necessary to combat the effects of quantization noise. Consider an analog to digital conversion scheme where the input signal is band limited. We can raw sample the signal with the sampling period TS. And as long as TS complies with the sampling theorem we don't have to worry about aliasing. And then we quantize the samples into the digital domain. The resultant signal can be seen as the sum of the true sample values. Plus a sequence of noise samples that are created by the quantizer. The standard model is that the noise sequence is an i.i.d process and it is independent of the underlying signal. This is a very strong assumption that is rarely true in practice. But for the time being let's assume that it works. If this is true the power spectral density of the noise is constant over the entire frequency band. And its magnitude is equal to delta square over 12. Where delta is the quantization step if we use a uniform quantizer. The key observation that leads to oversample analog to digital conversion. Is that the digital spectrum after sampling has a support that is inversely proportional to the sampling period. So if we sample fast with a very small sampling period, the support of the resulting spectrum will be narrow. Whereas if we sample with a large sampling period, the support will be large. This is true even if we considered input as a stochastic process and it will apply to the support of the power spectral density. So graphically, it looks like so. Imagine this is a signal sampled at the critical rate. So using the sampling frequency dictated by the sampling theorem. We know in this case that the support of the power spectral density will be full band, so from minus pi to pi. And it looks like so. And here we have plotted as well, the power spectral density of the noise. Now let's suppose that we're sampling at twice the minimum sampling rate. At that point, the new sampling period will be the previous sampling period divided by 2. And this will shrink the support of the power spectral density of the input to minus pi over 2 and pi over 2. So we're contracting the signal that used to be full band to a half-band signal. And we can continue with this process by increasing the sampling frequency. So at three times the minimal sampling frequency, the support will be one third. At four times, the support will be one fourth. You also see that the amplitude of the power spectral density increases linearly with the oversampling factor. As shown in the formula a couple of slides ago. The key observation is that even if we increase the sampling rate, the power spectral density of the noise does not change. Because we have assumed that the noise is independent of the underlying sample. You can immediately see the fallacy in this reasoning. When you sample really fast, the samples would be very, very close to each other. And so the error that the quantizer introduces for each sample will be strongly correlated. And so it would be very unlikely that the power spectral density is independent of sampling frequency. But for moderate oversampling factors, this is a reasonable approximation. And now of course we can use a low pass filter. To eliminate the out-of-band quantization noise, the noise that falls outside of the minus pi over capital N to pi over capital N interval. Which is where the signal resides. Then, once we have eliminated the out-of-band noise by filtering. We downsample the signal to bring it back to the nominal sampling frequency dictated by the sampling theorem. And in so doing, we scale down the signal according to the down sampling formula and we scale down the power of the noise. So in the end you have the same power spectral density as you would've had if you had sampled at the critical rate. But the noise power has been divided by a factor of capital N. The final scheme is like so. You have your band limited signal here. You have a sampler that operates at N times the minimal sampling frequency. Or conversely whose sampling period is 1 over N times the maximum sampling period. You have a quantizer. And then you have a low pass filter with cut off frequency pi over capital N followed by a down sampler by N. The signal to noise ratio at the output of this chain is N times the signal to noise ratio. That you would have had with a simple straightforward sampling and quantization. So that means that you gain about 3dBs per doubling of the sampling frequency. However, as we said, the key assumption of noise independence doesn't really hold for very high upsampling factors. We can quickly look at how over sample [INAUDIBLE] conversion works entirely in the time domain just to get some intuition. The over sampling factor of capital N basically produces N samples where we would've needed just one. Now if we assume that the quantization noise is uncorrelated between these extra samples. The low pass filtering operation is simply a local average. That by average in neighboring samples with uncorrelated noise will reduce the overall noise for this group of samples. And then by downsampling by capital N, we're discarding the redundant information introduced by oversampling. Another application where we can trade sampling rate for precision is in the conversion from the digital to analog domain. Here we're not concerned anymore with quantization errors. We are actually trying to use very cheap interpolation filters to produce an analog signal. This is particularly relevant in audio applications. And in fact, Oversampled D/A is the reason why we can have very cheap, and yet very high quality, audio consumer electronics today. So let's see how this works. We know that in the ideal case when we move from the digital to the analog domain, we should use sinc interpolation. Which in practice corresponds to using an analog low-pass filter with a very, very sharp, almost ideal, characteristic. Remember, this is the continuous time spectrum here, which is obtained from the discrete time spectrum. By simply selecting the base band component of the spectrum. Via the Fourier transform of the sinc interpolator that we know to be [INAUDIBLE] in the frequency domain. So graphically, if this is your digital spectrum, you know that in the continuous time domain, you will have all these repetitions. This is the positive frequency axis, right. And the sinc interpolator acts as an ideal low pass filter. That selects just the base band component and kills all the repetitions at high frequency. Now you understand for instance that in audio applications. These high-frequency repetitions of the spectrum would sound extremely disturbing. And even if these copies are beyond the limit of human hearing, they're really undesirable. Because they would overload the amplification stages of any music system like your stereo for instance. Now the problem is that using such a sharp low pass filter in digital to analog conversion. Means that you have to have an analog low pass filter with a very, very sharp characteristic. Now analog low pass filters are very expensive to manufacture with this precision. And also, the more precise the filter is, the more delicate it becomes. In the sense that, if you use analog components. You have a natural aging of the components of the capacitors, for instance, that are used in the filter. That will change the characteristic of the filter over time, or if the temperature changes, and so on and so forth. So if you want a very good digital to analog conversion, it seems that you have to invest a lot in the interpolation filter. And this indeed was the situation when the first digital audio devices came into the market in the 80s. Then people invented oversample digital to analog. Remember that in general, if we don't use a sinc interpolator, the spectrum of the reconstructed analog signal will have this form here. We still here have the digital spectrum mapped to the analog frequency axis. And then we multiply that by the Fourier transform of the interpolator. Which in this case is not necessarily sinc. It could be a rect, it could be a triangle, and so on and so forth. We have seen that the cheapest interpolator is the zero order hold that has a rectangular characteristic. Which in frequency becomes a sinc. And we have seen that because of this slow decay in frequency of the sync characteristic. This is not really a good interpolator to use in practice. We can see this in a plot very easily if we start from our digital spectrum. This is the digital spectrum now mapped to the analog frequency axis. And if we multiply that by the Fourier transform of the zero-order hold interpolator, we have this. Which results in two problems. The first one is that we have all these periods of high frequency stuff here that we really don't want. And then, of course, because the characteristic of the sinc infrequency is not flat on the past band. We have a distortion of the signal around the bass band. So if you were to compensate for the effects of using the zero ordered hold. We would have to use another filtering cascade, again an analog filter, therefore a very expensive filter, that does two things. It cuts away this high frequency component and compensates for the distortion introduced by the zero ordered hold. So we're actually worse off. In the sense that we now have to design an analog filter that's even more complicated than a simple low pass. All right, so now let's go back to the digital domain and consider a K times upsampled version of our discrete time sequence. We take our sequence. We introduce K minus 1 zeroes for every original sample. And then we do a low pass filter with a cut off frequency pi over K. Now here we are entirely in the digital domain so we can be as accurate as we want in the low pass filtering operation. And it's not going to be expensive. Just as a quick recap this is what happens when we over sample in the discrete time domain. This is our original spectrum. Oversample by a factor of 2 will contact the spectrum of course, bringing in copies from the left and from the right. And then the low pass filter will eliminate the copies that have been brought into the -pi, pi interval. And we're left with a spectral occupancy that is 1 over capital K, the original occupancy. Now we want to convert this to the analog domain. There is nothing particularly difficult. We just need to remember that we have artificially increased the underlying sampler rate by a factor of capital K. And so if we go through the math and we do a standard sinc-interpolation, everything is exactly the same. Except that now, all the frequencies have been multiplied by a factor K and the interpolation interval has been reduced by a factor of K. And if we do that and this is remember our upsampled discrete time spectrum. What happens in the analog domain is that again you map the spectrum to the analog frequency axis. The fact that the spectral occupancy in the visual domain is not 100% will create these gaps in the analog frequency. Which we will soon exploit for our purposes. But for the time being, let's just assume we're doing sinc interpolation. When that happens, again, the sinc will become [INAUDIBLE] in frequency. With a cutoff frequency now that is k times the original cutoff frequency. So nothing new here. Okay, now let's use the cheap zero ordered hold to convert this oversampled signal to the analog domain. Remember, we had two problems with the zero order hold. The first was of course, the spurious components in the high frequency bands. And the second one was the distortion for the baseband part, due to the non-flat characteristic of the sinc in frequency. We're still in the digital domain. So we can pre-compensate for the distortion in the baseband portion of spectrum. By multiplying the oversampled digital signal with a characteristic. That is the inverse of the sinc over the support of the signal. Again, we can do this cheaply in the digital domain. Because we're doing filters in software so we can achieve arbitrary characteristics with arbitrary precision. Once we have precompensated the signal in the digital domain, we go through the zero ordered hold and what we get is the following. So this is the precompensated spectrum here. The same spectrum mapped onto the analog frequency axis with the repetitions. And now the characteristic of the zero ordered holding frequency will look like so. Again, we have multiplied all the frequencies by a factor of two in the picture and by a factor of k in the general case. When we multiply the two together we still have of course some spurious frequency components. Which however are at a much higher frequency than before. And we have this gap here that we have created by oversampling that is going to make our life much, much easier. Because now what we can do is fit a very simple and gently sloping low pass filter, analog low pass filter. That can take all the space that it needs in order to transition from pass band to stop band. And then in the stop band, it will finally eliminate the remaining spurious high frequency components that are left there. So what we have done here is we have traded sampling frequency for economy in production. We will have to operate at a much higher rate than the one required by the original sampling rate of our discrete time sequence. And also the filtering operations in discrete time will have to go faster. But usually, that is a much cheaper option. Than having to design very high-quality analog filters for the last stages of the digital to analog converter.
