Welcome again to the course in audio signal processing for music applications. In the previous theory lecture I gave an overview of the feel of audio signal processing as an introduction and motivation to the course. Now I want to outline the specific topics that we will present in the course. Which even though cover a tiny part of the audio signal processing field. They're very relevant to understand many concepts and methods used in audio signal processing systems. Especially the ones for music applications. As introduction to the course this week we have given a context of the audio signal processing field. Given an outline of the course. And we will go over in the next class, over the basic mathematics that are required to follow the course. Then next week we will start by introducing the discrete Fourier transform. This transform is the basis for everything we will do. And, we will try to understand every component of it. So we'll go through the DFT equation, then we will go through complex exponentials, the scalar product in the DFT, the DFT of complex sinusoids, the DFT of real sinusoids. And we'll finish with the inverse-DFT. So to reconstruct the original signal. Then in in week three, in order to better understand the DFT, we will go over the basic theorems and properties of the Fourier transform. And these are the concept of linearity. Shift, symmetry and convolution. Then we'll talk about energy conservation and decibels. The concept of phase unwrapping and zero padding. The use of the Fast Fourier Transform. Which is the algorithm that we will be using to implement the DFT. And then the FFT together with the concept of zero-phase windowing. And finally we'll put it together into an analysis synthesis system. So then on week four, we will go to the Short-time Fourier Transform, which is the first version of the Fourier transform that can be used in real sounds. Those are some signals that change in time. We will present the first complete analysis synthesis system of the course that is of some practical use. So we'll start with the STFT equation, then talk about the analysis window, then about the FFT size and hop size. And the very common concept which is the time frequency compromise, which is fundamental to understand, the Short-time Fourier Transform. And finally, we will discuss the concept of the Inverse Short-time Fourier Transform. So with that we can, put together an analysis-synthesis system. Then, on week five we will go a step further and we will be building on top of the Short-time Fourier Transform. What we will call, the sinusoidal model. We'll present the, the equation of the sinusoidal model. Then we will introduce the concept of what are sinusoids seen from the frequency domain, seen from the spectrum. And then we will talk about how we will actually find those sinusoids, and the concept is going to be the, the idea of spectral peaks. And then we'll talk about how to, to analyze, identify this sinusoids in, as as time varying sinewaves in the spectrogram, in a time varying spectrum. And finally again, we'll talk about synthesis. So again, building up an analysis synthesis system, based on this concept of sinusoidal modeling. Then the next week on week six. We will be assuming that sound is harmonic. And then if we do that assumption, we can develop much more powerful model, what we will call the harmonic model. Which is a sinusoidal model, in which we add the constrains that all the sinusoids are multiple of a fundamental frequency. So we'll introduce the harmonic model equation, then we'll talk about three concepts that many times we confuse, which is the concept of sinusoids, the concept of partials and the concepts of harmonics. Then we will introduce the concept on monophonic and polyphonic signals. Because the analysis techniques we will be using are quite distinct if we apply them to monophonic or polyphonic signals. And finally we'll talk about how to detect the harmonics. To do that, we will complement this discussion with idea of fundamental frequency detection. A harmonic sound is one that has a fundamental frequency. So we need to understand these and be able to implement techniques to analyze the fundamental frequency. Then on week seven we will add some more complexity. Because many sounds cannot be well modeled with sinusoids or with harmonics. So we need to worry about the parts of sounds, attacks, non linerarities, noises. That are not well modeled with sinusoids. And for this, we introduce the idea of modelling sounds, with sinusoids plus residual. And in that concept, we need to introduce the concept of stochastic model. So how signals can be modeled using this, this concept of stochastic signals. Then we will actually talk about how to approximate sounds with these stochastic models. And then we will introduce the, this model of sinusoids or harmonics plus residual. So, the signal that is left that is not modeled well with sinusoids and harmonics. So therefore we'll, we'll need to talk about subtracting these residual, obtaining these residual. And then these residual, hopefully in many situations, can be a stochastic signal. And therefore then we can present these sinusoidal or harmonic plus a stochastic model. Therefore for that will need to introduce the, the concept of how to model the residual with this concept of stochastic signals. So in this this will be quite dense lecture, because we will be talking quite a bit about very different. Models and that complement each other, and that can be a little bit complex from this end of the first approximation. Okay, and then basically that's going to be all the models we will be talking about. I mean the, rest of the, classes, the, the next three classes, we will basically talk about applications and more kind of advanced topics. So, as I mentioned in this course we focus on two types of applications, one is sound transformations. So in this week eight, we will go over the models that we have presented. And show the types of transformations that are possible to do with them. So for example, we will talk about the Short-Time Fourier Transform, and what can we do in terms of transformation. So, specifically filtering or morphing. Then we will go over the sinusoidal model, and what type of transformations. Basically focusing on time and frequency scaling. Then, on the harmonics plus residual model. We will focus on the idea of pitch transposition. And finally on the most elaborative model that we will have introduced, which is the harmonics plus stochastic model, we will talk about time stretching and morphing. Then on week nine we will talk about the second major application we would like to focus on in this class. Which is the idea of sound description. So, with with what we have presented we can extract features of a sound in a way that are relevant to describe it. So we'll first focus on the idea of audio features that can be obtained from this spectral analysis that we have talked about. And then we will talk about how to use these type of techniques to describe more complex music or signals and concepts of like a whole collection of sounds. How do we handle the analysis of not just one sound, but a collection of sounds? 128 00:09:16,050 --> 00:09:20,099
And finally on the, the last week
of classes on the tenth week. We will review all the models and outline related topics that we have not been able to cover in class. We will present other Fourier-based sound models and we'll also present other models that are not Fourier-based, again so we'll review the class. And then go beyond this signal processing for music applications. Things that we have been talking. That's all in terms of references. As in every class all the content is available online and with open licenses so. Feel free to review all the content and have access to everything I have been mentioning. So this is all for this lecture we have done a quick outline of the course, going over the theory topics that we will cover. So from now on we are really starting going into signal processing. So, thank you for attention and I'll see you next class.