Blended
Basic tools for theoretical understanding and practical use of the main supervised and unsupervised learning algorithms.
This teaching unit offers a general introduction to the core topics of the School, together with a short review of key preliminary concepts relevant to the subsequent courses.
The focus will be on the task of image denoising, approached from both a theoretical and a computational standpoint. We will begin by formulating image denoising as a Bayesian inverse problem, highlighting connections to variational regularization theory and basic optimization methods. Then, we will reinterpret the problem within a statistical learning framework, demonstrating the effectiveness of end-to-end approaches based on neural networks with appropriate architectures.
The theoretical discussions will be complemented by numerical implementations using Python, leveraging the libraries NumPy, Matplotlib, and PyTorch.
Detailed program:
First half (1hr 15min) - Image denoising inverse problem : from Bayesian formulation to optimisation
Forward modeling: Gaussian noise, other additive noise models (heteroscedastic noise), Poisson noise.
Quantification of the degradation of an image: MSE, PSNR, SSIM.
Image denoising as an inverse problem.
Bayesian inverse problems: likelihood, prior, posterior, Bayes formula.
Maximum a posteriori estimation: connection with variational regularization + score function.
Comparison with different priors (Tikhonov-type, structured, sparsity-inducing).
For l^2 fidelity + l^2 prior: Gradient descent for solving the denoising problem.
Role of the regularisation parameter: tuning. Issues in handcrafted priors + parameter selection.
Second half (1hr 30min) - Black-box machine learning for image denoising
Image denoising as a regression learning problem.
Unsupervised and supervised datasets and (model blind) methods.
Neural networks as parametric hypothesis classes: bias-variance tradeoff, over-parametrization.
Network architectures for image processing: CNNs, UNets, attention layers,...
Training process: loss, backpropagation, optimizers, scheduling, batches, validation.
Basic image processing with numpy and matplotlib.
Examples of grayscale and color images and datasets.
Basic implementation of training and testing procedures via pytorch.