This aim is to introduce basic notions from machine learning. We will quickly focus on supervised learning in the purely data driven setting, i.e., there are no physics driven mechanistic models for how training data is generated. The aim is to survey some of the results and open problems associated with developing a mathematical and computational theory for deep learning in this setting.
Focus here is on applying machine learning to solve ill-posed inverse problems, i.e., to recover an operator that maps the data to signal. The starting point is to very briefly survey current regularization schemes emphasizing on the ability to account for a priori information. Next, is to outline challenges associated with using machine learning for solving ill-posed inverse problems followed by a survey on early attempts that are based on using it as a post- processing step. We conclude with outlining the limitations with this latter approach.
This lecture introduces specific deep neural networks for solving ill-posed inverse problems that account for the a priori information contained in a forward model. We outline the current approaches, point to open problems, and conclude with showing examples of their performance illustrated in tomographic image reconstruction.
What is an X-ray image? The Beer-Lambert law
Slice imaging and history of CT. Inverse problem of tomography
Are you a natural tomographer?
Filtered back-projection and the Radon transform
Applications
Why matrices for tomography instead of filtered back-projection?
Singular value decomposition and ill-posedness
Naive and regularized reconstructions for 12x12 pixel tomography
Review of basic regularization methods: Truncated SVD, Tikhonov regularization, total variation regularization, wavelet sparsity
A learning-based approach
Limited angle data and the Helsinki Tomography Challenge 2022
Nonlinear imaging: passive gamma emission tomography of spent nuclear fuel
More applications
Siiri Rautio, Salla Latva-Äijö, Elli Karvonen and Elena Morotti will help with the sessions.
Exercises on Monday: simple tomographic matrix models.
https://github.com/ssiltane/RICAM2022tomography
Lab class on Tuesday: working with open datasets from Helsinki
https://www.fips.fi/dataset.php
- Motivation, logarithmic barrier function, central path, neighbourhoods,
- path-following method, convergence proof, complexity of the algorithm,
- practical implementation issues.
- Quadratic Programming (QP) problems, primal-dual pair of QPs,
- Nonlinear (convex) inequality constraints,
- Second-Order Cone Programming,
- Semidefinite Programming,
- Newton method, logarithmic barrier function, self-concordant barriers.
- Sparse Approximations with IPMs:
modern applications of optimization which require a selection of a 'sparse' solution originating from computational statistics, signal or image processing, compressed sensing, machine learning, and discrete optimal transport, to mention just a few.
- Alternating Direction Method of Multipliers (ADMM).
Filippo Zanetti and Margherita Porcelli will help with the sessions
Exercise on Monday:
IPMs: From LP to QP
Conjugate Gradients for positive definite linear systems
Exercise on Tuesday:
Examples of IPMs in action:
Material-separating regularizer for multi-energy X-ray tomography
Semidefinite Programming: Matrix Completion
In this lecture we revise the basic notions on smoothness (Lipschitz continuity, Gateaux/Frechet differentiability..) and convex analysis (subdifferentials and subgradients, Fenchel conjugation, strong convexity..) that will be required for defining two basic algorithms solving convex optimisation problems: gradient descent (GD) and proximal gradient descent (GD).
In this lecture we will describe acceleration strategies for improving convergence performance of GD (Nesterov acceleration), PGD (FISTA) and show how strong convexity can be explicitly dealt with to define even faster acceleration schemes.
In this lecture we will give a focus on how sparsity appears in applications. We will comment on the use of l_1 VS l_0 optimisation-based approaches. For the latter ones, we will review Iterative hard thresholding and greedy algorithms, and introduce the notion of continuous relaxations, both for constrained and unconstrained l_0 problems. Applications to microscopy image analysis will be shown.
In this lab we will consider convex optimisation algorithms for solving problems with an l_2 data term and a sparsity promoting term for problems of image reconstruction such as molecule localisation (for undersampled, blurred and noisy data, with l_1 penalty). We will compare ISTA with its accelerated version (FISTA) and, for a slightly modified model, with its strongly convex variant.
Proof of convergence of FISTA in function values + strongly convex variant (with explicit knowledge of strong convexity parameter).