
FEDERICA PORTA
Ricercatore t.d. art. 24 c. 3 lett. B Dipartimento di Scienze Fisiche, Informatiche e Matematiche sede exMatematica

Home 
Curriculum(pdf) 
Didattica 
Pubblicazioni
2024
 A stochastic gradient method with variance control and variable learning rate for Deep Learning
[Articolo su rivista]
Franchini, G.; Porta, F.; Ruggiero, V.; Trombini, I.; Zanni, L.
abstract
In this paper we study a stochastic gradient algorithm which rules the increase of the minibatch size in a predefined fashion and automatically adjusts the learning rate by means of a monotone or non monotone line search procedure. The mini batch size is incremented at a suitable a priori rate throughout the iterative process in order that the variance of the stochastic gradients is progressively reduced. The a priori rate is not subject to restrictive assumptions, allowing for the possibility of a slow increase in the mini batch size. On the other hand, the learning rate can vary non monotonically throughout the iterations, as long as it is appropriately bounded. Convergence results for the proposed method are provided for both convex and non convex objective functions. Moreover it can be proved that the algorithm enjoys a global linear rate of convergence on strongly convex functions. The low per iteration cost, the limited memory requirements and the robustness against the hyperparameters setting make the suggested approach well suited for implementation within the deep learning framework, also for GPGPUequipped architectures. Numerical results on training deep neural networks for multiclass image classification show a promising behaviour of the proposed scheme with respect to similar state of the art competitors.
2024
 A variable metric proximal stochastic gradient method: An application to classification problems
[Articolo su rivista]
Cascarano, P.; Franchini, G.; Kobler, E.; Porta, F.; Sebastiani, A.
abstract
Due to the continued success of machine learning and deep learning in particular, supervised classification problems are ubiquitous in numerous scientific fields. Training these models typically involves the minimization of the empirical risk over large data sets along with a possibly nondifferentiable regularization. In this paper, we introduce a stochastic gradient method for the considered classification problem. To control the variance of the objective's gradients, we use an automatic sample size selection along with a variable metric to precondition the stochastic gradient directions. Further, we utilize a non monotone line search to automatize step size selection. Convergence results are provided for both convex and nonconvex objective functions. Extensive numerical experiments verify that the suggested approach performs on par with stateoftheart methods for training both statistical models for binary classification and artificial neural networks for multiclass image classification. The code is publicly available at https:// github .com /koblererich /lisavm.
2023
 A Line Search Based Proximal Stochastic Gradient Algorithm with Dynamical Variance Reduction
[Articolo su rivista]
Franchini, G.; Porta, F.; Ruggiero, V.; Trombini, I.
abstract
Many optimization problems arising from machine learning applications can be cast as the minimization of the sum of two functions: the first one typically represents the expected risk, and in practice it is replaced by the empirical risk, and the other one imposes a priori information on the solution. Since in general the first term is differentiable and the second one is convex, proximal gradient methods are very well suited to face such optimization problems. However, when dealing with largescale machine learning issues, the computation of the full gradient of the differentiable term can be prohibitively expensive by making these algorithms unsuitable. For this reason, proximal stochastic gradient methods have been extensively studied in the optimization area in the last decades. In this paper we develop a proximal stochastic gradient algorithm which is based on two main ingredients. We indeed combine a proper technique to dynamically reduce the variance of the stochastic gradients along the iterative process with a descent condition in expectation for the objective function, aimed to fix the value for the steplength parameter at each iteration. For general objective functionals, the a.s. convergence of the limit points of the sequence generated by the proposed scheme to stationary points can be proved. For convex objective functionals, both the a.s. convergence of the whole sequence of the iterates to a minimum point and an O(1 / k) convergence rate for the objective function values have been shown. The practical implementation of the proposed method does not need neither the computation of the exact gradient of the empirical risk during the iterations nor the tuning of an optimal value for the steplength. An extensive numerical experimentation highlights that the proposed approach appears robust with respect to the setting of the hyperparameters and competitive compared to stateoftheart methods.
2023
 Constrained and unconstrained deep image prior optimization models with automatic regularization
[Articolo su rivista]
Cascarano, Pasquale; Franchini, Giorgia; Kobler, Erich; Porta, Federica; Sebastiani, Andrea
abstract
Deep Image Prior (DIP) is currently among the most efficient unsupervised deep learning based methods for illposed inverse problems in imaging. This novel framework relies on the implicit regularization provided by representing images as the output of generative Convolutional Neural Network (CNN) architectures. So far, DIP has been shown to be an effective approach when combined with classical and novel regularizers. Unfortunately, to obtain appropriate solutions, all the models proposed up to now require an accurate estimate of the regularization parameter. To overcome this difficulty, we consider a locally adapted regularized unconstrained model whose local regularization parameters are automatically estimated for additively separable regularizers. Moreover, we propose a novel constrained formulation in analogy to Morozov's discrepancy principle which enables the application of a broader range of regularizers. Both the unconstrained and the constrained models are solved via the proximal gradient descentascent method. Numerical results demonstrate the robustness with respect to image content, noise levels and hyperparameters of the proposed models on both denoising and deblurring of simulated as well as real natural and medical images.
2023
 Correction to: A Line Search Based Proximal Stochastic Gradient Algorithm with Dynamical Variance Reduction (Journal of Scientific Computing, (2023), 94, 1, (23), 10.1007/s10915022020843)
[Articolo su rivista]
Franchini, G.; Porta, F.; Ruggiero, V.; Trombini, I.
abstract
We restated both the statement and the proof of Theorem 3. We stress that the proof only changes in obtaining the inequality (A11) from (A10), but for a better readability we report all the arguments of the proof.
2023
 Diagonal BarzilaiBorwein Rules in Stochastic GradientLike Methods
[Relazione in Atti di Convegno]
Franchini, G.; Porta, F.; Ruggiero, V.; Trombini, I.; Zanni, L.
abstract
2023
 Hybrid limited memory gradient projection methods for boxconstrained optimization problems
[Articolo su rivista]
Crisci, S.; Porta, F.; Ruggiero, V.; Zanni, L.
abstract
Gradient projection methods represent effective tools for solving largescale constrained
optimization problems thanks to their simple implementation and low computational
cost per iteration. Despite these good properties, a slow convergence rate
can affect gradient projection schemes, especially when high accurate solutions are
needed. A strategy to mitigate this drawback consists in properly selecting the values
for the steplength along the negative gradient. In this paper, we consider the
class of gradient projection methods with line search along the projected arc for boxconstrained
minimization problems and we analyse different strategies to define the
steplength. It is well known in the literature that steplength selection rules able to
approximate, at each iteration, the eigenvalues of the inverse of a suitable submatrix
of the Hessian of the objective function can improve the performance of gradient
projection methods. In this perspective, we propose an automatic hybrid steplength
selection technique that employs a proper alternation of standard Barzilai–Borwein
rules, when the final active set is not well approximated, and a generalized limited
memory strategy based on the Ritzlike values of the Hessian matrix restricted to the
inactive constraints, when the final active set is reached. Numerical experiments on
quadratic and nonquadratic test problems show the effectiveness of the proposed
steplength scheme.
2023
 Learning rate selection in stochastic gradient methods based on line search strategies
[Articolo su rivista]
Franchini, G.; Porta, F.; Ruggiero, V.; Trombini, I.; Zanni, L.
abstract
Finitesum problems appear as the sample average approximation of a stochastic optimization problem and often arise in machine learning applications with large scale data sets. A very popular approach to face finitesum problems is the stochastic gradient method. It is well known that a proper strategy to select the hyperparameters of this method (i.e. the set of apriori selected parameters) and, in particular, the learning rate, is needed to guarantee convergence properties and good practical performance. In this paper, we analyse standard and line search based updating rules to fix the learning rate sequence, also in relation to the size of the mini batch chosen to compute the current stochastic gradient. An extensive numerical experimentation is carried out in order to evaluate the effectiveness of the discussed strategies for convex and nonconvex finitesum test problems, highlighting that the line search based methods avoid expensive initial setting of the hyperparameters. The line search based approaches have also been applied to train a Convolutional Neural Network, providing very promising results.
2023
 Neural architecture search via standard machine learning methodologies
[Articolo su rivista]
Franchini, Giorgia; Ruggiero, Valeria; Porta, Federica; Zanni, Luca
abstract
In the context of deep learning, the more expensive computational phase is the full training of the learning methodology. Indeed, its effectiveness depends on the choice of proper values for the socalled hyperparameters, namely the parameters that are not trained during the learning process, and such a selection typically requires an extensive numerical investigation with the execution of a significant number of experimental trials. The aim of the paper is to investigate how to choose the hyperparameters related to both the architecture of a Convolutional Neural Network (CNN), such as the number of filters and the kernel size at each convolutional layer, and the optimisation algorithm employed to train the CNN itself, such as the steplength, the minibatch size and the potential adoption of variance reduction techniques. The main contribution of the paper consists in introducing an automatic Machine Learning technique to set these hyperparameters in such a way that a measure of the CNN performance can be optimised. In particular, given a set of values for the hyperparameters, we propose a lowcost strategy to predict the performance of the corresponding CNN, based on its behavior after only few steps of the training process. To achieve this goal, we generate a dataset whose input samples are provided by a limited number of hyperparameter configurations together with the corresponding CNN measures of performance obtained with only few steps of the CNN training process, while the label of each input sample is the performance corresponding to a complete training of the CNN. Such dataset is used as training set for a Support Vector Machines for Regression and/or Random Forest techniques to predict the performance of the considered learning methodology, given its performance at the initial iterations of its learning process. Furthermore, by a probabilistic exploration of the hyperparameter space, we are able to find, at a quite low cost, the setting of a CNN hyperparameters which provides the optimal performance. The results of an extensive numerical experimentation, carried out on CNNs, together with the use of our performance predictor with NASBench101, highlight how the proposed methodology for the hyperparameter setting appears very promising.
2023
 Piecewise Constant Image Segmentation with a Deep Image Prior Approach
[Relazione in Atti di Convegno]
Benfenati, A.; Catozzi, A.; Franchini, G.; Porta, F.
abstract
Image segmentation is a key topic in image processing and computer vision and several approaches have been proposed in the literature to address it. The formulation of the image segmentation problem as the minimization of the MumfordShah energy has been one of the most commonly used techniques in the last past decades. More recently, deep learning methods have yielded a new generation of image segmentation models with remarkable performance. In this paper we propose an unsupervised deep learning approach for piecewise image segmentation based on the so called Deep Image Prior by parameterizing the MumfordShah functional in terms of the weights of a convolutional neural network. Several numerical experiments on both biomedical and natural images highlight the goodness of the suggested approach. The implicit regularization provided by the Deep Image Prior model allows to also consider noisy input images and to investigate the robustness of the proposed technique with respect to the level of noise.
2022
 Biomedical Image Classification via Dynamically Early Stopped Artificial Neural Network
[Articolo su rivista]
Franchini, Giorgia; Verucchi, Micaela; Catozzi, Ambra; Porta, Federica; Prato, Marco
abstract
It is well known that biomedical imaging analysis plays a crucial role in the healthcare sector and produces a huge quantity of data. These data can be exploited to study diseases and their evolution in a deeper way or to predict their onsets. In particular, image classification represents one of the main problems in the biomedical imaging context. Due to the data complexity, biomedical image classification can be carried out by trainable mathematical models, such as artificial neural networks. When employing a neural network, one of the main challenges is to determine the optimal duration of the training phase to achieve the best performance. This paper introduces a new adaptive early stopping technique to set the optimal training time based on dynamic selection strategies to fix the learning rate and the minibatch size of the stochastic gradient method exploited as the optimizer. The numerical experiments, carried out on different artificial neural networks for image classification, show that the developed adaptive early stopping procedure leads to the same literature performance while finalizing the training in fewer epochs. The numerical examples have been performed on the CIFAR100 dataset and on two distinct MedMNIST2D datasets which are the largescale lightweight benchmark for biomedical image classification.
2022
 On the FirstOrder Optimization Methods in Deep Image Prior
[Articolo su rivista]
Cascarano, P.; Franchini, G.; Porta, F.; Sebastiani, A.
abstract
Deep learning methods have stateofTheArt performances in many image restoration tasks. Their effectiveness is mostly related to the size of the dataset used for the training. Deep image prior (DIP) is an energyfunction framework which eliminates the dependency on the training set, by considering the structure of a neural network as an handcrafted prior offering high impedance to noise and low impedance to signal. In this paper, we analyze and compare the use of different optimization schemes inside the DIP framework for the denoising task.
2022
 On the convergence properties of scaled gradient projection methods with nonmonotone Armijo–like line searches
[Articolo su rivista]
Crisci, S.; Porta, F.; Ruggiero, V.; Zanni, L.
abstract
In order to solve constrained optimization problems on convex sets, the class of scaled gradient projection methods is often exploited in combination with nonmonotone Armijo–like line search strategies. These techniques are adopted for efficiently selecting the steplength parameter and can be realized by means of two different approaches: either the one along the arc or the one along the feasible directions. In this paper we deeply analyze the convergence properties of the scaled gradient projection methods equipped with the nonmonotone version of both these Armijo–like line searches. To the best of our knowledge, not all the convergence results proved for either the nonscaled or the monotone gradient projection algorithm have been also stated for the nonmonotone and scaled counterpart. The goal of this paper is to fill this gap of knowledge by detailing which hypotheses are needed to guarantee both the stationarity of the limit points and the convergence of the sequence generated by the nonmonotone scaled gradient projection schemes. Moreover, in the case of polyhedral constraint set, we discuss the identification of the active set at the solution for the sequence generated by the along the arc approach. Several numerical experiments on quadratic and nonquadratic optimization problems have been carried out in order to compare the behaviour of the considered scaled gradient projection methods.
2021
 Combining Weighted Total Variation and Deep Image Prior for natural and medical image restoration via ADMM
[Relazione in Atti di Convegno]
Cascarano, P.; Sebastiani, A.; Comes, M. C.; Franchini, G.; Porta, F.
abstract
In the last decades, unsupervised deep learning based methods have caught researchers' attention, since in many real applications, such as medical imaging, collecting a large amount of training examples is not always feasible. Moreover, the construction of a good training set is time consuming and hard because the selected data have to be enough representative for the task. In this paper, we focus on the Deep Image Prior (DIP) framework and we propose to combine it with a spacevariant Total Variation regularizer with an automatic estimation of the local regularization parameters. Differently from other existing approaches, we solve the arising minimization problem via the flexible Alternating Direction Method of Multipliers (ADMM). Furthermore, we provide a specific implementation also for the standard isotropic Total Variation. The promising performances of the proposed approach, in terms of PSNR and SSIM values, are addressed through several experiments on simulated as well as real natural and medical corrupted images.
2021
 Variable metric techniques for forward–backward methods in imaging
[Articolo su rivista]
Bonettini, S.; Porta, F.; Ruggiero, V.; Zanni, L.
abstract
Variable metric techniques are a crucial ingredient in many first order optimization algorithms. In practice, they consist in a rule for computing, at each iteration, a suitable symmetric, positive definite scaling matrix to be multiplied to the gradient vector. Besides quasiNewton BFGS techniques, which represented the stateoftheart since the 70's, new approaches have been proposed in the last decade in the framework of imaging problems expressed in variational form. Such recent approaches are appealing since they can be applied to large scale problems without adding significant computational costs and they produce an impressive improvement in the practical performances of first order methods. These scaling strategies are strictly connected to the shape of the specific objective function and constraints of the optimization problem they are applied to; therefore, they are able to effectively capture the problem features. On the other side, this strict problem dependence makes difficult extending the existing techniques to more general problems. Moreover, in spite of the experimental evidence of their practical effectiveness, their theoretical properties are not well understood. The aim of this paper is to investigate these issues; in particular, we develop a unified framework for scaling techniques, multiplicative algorithms and the Majorization–Minimization approach. With this inspiration, we propose a scaling matrix rule for variable metric first order methods applied to nonnegatively constrained problems exploiting a suitable structure of the objective function. Finally, we evaluate the effectiveness of the proposed approach on some image restoration problems.
2020
 A Limited Memory Gradient Projection Method for BoxConstrained Quadratic Optimization Problems
[Relazione in Atti di Convegno]
Crisci, S.; Porta, F.; Ruggiero, V.; Zanni, L.
abstract
Gradient Projection (GP) methods are a very popular tool to address boxconstrained quadratic problems thanks to their simple implementation and low computational cost per iteration with respect, for example, to Newton approaches. It is however possible to include, in GP schemes, some second order information about the problem by means of a clever choice of the steplength parameter which controls the decrease along the antigradient direction. Borrowing the analysis developed by Barzilai and Borwein (BB) for an unconstrained quadratic programming problem, in 2012 Roger Fletcher proposed a limited memory steepest descent (LMSD) method able to effectively sweep the spectrum of the Hessian matrix of the quadratic function to optimize. In this work we analyze how to extend the Fletcher’s steplength selection rule to GP methods employed to solve boxconstrained quadratic problems. Particularly, we suggest a way to take into account the lower and the upper bounds in the steplength definition, providing also a theoretical and numerical evaluation of our approach.
2020
 Spectral properties of barzilaiborwein rules in solving singly linearly constrained optimization problems subject to lower and upper bounds
[Articolo su rivista]
Crisci, S.; Porta, F.; Ruggiero, V.; Zanni, L.
abstract
In 1988, Barzilai and Borwein published a pioneering paper which opened the way to inexpensively accelerate firstorder. In more detail, in the framework of unconstrained optimization, Barzilai and Borwein developed two strategies to select the step length in gradient descent methods with the aim of encoding some secondorder information of the problem without computing and/or employing the Hessian matrix of the objective function. Starting from these ideas, several effcient step length techniques have been suggested in the last decades in order to make gradient descent methods more and also more appealing for problems which handle largescale data and require real time solutions. Typically, these new step length selection rules have been tuned in the quadratic unconstrained framework for sweeping the spectrum of the Hessian matrix, and then applied also to nonquadratic constrained problems, without any substantial modification, by showing them to be very effiective anyway. In this paper, we deeply analyze how, in quadratic and nonquadratic mini mization problems, the presence of a feasible region, expressed by a single linear equality constraint together with lower and upper bounds, inuences the spectral properties of the original BarzilaiBorwein (BB) rules, generalizing recent results provided for boxconstrained quadratic problems. This analysis gives rise to modified BB approaches able not only to capture secondorder informa tion but also to exploit the nature of the feasible region. We show the benefits gained by the new step length rules on a set of test problems arising also from machine learning and image processing applications.
2019
 Recent advances in variable metric firstorder methods
[Capitolo/Saggio]
Bonettini, S.; Porta, F.; Prato, M.; Rebegoldi, S.; Ruggiero, V.; Zanni, L.
abstract
Minimization problems often occur in modeling phenomena dealing with reallife applications that nowadays handle largescale data and require realtime solutions. For these reasons, among all possible iterative schemes, firstorder algorithms represent a powerful tool in solving such optimization problems since they admit a relatively simple implementation and avoid onerous computations during the iterations. On the other hand, a well known drawback of these methods is a possible poor convergence rate, especially showed when an high accurate solution is required. Consequently, the acceleration of firstorder approaches is a very discussed field which has experienced several efforts from many researchers in the last decades. The possibility of considering a variable underlying metric changing at each iteration and aimed to catch local properties of the starting problem has been proved to be effective in speeding up firstorder methods. In this work we deeply analyze a possible way to include a variable metric in firstorder methods for the minimization of a functional which can be expressed as the sum of a differentiable term and a nondifferentiable one. Particularly, the strategy discussed can be realized by means of a suitable sequence of symmetric and positive definite matrices belonging to a compact set, together with an Armijolike linesearch procedure to select the steplength along the descent direction ensuring a sufficient decrease of the objective function.
2018
 A nonsmooth regularization approach based on shearlets for Poisson noise removal in ROI tomography
[Articolo su rivista]
Bubba, Tatiana Alessandra; Porta, Federica; Zanghirati, Gaetano; Bonettini, Silvia
abstract
Due to its potential to lower exposure to Xray radiation and reduce the scanning time, regionofinterest (ROI) computed tomography (CT) is particularly appealing for a wide range of biomedical applications. To overcome the severe illposedness caused by the truncation of projection measurements, ad hoc strategies are required, since traditional CT reconstruction algorithms result in instability to noise, and may give inaccurate results for small ROI. To handle this difficulty, we propose a nonsmooth convex optimization model based on ℓ1 shearlet regularization, whose solution is addressed by means of the variable metric inexact line search algorithm (VMILA), a proximalgradient method that enables the inexact computation of the proximal point defining the descent direction. We compare the reconstruction performance of our strategy against a smooth total variation (sTV) approach, by using both Poisson noisy simulated data and real data from fanbeam CT geometry. The results show that, while for synthetic data both shearets and sTV perform well, for real data, the proposed nonsmooth shearletbased approach outperforms sTV, since the localization and directional properties of shearlets allow to detect finer structures of a textured image. Finally, our approach appears to be insensitive to the ROI size and location.
2018
 Serial and parallel approaches for image segmentation by numerical minimization of a secondorder functional
[Articolo su rivista]
Zanella, Riccardo; Porta, F.; Ruggiero, Valeria; Zanetti, M.
abstract
Because of its attractive features, second order segmentation has shown to be a promising tool in remote sensing. A known drawback about its implementation is computational complexity, above all for large set of data. Recently in Zanetti et al. [1], an efficient version of the blockcoordinate descent algorithm (BCDA) has been proposed for the minimization of a second order elliptic approximation of the Blake–Zissermann functional. Although the parallelization of linear algebra operations is expected to increase the performance of BCDA when addressing the segmentation of largesize gridded data (e.g., fullscene images or Digital Surface Models (DSMs)), numerical evidence shows that this is not sufficient to get significant reduction of computational time. Therefore a novel approach is proposed which exploits a decomposition technique of the image domain into tiles. The solution can be computed by applying BCDA on each tile in parallel way and combining the partial results corresponding to the different blocks of variables through a proper interconnection rule. We prove that this parallel method (OPARBCDA) generates a sequence of iterates which converges to a critical point of the functional on the level set devised by the starting point. Furthermore, we show that the parallel method can be efficiently implemented even in a commodity multicore CPU. Numerical results are provided to evaluate the efficiency of the parallel scheme on large images in terms of computational cost and its effectiveness with respect to the behavior on the tile junctions.
2017
 On the convergence of a linesearch based proximalgradient method for nonconvex optimization
[Articolo su rivista]
Bonettini, Silvia; Loris, Ignace; Porta, Federica; Prato, Marco; Rebegoldi, Simone
abstract
We consider a variable metric linesearch based proximal gradient method for the minimization of the sum of a smooth, possibly nonconvex function plus a convex, possibly nonsmooth term. We prove convergence of this iterative algorithm to a critical point if the objective function satisfies the Kurdyka Lojasiewicz property at each point of its domain, under the assumption that a limit point exists. The proposed method is applied to a wide collection of image processing problems and our numerical tests show that our algorithm results to be flexible, robust and competitive when compared to recently proposed approaches able to address the optimization problems arising in the considered applications.
2017
 Runge–Kuttalike scaling techniques for firstorder methods in convex optimization
[Articolo su rivista]
Porta, Federica; Cornelio, Anastasia; Ruggiero, Valeria
abstract
It is well known that there is a strong connection between time integration and convex optimization. In this work, inspired by the equivalence between the forward Euler scheme and the gradient descent method, we broaden our analysis to the family of Runge–Kutta methods and show that they enjoy a natural interpretation as firstorder optimization algorithms. The strategies intrinsically suggested by Runge–Kutta methods are exploited in order to detail novel proposal for either scaling or preconditioning gradientlike approaches, whose convergence is ensured by the stability condition for Runge–Kutta schemes. The theoretical analysis is supported by the numerical experiments carried out on some test problems arising from suitable applications where the proposed techniques can be efficiently employed.
2016
 A Variable Metric ForwardBackward Method with Extrapolation
[Articolo su rivista]
Bonettini, Silvia; Porta, Federica; Ruggiero, V.
abstract
Forwardbackward methods are a very useful tool for the minimization of a functional given by the sum of a differentiable term and a nondifferentiable one, and their investigation has comprised several efforts from many researchers in the last decade. In this paper we focus on the convex case and, inspired by recent approaches for accelerating firstorder iterative schemes, we develop a scaled inertial forwardbackward algorithm which is based on a metric changing at each iteration and on a suitable extrapolation step. Unlike standard forwardbackward methods with extrapolation, our scheme is able to handle functions whose domain is not the entire space. Both an O(1/k^2) convergence rate estimate on the objective function values and the convergence of the sequence of the iterates are proved. Numerical experiments on several test problems arising from image processing, compressed sensing, and statistical inference show the effectiveness of the proposed method in comparison to wellperforming stateoftheart algorithms.
2016
 On the constrained minimization of smooth Kurdyka– Lojasiewicz functions with the scaled gradient projection method
[Relazione in Atti di Convegno]
Prato, Marco; Bonettini, Silvia; Loris, Ignace; Porta, Federica; Rebegoldi, Simone
abstract
The scaled gradient projection (SGP) method is a firstorder optimization method applicable to the constrained minimization of smooth functions and exploiting a scaling matrix
multiplying the gradient and a variable steplength parameter to improve the convergence of the scheme. For a general nonconvex function, the limit points of the sequence generated by SGP have been proved to be stationary, while in the convex case and with some restrictions on the choice of the scaling matrix the sequence itself converges to a constrained minimum point. In this paper we extend these convergence results by showing that the SGP sequence converges to a limit point provided that the objective function satisfies the Kurdyka– Lojasiewicz property at each point of its domain and its gradient is Lipschitz continuous.
2016
 The ROI CT problem: a shearletbased regularization approach
[Relazione in Atti di Convegno]
Bubba, TATIANA ALESSANDRA; Porta, Federica; Zanghirati, Gaetano; Bonettini, Silvia
abstract
The possibility to significantly reduce the Xray radiation dose and shorten the scanning time is particularly appealing, especially for the medical imaging community. Regionofinterest Computed Tomography (ROI CT) has this potential and, for this reason, is currently receiving increasing attention. Due to the truncation of projection images, ROI CT is a rather challenging problem. Indeed, the ROI reconstruction problem is severely illposed in general and naive local reconstruction algorithms tend to be very unstable. To obtain a stable and reliable reconstruction, under suitable noise circumstances, we formulate the ROI CT problem as a convex optimization problem with a regularization term based on shearlets, and possibly nonsmooth. For the solution, we propose and analyze an iterative approach based on the variable metric inexact linesearch algorithm (VMILA). The reconstruction performance of VMILA is compared against different regularization conditions, in the case of fanbeam CT simulated data. The numerical tests show that our approach is insensitive to the location of the ROI and
remains very stable also when the ROI size is rather small.
2016
 Variable metric inexact linesearch based methods for nonsmooth optimization
[Articolo su rivista]
Bonettini, Silvia; Loris, Ignace; Porta, Federica; Prato, Marco
abstract
We develop a new proximalgradient method for minimizing the sum of a differentiable, possibly nonconvex, function plus a convex, possibly non differentiable, function. The key features of the proposed method are the definition of a suitable descent direction, based on the proximal operator associated to the convex part of the objective function, and an Armijolike rule to determine the step size along this direction ensuring the sufficient decrease of the objective function. In this frame, we especially address the possibility of adopting a metric which may change at each iteration and an inexact computation of the proximal point defining the descent direction. For the more general nonconvex case, we prove that all limit points of the iterates sequence are stationary, while for convex objective functions we prove the convergence of the whole sequence to a minimizer, under the assumption that a minimizer exists. In the latter case, assuming also that the gradient of the smooth part of the objective function is Lipschitz, we also give a convergence rate estimate, showing the O(1/k) complexity with respect to the function values. We also discuss verifiable sufficient conditions for the inexact proximal point and we present the results of two numerical tests on total variation based image restoration problems, showing that the proposed approach is competitive with other stateoftheart methods.
2015
 A convergent leastsquares regularized blind deconvolution approach
[Articolo su rivista]
Cornelio, Anastasia; Porta, Federica; Prato, Marco
abstract
The aim of this work is to present a new and efficient optimization method for the solution of blind deconvolution problems with data corrupted by Gaussian noise, which can be reformulated as a constrained minimization problem whose unknowns are the point spread function (PSF) of the acquisition system and the true image. The objective function we consider is the weighted sum of the leastsquares fittodata discrepancy and possible regularization terms accounting for specific features to be preserved in both the image and the PSF. The solution of the corresponding minimization problem is addressed by means of a proximal alternating linearized minimization (PALM) algorithm, in which the updating procedure is made up of one step of a gradient projection method along the arc and the choice of the parameter identifying the steplength in the descent direction is performed
automatically by exploiting the optimality conditions of the problem. The resulting approach is a particular case of a general scheme whose convergence to stationary points of the constrained minimization problem has been recently proved. The effectiveness of the iterative method is validated in several numerical simulations in image reconstruction problems.
2015
 A new steplength selection for scaled gradient methods with application to image deblurring
[Articolo su rivista]
Porta, Federica; Prato, Marco; Zanni, Luca
abstract
Gradient methods are frequently used in large scale image deblurring problems since they avoid the onerous computation of the Hessian matrix of the objective function. Second order information is typically sought by a clever choice of the steplength parameter defining the descent direction, as in the case of the wellknown Barzilai and Borwein rules. In a recent paper, a strategy for the steplength selection approximating the inverse of some eigenvalues of the Hessian matrix has been proposed for gradient methods applied to unconstrained minimization problems. In the quadratic case, this approach is based on a Lanczos process applied every m iterations to the matrix of the gradients computed in the previous m iterations, but the idea can be extended to a general objective function. In this paper we extend this rule to the case of scaled gradient projection methods applied
to constrained minimization problems, and we test the effectiveness of the proposed strategy in image deblurring problems in both the presence and the absence of an
explicit edgepreserving regularization term.
2015
 Limitedmemory scaled gradient projection methods for realtime image deconvolution in microscopy
[Articolo su rivista]
Porta, Federica; Zanella, R.; Zanghirati, G.; Zanni, Luca
abstract
Gradient projection methods have given rise to effective tools for image deconvolution in
several relevant areas, such as microscopy, medical imaging and astronomy. Due to the
large scale of the optimization problems arising in nowadays imaging applications and
to the growing request of realtime reconstructions, an interesting challenge to be faced
consists in designing new acceleration techniques for the gradient schemes, able to
preserve their simplicity and low computational cost of each iteration. In this work we propose
an acceleration strategy for a stateoftheart scaled gradient projection method for
image deconvolution in microscopy. The acceleration idea is derived by adapting a steplength
selection rule, recently introduced for limitedmemory steepest descent methods
in unconstrained optimization, to the special constrained optimization framework arising
in image reconstruction. We describe how important issues related to the generalization of
the steplength rule to the imaging optimization problem have been faced and we evaluate
the improvements due to the acceleration strategy by numerical experiments on
largescale image deconvolution problems.
2015
 On some steplength approaches for proximal algorithms
[Articolo su rivista]
Porta, Federica; Loris, Ignace
abstract
We discuss a number of novel steplength selection schemes for proximalbased convex optimization algorithms. In particular, we consider the problem where the Lipschitz constant of the gradient of the smooth part of the objective function is unknown. We generalize two optimization algorithms of Khobotov type and prove convergence. We also take into account possible inaccurate computation of the proximal operator of the nonsmooth part of the objective function. Secondly, we show convergence of an iterative algorithm with Armijotype steplength rule, and discuss its use with an approximate computation of the proximal operator. Numerical experiments show the efficiency of the methods in comparison to some existing schemes.
2013
 Filter factor analysis of scaled gradient methods for linear least squares
[Relazione in Atti di Convegno]
Porta, F.; Cornelio, A.; Zanni, L.; Prato, M.
abstract
A typical way to compute a meaningful solution of a linear least squares problem involves the introduction of a filter factors array, whose aim is to avoid noise amplification due to the presence of small singular values. Beyond the classical direct regularization approaches, iterative gradient methods can be thought as filtering methods, due to their typical capability to recover the desired components of the true solution at the first iterations. For an iterative method, regularization is achieved by stopping the procedure before the noise introduces artifacts, making the iteration number playing the role of the regularization parameter. In this paper we want to investigate the filtering and regularizing effects of some firstorder algorithms, showing in particular which benefits can be gained in recovering the filters of the true solution by means of a suitable scaling matrix.
2013
 On the filtering effect of iterative regularization algorithms for discrete inverse problems
[Articolo su rivista]
Cornelio, Anastasia; Porta, Federica; Prato, Marco; Zanni, Luca
abstract
Many realworld applications are addressed through a linear leastsquares
problem formulation, whose solution is calculated by means of an iterative approach. A huge amount of studies has been carried out in the optimization field to provide the fastest methods for the reconstruction of the solution, involving choices of adaptive parameters and scaling matrices. However, in presence of an illconditioned model and real data, the need of a regularized solution instead of the leastsquares one changed the point of view in favour of iterative algorithms able to combine a fast execution with a stable behaviour with respect to the restoration error. In this paper we analyze some classical and recent gradient approaches for the linear leastsquares problem by looking at their way of filtering the singular values, showing in particular the effects of scaling matrices and nonnegative constraints in recovering the correct filters of the solution. An original analysis of the filtering effect for the image deblurring problem with Gaussian noise on the data is also provided.