# THRESHOLDING ESTIMATORS FOR MINIMAX RESTORATION AND DECONVOLUTION. Jérôme Kalifa and Stéphane Mallat - PDF

Description
THRESHOLDIG ESTIMATORS FOR MIIMA RESTORATIO AD DECOVOLUTIO Jérôme Kalifa an Stéphane Mallat Centre e Mathématiques Appliquées Ecole Polytechnique, Plateau e Saclay 9118 Palaiseau Ceex, France ABSTRACT

View again

All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Information
Category:

## Shopping

Publish on:

Views: 15 | Pages: 5

Share
Transcript
THRESHOLDIG ESTIMATORS FOR MIIMA RESTORATIO AD DECOVOLUTIO Jérôme Kalifa an Stéphane Mallat Centre e Mathématiques Appliquées Ecole Polytechnique, Plateau e Saclay 9118 Palaiseau Ceex, France ABSTRACT Inverting the istortion of signals an images in presence of aitive noise is often numerically unstable. To solve these ill-pose inverse problems, we stuy linear an non-linear iagonal estimators in an orthogonal basis. General conitions are given to buil nearly minimax optimal estimators with a thresholing in an orthogonal basis. As an application, we stuy the econvolution of boune variation signals, with numerical results on the eblurring of satellite images. 1. ITRODUCTIO We consier a measurement evice that egraes a signal f of size with a linear operator U an as a Gaussian white noise W of variance ff. The measure signal Y is therefore relate to the original signal f following Y = Uf + W; (1) We suppose that U an ff have been calculate through a calibration proceure. Applying the inverse U 1 to Y yiels an equivalent enoising problem = U 1 Y = f + U 1 W = f + Z () The resulting noise Z is not white but remains Gaussian because U 1 is linear. Its covariance operator K is K = ff U 1;Λ U 1 ; (3) where A Λ is the ajoint of an operator A. When the inverse U 1 is not boune, the resulting noise Z = U 1 W is amplie by a factor that tens to innity. Fining an estimate ~ F of the signal f is an ill-pose inverse problem. To buil efcient estimators, we nee to introuce some prior information on our signals. A Bayes estimator supposes that we know the prior probability istribution of the signals to estimate an minimizes the average estimation error. However, it is rare that we know the probability istribution of complex signals such as natural images. The prior information often enes a set where the signals are guarantee to remain, without specifying their probability istribution in. Minimax estimation tries to minimize the maximum estimation error for all signals in. Donoho an Johnstone have obtaine general minimax optimality results to estimate signals contaminate by white Gaussian noise with thresholing estimators in orthogonal bases [DJ94]. To obtain similar results when estimating signals contaminate by non white noises, one nees to aapt the basis to the covariance properties of the noise. Section shows that thresholing estimators are quasiminimax optimal if the basis nearly iagonalizes the covariance of the noise an if it concentrates the energy of the signal on a few coefcients. As an application, we stuy in section 3 the econvolution of boune variation signals, with an application to the eblurring of satellite images.. MIIMA ESTIMATIO I GAUSSIA OISE We consier the generic inverse problem of equation (1), equivalent to the estimation of a signal f contaminate by an aitive Gaussian noise Z = U 1 W = f + Z The ranom vector Z is characterize by its covariance operator K, an we suppose that EfZ[n]g =0. The risk of an estimation ~ F = D is r(d; f) =EfkD fk g The expecte risk over a set cannot be compute because we o not know the probability istribution of signals in. To control the risk for any f, we try to minimize the maximum risk r(d; ) = sup EfkD fk g f Let O n be the set of all linear an non-linear operators from C to C.Theminimax risk is the lower boun compute over all operators D r n( ) = inf r(d; ) DOn In practice, we must n D that is simple to implement an such that r(d; ) is close to the minimax risk r n( ). As a rst step, one can simplify this problem by restricting D to be a linear operator. Let O l be the set of all linear operators from C to C.The linear minimax risk over is the lower boun r l ( ) = inf r(d; ) DOl We shall see when this strategy is efcient, i.e. when r l ( ) is of the same orer as r n( ). .1. Diagonal Estimation When the aitive noise is white, Donoho an Johnstone [DJ94] prove that non linear iagonal estimators in an orthonormal basis B = fg mg 0»m are nearly minimax optimal if the basis provies a sparse signal representation, which means that the basis concentrates the energy of the signal on a few coefcients. When the noise is not white, the coefcients of the noise have a variance that epens upon each g m ff m = EfjZ B[m]j g = hkg m;g mi The basis choice must therefore epen on the covariance K. We stuy the risk of estimators that are iagonal in B ~F = D = m( B[m]) g m (4) If m( B[m]) = a[m] B[m], one can verify that the minimum risk Efk ~ F fk g is achieve by the following attenuation an a[m] = Efk ~ F fk g = r inf (f) = jf B[m]j ; (5) jf B[m]j + ffm ff m jf B[m]j ff m + jf B[m]j (6) Over a signal set, the maximum risk of this attenuation is r inf ( ) = sup f r inf (f). The attenuation (5) is calle an oracle attenuation because it uses information normally not available, as a[m] epens upon jf B[m]j which is not known in practice. The risk r inf ( ) is thus only a lower boun for the minimax risk of iagonal estimators. We shall see that a simple thresholing estimator has a maximum risk that is close to r inf ( ). A thresholing estimator is ene by ~F = D = ρ T m( B[m]) g m ; (7) where ρ T m(x) is for example a har thresholing function ρ T m(x) =ρ x if jxj Tm 0 if jxj»t m ; (8) The risk of this thresholing estimator is r t(f) =r(d; f) = Efjf B[m] ρ T m( B[m])j g Donoho an Johnstone stuie thresholing estimators when T m = ff m p loge. If the signals belong to a set, the threshol values are improve by consiering the maximum of signal coefcients s B[m] = sup f jf B[m]j; ifs B[m]» ff m then setting B[m] to zero yiels a risk jf B[m]j that is always smaller than the risk ff m of keeping it. This is one by choosing T m = 1 to guarantee that ρ T m( B[m]) = 0. Threshols are thus ene by We shall stuy in which case thresholing estimators are close to minimax optimality, an compare them with linear estimators. To analyse the properties of linear an non-linear estimators, we introuce orthosymmetric sets. is orthosymmetric in B if for any f an for any a[m] with ja[m]j»1 then a[m] f B[m] g m This means that the set is elongate along the irections of the vectors g m of B. The linear vs non-linear iagonal estimation issue epens on the size of the orthosymmetric set as compare to its quaratic convex hull, ene as following The square of a set in the basis B is ene by ( ) B = f ~ f ~ f = jf B[m]j g m with f g (10) We say that is quaratically convex in B if ( ) B is a convex set. The quaratic convex hull QH[ ] of in the basis B is ene by QH[ ] = n f jf B[m]j is in the convex hull of ( ) B o (11) It is the largest set whose square (QH[ ]) B is equal to the convex hull of ( ) B... early Diagonal Covariance Donoho an Johnstone [DJ94] obtaine minimax estimation results on non linear thresholing estimators when the aitive noise is white. To obtain similar results when the noise Z is not white, we nee to n a basis B that transforms the noise into nearly inepenent coefcients. This approach was stuie by Donoho for some specic econvolution problems where wavelet bases are aapte [Don95], which is not the case for hyperbolic econvolution such as eblurring in section 3. We give more general conitions on the orthogonal basis B to obtain nearly minimax thresholing estimators [KM99]. Since the noise Z is Gaussian, the coefcients Z B[m] are nearly inepenent is they are nearly uncorrelate, which means that its covariance K is nearly iagonal in B. This approximate iagonalization is measure by preconitioning K with its iagonal. We enote by K the iagonal operator in the basis B, whose iagonal is equal to the iagonal of K. The iagonal coefcients of K an K are thus ffm = EfjZ B[m]j g.letk 1 be the inverse of K, an K 1= be the iagonal matrix whose coefcients are ff m. Theorem 1 computes lower an upper bouns of the minimax risks with a conitioning factor ene with the operator sup norm k k S. Theorem 1 The conitioning factor satises B = kk 1= K 1 K 1= k S 1 If is orthosymmetric in B then T m = ρ ffm p loge if ff m s B[m] 1 if ff m s B[m] (9) 1 B r inf (QH[ ])» r l ( )» r inf (QH[ ]) (1) an 1 r inf ( )» r n( )» r t( )» ( log 15 e +1) μff + r inf ( ) B (13) One can verify that B =1ifan only if K = K an is thus iagonal in B. The closer B is to 1 the more iagonal K. The main ifculty is to n a basis B which nearly iagonalizes the covariance of the noise an provies sparse signal representations so that is orthosymmetric or can be embee in two close orthosymmetric sets. If the basis B nearly iagonalizes K so that B is of the orer of 1 then r l ( ) is of the orer of r inf (QH[ ]), whereas r n( ) an r t( ) are of the orer of r inf ( ).If is quaratically convex then =QH[ ] so the linear an non-linear minimax risks are close. Otherwise its quaratic hull QH[ ] may be much bigger than. When is strongly elongate in the irections of the basis vectors g m, a thresholing estimation in B may signicantly outperform an optimal linear estimation. convolution. In the iscrete Fourier basis, the oracle risk (6) is rewritten ffk 1 j r ^f [k]j inf (f) = ffk + 1 j ^f [k]j (15) k=0 We enote by QH[ ] the quaratic convex hull of in the iscrete Fourier basis. Theorem Let be a translation invariant set. The minimax linear risk for estimating f from = f + Z is reache by circular convolutions an r l ( ) = r inf (QH[ ]) (16) If is close an boune, then there exists x QH[ ] such that r inf (x) =r inf (QH[ ]). One can verify that the minimax linear estimator is ~ F = DY = ~ Y, with 3. DECOVOLUTIO The restoration of signals egrae by a convolution operator U is a generic inverse problem that is often encountere in signal processing. The convolution is suppose to be circular to avoi borer problems. The goal is to estimate f from Y = f ~ u + W The circular convolution is iagonal in the iscrete Fourier basis B = fg k [n]g 0»k .TheinverseofU is U 1 f = f ~u 1 where the iscrete Fourier transform of u 1 is u 1 [k] = 1. The econvolve ata bu[k] is = U 1 Y = Y ~ u 1 = f + Z The noise Z = U 1 W is circular stationary. Its covariance K is a circular convolution with ff u 1 ~ u 1,whereu 1 [n] = u 1 [ n]. The Karhunen-Loève basis which iagonalizes K is therefore the iscrete Fourier basis B. The eigenvalues of K are ff k = ff j^u[k]j.when^u[k] =0we formally set ff k = 1. When the convolution lter is a low-pass lter with a zero at high frequency, the econvolution problem is highly unstable. Suppose that the iscrete Fourier transform ^u[k] has a zero of orer p 1 at the highest frequency k = ±= j^u[k]j ο k p 1 (14) The noise variance ff k has a hyperbolic growth when the frequency k is in the neighborhoo of ±=. This is calle a hyperbolic econvolution problem of egree p Linear Deconvolution In many econvolution problems the set is translation invariant, which means that if b then any translation of b moulo also belongs to. Since the amplie noise Z is circular stationary the whole estimation problem is translation invariant. In this case, the linear estimator that achieves the minimax linear risk is iagonal in the iscrete Fourier basis. It is therefore a circular ^[k] = 1 j^x[k]j ^u Λ [k] ff + 1 j^x[k]j j^u[k]j (17) If ffk = ff j^u[k]j 1 j^x[k]j then ^[k] ß ^u 1 [k], butif ffk fl 1 j^x[k]j then ^[k] ß 0. The lter is thus a regularize inverse of u. The total variation of a iscrete signal f of size is ene with kfk V = n=0 jf[n] f[n 1]j (18) The total variation measures the amplitue of all signal oscillations an is well suite to moel the spatial inhomogeneity of piecewise regular signals. Boune variation signals may inclue sharp transitions such as iscontinuities. A set V of boune variation signals of perio is ene by V = ( f kfk V = n=0 f[n] f[n 1]» C Theorem can be applie to the set V which is inee translation invariant [KM99]. Theorem 3 For a hyperbolic econvolution of egree p, if1» C=ff» then r l ( V ) ff ο ) (p 1)=p C (19) 1= ff For a constant signal to noise ratio C =( ff ) ο 1, (19) implies that r l ( V ) ff ο 1 (0) Despite the fact that ff ecreases an increases the normalize linear minimax risk remains of the orer of 1. 3.. Thresholing Deconvolution An efcient thresholing estimator is implemente in a basis B which enes a sparse representation of signals in V an which nearly iagonalizes K. The covariance operator K is iagonalize in the iscrete Fourier basis an its eigenvalues are ff k = ff j^u[k]j ο k p ff 1 (1) The iscrete Fourier basis is not appropriate for the thresholing algorithm because it oes not approximate efciently boune variation signals. Perioic wavelet bases provie efcient approximations of boune variation signals, but a wavelet basis fails to approximatively iagonalize K. The iscrete Fourier transforms of these wavelets have an energy mostly concentrate on yaic intervals, as illustrate by Figure 1. On all scales but the nest, (1) shows that the eigenvalues ff k remain of the orer of ff. These wavelets are therefore approximate eigenvectors of K. At the nest scale, the wavelets have an energy mainly concentrate in the higher frequency ban [=4;=], whereff k varies by a huge factor of the orer of r. To construct a basis of approximate eigenvectors of K, the nest scale wavelets must be replace by vectors that have a Fourier transform concentrate in subintervals of [=4;=] where ff k varies by a factor that oes not grow with. We replace the nest scale wavelets by wavelet packets [Wic94] whose iscrete Fourier transform support ecrease exponentially at high frequencies, while keeping a small spatial support (an hence the largest possible frequency support) to efciently approximate piecewise regular signals. The optimal traeoff is obtaine by particular wavelet packets illustrate in gure 1, calle mirror wavelets because of their frequency istribution symmetric with respect to wavelets. More etails can be foun in [KM99]. To prove that the covariance K is almost iagonalize in B for Theorem 4 Let B a mirror wavelet basis constructe with a conjugate mirror lter that enes a wavelet that is C q with q vanishing moments. For a hyperbolic econvolution of egree p q, if 1» C=ff» p+ 1 then r n( V ) ο rt( V ) ο ff ff 4p=(p+1) C ff (loge ) 1=(p+1) () This theorem proves that a thresholing estimator in a mirror wavelet basis yiels a quasi-minimax econvolution estimator for boune variation signals. If we suppose that the signal to noise ratio C =(ff ) ο 1 then r n( V ) ο rt( V ) ο ff ff 1=(p+1) loge (3) As oppose to the normalize linear minimax risk (0) which remains of the orer of 1, the thresholing risk in a mirror wavelet basis converges to zero as increases. The larger the number p of zeros of the low-pass lter ^u[k] at k = ±= the slower the risk ecay Deconvolution of Satellite Images early optimal econvolution of boune variation images can be calculate with a separable extension of the econvolution estimator in a mirror wavelet basis. Such a restoration algorithm is use by the French Spatial Agency (CES) for the prouction of satellite images. The satellite movement an the imperfection of the optics prouces a blur, to which is ae a Gaussian white noise ue to the electronic of the photoreceptors. A calibration proceure measures the impulse response u of the blur an the noise variance ff. The image (b) is a simulate satellite image provie by the CES, which is calculate from an airplane image shown in Figure (a). The impulse response is a separable low-pass lter Uf[n 1;n ]=f ~ u[n 1;n ] with u[n 1;n ]=u 1[n 1] u [n ] σ 0 /4 / FIG. 1 Frequency ecomposition inuce by a mirror wavelet basis. The variance ff k of the noise has a hyperbolic growth but varies by a boune factor on the frequency support of each mirror wavelet. all, the asymptotic behavior of the iscrete wavelets an mirror wavelets must be controlle. The following theorem thus supposes that these wavelets an wavelet packets are constructe with a conjugate mirror lter which yiels a continuous time wavelet that has q pvanishing moments an which is C q. The near iagonalization is verie to prove that a thresholing estimator in a mirror wavelet basis has a risk whose ecay is equivalent to the non-linear minimax risk. σ k k The iscrete Fourier transform of u 1 an u have respectively a zero of orer p 1 an p at ±= ^u 1[k 1] ο k1 1 p1 an ^u [k ] ο k 1 p Most satellite images are well moele by boune variation images. For a square iscrete image of pixels, the total variation is ene by kfk V = 1 n 1 =0 n =0 f[n1;n ] f[n 1 1;n ] f[n 1;n ] f[n 1;n 1] + 1 We say that an image has a boune variation if kfk V is boune by a constant inepenent of the resolution. Let V be the set of images that have a total variation boune by C V = n f kfk V» C o Boune variation plays an important role in image processing, where its value epens on the length of the image level sets. The econvolve noise has a covariance K that is iagonalize in a two-imensional iscrete Fourier basis. The eigenvalues are ff k 1 ;k = ff j^u 1[k 1]j j^u [k ]j ο ff k 1 1 p1 k 1 p (4) The main ifculty is again to n an orthonormal basis which provies a sparse representation of boune variation images an which nearly iagonalizes the noise covariance K. Each vector of such a basis shoul have a Fourier transform whose energy is concentrate in a frequency omain where the eigenvectors ffk 1 ;k vary at most by a constant factor. Rougé [Rou97] has emonstrate numerically that efcient econvolution estimations can be performe with a thresholing in a wavelet packet basis. This algorithm is inspire by his approach although the chosen basis is ifferent. At low frequencies (k 1;k ) [0;=4] the eigenvalues remain approximatively constant ffk 1 ;k ο ff. This frequency square can be covere with a separable iscrete wavelet basis. The remaining high frequency annulus is covere by two-imensional mirror wavelets that are separable proucts of two one-imensional mirror wavelets. One can verify that the union of these two families ene an orthonormal basis of images of pixels. This twoimensional mirror wavelet basis is an anisotropic wavelet packet basis, in which ecomposing a signal with a lter bank requires O( ) operations [Wic94]. One can prove that there exists such that kk 1= K 1 K 1= k S». A thresholing estimator in B has a risk r t( V ) close to the non-linear minimax risk r n( V ) an that converges to zero as increases, whereas a linear minimax estimator oes not reuce the original noise energy ff by more than a constant. Theorem 5 For a separable hyperbolic econvolution of egree p = max(p 1;p ) 3=, ifc =( ff ) ο 1 then r l ( V ) rn( V ) ο 1 an ο rt( V ) ο ff ff ff loge 1 p+1 Figure (c) shows an example of econvolution calculate in the mirror wavelet basis. This can be compare with the linear estimation in Figure (), calculate with a circular convolution estimator whose maximum risk over boune variation images is close to the minimax linear risk. The linear econvolution sharpens the image but leaves a visible noise in the regular parts of the image. The thresholing algorithm removes completely the noise in these regions while improving the restoration of eges an oscillatory parts. This algorithm was chosen among several competing algorithms by photointerpreters of the French spatial agency (CES) to perform the econvolution of satellite images, an it is now integrate in the CES satellite image acquisition channel. 4. COCLUSIO We have built a theoretical framework for minimax optimal restoration of signals an images in the case of ill-pose inverse problems. One can perform an optimal restoration if one can n an orthogonal basis which can both compress the signal to estimate on a few coefcients an nearly iagonalize the covariance of the non-white Gaussian noise obtaine after applying the inverse of the egraation operator. The use of this approach to solve hyperbolic econvolution of signals an images leas to the creation of mirror wavelet bases in which a simple thresholing proceure on (a) (c) FIG. (a) Original airplane image. (b) Simulation of a satellite image (SR = 31.1b). (c) Deconvolution with a thresholing in a mirror wavelet basis (34.1b). () early minimax optimal lin
Related Search
Similar documents

View more...