Wavelet deconvolution in a periodic setting

Abstract

Summary. Deconvolution problems are naturally represented in the Fourier domain, whereas thresholding in wavelet bases is known to have broad adaptivity properties. We study a method which combines both fast Fourier and fast wavelet transforms and can recover a blurred function observed in white noise with O{n  log (n)²} steps. In the periodic setting, the method applies to most deconvolution problems, including certain ‘boxcar’ kernels, which are important as a model of motion blur, but having poor Fourier characteristics. Asymptotic theory informs the choice of tuning parameters and yields adaptivity properties for the method over a wide class of measures of error and classes of function. The method is tested on simulated light detection and ranging data suggested by underwater remote sensing. Both visual and numerical results show an improvement over competing approaches. Finally, the theory behind our estimation paradigm gives a complete characterization of the ‘maxiset’ of the method: the set of functions where the method attains a near optimal rate of convergence for a variety of L^p loss functions.

1. Deconvolution in white noise

Suppose that we observe the random process Y_n(·),

(1)

where σ is a positive constant, W(·) is Gaussian white noise and

(2)

Our goal is to recover the unknown function f from the noisy blurred observations (1). The blurring function g in convolution (2) is assumed to be known. Further, we assume that the function f is periodic on the unit interval T and that g has a certain degree of smoothness. There is an extensive statistical literature on deconvolution problems; in particular wavelet methods have received considerable attention over the last decade. References that are particularly relevant to the present work include Donoho (1995), Abramovich and Silverman (1998), Pensky and Vidakovic (1999), Fan and Koo (2002), Kalifa and Mallat (2003) and Neelamani et al. (2003): these works in turn contain further references to previous literature.

An important application setting that is modelled by expression (1) is that of motion blur in signals or images; see for example Bertero and Boccacci (1998). Here g is taken as a ‘boxcar’g(x)=(2a)⁻¹ 𝕀_[−a,a](x), of half‐width a. Owing to oscillations in the Fourier coefficients of g, this situation escapes the assumptions of much wavelet literature, but recent work of Neelamani et al. (2004) studied it explicitly with their ForWaRD algorithm.

Our aim in this paper is to study a wavelet deconvolution algorithm which can be applied to many deconvolution problems including certain cases of boxcar blur. We are particularly interested in obtaining adaptivity properties relative to a variety of error measures and function classes. Our theoretical investigation is conducted by using model (1), but examples and software are provided for data sampled at a discrete set of n equally spaced points.

For ordinary smooth convolution where the Fourier coefficients of g decay in a polynomial fashion, |g_l|∼C|l|^−ν, our proposal can recover the unknown function f with an accuracy of order

(3)

performance being measured in an integrated L^p‐metric, for any p>1. Here n denotes the usual sample size and s plays the role of a smoothness index for our target function f (taken in a large class which includes spatially inhomogeneous functions). For boxcar blur, we show that rate (3) holds with , provided that the boxcar width is ‘badly approximated’ by rational numbers. This notion is recalled in Section 2.2; it includes boxcars of width that is given by quadratic irrational numbers such as √5; see also remark 8 following proposition 2.

Our rate result (3) is established for a class of convolution operators satisfying a decay condition when averaged over dyadic Fourier blocks. Thus, if (g_l) denote the Fourier coefficients of g, we suppose that for some fixed r>0

(condition C). (The notation a_j≅b_j means that there are constants c₀ and c₁ such that, for all j, c₀a_j/b_jc₁.) Condition C typically holds for ordinary ‘smooth’ convolution and also covers certain oscillatory cases, such as arise with irrational boxcar blur; see for example Fig. 2 in Section 2.1 and proposition 2. It does not include ‘supersmooth’ kernels, such as the Gaussian kernel, with exponential or faster Fourier decay.

Figure 2

(a) Log‐spectrum of a Γ(1, 0.0065) probability distribution function (smooth blur) and (b) log‐spectrum of a boxcar function g(x)=(1/2a) 𝕀_[−a,a](x) with a=1/√353 (boxcar blur)

For both theoretical and practical convenience, we use band‐limited wavelet basis functions, and in particular the (periodized) Meyer wavelet basis for which fast algorithms are available; Kolaczyk (1994) and Donoho and Raimondo (2004). Our method can thus perform deconvolution in O{n pt log (n)²} steps. The WaveD software package that was used to prepare most figures and tables in this paper is available at http://www.usyd.edu.au:8000/u/marcr/. It is intended for use with Wavelab; see Buckheit et al. (1995).

We begin in Section 2 by describing an application of statistical deconvolution to remote sensing. There follow short reviews of continued fractions, periodized Meyer wavelets, Besov spaces and wavelet shrinkage. Section 3 describes our method specifically, and its relationship with the wavelet–vaguelette approach of Donoho (1995) and Abramovich and Silverman (1998). Section 4 is concerned with numerical performance and competing approaches. In the implementation of our method the choice of tuning parameters is informed by asymptotic minimax theory: this is discussed in Section 5. Proofs are summarized in Appendices A and B.

2. Motivations and preliminaries

2.1. Illustration from remote sensing

Deconvolution is a common problem in many areas of signal and image processing; see for example Jain (1989). Here we shall focus on light detection and ranging (LIDAR) remote sensing as in Je Park et al. (1997) and Harsdorf and Reuter (2000). LIDAR uses a laser device which emits pulses, reflections of which are gathered by a telescope that is aligned with the laser. The return signal is used to determine the distance and position of the reflecting material. Accordingly, the distance resolution is limited by the time resolution of the LIDAR instrument. If the system response function of the LIDAR detector is longer than the time resolution interval the measured LIDAR signal is blurred and the effective accuracy of the LIDAR decreases. This loss of precision can be corrected by deconvolution. In practice, measured LIDAR signals are corrupted by additional noise which renders direct deconvolution impossible. Borrowed from Harsdorf and Reuter (2000), we have depicted an ideal LIDAR signal in Fig. 1; this will be our target function f for numerical illustrations throughout this paper. The system response function of the LIDAR detector (denoted g(t) in expression (1)) is calibrated a posteriori once the LIDAR instrument has been built. We follow Harsdorf and Reuter (2000) and use system response functions that have a strong low pass characteristic. In the WaveD software that was used to plot most figures in this paper, the system response function parameters can be changed by the user to accommodate different calibration settings. In Fig. 2, we illustrate a smooth blur and a boxcar blur scenario in the Fourier domain. These two examples of system response function shapes illustrate the possibilities that are offered by our assumption C, under which near optimal rates are achievable. Finally, Fig. 3 shows artificial LIDAR data for a combination of different noise levels and system response functions.

Figure 1

Ideal LIDAR signal as in Harsdorf and Reuter (2000), corresponding to data for underwater LIDAR

Figure 3

Simulated LIDAR signals (1) with t_i = i/n, n = 2048 , corresponding to the system response func‐ tions of Fig. 2: (a) smooth blur with low (standard deviation sd = 0.05) noise level; (b) smooth blur with medium (sd = 0.5) noise level; (c) smooth blur with high (sd = 1) noise level; (d) boxcar blur with low (sd = 0.05) noise level; (e) boxcar blur with medium (sd = 0.5) noise level; (f) boxcar blur with high (sd = 1) noise level

2.2. Boxcar blur and the continued fractions algorithm

A boxcar function is the indicator of an interval g(x) = (1/2a) 𝕀_[−a,a](x) where the parameter a indicates a preferred spatial scale. The Fourier coefficients of such a boxcar are given by

(4)

The convolution problem (1) that is associated with the boxcar, later referred to as boxcar blur, has the problem that, for rationala=p/q, the coefficients g_k vanish for any integer k multiple of q. Hence, even without noise some frequencies are lost and f cannot be fully recovered. The problem is less severe for irrational numbers, and particularly for those which are ‘badly approximable’ (BA) by rational numbers. We briefly review the key tool in constructing such numbers.

2.2.1. The continued fractions algorithm

Let a₀ be an integer and a₁,a₂,… be strictly positive integers. Define sequences (p_k) and (q_k) recursively by p₀/q₀ = a₀, p₁/q₁ = a₀ + 1/a₁ and p₂/q₂ = a₀ + 1/(a₁ + 1/a₂), and for n  2 let

(5)

The sequence of rational numbers (p_k/q_k) that is constructed in this way has very special properties, the first of which being that

(6)

In fact, any real number a that is not an integer may be uniquely determined by its continued fraction expansion. To every real number corresponds a unique sequence (a_k) and vice versa:

(7)

For rational numbers the expansion stops as in equation (6) and a_k=0 for later k, whereas for irrational number a the sequence of (a_k),a_k>0, is infinite. The rational numbers (p_k(a)/q_k(a)) defined at expression (5) are called the convergents of a. For any irrational number a, the convergents have the property of best approximation: for n1,

(8)

where ║x║ denotes the distance from x ∈ ℝ to the nearest integer. The study of such Diophantine approximations plays a central role in our analysis of the boxcar blur, since from equation (4)

(9)

We recall some basic properties, referring to Lang (1966) and Khinchin (1992) for further details. The quality of best approximation satisfies

(10)

• (a)

  The denominators q_n grow at least geometrically:

  (11)

• (b)

  For all n0,

  

Hence, the size of the elements in the continued fractions algorithm (5) determines the quality of best rational approximation to a. It is customary to define families of irrational numbers a according to the size of their elements as follows.

Definition 1. An irrational number a is called BA if

Definition 2. A rational number a is called BA of order n if a is the convergent of order k of a BA number (a=p_k/q_k) and if q_k−1n<q_k.

The set of all BAs contains quadratic irrational numbers (e.g. √5). For the boxcar blur, we prove (proposition 2 in Section 4) that condition C holds with for any scale a chosen in the set of BAs. In the finite sample implementation (of size n) of model (1), our method will remain numerically stable for any scale a that is chosen in the set of BA rational numbers of order at least n (see remark 8 below proposition 2) and satisfying a uniform bound (in a) on  sup _n{a_n(a)}. We refer to Johnstone and Raimondo (2002) for a discussion of cases outside the BA numbers.

2.3. Periodized Meyer wavelet transforms

Let (φ,ψ) denote the Meyer scaling and wavelet function; see Meyer (1992) or Mallat (1999). As usual,

(12)

is the dilated and translated wavelet at resolution level j and time position k/2^j; here and below κ denotes the bivariate index (j,k). The functions φ are defined similarly. Such wavelet functions define a multiresolution analysis of L²(ℝ); for any f ∈ L²(ℝ) the following expansion holds:

(13)

where

(14)

are the wavelet coefficients of f. Quite naturally, we can define a similar multiresolution analysis for periodic functions in L²(T), T=[0,1]. This is done by periodizing the basis functions

(15)

Here and in the rest of the paper (Φ,Ψ) will denote the periodized Meyer scaling and wavelet functions (Fig. 4). Thus, for any periodic function f an expansion that is similar to equation (13) holds with periodized basis functions (Φ,Ψ) and bivariate index κ restricted to the set I={(j,k):j0 and k=0,1,…,2^j−1}. We use this basis for the following reasons.

Figure 4

Periodized Meyer scaling and wavelet function: (a) Φ_3,4; (b) Ψ_4,5

• (a)

  The Meyer wavelet is band limited. In particular, we have , where denotes the Fourier transform of ψ.

• (b)

  An efficient algorithm, due to Kolaczyk (1994), is available to compute the periodized Meyer wavelet transforms. It requires only O{n pt log (n)²} steps to derive an empirical version of the coefficients (14) from a sample of size n of f.

Band‐limited wavelets have been used in the deconvolution setting by Walter and Shen (1999), Shen and Walter (2002), Pensky and Vidakovic (1999) and Fan and Koo (2002). General information on band‐limited wavelet bases may be found, for example, in Mallat (1999), Hernàndez and Weiss (1996) and Walter (1994).

2.4. A wide class of target functions

Let us first introduce the standard Besov spaces of periodic functions and r1. For this, define for every measurable function f

then, recursively, and similarly for positive integer N. Let

Then, for N>s, we define

(with the usual modifications for r=∞ or π=∞).

In what follows, we shall also write Ψ₋₁ = Φ. In this setting, recall that the Besov spaces are characterized by the behaviour of the wavelet coefficients (as soon as the wavelet is periodic and has enough smoothness and vanishing moments). In particular, for f ∈ L^π(T),

(16)

The Besov spaces have proved to be an interesting scale for studying the properties of statistical procedures. The index s indicates the degree of smoothness of the function. Owing to the differential averaging effects of the integration parameters π and r, the Besov spaces capture a variety of smoothness features in a function including spatially inhomogeneous behaviour; see Donoho et al. (1995).

2.5. Wavelet shrinkage

Wavelet shrinkage is now a well‐established statistical procedure for nonparametric estimation. A wavelet estimator of an unknown function f ∈ L²(T), based on hard thresholding, is given by

(17)

where and are estimated wavelet coefficients and I₀ and I₁ are sets of indices. I₀ ={(j₀,k):k=0,1,…,2^j0−1} corresponds to a coarse resolution level j₀ and I₁={(j,k):k=0,1,…,2^j−1,j₀jj₁} indexes details up to a fine resolution level j₁. The procedure (17) is non‐linear since only large coefficients are kept; here λ_j is a threshold parameter. The choices of parameters j₀, j₁ and λ_j as well as estimators and depend on the problem at hand. For deconvolution problem (1) this will be discussed in the next section.

3. Wavelet deconvolution in the Fourier domain

3.1. Inverse estimation paradigm

Since the Fourier transform interchanges convolution and multiplication, it is natural to employ Fourier representations for the deconvoluation problem. Let e_l(t) = exp (2πilt), l  ∈  ℤ, and write f_l=〈f,e_l〉 and g_l=〈g,e_l〉 for the Fourier coefficients of f and g respectively where . Letting h=f * g we have

(18)

For the (real‐valued) random processes Y_n and W we write, with a slight abuse of notation, y_l=〈e_l,Y_n〉=∫e_l dY_n and z_l=〈e_l,W〉=∫e_l dW. Calculating Fourier coefficients in expression (1):

(19)

where z_l are zero‐mean Gaussian random variables. We denote by the Fourier coefficients of Ψ, i.e. . Combining equation (18) with Plancherel's identity we obtain

(20)

Noting that , we can recover wavelet coefficients

(21)

Here g_l and are known Fourier coefficients but the h_ls are not directly observable; in equation (19) we take y_l as an (unbiased) estimator of h_l and let

(22)

be our estimator of β which can be computed from the observations (1). Of course, an estimator of α is defined in a similar fashion with Φ in place of Ψ.

Let —it is easily seen that this set does not depend on k. Indeed, from the compact support of the Meyer wavelet, we have

3.2. The wavelet deconvolution method

For deconvolution problems (1) we shall use wavelet‐based estimator (17) with coefficients (22). Estimator (17) requires three input parameters: j₀,j₁ and λ. The coarse scale has the default value j₀=3 in software and is not important in the asymptotic theory. To specify the more critical thresholds λ_j and finest scale j₁, set

(23)

where |C_j| denotes the cardinality of C_j. Then, for the thresholds,

(24)

where the default value of η is √2 in software—this is discussed further below. Finally, using the decay parameter ν from assumption C, the finest scale j₁ is determined from

(25)

If it is necessary to compute an estimate of the noise standard deviation σ, we adapt the method of Donoho et al. (1995) that was developed for direct data. If y_J,k = 〈Y_n,Ψ_J,k〉 denote the finest scale wavelet coefficients of the observed data, then , where mad is the median absolute deviation.

We summarize the main steps of our wavelet deconvolution method, and we illustrate it in Fig. 5. Here and in the rest of the paper we refer to this as the WaveD method.

Figure 5

WaveD method applied to the LIDAR signals of Fig. 3 (smooth blur) (the threshold values are summarized in Table 1; levels 4–7 are shown as 100% thresholding occurs at level 8 or below): (a) low noise, estimated wavelet coefficients (22); (b) low noise, estimated wavelet coefficients (22) after shrinkage (24) with η_S = √2; (c) low noise, estimated LIDAR signal; (d) medium noise, estimated wavelet coefficients (22); (e) medium noise, estimated wavelet coefficients (22) after shrinkage (24) with η_S = √2; (f) medium noise, estimated LIDAR signal; (g) high noise, estimated wavelet coefficients (22); (h) high noise, estimated wavelet coefficients (22) after shrinkage (24) with η_S = √2; (i) high noise, estimated LIDAR signal

• (a)

  Compute Fourier coefficients y_l and g_l and recover wavelet coefficients (22) by using Kolaczyck's algorithm (which requires only O{n pt log (n)²} operations); see Figs 5(a), 5(d) and 5(g).

• (b)

  Compute, if needed, an estimate of the noise standard deviation as described above. Find thresholds λ_j:=λ_n,j as shown in equation (24) and illustrated in Table 1 for the boxcar.

• (c)

  Apply hard thresholding ; see Figs 5(b), 5(e) and 5(h). Finally, invert the wavelet transform to obtain an estimate of f; see Figs 5(c), 5(f) and 5(i). (In the WaveD software the (default) maximum resolution level j₁ is determined from the data as follows: j₁ is set to be the level preceeding j(100%) where j(100%) is the smallest level where 100% of thresholding occurs; see Table 1.)

• (d)

  Cycle‐spin the WaveD estimator in the fashion of Coifman and Donoho (1995) (optional). This improves visual and numerical performance and was used in Fig. 7 and Table 2 later. We refer to Donoho and Raimondo (2004) for an efficient algorithm which cycle‐spins the WaveD estimator over all circulant shifts.

Table 1. Level‐by‐level thresholds (smooth blur): defined by expression (24)†

Noise level	η	Thresholds for the following levels of j:
		j = 5	j = 6	j = 7	j = 8
Low ()	√2	0.0036 (31.35%)	0.0066 (62.5%)	0.0128 (95.31%)	0.0242 (100%)
Medium ()	√2	0.0375 (68.75%)	0.0688 (98.44%)	0.1328 (100%)	Maximum level: j1=6
High ()	√2	0.0729 (90.63%)	0.1338 (100%)	Maximum level: j1=5
Low ()	√6	0.0063 (31.25%)	0.0115 (81.25%)	0.0222 (99.22%)	0.0418 (100%)
Medium ()	√6	0.0649 (87.5%)	0.1192 (100%)	Maximum level: j1=5
High ()	√6	0.1263 (96.88%)	0.2318 (100%)	Maximum level: j1=5

• †Indicated in parentheses are the corresponding fractions of shrunken coefficients. The first three rows correspond to the smaller choice η_S=√2. The last three rows correspond to the conservative choice η_L=√{2(2ν+1)} with ν=1. (In the WaveD software the (default) maximum resolution level (25) is set to be the level preceeding j(100%) where j(100%) is the smallest level where 100% of thresholding occurs.)

Figure 7

(a) WaveD (low noise scenario); (b) WaveD (medium noise scenario); (c) WaveD (high noise scenario); (d) ForWaRD (low noise scenario); (e) ForWaRD (medium noise scenario); (f) ForWaRD (high noise scenario); (g) FoRD (low noise scenario); (h) FoRD (medium noise scenario); (i) FoRD (high noise scenario)

Table 2. Monte Carlo approximations to MISE =

Method	Blur	Means for the following levels of noise:
		σ low=0.05	σ med=0.5	σ high=1	σ lim=1.25
WaveD	Smooth	0.0024	0.0180	0.0388	0.0519
ForWaRD	Smooth	0.0027	0.0208	0.0642	0.0950
FoRD	Smooth	0.0084	0.0906	0.3201	0.3352
WaveD	Boxcar	0.0223	0.0753	0.0831	0.0900
ForWaRD‡	Boxcar	0.0110	0.0573	0.0906	0.1030
FoRD	Boxcar	0.0237	0.0950	0.3470	0.3610

• †The results are means of 1000 independent simulations of model (1) with n=2048 as in Fig. 3. For each scenario, numbers in bold indicate the method which has the smallest MSE.

3.3. Connection with the wavelet–vaguelette decomposition

Donoho (1995) gave the first discussion of wavelet thresholding in linear inverse problems and introduced the wavelet–vaguelette decomposition (WVD). Specialized to convolution operators on the circle, the WVD structure in part postulates the existence of biorthogonal systems (𝒰) and (𝒱) and pseudosingular values κ_j (not depending on the spatial index k) such that

where . In terms of Fourier coefficients, and setting s_jl=g_l/κ_j,

(26)

The WVD class of estimators takes a co‐ordinatewise thresholding rule δ(x,t_j) and level‐ dependent thresholds (t_j) and sets . If we observe that

then it is clear that our estimator (17)formally can be viewed as being consistent with the WVD recipe. However, the implementation of the estimator differs here—in the WaveD scenario (22) the functions 𝒰 and 𝒱 are not constructed explicitly, and the coefficients are instead evaluated in the Fourier domain using the original wavelets Ψ and filter g.

The key additional property that is needed to establish a WVD is that the systems (𝒰) and (𝒱) each form Riesz bases—this property allows lower bounds and hence minimax rates of convergence to be established over the Besov classes that are considered in this paper. The lower bound arguments are given in detail in Donoho (1995) for L₂‐loss and, as noted there, the methods can be extended to more general loss measures.

For the dilation homogeneous operators on ℝ that were principally studied in Donoho (1995) and Abramovich and Silverman (1998), the vaguelette systems are multiples of translates and dilates of a single mother vaguelette 𝒰_0,0 or 𝒱_0,0, and the Riesz basis property can be established as in Donoho (1995). See also Lee and Lucier (2001).

This dilation structure is no longer available for the candidate vaguelettes (26) corresponding to convolution on the circle (e.g. Fig. 6). Nevertheless, we show in Appendix B that (𝒰) and (𝒱) are Riesz bases if and condition C holds and if in addition we have, for constants C₀ and C that are independent of j,

(27)

where the difference operator Δs_jl=s_j,l+1−s_jl and Δ²s_jl=Δ(Δs_jl).

Figure 6

Candidate vaguelettes corresponding to the boxcar convolution of Fig. 2: (a) 𝒱_4,5; (b) 𝒰_4,5; (c) 𝒱_5,6; (d) 𝒰_5,6

These conditions hold if g_l ≡ C|l|^−ν or more generally if c₀|l|^−ν  |g_l|  c₁|l|^−ν and Δ^rg_l |l|^−ν−r for r = 1 and r = 2. For such convolution operators, we obtain lower bounds to the rates of convergence over Besov spaces via the methods of Donoho (1995).

However, the sufficient conditions (27) are not satisfied by the boxcar kernel, owing to the oscillations in g_l which inflate Δg_l. We do not yet know whether the systems (𝒰) and (𝒱) form Riesz bases in this case, and so yield vaguelettes; see Fig. 6. Thus the only lower bounds that are currently available for this kernel are those which were established for Fourier hyperrectangles and ellipsoids in Johnstone and Raimondo (2004).

3.4. Remarks on tuning

Asymptotic minimax theory gives insight into the choice of tuning parameters. For example, in direct estimation problems Donoho et al. (1995) have shown near optimality of

(28)

where is an estimated scale from the data and

(29)

For deconvolution problems (1), our main result (proposition 1 in Section 4.1) states that, for any constant η2√{8π(p∨2)} that is sufficiently large, the choices (24) and (25) are near optimal for a wide variety of target function (Appendix B.1) and L^p loss functions.

It may be seen that the finest scale j₁ that is suggested by equation (25) is considerably smaller than that given by equation (29) in the direct case. The size of the thresholds in equation (24) may in principle be larger than in equation (28), but in practice smaller thresholds than suggested by the proof may be desirable.

Again, it is interesting to compare our results with the WVD approach. For dilation ho‐ mogeneous operators with index ν, Abramovich and Silverman (1998) showed that, if (soft) thresholding is to be used at all levels j< log ₂(n), then near optimality of expected mean‐ squared estimation error rates required the use of higher thresholds, which in our notation are given by

(30)

We have tested our method with two choices of the parameter η: η_L=√{2(2ν+1)} as in Abramovich and Silverman (1998) and a smaller value η_S=√2 that is similar to that of direct estimation (28). The values of corresponding thresholds (for smooth blur) are given in Table 1. In our simulation study (Section 4), we used the WaveD estimator with η_S as it led to slightly better results than the conservative choice η_L.

4. Numerical performances of the wavelet deconvolution method

We compare several approaches to deconvolution which differ in the degree to which Fourier and wavelet filtering are balanced. On the one hand, we have Wiener‐filter‐like methods which have no wavelet component but use only Fourier inversion together with a regularization parameter (below referred to as the FoRD method). On the other hand we have wavelet decomposition approaches (like the WaveD method) where we perform Fourier inversion with no regularization but use wavelet smoothing to remove noise. Between those two approaches lies the ForWaRD method which combines Fourier regularization with wavelet smoothing. We give only a brief description of the ForWaRD and FoRD methods, referring to Neelamani et al. (2004) and references therein for further details.

4.1. Fourier regularized deconvolution

The FoRD estimator of f is defined in the Fourier domain:

(31)

where

(32)

Then we take as an estimator of f by using the Fourier series with coefficients . Here α is a regularization parameter which balances the suppression of noise with signal distortion. Small values of α give an unbiased but noisy estimate whereas large values of α suppress the noise but also distort the signal. We use the terminology FoRD of Neelamani et al. (2004) since our comparisons below use the regularization parameter choice in code that was graciously provided by R. Neelamani.

4.2. Wavelet‐regularized deconvolution

First apply filtering (31), deriving an estimator of f. In the second step, we further smooth by using data‐driven level‐dependent wavelet thresholding (17). Here also α plays the role of a regularization parameter which balances the level of noise and signal distortion. For α=0 the ForWaRD estimator is similar to a compactly supported WVD estimator of f whereas for α>0 the ForWaRD estimator is a hybrid of the FoRD and WVD methods. Although it is difficult to derive an optimal choice of the regularization parameter α (see Neelamani et al. (2004)), a data‐driven algorithm to compute the ForWaRD estimator is available at http://www.dsp.rice.edu/software/ward.shtml.

We compared the WaveD method with the FoRD and ForWaRD methods in a simulation study using the LIDAR target that is depicted in Fig. 1. Performance was tested with different blurring types and noise levels as illustrated in Fig. 3. For each combination of noise level and blurring type we computed the Monte Carlo approximation to the . Our results are illustrated in Fig. 7 for smooth blur and summarized in Table 2.

4.3. Analysis of the results

Regularized Fourier filtering tends to distort the original signal and the superiority of the WaveD method becomes more apparent as the noise level increases. For smooth blur WaveD outperformed both ForWaRD and FoRD in all cases with larger margins in high noise scenarios. For all the methods we observe smaller margins and poorer performances in the boxcar blur scenario (which confirms a larger DIP). For boxcar blur WaveD outperformed ForWaRD for larger noise levels whereas ForWaRD outperformed WaveD for smaller noise levels. Both WaveD and ForWaRD outperformed FoRD, whose performance is limited because of its linear nature.

Remark 1. Cycle spinning the WaveD estimator mitigates its lack of translation invariance and leads to a better visual appearance and smaller MSE. We refer to Donoho and Raimondo (2004) for an efficient algorithm which cycle‐spins the WaveD estimator over all circulant shifts (used in the computations for Table 2 and Fig. 7).

Remark 2. Boundary corrections to deal with non‐periodic signals and an extension to the deconvolution of two‐dimensional data are currently under investigation by the authors.

5. Asymptotic theory

5.1. Near optimality for a wide range of smoothness classes

Proposition 1. Suppose that we observe the random process (1) with σ=1, under assumption C. Let p>1 be an arbitrary number. If f belongs to with π1, s1/π and

then, for η2√{8π(p∨2)} the wavelet‐based estimator (17) with threshold (24) with and maximum level (25) is such that

(33)

(34)

and

(35)

Remark 3. There is an ‘elbow effect’ or ‘phase transition’ in the rates of convergence, switching from condition (34) to condition (35) as the assumed smoothness decreases. The existence of this effect is familiar from the direct observation case (e.g. Donoho et al. (1995) and references cited therein, where conditions (34) and (35) are respectively referred to as the ‘dense’ and ‘sparse’ cases). The additional presence of the parameter ν makes the sparse case (35) relevant even for quadratic loss, p=2.

Remark 4. For p = 2 and smooth convolutions, the rate α and dense case condition (34) are consistent with results of Donoho (1995), Abramovich and Silverman (1998) and Fan and Koo (2002). Pensky and Vidakovic (1999) obtained similar rates (in the density model) with the additional restriction π=2 (Hilbert–Sobolev function classes) so that constraint (34) does not appear.

Remark 5. For p=2 Kalifa and Mallat (2003) proposed a related procedure which can be applied for hyperbolic convolution where the convolution kernel depends on the sample size. Such convolutions do not satisfy condition C; hence their results are not directly comparable with ours.

Remark 6. For p = 2 and severely ill‐posed convolutions such as boxcar blur, our results agree with the degree of ill‐posedness that was derived in Johnstone and Raimondo (2002).

Remark 7. For p≠2 our result seems to be new in the deconvolution context.

The following proposition gives some examples of blurring type where near optimal results of proposition 1 are achievable by our estimator.

Proposition 2.

• (a)

  For ordinary smooth convolution, |g_l|∼C|l|^−ν, where g_l denotes the Fourier coefficients of g and ν>0, assumption C is satisfied.

• (b)

  For boxcar blur, g(x) = (1/2a)𝕀_[−a,a](x), where a is a BA number, assumption C is satisfied with .

Remark 8. Combining the results of propositions 1 and 2 for boxcar blur, we see that rates (34) and (35) hold with provided that the boxcar width is a BA irrational number. In the finite sample implementation of model (1) on a computer, Fourier coefficients g_l are computed up to l = n or l = −n, or more precisely for blocks C_j that are wholly contained in [−n,n]. Hence condition C needs only to be satisfied for any j>0 where 2^j+rn. An examination of the proof of proposition 2, part (b), shows that the latter condition holds for those BA rational numbers that are of order greater than n discussed in Section 2.2.

Remark 9. For almost all irrational numbers (i.e. except for a set of Lebesgue measure 0), the boxcar blur is also known to have a degree of ill‐posedness of , ignoring logarithmic terms (Johnstone and Raimondo, 2002). Whether the WaveD estimator can be tuned to achieve rates that are similar to expression (3) for boxcar blur in the almost all case remains open.

Remark 10. In the direct estimation setting, alternatives to co‐ordinatewise non‐linear thresholding also have broad adaptivity properties (e.g. Efromovich (1999)); whether such results extend to the deconvolution is an issue for further work.

5.2. The maxiset approach

Near optimal properties of our proposal are direct applications of the following theorem which has been borrowed from Kerkyacharian and Picard (2000). This theorem gives the ‘maxiset’ (condition (40)) for a general wavelet estimator of the form (39). It will be applied directly to our procedure as outlined in Appendix A. We refer to Appendix B for condition (63) (known as the Temlyakov property). First, we introduce some notation: μ will denote the measure such that, for j ∈ ℕ, k ∈ ℕ,

Theorem 1.  Let p > 1, 0 < q < p, {ψ_j,k,j  −1,k = 0,1,…,2^j} be a periodized wavelet basis of L²(T) and σ_j be a positive sequence such that the heteroscedastic basis σ_jψ_j,k satisfies property (63) in Appendix B. Suppose that Λ_n is a set of pairs (j,k) and c_n is a deterministic sequence tending to 0 with

(36)

If, for any pair κ=(j,k) ∈ Λ_n, we have

(37)

(38)

for some positive constants η and C, then the wavelet‐based estimator

(39)

is such that, for all positive integers n,

if and only if

(40)

Remark 11. Through condition (40) and in the light of Appendix B.1 the theorem gives the maxiset of the method, i.e. the set of functions where the method attains a given rate of convergence. This way of measuring the performances of statistical procedures has been particularly successful in the nonparametric framework. It has often the advantage of giving less arbitrary and pessimistic comparisons of procedures than the minimax approach.

Remark 12. We shall prove (Appendix B.1) that the Besov spaces are included in the maxiset defined in expression (40) for q chosen such that provides the rate that is given in inequality (33). In particular, for appropriate choices of j₁ and Λ_n (see expression (42)), we have that

(41)

where P_Vj1 denotes the projection on the space V_j1 of the multiresolution analysis that is associated with the wavelet basis. In this case it appears more clearly that the second part of condition (40) is directly linked to standard conditions for membership in Besov spaces. This part is responsible for the condition s1/π in the assumptions of proposition 1.

Acknowledgements

We are grateful to all the referees, who provided us with helpful suggestions that have improved the original version significantly. This project began while Gérard Kerkyacharian and Dominique Picard visited the University of Sydney, partly funded by the University of Sydney. IMJ was supported in part by National Science Foundation grant DMS 00‐72661 and National Institutes of Health grant ROI EB001988‐08.

Appendices

Appendix A: Proofs

A.1. Outline of the proof of proposition 1

We shall prove that proposition 1 follows from theorem 1. For this, we shall consider the wavelet‐based estimator (17) with threshold (24) and maximum level (25). In the light of theorem 1, that is to say that σ_j=τ_j as in equation (23) and

(42)

In this setting, and under assumption C, we shall prove the following claims.

• (a)

  Inequalities (37) and (38) hold with η2√{8π(p∨2)} (claim 1).

• (b)

  The basis (σ_jΨ_jk) satisfies condition (63) (see Appendix B.2) as soon as there is a constant C such that, for any finite subset Λ of ℕ,

  (43)
  (44)
  (claim 2). Note that for p=2 condition (63) holds without any condition on σ_j.

• (c)

  Conditions (36), (43) and (44) are satisfied (claim 3).

Hence, under the assumptions of proposition 1, theorem 1 applies to the wavelet‐based estimator (17) which combined with remarks following theorem 1 gives the rate (33). To complete the proof we shall now prove the claims.

A.1.1. Proof of claim 1

First, we derive the bias and variance of . Taking the expectation in equation (19),

(45)

under the assumptions of proposition 1 we have σ=1. It follows that

as the z_ls are independent and identically distributed standard Gaussian random variables:

(46)

Note that and that

(47)

so since, for the Meyer wavelet, . Using definition (23) and recalling that for the Meyer wavelet |C_j|=4π·2^j:

(48)

after recalling the notation of equation (23). As the s are Gaussian,

(49)

which combined with expression (48) yields

(50)

hence proving inequality (37). Let Z∼𝒩(0,1); by using expression (48) we have that

Hence, for η2√{8π(p∨2)} we have proved that

(51)

which proves inequality (38) for the WaveD threshold (24) with σ=1.

A.1.2. Proof of claim 2

The proof of claim 2 is a direct application of theorem 2 (see Appendix B).

A.1.3. Proof of claim 3

Clearly conditions (43) and (44) will be true for any σ_j of the form 2^jν, which follows from assumption C if σ_j=τ_j. Next we prove inequality (36); under assumption C,σ_j≅2^jνC,ν>0, we have

For p>1, pν+p/2>1 is equivalent to . Now by equation (25)

which proves inequality (36).

A.1.4. Proof of proposition 2

Recall definition (23):

Here Ψ is band limited: hence C_j={l:2^j|l|2^j+r}, for some fixed r>0. To simplify the exposition we shall further assume that C_j={l:2^jl2^j+r}, noting that, by symmetry, bounds below hold for negative values of l also. Under assumption (a): |g_l|∼C|l|^ν,

(52)

which proves proposition 2, part (a). Under assumption (b), we shall prove that

(53)

which by identification, 2^2jν=2^3j, shows that condition C holds with . Result (53) follows from condition (9) and the following lemma (see Johnstone and Raimondo (2002)). We refer to Section 2.2 for the notion of a BA number.

Lemma 1. Let p/q and p^′/q^′ be successive principal convergents in the continued fraction expansion of a real number a. Let q4 and N be a non‐negative integer with N+q<q^′. Then, for BA number a,

(54)

Starting at equation (23) and using condition (9), we see that

(55)

Our task, then, is to show that Σ_l ∈ Cj║la║⁻²≅2^2j.

We consider first the upper bound. Let m be the smallest index such that q_m2^j. Recall that C_j={l:2^jl2^j+2}. The geometric growth of the denominators q_n (compare expression (11)) implies that q_m+2r2^rq_m>2^j+r, so

Introduce intervals D₀=ℕ∩[1,q_m) and D=ℕ∩[q_m+τ−1,q_m+τ) for τ=1,…,4 which together cover C_j. Since a is BA, there is an integer K=K(a) such that q_n+1Kq_n for all n. Hence there are at most K disjoint blocks of length q_m+τ−1 that cover D. Apply lemma 1 to each of these blocks:

whereas . Since q_m+τ−1K^τq_m−1, we combine over τ to obtain

Noting that q_m−12^j, we recover the upper bound.

For the lower bound, a little care is needed to construct intervals [N+1,N+q]⊂C_j on which to apply the lower bound of condition (54) in lemma 1. Define q_m as before. Set N=2^j and consider the following three cases.

• (a)

  For q_m>2^j+1, set q=q_m−1. Since q_m−12^j, we have N+q=2^j+q_m−12^j+1<q_m=q^′ and so [N+1,N+q]⊂C_j. In addition,

• (b)

  For q_m2^j+1 and q_m+12q_m+q_m−1 (where the second condition corresponds to a_m+12 in expression (5)), set q=q_m. We have N+q=2^j+q_m3×2^j so [N+1,N+q]⊂C_j and N+q<2q_m<q_m+1=q^′.

• (c)

  Finally, suppose that q_m2^j+1 and q_m+1=q_m+q_m−1. Now set q=q_m+1, so that N+q=2^j+q_m+12^j+q_m+q_m−12^j+2^j+1+2^j4×2^j. In addition N+q<q_m+q_m+1q_m+2=q^′, and .

In each of cases (a)–(c), we have

Appendix B

B.1. Embedding of Besov spaces

Our aim here is to investigate which particular periodic Besov space may be embedded in the spaces l_q,∞(μ) as well as imply the condition

(56)

Let us recall that we shall concentrate on the case where

First, we observe that condition (56) will be satisfied when f belongs to . Hence, we only need to prove that is included in . For this we shall use two types of Besov embeddings, setting appropriate conditions on s, π, r and q.

• (a)

  In the periodic setting, we have

  (57)

• (b)

  In the general case, we have the standard ‘Sobolev embeddings’

  (58)

To prove condition (56), we are interested in taking ρ=p. For the case pπ, s>0 implies that only the dense case (34) can occur; hence we need to prove that . This is always true for s>0 since 1−q/p=2s/(1+2ν+2s).

For the case p>π, we must prove that, in the dense case (34), . This is equivalent to 2sσ^′+(1+2ν)(1/p−1/π)0. But in this case the left‐hand side is greater than (1+2ν)(p/π−1)(s−1/π)≥0. In the sparse case (35), we must check that σ^′≥(2ν+1)s^′/(1+2ν+2s^′), but this is equivalent to (2ν+1)p/{(2ν+1)p−2+2pσ^′}1 or s1/π.

Now let us turn to the problem of embedding a particular space into l_q,∞(μ). First let us mention that we shall simplify the problem by considering the embedding into

Using Markov inequality, l_q(μ)⊂l_q,∞(μ). We observe that in the dense case where

(59)

we have

hence deriving the advertised rate of convergence since here p−q=2sp/{1+2(ν+s)}.

It remains to study the more intricate cases where we do not have π=r=q.

Proposition 3.

• (a)

  Let q be defined by the relation (59); if 0<rq and

  (60)
  then
  

• (b)

  Let q be defined by

  (61)
  if 0<rq and
  (62)
  then
  

Remark 13. Case (62) implies that

For this not a restriction, since we are considering 1<p<∞. Moreover, in this case the first member of inequality (62) is always true if we deal with 1π, as

Proof. We have

recalling that q has been chosen in such a way that

and using expression (16), we obtain the following characterization of :

We shall now use embeddings (57) and (58), taking ρ=q.

• (a)

  If and rq we have qπ; hence, using condition (57),

  
  (Let us observe in addition that p>q⇔s>0.)

• (b)

  If and rq we shall use condition (58) to find an embedding with a different order of smoothness. This explains our definition of q. Solving

  
  and using condition (58) gives
  
  We now must check that p>q, but this is equivalent to

B.2. Temlyakov inequalities

Let us recall the Temlyakov property for a basis e_n(x) in L^p: there are absolute constants c and C such that, for all Λ⊂ℕ,

or, equivalently,

(63)

Obviously the left‐hand side is always true for p2 with c=1, whereas the right‐hand side is always true for p2 with C=1. In this section, we shall prove the following result.

Theorem 2. Let φ be a scaling function of a multiresolution analysis and ψ the associated wavelet. Let us assume that

If there is a constant C<∞ such that for all A⊂ℕ

(64)

then the weighted wavelet basis {2^j/2σ_j ψ(2^jx−k), j ∈ ℕ, k ∈ ℤ}∪{σ₀ φ(x−k),k ∈ ℤ} satisfies the Temlyakov property.

Proof. We start by proving the theorem for the Haar basis. We introduce the weighted Haar basis (2^j/2σ_jh) where as usual h(x)=h_j,k(x)=h(2^jx−k) and h(x)=1_[0,1](2x)−1_[0,1](2x−1).

Let us suppose first that p2 and there exists C<∞ such that, for all A⊂ℕ,

(65)

Typically this is true when σ_j=2^jν. If inequality (65) is true, we have for all Λ⊂ℕ×ℤ, pointwise,

so in this case

Using inequality (65) for p2,

Now we suppose that p2 and that there exists C^′<∞ such that, for all A⊂ℕ,

(66)

Then again we have pointwise, for all Λ⊂ℕ×ℤ,

so in this case

using inequality (66) for p2,

Now we shall extend this result to a general wavelet by using the transfer lemma (below). For any locally measurable function let us recall the definition of the Hardy–Littlewood maximal function. Let I denote an interval of ℝ and |I| its Lebesgue measure: for all x ∈ ℝ,

Lemma 2 (transfer). Let us consider two sequences of functions (f_n(x))_{n ∈ ℕ} and (e_n(x))_{n ∈ ℕ}. Suppose that the sequence (f_n(x))_{n ∈ ℕ} satisfies the Temlyakov property and that there exists A<∞ such that for all n ∈ ℕ

Then the sequence (e_n(x))_{n ∈ ℕ} satisfies the Temlyakov property also.

Theorem 2 follows from lemma 2 since, for f = 1_[0,1],f^*(x) ≅ c(1 ∧ 1/|x|), and obviously, for all x ∈ ℝ,|f(x)|f^*(x). Combining this with the assumption of theorem 2 we have that |ψ(x)|C h^*(x) and |h(x)|C ψ^*(x), which is obvious.

To complete the proof we derive the lemma.

Proof. The key tool for deriving the transfer lemma is the Fefferman–Stein inequality (Fefferman and Stein, 1971): for all p,q,1<p<∞,1<q∞, there is a positive constant C_p,q<∞ such that

Using our assumption and the previous inequality we have, for all q, 1<q∞, and all Λ, Λ⊂ℕ,

So for all q,1<q∞,

Using the previous computation for q=2 and q=p, we have

and so

B.3. Vaguelette properties

We use (u,κ ∈ I) to denote a generic system of candidate vaguelettes on the circle T. With , then stands for in the case of the (𝒱) system and in the case of the (𝒰) system.

Adapting the definition of Meyer and Coifman (1997), chapter 8, page 56, we say that {u,κ ∈ I} is a system of periodic vaguelettes on T if there are exponents 0<β<α and a constant C such that

• (a)

  |u(t)|2^j/2C(1+|2^jt−k|)^−1−α, t ∈ T,

• (b)

  ∫_Tu(t) dt=0 and

• (c)

  |u(s)−u(t)|2^j(1/2+β)C|s−t|^β,s,t ∈ T.

(In what follows, α=1 and 0<β<1.)

The proof of Meyer and Coifman (1997), theorem 2, page 56, goes through essentially unchanged for L₂(T) under conditions (a)–(c) and so, for every sequence (α),

From the remarks around Donoho (1995), theorem 2, this is sufficient for the Riesz basis property.

It remains, then, to verify conditions (a)–(c). Condition (b) is immediate, since for the Meyer wavelet . For the Hölder condition (c),

(67)

as follows by considering |s−t|⁻¹|l| and |s−t|⁻¹>|l| separately. Writing , we have

If |g_l|∼C|l|^−ν, then set κ_j=2^−jνC and observe that, for l ∈ C_j, we have s_jl≅1 and so for either (𝒰) or (𝒱) systems |γ_l|≅1 for l ∈ C_j. Combining with the two previous displays,

(68)

For condition (a), we first observe that u_jk(t)=u_j,0(t−2^−jk), and so it suffices to show, for κ=(j,0), that

(69)

For , we have t²∼|1− exp (−2πit)|² and, setting Δf_l=f_l+1−f_l and Δ²f_l=Δ(Δf_l),

Set ; from formula (47) for , we obtain

with

If we suppose that

(70)

then from our previous remarks

If |t|2^−j, we simply bound and retracing the argument from condition (67) to condition (68) with β=0 we obtain condition (69).

To establish condition (70), observe first that, since , we have for r=0,1,2. Some calculation shows that

and now condition (70) may be seen to follow from condition (27).