- Email: [email protected]

Estimating the amount of defocus through a wavelet transform approach Jinlong Lin *, Chao Zhang, Qingyun Shi Center for Information Science, Room 306, Building 18, Wei Xiuyan, Peking University, Beijing 100871, PR China Received 11 August 2002; received in revised form 1 July 2003

Abstract This paper presents a new technique for defocus estimation of a captured image. In our method, a ratio of the wavelet coeﬃcients of high frequency correspond to a same image point at two diﬀerent levels is used. For an edge point, it is shown that the ratio is related to the amount of defocus. Let a be the ratio of wavelet coeﬃcients at the ﬁrst level to that at the second level. The value of a decreases as the amount of defocus increase. In our experiments of iris image analysis, when a is larger than 0.5 the number of feature points in an image almost remain constant. It means that the image is little defocused and available for image recognition. Compared with Fourier methods, this technique is more robust. In addition, this method is fast enough to be used in auto-focus system for tracking moving objects. 2003 Elsevier B.V. All rights reserved. Keywords: Defocus estimation; Wavelet transform; Auto-focus

1. Introduction Defocus estimation is one of the most important tasks in computer vision for applications of image recognition. Heavily defocused images, which lose detailed information, make image recognition diﬃcult or impossible. Because of diﬃculty to determine camera parameters, computation complexity and noise, some methods (Tekalp, 1995) introduced to recover details from

* Corresponding author. Tel.: +86-1062755910; fax: +861082886123. E-mail addresses: [email protected], [email protected] com (J. Lin).

defocused image are unavailable for dynamic and real-time applications. Estimating the amount of defocus of images to choose the high quality image is essential for image recognition. In addition, defocus estimation methods can be used in autofocus optical systems for tracking moving object. Generally, the defocus estimation algorithms are from either the imageÕs power spectrum in frequency domain, or from the imageÕs pointspread function in spatial domain (Hofeva, 1994). Subbarao (1990) estimated blur through Fourier transform (FT) and determined the distance between surface patches of a scene and a camera system. A ratio of the Fourier powers between the two images is used to measure the amount of defocus. Pentland (1987) proposed two methods

0167-8655/$ - see front matter 2003 Elsevier B.V. All rights reserved. doi:10.1016/j.patrec.2003.11.003

408

J. Lin et al. / Pattern Recognition Letters 25 (2004) 407–411

to measure the amount of defocus. The ﬁrst method requires only one image and is based on measuring the width of edges that have step discontinuity in focused image. The defocused edge is modeled as the result of convolving a focused image with a point-spread function that is assumed to be a Gaussian distribution with spatial parameter r. The parameter r is used as the measure of blur, and has a correspondence to the depth of defocus. The second method is similar to SubbaraoÕs. Otherwise, edge detection methods were used to estimate blur (Elder and Zucker, 1998). But, edge detection itself is a rough task. For some applications, there is no need to measure the distance between the scene and camera, and it is very diﬃcult to determine the parameters of camera setting. We only need to estimate the amount of defocus of every image captured and choose the available one at real time. PentlandÕs ﬁrst method is very time-consuming and his second method requires two images with diﬀerent aperture diameter. There are not applicable to real time applications. On the other hand, for low SNR images, with large proportion of power at high frequencies caused by noise, it is diﬃcult to estimate the amount of defocus through FT approach well and truly. To solve the problems, a fast method of measuring the amount of defocus is proposed in this paper. We use parameter a that is the ratio of the value of wavelet coeﬃcients at the ﬁrst level to that at the second level as the measure of defocus. The value of a decreases as the amount of defocus increase. In our experiments of iris image analysis, when a is larger than 0.5 the number of feature points in an image almost remain constant. It means that the image is little defocus and available for image recognition. Compared with Fourier methods, this technique is more robust. In addition, this method is fast enough to be used in auto-focus system for tracking moving objects. This paper is organized as follows: Section 2 overviews the eﬀects on the value of wavelet coeﬃcients of defocusing. Section 3 describes the method of estimating the amount of defocus. Section 4 presents experimental results. This paper is concluded in Section 5.

2. Relationship between the value of wavelet coefﬁcients and the amount of defocus For orthogonal dyadic wavelet bases, a signal f0 can be decomposed as following: f0 ¼

N X

gk þ fN

ð1Þ

k¼1

where k is the level of wavelet transform, and 2N is the smallest scale. Let X Cnk uk;n ð2Þ fk ¼ n2Z

gk ¼

X

dnk wk;n

ð3Þ

n2Z

then 1 X k1 Cj hj2n Cnk ¼ pﬃﬃﬃ 2 j2Z

ð4Þ

1 X k1 dnk ¼ pﬃﬃﬃ Cj gj2n 2 j2Z

ð5Þ

where uk;n and wk;n are scale function and wavelet function, respectively. hn and gn are wavelet ﬁlter coeﬃcients. Based on above equations, a discrete signal C 0 can be decomposed as d 1 ; d 2 ; . . . ; d N and C N through N levels WT. d 1 ; d 2 ; . . . and d N are wavelet coeﬃcients corresponding to diﬀerent levels and C N is the result of the N th scale analysis. For a particular wavelet function, a wavelet scale has certain relationship with one Fourier frequency band (Torrence and Compo, 1998). The wavelet function wk;n is a band-pass ﬁlter and the value of coeﬃcients d 1 ; d 2 ; . . . ;d N correspond to the power of diﬀerent frequency band portions of a signal. d 1 is correspond to high frequency band and d N is correspond to the lowest frequency band. When an image is defocused, edges in it are smoothed and widened. The value of d k corresponding to high frequency band decreases, and that corresponding to low frequency band increases. A step discontinuous edge is shown in Fig. 1(a). It is smoothed by convolved with Gaussian distribution function at diﬀerent parameter r. The

J. Lin et al. / Pattern Recognition Letters 25 (2004) 407–411

409

the increasing of r. We choose wavelet coeﬃcients of the lowest two levels and the ratio of their value is used to be the measure of defocus e.g., a¼

Fig. 1. The relationships between the value of wavelet coeﬃcients and the amount of defocus: (a) a step discontinuity edge, (b) plots of jd 1 j, jd 2 j, jd 3 j, jd 4 j against to Gausses parameter r.

plots of the ﬁrst four level wavelet coeﬃcients at point A against r are shown in Fig. 1(b). With the increasing of r, jd 1 j decreases monotonously and that jd 2 j, jd 3 j and jd 4 j increase ﬁrst and than decrease after reaching their acmes. Fig. 1(b) shows that wavelet transform has characters of a bandpass ﬁlter.

jd 1 j jd 2 j

ð6Þ

For image analysis, we calculate a of diﬀerent edges in an image and use the maximal value to measure the amount of defocus of the image. The a of an edge is the average ratio of all points on the edge. Let the number of edges in an image is M, and there are K points on an edge. The a of the mth edge is: 1 X jdk1 j am ¼ ð7Þ K k jdk2 j The a of the image is a ¼ max fam g m

ð8Þ

The calculation process is as follows: Step 1: Make a two-level wavelet transform of an image. The map of wavelet coeﬃcient is as Fig. 3. Step 2: Find available edges in LH1 and HL1. Step 3: Calculate the a of every edge. Step 4: Search for the maximal a.

3. Measure of defocus Further, we calculate the value of jd 1 j=jd 2 j, 2 jd j=jd 3 j and jd 3 j=jd 4 j and the plots of these value against Gaussian parameter r are shown in Fig. 2. It is shown that the ratio of the value of lower level wavelet coeﬃcients to that of high level wavelet coeﬃcients decreases monotonously with

Fig. 2. Plots of the ratio against to Gaussian parameter r.

If a focused or little defocused edge existed in an image, the image is available.

4. Experimental results To keep scene unchanged, we only adjusted focus distance to change the defocus degree. In our

Fig. 3. The map of wavelet coeﬃcients after a two-level wavelet transform.

410

J. Lin et al. / Pattern Recognition Letters 25 (2004) 407–411

experiments, objects were set 100 cm from the lens of the camera. Changing focus distance from 100 to 10 cm, we got a series of images with diﬀerent amount of defocus. At 100 cm, the image is focused and the amount of defocus increases as focus distance decrease. Fig. 4 shows three images cap-

Fig. 4. Images captured at focus distance (a) d f ¼ 100 cm, (b) d f ¼ 80 cm, (c) d f ¼ 50 cm.

tured at diﬀerent focus distance. The values of a of the series of images are shown in Fig. 5. To evaluate the capability of noise immunity of the method, we added Gaussian noise into above images and calculated the value of a. The results are also in Fig. 5(a). For comparison, we estimate the blur with a Fourier method, the ratio of power of high frequencies portion to that of low frequencies portion denotes the amount of defocus. Fig. 5(b) shows the relationships between the ratio (aF ) and defocus degree with diﬀerent noise intensity. In Fig. 5(a), the noiseÕs eﬀect on a is insigniﬁcant. In Fig. 5(b), noise inﬂuences the value of ratio (aF ) so heavy that it is not able to measure the amount of defocus. It shows that the wavelet transform approach can estimate the amount of defocus credibly even though in heavy noise situation and is more robust than the Fourier method. We tested the wavelet transform method with images of diﬀerent content such as portrait, text,

Fig. 5. The value of a of images captured at diﬀerent focused distance: (a) wavelet transform method, (b) Fourier transform method.

J. Lin et al. / Pattern Recognition Letters 25 (2004) 407–411

411

5. Conclusion In this paper, we introduced a wavelet transform method to estimate the amount of defocus. The ratio of values of wavelet coeﬃcients of the lowest two levels is used as the measure of defocus. Experimental results indicate that the new method is available and robust. Fig. 6. The number of feature points at diﬀerent value of a.

References landscape, and iris, the results are consistent with above. To evaluate the eﬀect of defocus on our iris recognition system, we extracted feature points in a series of images of an iris with diﬀerent value of a, the result is shown in Fig. 6. It shows that if the value of a is larger than 0.5, the iris image is available for recognition. In our experiments, the size of an image is 640 · 480, and the computer is PIII 650. The time of analyzing one image is about 40 ms. So, this method is fast enough to be used in auto-focus system for tracking moving objects. In all above experiments, Daubchies compact wavelet ﬁlter (Cohen et al., 1992) was used. In addition, we used Harr wavelet ﬁlter to do the same experiments, and the results were similar.

Cohen, A., Daubechies, I., Feauveau, J., 1992. Biorthogonal bases of compactly supported wavelets. Comm. Pure Appl. Math. 45, 485–560. Elder, J.H., Zucker, S.W., 1998. Local scale control for edge detection and blur estimation. IEEE Trans. Pattern Anal. Machine Intell. 20 (7), 699–716. Hofeva, L.F., 1994. Range estimation from camera blur by regularized adaptive identiﬁcation. Int. J. Pattern Recognit. Artiﬁcial Intell. 8, 1273–1300. Pentland, A.P., 1987. A new sense for depth of ﬁeld. IEEE Trans. Pattern Anal. Machine Intell. PAMI-9 (4), 523–531. Subbarao, M., 1990. Method and apparatus for determining the distance between surface patches of a three-dimensional spatial scene and a camera system, US Pat. No. 4,965,840. Tekalp, A.M., 1995. Digital Video Process.. Prentice Hall. pp. 283–291. Torrence, C., Compo, G.P., 1998. A practical guide to wavelet analysis. Bulletin of the American Meteorological Society 79 (1), 61–78.