Optical processing of speckle images with bacteriorhodopsin for pattern recognition

Optical processing of speckle images with bacteriorhodopsin for pattern recognition

Oprics and Lasers in Engineering 23 (1995) 121-136 Elsevier Science Limited Printed in Northern Ireland 0143-8166(95)ooo10-0 Optical Processing of Sp...

861KB Sizes 0 Downloads 30 Views

Oprics and Lasers in Engineering 23 (1995) 121-136 Elsevier Science Limited Printed in Northern Ireland 0143-8166(95)ooo10-0

Optical Processing of Speckle Images with Bacteriorhodopsin for Pattern Recognition

John D. Downie NASA Ames Research Center, M/S 269-3, Moffett Field, CA 94035, USA

ABSTRACT Logarithmic processing of images with multiplicative noise characteristics can be utilized to transform an image into one with an additive noise distribution. This simplifies subsequent image processing steps for applications such as image restoration or correlation for pattern recognition. One particularly common form of multiplicative noise is speckle, for which the logarithmic operation not only produces additive noise, but also makes it of constant variance (signal-independent). I examine the optical transmission properties of some bacteriorhodopsin films here and find them well suited to implement such a pointwise logarithmic transformation optically in a parallel fashion. I present experimental results of the optical conversion of speckle images into transformed images with additive, signal-independent noise statistics using the real-time photochromic properties of bacteriorhodopsin. I provide an example of improved correlation performance in terms of correlation peak signal-to-noise for such a transformed speckle image.

1 INTRODUCTION Logarithmic image transformations may be useful any time the noise on an image can be described as multiplicative in nature.’ This nonlinear operation on the image separates the noise from the underlying object function by producing a new image with additive noise. Traditional linear filtering techniques can then be applied for purposes of image restoration, reconstruction, etc. One quite commonly occurring type of 121


J. D. Downie

multiplicative image noise is speckle, which is found during imaging with spatially coherent light or radiation when the object of interest has a random surface roughness of the order of a wavelength, and the imaging system cannot resolve the microscale of the object’s roughness.’ Speckle can be observed in images from coherent synthetic aperture radar, laser and sonar imaging systems, ultrasonic imaging systems that employ coherent acoustic irradiation, and holographic and interferometric images to name only a few examples. In addition to facilitating the easy application of linear filters for image processing, we have also recently shown that object detection and recognition of speckle images through correlation with a matched filter is significantly improved after preprocessing the image with a logarithmic transformation to make the noise additive and signal-independent.3 In this paper I examine the particular case of speckle noise, and propose and present experimental results of the optical logarithmic transformation of speckle images using the real-time photochromic properties of bacteriorhodopsin. Certainly, the logarithmic transformation can be performed digitally with relative ease, but there may be situations such as the input of speckle images into an optical correlator where it would be advantageous to perform all processing operations optically. Bacteriorhodopsin (BR) films can accomplish this particular transformation in real-time and in parallel, and are thus an attractive option. In addition, the extremely high spatial resolution of BR films offers an advantage over digital processing. The logarithmic transformation has been previously implemented optically using a halftone screen process,’ but this approach suffers from the necessary time delay due to removal and development of the photographic film behind the halftone screen. Another technique was also presented recently using two phase conjugators in series to produce a photorefractive quadratic nonlinear processor for signal recovery from speckle noise,4 but this system produces additive noise that is still signal-dependent and is also more complicated than the simple BR system that I will present here. The remainder of this paper is divided into four parts. In Section 2 I briefly review the properties of speckle noise and the logarithmic transformation. In Section 3 I discuss the background and characteristics of bacteriorhodopsin. Then in Section 4 I discuss the experiments performed, present the experimental data and present simulation results of the correlations of speckle and transformed images obtained experimentally. In the last section I conclude and summarize the results.

Optical processing of speckle images with bacteriorhodopsin





The statistical properties of speckle noise have been thoroughly studied in the past, and it is well known that the image intensity generally follows a negative exponential probability density function (pdf).5 An image model that has been widely used to describe speckle images is the multiplicative model &(X, Y) = ‘&




where T~~(x,y) is the noisy speckle image intensity, s(x, y) is the incoherent intensity image of the object, n,(x, y) is the noise function and K is a proportionality factor dependent on system parameters.“.’ We will assume here that K equals 1 without loss of generality. We note that eqn (1) is strictly true only for spatially uniform areas of the object, and is a less accurate model for image regions that contain spatial details smaller than the resolution of the coherent system.’ Assuming the model of eqn (1) and negative exponential statistics for T~~(x,y), then the random variable n,(x, y) necessarily also has a negative exponential pdf:

Pnbb(x~ y>l=

exp {-n,(x,



It is then easy to show that both the mean and the standard deviation of Y~,~,,(x, y) are equal to 1, implying that the mean and standard deviation of rJ_x, y) are proportional to s(x, y), which clearly makes the noise signal-dependent in nature. While it is clear that a logarithmic transformation of an image described by eqn (1) will change the noise from multiplicative to additive as log &7(x,

Y )I=

1% -W

Y )> + 1% {%AX,

Y )I,


it is not as obvious that the new noise will also be signal-independent in nature. The theory behind this assertion was developed in previous work generally concerned with transforming any noisy image r(x, y) with signal-dependent noise in a pointwise fashion into another function p(x, y) such that p(x, y) will contain statistically signal-independent noise (i.e. constant variance noise) over a wide range of image values.8*9 The general nonlinear transformation derived in that work is expressed as


J. D. Downie

where D(e) represents the functional dependence deviation a, on the image mean pL,as given by

of the image standard

0, = D(/&),


and K is an arbitrary constant. For speckle noise, a, = p,, and therefore D(r) = r, and we thus arrive at P (x7 Y > = In b&

Y >I


as the image transformation that makes the noise both additive and signal-independent. This considerably simplifies the image processing for restoration purposes, and also produces an image better suited for correlation with a matched filter for pattern detection.



For digital image processing applications, the logarithmic operation would be performed digitally. However, for some optical processing applications such as correlation, it may be more efficient to also perform the logarithmic preprocessing of the images optically. In that case, we require an optical material that exhibits a logarithmic transfer function of some kind. Bacteriorhodopsin (BR) is an organicallyderived material that demonstrates many desirable properties for use as a real-time optically addressed spatial light modulator for optical processing application.lOv” These include photochromism, second order optical nonlinearity, photoelectric effect and photoinduced anisotropy. Some previously studied applications for BR films include Fourierplane spatial filtering,‘* optical limiting,13 fast photodetectors,‘4 realtime holography for optical correlation,‘5 and holographic imaging through a time-varying thin phase aberration.‘6 For the application of logarithmic image processing discussed here, I utilize the photochromic nature of the material. BR is a protein molecule found in the photosynthetic system of a salt-marsh bacterium called H&bacterium salinarium. The BR molecule is located in a membrane commonly called the purple membrane (PM). To the bacterium, BR is important in an oxygen-deficient environment, as the BR molecules function as light-driven proton pumps which transport protons across the cell membrane. This creates a proton gradient which in turn generates an electrochemical potential used by the organism to synthesize adenosine triphosphate (ATP). Effectively, BR is used by the bacterium to directly convert sunlight into chemical energy. The absorption of light also initiates a photocycle

Optical processing of speckle images with bacteriorhodopsin


Fig. 1. Schematic of BR photocycle. Numbers in parentheses indicate peak wavelength of absorption spectrum at the bR-state and M-state. The transition time from bR+M is --SOPS, and the thermal relaxation time from M-, bR is -1Oms for BR in its native form.

in the BR molecule in addition to the transportation of protons. It is this photocycle which makes it a potentially useful material as an optically-addressed spatial light modulator. This is especially true since the photocycle continues to operate after the BR is extracted from the bacterium in the form of two-dimensional crystalline sheets of PM. A schematic diagram of the BR photocycle is shown in Fig. 1. In the dark, the molecule is initially in the bR-state, which has an absorption spectrum peaking at about 570 nm. Upon absorption of a photon in this band, the molecule quickly passes through several intermediate states via thermal relaxation until it reaches the M-state, in which its absorption spectrum is shifted about 160 nm towards the blue from the initial bR-state. In this state the peak absorption wavelength lies at about 412nm, and even though the spectra of both the bR- and M-states are fairly broad, there is little overlap between them. Thus, images may be written in a film of BR based on the conversion of molecules from the bR-state to the M-state. In its native form, at nominal pH, relative humidity and ambient temperature, the lifetime of the M-state is about 10ms. However, the image lifetime can be increased by several orders of magnitude using chemical” and mutagenicI methods. The M-state can also be stabilized by temperature control at -4O’C.” Additionally, one can actively erase images by exposing the material in the M-state to blue light, which directly

J. D. Downie


returns the molecule to the bR-state. Thus, BR films can essentially be used as real-time, erasable optical photographic or holographic films which require no development process, although in general the images recorded are not permanent. Other characteristics of importance for the application considered include a potentially fast image-writing rate limited to about 50 ps by the intrinsic photocycle time, and very high spatial resolution of several thousands of line pairs/mm due to the small molecular size of BR.




The film that we used in this experiment was prepared from the wildtype form of BR by Bend Research, Inc. After extraction from the bacteria, the PM material was suspended in a polymer solution and deposited on an optically flat substrate. The film was dried in a controlled humidity environment to lower the water content. The M-state lifetime (image lifetime) was lengthened to approximately 10s by means of chemical additives, producing a solution pH of about 9.0. Finally, the film was sealed between the substrate and another glass plate to both protect the film and ensure a uniform thickness across the aperture of 1 inch diameter. The film thickness was approximately 100 pm and the optical density (OD) of the film in the bR-state at the absorption peak wavelength was 2.4.

4.1 Transmission


The light transmission characteristics of the film were measured experimentally using a variable-strength write beam and a weak read beam whose intensity was small enough so as to not affect the transmission measurement. The experimental set-up is shown schematically in Fig. 2. Both write and readout beams are plane waves from an argon ion laser at 514.5 nm. The film is first exposed to the write beam, which causes part of the bR-state population to transverse the photocycle and switch predominantly to the M-state. In a steady-state condition, the read beam is passed through the exposed area of the film, and the transmitted intensity is measured with a photodetector. The data describing the relationship between transmitted intensity of the read beam Iread and the incident power density of the write beam E,,if,


Optical processing of speckle images with bacteriorhodopsin

detector RP




Fig. 2. Schematic of optical set-up for making measurements of transmission characteristics of BR film. The wavelengths of the write beam and the readout beam were both 514.5 nm.

are given in Fig. 3. It is clear that the function is logarithmic central part of the curve, i.e. Ztrans,read







over the


for some constants K and c. Thus, in general the transmittance of the weak readout beam is controlled by the power density of the write beam. This is, of course, the principle on which the basis of using BR films as optically addressed spatial light modulators is founded. For general use, the write and read beams may be either coherent or incoherent. For example, incoherent-to-coherent light conversion can


z : z ._ E z

‘0.75 0.5 -

Write beam power density


Fig. 3. Transmittance of read beam as a function of write beam power density for wildtype BR film. Both beams have wavelength 514.5 nm.

J. D. Downie


be obtained by making the write beam incoherent and reading out with a weak coherent beam. For our application here of transforming speckle images, both the writing and reading beams are coherent. The transmittance relation disilayed in Fig. 3 and expressed in eqn (7) shows that this film exhibits a logarithmic transfer function, which is exactly that required for the transformation of images with multiplicative noise such as speckle. In addition, the dynamic range of the write power over which this property holds is large (> two orders of magnitude). I note, however, that the transmittance relation is dependent upon many variables, including film thickness, optical density, M-state lifetime and readout wavelength.** Films with different characteristics will display significantly different transmission properties. 4.2 Speckle image transformation Given the transmittance results of Fig. 3, I then performed an experiment to optically implement the logarithmic transformation of an image with the BR film. The optical set-up is shown in Fig. 4. The argon laser beam is collimated and then split into two beams, each individually controlled with an electronic shutter. With shutter A open and shutter B closed, a transparency in contact with a diffuser is illuminated and an imaging system forms the speckle image of the transparency on the BR film. The power level of the exposing beam is set to ensure that we are operating in the cenr ral logarithmic region of the transmission curve of Fig. 3. The exposure time is approximately 1 s. Immediately after recording the speckle image in the film, shutter A

CCD camera


w Ibt

lens 1

BR film 5s \


Object and diffuser mirror

V lens 2 shutter 5 II

I 5s

shutter A


of speckle image with BR film. Fig. 4. Optical set-up for logarithmic transformation Shutter A is opened during the write process and shutter B is opened during the read process.

Optical processing of speckle images with bacteriorhodopsin


is closed, shutter B is opened and the beam transmitted through the BR film is imaged into a CCD camera. The read beam power level is again weak to ensure that it does not affect the image written in the BR film and also so that it will be within the linear operating range of the CCD camera. Thus, the readout beam transmittance is spatially controlled by the intensity distribution of the writing beam which contains the image information. We evaluated the performance of the BR film with a two-tone object encoded on the transparency. The right half of the object is clear and the left half is a gray level with intensity transmittance at 514.5 nm approximately one-third that of the clear side. The speckle image of this object as obtained at the BR film plane is shown in Fig. 5a. The image obtained by the CCD camera upon transmission of the read beam through the BR film after recording is shown in Fig. 5b. We analysed both images in Fig. 5 and determined the first- and second-order statistics. These results are presented in Table 1. It is clear that the speckle image has statistics consistent with a negative exponential pdf, as the mean and standard deviation of each area of the image are essentially equal, even though the mean is greater for the right half by a factor of about 3. On the other hand, the image transmitted by the BR film shows quite different statistics; the means of the two halves are different but the standard deviations are almost exactly equal. Thus, the noise of the transmitted image is indeed found to be signal-independent. It can also be easily verified that the ratio of means of the two halves of the BR-transmitted image is consistent with that expected from the logarithmic transformation of the speckle image. As further evidence of the appropriate nature of the pdfs of the speckle image and the transformed image, the histograms of the right-hand sides of Fig. 5a and b are presented in Fig. 6, which demonstrates that the speckle image pdf closely resembles a negative exponential, while the transformed image noise is clearly additive and nearly Gaussian. 4.3 Correlation of speckle and transformed images Finally, as a demonstration of the advantage afforded by transforming speckle images that are submitted to an optical correlator for pattern detection and recognition, I performed a similar experiment to that discussed above with the two-tone object. In this case, I used a simple binary object corresponding to the letter ‘T’. The object was light on a dark background. As before, I captured the original speckle image T~~:,(x, y) of the object placed in contact with a diffuser, and also an


J. D. Downie


(W Fig. 5.

(a) Original speckle image of two-tone object. (b) Logarithmically image of object obtained through BR film.


image of the liglht transmitted by the BR after recording the speckle image in the BR film. These two images are shown in Fig. 7. I digitally simula .ted the (optical correlation of these images with a binary phase- ,only filter (BPOF)19 designed for the binary image. The BPOF

Optical processing of speckle images with bacteriorhodopsin


TABLE 1 Relative Statistics of Speckle Image and Transformed Image (absolute numbers are unrelated between two images) Speckle image

Mean I_L St. dev. (T

Image transmitted by BR

leji half

right half

left half

right half

31.9 29.0

98.5 96.8

90.0 37.4

121.7 37.4

design is expressed as Hb&u, V) = sign [real {S*(u, u)}],


where S*(u, u) is the conjugate of the Fourier transform of the binary object function s(x, y) corresponding to the letter ‘T’. Since in this case s(x, y) is binary, the BPOF as given in eqn (8) is the appropriate design for correlation with both s(x, y) and the transformed object log {s(x, y)}.” The correlation operation performed optically is given as 4x9 y) = F-‘-M4




where F-‘{ } represents the inverse Fourier transform and R(u, u) is the Fourier transform of the input image r(x, y), representing either the speckle or transformed image. The correlation intensities of the BPOF H,,,,&u, u) with the speckle image r&x, y) and the image transmitted by the BR film r&(x, y) a log {T&, y)} are displayed in pseudo-three-dimensional forms in Fig. 8a and b, respectively. The correlation function of the BR-transformed image is clearly smoother and less cluttered than that of the original speckle image. This indicates that the additive image noise in the transformed image is less degrading to the correlation function for this filter than the speckle noise. Of course, these two images are only sample realizations of an infinite number of possible noise patterns. In practice, one often uses the amplitude correlation peak value co as a measure of the match between the input image and the filter. We have estimated the signal-to-noise ratio (SNR) of the correlation peak co for both image types with the filter &,,&u, u) based on the statistics of the noise distributions of the images. The correlation peak SNR is defined as SNR=z,

J. D. Downie





loooo I









BR transformed



1? a!




X ._ P

t “E z


0 16




Fig. 6.

(a) Histogram





of right half of speckle image. (b) Histogram image transmitted by BR film.

of right half of

where E[-] represents the expected value and var [.I represents the variance. The calculated SNR values are given in Table 2, which shows that the correlation peak SNR of the transformed image exceeds that of the speckle image by about 11 dB, a result that agrees qualitatively with earlier theory and simulation predictions.‘**”

Optical processing of speckle images with bacteriorhodopsin



(W Fig. 7.

(a) Speckle image T~,,(x,y) used for optical correlation simulation. trAnsformed image r&(x, y) used for optical correlation simulation.


(b) BR


The application of a logarithmic transformation to images with multiplicative noise is useful to make the noise additive in nature. For the particular case of speckle noise, the logarithmic transform not only

J. D. Downie



(W Fig. 8.

(a) Correlation function of binary phase-only filter HbPO,with original speckle image r&, y). (b) Correlation function of binary phase-only filter H,& with BR transformed image r&(x, y). TABLE 2 Estimated SNR Values of Correlation Peaks for Speckle Image and Transformed Image of Letter ‘T’ Object with Binary Phase-only Filter Speckle image

Image transmitted by BR

14.5 dB

25.5 dB

Optical processing of speckle images with bacteriorhodopsin


produces additive noise, but also makes it signal-independent. Previous work has shown such transformed speckle images to be easier to filter for image reconstruction and also more robust for correlating with a matched filter in an optical correlator system. I have experimentally demonstrated here the optical logarithmic transformation of a speckle image using the photochromic characteristics of BR film, and the subsequent simulated improvement in correlation performance in terms of peak SNR. BR films have very high spatial resolution, require no external processing and may be cycled through the write/read/erase process millions of times without degradation. They thus have the potential to perform rapid optical logarithmic transformations for images with multiplicative noise such as may be input to an optical correlator, or when optical processing of the image may be advantageous to system performance. ACKNOWLEDGEMENT This work was supported by the NASA Office of Advanced and Technology under RTOP 233-02-05-06.


REFERENCES 1. Kate, H. & Goodman, J. W., Nonlinear filtering in coherent optical systems through halftone screen processes. Appl. Opt., 14 (1975), 1813-

1824. 2. Dainty, J. C. (ed.), Laser Speckle and Related Phenomena. Springer, New York, 1975.

3. Downie, J. D. & Walkup, J. F., Optimal correlation filters for images with signal-dependent noise. J. Opt. Sot. Am., All (1994), 1599-1609. J., Biernacki, A. M., Woods, C. L. & Cronin-Golomb, M., Photorefractive quadratic processor for signal recovery from multiplicative complex noise. Opt. Engng, 32 (1993), 2872-2876. Goodman, J. W., Statistical Optics. John Wiley, New York, 1985. Arsenault, H. H. & April, G., Properties of speckle integrated with a finite aperture and logarithmically transformed. J. Opt. Sot. Am., 66 (1976), 1160-1163. Tur, M., Chin, K. C. & Goodman, J. ‘W., When is speckle noise multiplicative? Appl. Opt., 21 (1982), 1157-1159. Arsenault, H. H. & Denis, M., Integral expression for transforming signal-dependent noise into signal-independent noise. Opt. Lett., 6 (1981),

4. Khoury,

5. 6.

7. 8.

210-212. 9. Prucnal,


P. R. & Saleh, B. E. A., Transformation of image-signalnoise into image-signal-independent noise. Opt. Lett., 6 (1981),


10. Brauchle, C., Hampp, bacteriorhodopsin and 420-428.

N. & Oesterhelt, D., Optical applications of its mutated variants. Adu. Mater., 3 (1991),

J. D. Downie


11. Birge, R. R., Photophysics and molecular electronic applications of the rhodopsins. Ann. Rev. Phys. Chem., 41 (1990), 683-733. 12. Thoma, R., Hampp, N., Brauchle, C. & Oesterhelt, D., Bacteriorhodopsin films as spatial light modulators for nonlinear-optical filtering. Opt. Lert., 16 (1991), 651-653. 13. Song, Q. W., Zhang, C., Gross, R. & Birge, R., Optical limiting by chemically enhanced bacteriorhodopsin films. Opt. Lett., 18 (1993), 775777. 14. Trissl, H. W., Garter, W. & Leible, W., Reversed picosecond charge displacement from the photoproduct K of bacteriorhodopsin demonstrated photoelectrically. Chem. Phys. Let?,, 158 (1989), 515-518. 15. Hampp, N., Thoma, R., Oesterhelt, D. & Brauchle, C., Biological photochrome bacteriorhodopsin and its genetic variant ASp96+ Asn as media for optical pattern recognition. Appl. Opt., 31 (1992), 1834-1841. 16. Downie, J. D., Real-time holographic image correction using bacteriorhodopsin. Appl. Opt., 33 (1994), 4353-4357. 17. Miller, A. & Oesterhelt, D., Kinetic optimization of bacteriorhodopsin by aspartic acid 96 as an internal proton donor. Biochim. Biophys. Acta, 1020 (1990), 57-64.

18. Song, Q. W., Zhang, C., Blumer, R., Gross, R. B., Chen, Z. & Birge, R. R., Chemically enhanced bacteriorhodopsin thin-film spatial light modulator. Opt. Len., 18 (1993), 1373-1375. 19. Psaltis, D., Paek, E. G. & Venkatesh, S. S., Optical image correlation with binary spatial light modulator. Opt. Engng, 23 (1984), 698-704. 20. Downie, J. D. & Walkup, J. F., Correlation of images with signaldependent noise. Proc. Euro-American Workshop on Optical Pattern Recognition, La Rochelle, France, June 1994.