A taxonomy of texture models

Transkript

1 INF Lecture Texture based on filtering Based on: M. Tuceryan and A. Jain, Texture analysis, in Handbook of Pattern Recognition and Computer Vision, C. Chen, L. Pau, W. Wang (editors) T. Randen and J. H. Husøy, Filtering for texture classification: a comparative study, IEEE Tr. PAMI, vol. 21, no. 4. April A. Jain and F. Farrokhnia, Unsupervised texture segmentation using Gabor filters, Pattern Recognition, vol. 24, Texture INF AS 1 A taxonomy of texture models We can characterize texture models into different groups: Statistical models GLCM, GLRL, Autocorrelation features Geometrical models Voronoi tesselation, structural models Model-based methods Markov random field models Fractals Signal-processing methods Frequency-based methods like wavelets, Gabor filters, filter banks, etc. Texture INF AS 2

2 Texture based on filtering To discriminate textures containing structures with different spatial frequencies or different orientations, spatial filtering methods are useful. The most common approach is to set up a filter bank with different filter, and compute the response to a set of filters with covering different ranges of the frequence spectrum. A special feature extraction function is then used to combine the filter outputs to texture descriptors. A simple approach is to use edge detection filters. This frequency-based approach is best suited to texture which can be identified as belonging to different regions of the Fourier spectrum. Texture INF AS 3 Background: spatial frequencies and Fourier transform Nature contains structures with an infinite set of frequencies. Fourier theory: an image can be described by a finite sum of cosinus and sinus functions with different frequencies. A digital image will consist of a finite set of frequencies. Texture INF AS 4

3 Sum av sinusoider + = + + = = Texture INF AS 5 Fourier-teori: introduksjon Alle periodiske funksjoner kan uttrykkes som en sum av sinus og cosinusfuksjoner med ulik frekvens og amplitude. Vi antar at bildene repereres i et grid slik at vi får en periodisk sekvens. Sinus og cosinusfunksjonene utgjør basisvektorer, og bildet kan uttrykkes vha. dem vektet med ulike koeffisienter. Bildet kan representeres vha. disse koeffisientene. Texture INF AS 6

4 2D Diskret Fouriertransform (DFT) f(x,y) er et N N-bilde Definisjon: F( u, v) = Inverstransform: f ( x, y) = Kan også skrives: 1 N 1 N N 1 N 1 x= 0 y= 0 N 1N 1 u= 0v= 0 f ( x, y) e F( u, v) e 2π ( ux+ vy) j N 2π ( ux+ vy) j N F( u, v) = 1 N N 1 N 1 x= 0 y= 0 2π ( ux + vy) f ( x, y) cos N 2π ( ux + vy) j sin N Texture INF AS 7 Bildet f(x,y) er et N N bilde. F(u,v) er et N N bilde med komplekse verdier. F(u,v) kan representeres som et bilde med to bånd, enten: R(u,v) - Realdelen I(u,v) - Imaginærdelen eller F(u,v) - amplituden eller spekteret Forteller noe om hvilke frekvenser bildet inneholder φ(u,v) - fasen Visuelt ser fasebildet ut som støy, men fasen inneholder informasjon om hvor i bildet objektene er. Texture INF AS 8

5 Sinus image Fourier spectrum Texture INF AS 9 Eksempler Texture INF AS 10

6 Texture INF AS 11 Filtrering i frekvensdomenet Konvolusjon i bildedomenet er ekvivalent med multiplikasjon i frekvensdomenet-> romlig filtrering kan gjøres i frekvensdomenet, og vi kan designe filtere som plukket ut bestemte frekvenser. Implementasjon av filtrering i frekvensdomenet: 1. Beregn DFT av bildet 2. Beregn DFT av filterkjernen 3. Multipliser de to transformerte leddene 4. Transformer resultatet tilbake til bildedomenet vha. invers DFT (IDFT) Det er filterdesign i frekvensdomenet vi nå skal bruke til tekstur. Texture INF AS 12

7 Texture INF AS 13 Repetisjon filtere i frekvensdomenet Lavpassfiltere Høypassfiltere Båndpassfiltere Båndstoppfiltere Texture INF AS 14

8 Texture INF AS 15 Texture based on filter banks Original 1D profile Filtered profile Nonlinear transform Smoothed Resulting 2D feature image Texture INF AS 16

9 Designing the filters in the filter bank A filter bank is a collection of spatial filters which covers the most interesting parts of the frequency domain. To detect a set of textures, a filter bank with filters that are tailored to the frequency characteristics of the texture is needed. The main idea is to partition the frequency space into different regions and apply one filter for each region. How do we partion the frequency domain, and how many filters do we use? Can we use prior knowledge about the textures to tailor the filters? Texture INF AS 17 Unsupervised filter banks Unsupervised means that no information about the textures is used when selecting the filter banks. Several approaches have been tried: Laws filter masks Ring and wedge filters Gabor filter banks Wavelet transform Discrete Cosine Transform Quadrature Mirror Filters Texture INF AS 18

10 Ring and wedge filters Coggins and Jain (1985) used ring and wedge filters filters 7 ring filters representing different spatial frequencies. These do not give any information about texture orientation. 4 wedge filters gives some broad information about texture orientation. All filters have Gaussian cross-section profiles. Texture INF AS 19 Frequency response - ring and wedge filters Texture INF AS 20

11 Gabor filter banks Based on phychophysical experiments for the human visual system. Campbell and Robson (1968) proposed that the visual system decomposes the retinal image into a number of filtered images, each of which contains intensity variations over a narrow range of frequency/size and orientation. The different filters can be seen a multi-channel filtering. Texture INF AS 21 Gabor filter kernels We consider even-symmetric Gabor-filters of the following 2 2 form: 1 x y 2 2 σ x σ y h( x, y) = e cos(2πf0 x) + 2 This yields a filter with orientation 0. Other orientations are found by rotating the reference coordinate system x,y. (Orientations 0, 45, 90, and 135 are often used.) The Fourier-transform of a Gaussian function is a Gaussian, thus the filter frequency respons for each filter is a Gaussian function with a given center frequency and width. Gabor filters are claimed to give optimal localization properties both in the spatial and in the frequency domain (mainly because of their shape (Gaussian)). Different filter parameters can be choosen. Texture INF AS 22

12 Frequency respons for a Gabor filter bank Jain and Farroknia suggests a set of filters with center frequencies ,,,, and orientations 0, 45, 90, and 135 This gives a almost uniform coverage of the spectrum. Texture INF AS 23 Wavelet filter banks The Gabor function in the spatial domain is an example of a wavelet. The discrete wavelet transform corresponds to a filter bank with particular filter parameters and subband decomposition. There are many familites of wavelets. A common family is the Daubechies family (Daubechies 1992) Texture INF AS 24

13 From the output of a filter bank to texture features The result after applying a filter bank with M filters to an image is M filtered images. If texture is computed in a local window, M subimages result from each window position. These cannot be used as feature vectors directly. We try some kind of feature extraction to the filtered images. We are looking for a feature extraction step that will give constant feature values for regions with equal texture, and different for region with different texture. There is no evident way to do this. A common approach is to first perform a non-linear transform, then to smooth the resulting image. The success of the texture model will depend on the success of this step. Texture INF AS 25 Jain and Farrokhnias feature extraction approach First, each filter is subjected to a nonlinear transform using a tanh function 1 e ψ ( t) = tanh( αt) = 1+ e 2αt 2αt whereα is a constant This results in a threshold-like function and gradual changes in the filtered images are converted to squarelike blobs. Then, they compute the average deviation from the mean in small overlapping windows 1 ek ( x, y) = ψ M ( r ( a, b) ) where r is the filtered image no. k 2 k k ( a, b) This is similar to Law s texture feature. Texture INF AS 26

14 The smoothing step The method is sensitive to the window size M. In filter design, there is a trade-off between good frequency localization (filters with large spatial width) and filters with high spatial resolution (filters which covers a broad part of the frequency spectrum). Gaussian filters are better than rectangular filters because the Fourier transform of a Gaussian function is another Gaussian function (and we get better trade-off between spatial resolution and frequency localization). Too smooth low frequencies, we need a filter with large window size, while smaller window sizes can be used for higher frequencies. Thus, let the filter size be a function of the center frequency of the window. Texture INF AS 27 Let f 0 be the radial center frequency for the a bandpass filter in the filter bank. Use the following corresponding Gaussian lowpass filter h G ( n) = 1 e 2πσ where σ s = 2 s 1 2 f 1 n 2 2σ 0 2 s Other choices are also used. Texture INF AS 28

15 The texture segmentation or classification step After the filtering, nonlinear transform and smoothing, a set of K feature images result. How do we use these to discriminate between various textures? It is possible to use them as input to a regular feature selection process. They can be used either individually or as a multivariate feature vector. Either unsupervised texture segmentation or supervised texture classification can be applied. Texture INF AS 29 Texture INF AS 30

16 Frequency-based texture computation: when does it work? Scientists often report good results for texture patches like Broadatz-images which contains large regions of different textures with different orientation. Such synthetic texture patches have large regions and sharp borders between different textures. This is often not the case in real applications! A keyword is orientation: do the textures we want to discriminate have different orientations, or are they isotropic? For isotropic textures, filtering methods are often not so good. Texture INF AS 31