OPPA European Social Fund Prague & EU: We invest in your future.

Transkript

1 OPPA European Social Fund Prague & EU: We invest in your future.

2 Talk Outline appearance based tracking patch similarity using histogram tracking by mean shift experiments, discussion Mean shift Tomáš Svoboda, Czech Technical University in Prague, Center for Machine Perception Last update: April 8, 23 Please note that the lecture will be accompanied be several sketches and derivations on the blackboard and few live-interactive demos in Matlab

3 Appearance based tracking 2/2 2 2 illustration from []

4 Histogram based representation 3/2 image patch 3 histogram for 6 bins frequency intensity

5 Patch comparison 4/2 image patch 3 histogram for 6 bins frequency image patch frequency intensity histogram for 6 bins intensity

6 histogram difference assume normalized histograms, i.e m u= p u = 5 image patch 3 25 histogram for 6 bins 5/2 2 d = ρ[p, q] 5 2 frequency 5 where ρ[p, q] is the Bhattacharyya coefficient intensity ρ[p, q] = m u= pu q u 5 5 image patch frequency histogram for 6 bins intensity

7 Similarity measured by the Bhattacharyya coefficient 6/ Similarity surface of the frame The object is the 4 plate and the model is histogram of image intensities. s(y) = m u= pu (y)q u where p(y) is the histogram of image patch at position y and q is the histogram of the template.

8 Problem: finding modes in probability density 7/ Similarity surface of the frame the complete enumeration of similarity surface can be costly, can we do it faster and more elegantly?

9 Density Gradient Estimation 8/2

10 Mean shift procedure 9/2 3 3 Figure borrowed from [4]

11 Meanshift segmentation of colours - color distribution /2 4 4 Figure from [2]

12 Meanshift segmentation of colours - color modes seeking /2 5 5 Figure from [2]

13 Mean shift segmentation - intensity and space 2/2 u, v are here spatial pixel coordinates different normalization for intensity and spatial coordinates

14 Multivariate kernel density estimator 3/2 Normal kernel, K N (x) = exp( 2 x 2 ) Epanechnikov kernel, K E (x) = x Given n data points x i in d-dimensional space R d. f h,k (x) = nh d n i= K ( ) x xi h

15 Multivariate kernel density estimator 3/2 Normal kernel, K N (x) = exp( 2 x 2 ) Epanechnikov kernel, K E (x) = x Given n data points x i in d-dimensional space R d. f h,k (x) = nh d n i= K ( ) x xi h looking for extremum of fh,k(x)

16 Multivariate kernel density estimator 3/2 Normal kernel, K N (x) = exp( 2 x 2 ) Epanechnikov kernel, K E (x) = x Given n data points x i in d-dimensional space R d. f h,k (x) = nh d n i= K ( ) x xi h looking for extremum of fh,k(x) gradient fh,k(x) =

17 Kernels 4/2 Normal kernel, K N (x) = exp( 2 x 2 ) Epanechnikov kernel, K E (x) = x Can be seen as membership function. Remind Kernel density estimation (Parzen method). Remember convolution? 6 6 Taken from

18 Mean-shift iterations 5/2 Assuming a reasonable differentiable kernel K, iterate till convergence: y k+ = n i= x ig( y k x i 2 ) n i= g( y k x i 2 ) g is the derivative of kernel profile.

19 Mean-shift tracking - ratio histogram Ratio histogram: r u = min ( ) qu, p u 6/2 where q is the histogram of the target and p is the histogram of the current frame. w i = r b(xi ) (just binning) Image intensities (or colors) are tranformed into weights, w i, by back projection of the ratio histogram. Mean-shift iterations: y k+ = n i= w ix i g( y k x i 2 ) n i= w ig( y k x i 2 )

20 Mean-shift tracking - Bhattacharya coeficient model, coordinates x i centered at : 7/2 q u = C n i= k( x i 2 )δ(b(x i ) u) target candidate centered at y: n h p u (y) = C h i= k ( y x i h 2) δ(b(x i ) u)

21 ms tracking - object and its model 8/2 7 7 Figure from [5]

22 ms tracking - iterations 9/2 8 8 Figure from [5]

23 Mean-shift originally from [3]. References 2/2 [] Robert Collins. CSE/EE486 Computer Vision I. slides, web page. Robert kindly gave general permission to reuse the material. [2] Dorin Comaniciu and Peter Meer. Mean shift: A robust approach toward feature space analysis. IEEE Transactions on Pattern Analysis and Machine Analysis, 24(5):63 69, May 22. [3] Keinosuke Fukunaga and Larry D. Hostetler. The estimation of the gradient of a density function, with appilcations in pattern recognition. IEEE Transactions on Information Theory, 2():32 4, January 975. [4] Milan Šonka, Václav Hlaváč, and Roger Boyle. Image Processing, Analysis and Machine Vision. Thomson, 3rd edition, 27. [5] Tomáš Svoboda, Jan Kybic, and Václav Hlaváč. Image Processing, Analysis and Machine Vision. A MATLAB Companion. Thomson, 27. Accompanying www site

24 End 2/2

25

26 image patch 3 histogram for 6 bins frequency intensity

27 image patch 3 histogram for 6 bins frequency intensity

28 image patch 8 7 histogram for 6 bins frequency intensity

29 image patch 3 histogram for 6 bins frequency intensity

30 image patch 8 7 histogram for 6 bins frequency intensity

31

32 Similarity surface of the frame

33

34 Similarity surface of the frame

35

36

37

38

39 Normal kernel, K N (x) = exp( 2 x 2 )

40 Epanechnikov kernel, K E (x) = x

41

42 Normal kernel, K N (x) = exp( 2 x 2 )

43 Epanechnikov kernel, K E (x) = x

44

45

46

47

48

49 OPPA European Social Fund Prague & EU: We invest in your future.