Lab 2 04.02.2016
Ressurser OpenCV documentation: http://opencv.org/documentation.html Eigen documentation : http://eigen.tuxfamily.org/dox/ Quick reference quide: http://eigen.tuxfamily.org/dox/group QuickRefPage.html C++: http://en.cppreference.com/w/ Image Watch: An image debugger plug-in for Visual Studio Download directly from Visual Studio: Tools Extensions and Updates Online Search for Image Watch 2
Litt om cv::mat 3
Huskeliste Lage prosjekt Konstruere en cv::mat Datatyper, lage «vinduer» MatCommaInitializer Hente og endre piksler at< >() foreach Regne på matriser Operasjoner, MatExpr, 4
Perspektivkameramodellen 5
The perspective camera model C is a standard coordinate system for a camera x C - Right y C - Down z C - Forward Perspective Camera C q Q Point observed by the camera C y C V ξ C V x C z C x V W ξ V W z W x W y W V is a standard coordinate system for a vehicle x V - Forward y V - Right z V - Down z V Vehicle y V World W is a local NED coordinate system (North East Down) 6
The perspective camera model The point Q Position W q = 27.3, 20.9, 11.5 T The vehicle 3m wide, 6m long The origin of V is chosen to be at the center of the vehicle, 1m above the ground Pose relative to the world: x = 6.7m y = 2.4m z = 14.0m rrrr = 3.7 ppppp = 9.3 heeeeee = 307.6 The camera The optical center is 2m in front of and 1m to the left of the vehicles center The optical center is 4m above ground The y-axis of C is perpendicular to the xy-plane of V The optical axis, i.e. the z-axis of C, is rotated 4.7 to the right of the x-axis of V The camera calibration matrix is 1028 0 400 K = 0 1028 300 0 0 1 7
The perspective camera model Problem In which pixel of the image will we observe the point Q? Sub-problems 1. W W Represent ξ V as a SE(3) object T V 2. V V Represent ξ C as a SE(3) object T C 3. W W Represent ξ C as a SE(3) object T C 4. Determine the camera matrix P = K R t 5. Determine the pixel u = u, v that the point Q projects to according to the perspective camera model 8
Sub-problem 1 Represent W W ξ V as a SE(3) object T V Sophus::SE3d objects can be initialized with a Eigen::Matrix3d rotation matrix R and a Eigen::Vector3d translation vector t Roll, pitch, heading relates to the zyx-rotation sequence, so R = R z R y R x A basic rotation matrix like R x θ can be created by Eigen::AngleAxisd(theta * M_PI / 180, Eigen::Vector3d::UnitX()) // Visualization of world and vehicle cv::mat cv_t_w_v, cv_r_w_v; cv::eigen2cv(t_w_v, cv_t_w_v); cv::eigen2cv(r_w_v, cv_r_w_v); cv::affine3d cv_t_w_v(cv_r_w_v, cv_t_w_v); cv::viz::viz3d my_window("window 1"); my_window.showwidget("world-axes", cv::viz::wcoordinatesystem(3.0)); my_window.showwidget("vehicle-axes", cv::viz::wcoordinatesystem(3.0), cv_t_w_v); my_window.showwidget("vehicle", cv::viz::wcube(cv::vec3d(-3.0, -1.5, -1), cv::vec3d(3.0, 1.5, 1.0)), cv_t_w_v); my_window.spin(); 9
Sub-problem 2 and 3 V V Represent ξ C as a SE(3) object T C Which basic rotations must V undergo in order to coincide with C? Two basic rotations is enough Represent W W ξ C as a SE(3) object T C Recall that W W T C = T V V T C // Visualization of camera frustum cv::mat cv_t_w_c, cv_r_w_c, cv_k; cv::eigen2cv(k, cv_k); cv::eigen2cv(t_w_c.translation(), cv_t_w_c); cv::eigen2cv(t_w_c.rotationmatrix(), cv_r_w_c); cv::affine3d cv_t_w_c(cv_r_w_c, cv_t_w_c); my_window.showwidget("camera_frustum", cv::viz::wcameraposition(cv_k, 1.0, cv::viz::color::red()), cv_t_w_c); 10
Sub-problem 4 and 5 Determine the camera matrix P = K R t C C Recall that in the perspective camera model R = R W and t = t W, so we can not read R and W t directly from T C Determine the pixel u = model Recall that u = PX u, v that the point Q projects to according to the perspective camera // Visualization of point (as a small sphere) my_window.showwidget("q", cv::viz::wsphere({ Q_W(0), Q_W(1), Q_W(2) }, 0.1)); 11
Laplace blending 12
Steg 1: Lag nytt prosjekt og vis frem bilder Kopier «opencv_project_template» og gi nytt navn Husk å endre «PROJECT_NAME» i CMakeLists.txt Lag nytt prosjekt med Cmake Skriv et program som leser to bilder img_1: free_cat.jpg img_2: free_tiger.jpg Bildene bør konverteres til flyttallsbilder: cv::imread( ).convertto(img_1, CV_32F, 1.0/255.0); Vis bildene frem cv::namedwindow() cv::imshow() 13
Steg 2: Enkel blanding av bilder Samregistrer bildene ved å angi tre punktkorrespondanser cv::point2f pts_1[] = {{321, 200}, {647, 200}, {476, 509}}; cv::point2f pts_2[] = {{441, 726}, {780, 711}, {615, 1142}}; cv::mat trans_mat = cv::getaffinetransform(pts_2, pts_1); cv::warpaffine(img_2, img_2, trans_mat, img_1.size()); Lag maske med rampe cv::mat mask = cv::mat::zeros(img_1.size(), CV_32FC1); cv::rectangle(mask, cv::rect{img_1.cols/2, 0, img_1.cols/2 + 1, img_1.rows}, 1, CV_FILLED); cv::blur(mask, mask, cv::size{3, 3}); 14
Steg 2: Enkel blanding av bilder Lag funksjon som gjør enkel blanding av to bilder vektet med masken cv::mat linear_blend(cv::mat& img_1, cv::mat& img_2, cv::mat& mask) Tips: cv::blendlinear() Bruk funksjonen og vis frem resultatet Prøv med forskjellige masker Andre sømmer, sirkler, større glattefilter 15
Steg 3: Laplaceblanding Lag funksjonen std::vector<cv::mat> construct_gaussian_pyramid(cv::mat& img) cv::pyrdown() 16
Steg 3: Laplaceblanding Lag funksjonen std::vector<cv::mat> construct_laplacian_pyramid(cv::mat& img) Bruk construct_gaussian_pyramid cv::pyrup() 17
Steg 3: Laplaceblanding Lag funksjonen cv::mat collapse_pyramid(std::vector<cv::mat>& pyr) cv::pyrup() 18
Steg 3: Laplaceblanding Lag funksjonen cv::mat laplace_blending(cv::mat& img_1, cv::mat& img_2, cv::mat& mask) For eksempel slik: std::vector<cv::mat> mask_pyr = construct_gaussian_pyramid(mask); std::vector<cv::mat> img_1_pyr = construct_laplacian_pyramid(img_1); std::vector<cv::mat> img_2_pyr = construct_laplacian_pyramid(img_2); std::vector<cv::mat> blend_pyr(img_1_pyr.size()); for (int i = 0; i < img_1_pyr.size(); i++) { // TODO: Perform linear blend on each level. } return collapse_pyramid(blend_pyr); 19
Steg 3: Laplaceblanding Bruk laplace_blending(img_1, img_2, mask) Vis frem resultat Sammenlign med enkel blending 20
Steg 4: Moroplukk Prøv andre bilder Ta bilder med kameraet Finn bilder på nett Angi nye punkter for samregistrering Andre masker Last ned GIMP for å tegne finere masker 21
Steg 5: Dypdykk Implementer pyramiden selv Ikke bruk cv::pyrdown() eller cv::pyrup() Ta en titt på cv::seamlessclone() Prøv å implementere warpingen selv 22