https://blog.csdn.net/shadow_guo/article/details/44312691
基于SURF特征提取和金字塔LK光流法的单目视觉三维重建
1. 单目视觉三维重建问题
在前面的文章中,笔者用SIFT提取特征后用radio测试剔除了匹配中异常的特征点,然后根据匹配合格的特征点计算基础矩阵和本征矩阵,对本征矩阵SVD分解来估计和构造透视矩阵,根据透视矩阵和齐次坐标变换后的特征点三角化获得特征点在三维空间中的坐标。
(1)找不到外极线
对于运动范围过大的两幅图像,有可能计算不出外极线。经过试验发现,运动范围过大的两帧图像由于SIFT特征点检测后特征点的个数大幅下降,或句话说,SIFT检测特征点没什么问题,但radio测试踢掉了好多异常特征点,特征点个数的减少造成基础矩阵计算得不准确,所以计算外极线时会出现找不到的情况。
(2)仿射结构变化敏感
SIFT检测特征点是很精确的,但为什么检测出的特征点在估计仿射结构时会出现外极点和外极线跳动大的情况呢?个人认为有以下几个方面原因:
a)SIFT检测过精确:SIFT的精确检测剔除了很多本可以匹配的特征点,特征点过少会造成外极线检测误差大,换句话说,SIFT的精确检测结果有可能造成“过拟合”问题;不过可以试试改改SIFT库函数的输入参数,可能会解决;
b)摄像头标定的参数不准确:径向畸变略大也会导致出现扭曲的图像特征点,SIFT检测时出现误检测;
c)图像噪声未补偿:高速运动中的图像需要适当的运动补偿,如果摄像机和跟踪的对象以不同的速度运动,前景和背景同时运动,必然会产生模糊的图像使SIFT特征点检测不准确。
主要出现的问题在a)。b)多次标定摄像头可以解决;c)肉眼观察得到的图像即可判断是否出现问题。
2. 解决单目视觉三维重建问题
(1)SURF特征提取
(2)金字塔Lucas-Kanade光流法
3. 代码实现和改进
(1)代码改进
(2)代码实现
- import cv2
- import math
- import numpy as np
- from match import *
- ################################################################################
- print 'Load Image'
- img1 = cv2.imread('images/cat_1.bmp') #query image
- img2 = cv2.imread('images/cat_2.bmp') #train image
- rows, cols, channels = img1.shape
- img1 = cv2.cvtColor(img1, cv2.COLOR_BGR2RGB)
- img2 = cv2.cvtColor(img2, cv2.COLOR_BGR2RGB)
- imgGray1 = cv2.cvtColor(img1, cv2.COLOR_RGB2GRAY)
- imgGray2 = cv2.cvtColor(img2, cv2.COLOR_RGB2GRAY)
- ################################################################################
- print 'SURF Feature Detection'
- # initialize ORB object with default values
- surf = cv2.SURF(800)
- # find keypoints
- keypoint1, descriptor1 = surf.detectAndCompute(imgGray1, None)
- keypoint2, descriptor2 = surf.detectAndCompute(imgGray2, None)
- ################################################################################
- def keypointToPoint(keypoint):
- '''''
- from keypoints to points
- '''
- point = np.zeros(len(keypoint) * 2, np.float32)
- for i in range(len(keypoint)):
- point[i * 2] = keypoint[i].pt[0]
- point[i * 2 + 1] = keypoint[i].pt[1]
- point = point.reshape(-1,2)
- return point
- point1 = keypointToPoint(keypoint1)
- rightFeatures = keypointToPoint(keypoint2)
- ################################################################################
- print 'Calculate the Optical Flow Field'
- # how each left points moved across the 2 images
- lkParams = dict(winSize=(15,15), maxLevel=2, criteria=(3L,10,0.03))
- point2, status, error = cv2.calcOpticalFlowPyrLK(imgGray1, imgGray2, point1, None, **lkParams)
- # filter out points with high error
- rightLowErrPoints = {}
- for i in range(len(point2)):
- if status[i][0] == 1 and error[i][0] < 12:
- rightLowErrPoints[i] = point2[i]
- else:
- status[i] = 0
- bf = cv2.BFMatcher(cv2.NORM_L2, crossCheck=True)
- matches = bf.match(descriptor1, descriptor2)
- print 'matches:', len(matches)
- dist = []
- for m in matches:
- dist.append(m.distance)
- # distance threshold
- thresDist = np.median(dist)
- good = []
- for m in matches:
- if m.distance < thresDist:
- good.append(m)
- print 'Good Matches:', len(good)
- ################################################################################
- # select keypoints from good matches
- points1 = []
- points2 = []
- for m in good:
- points1.append(keypoint1[m.queryIdx].pt)
- points2.append(keypoint2[m.trainIdx].pt)
- points1 = np.float32(points1)
- points2 = np.float32(points2)
- ################################################################################
- # combine two images into one
- view = drawMatches(img1, img2, points1, points2, colors)
- img5, img3 = drawEpilines(img1, img2, points1, points2)
- displayMatchImage(view, img5, img3)
- # camera matrix from calibration
- K = np.array([[517.67386649, 0.0, 268.65952163], [0.0, 519.75461699, 215.58959128], [0.0, 0.0, 1.0]])
- P, P1, E = calcPespectiveMat(K, F)
- pointCloudX, pointCloudY, pointCloudZ, reprojError = triangulatePoints(points1, points2, K, E, P, P1)
- positionX, positionY, positionZ = transformToPosition(pointCloudX, pointCloudY, pointCloudZ, P1, K, scale=10.0)
- plotPointCloud(positionX, positionY, positionZ, colors)
- ################################################################################
- print 'Goodbye!'