针孔投影的反向旋转和平移

Question

所以，我觉得这么长时间试图解决这个问题真的很愚蠢。我真的需要帮助。

我正在尝试使用针孔透视投影将相机像素坐标投影到世界坐标。但不知何故，我产生的光线有奇怪的旋转（方向）。所以函数是：

def pixel_coords_to_world_coords_ray(pixel_coords, camera_matrix, distortion_coeffs, R_world_to_camera, t_world_to_camera, plane_normal_world, plane_distance_world):
    
    pixel_coords_undistorted, newcameramtx = undistort_contour_pixel(pixel_coords, camera_matrix, distortion_coeffs)
    pixel_coords_undistorted = np.squeeze(pixel_coords_undistorted, axis=1)
    pixel_coords_homogeneous = np.hstack((pixel_coords_undistorted, np.ones((len(pixel_coords_undistorted), 1))))
    
    # Invert the camera matrix to get pixel to ray conversion matrix
    camera_matrix_inv = np.linalg.inv(newcameramtx)
    
    # Compute the light ray direction in the camera frame.
    ray_directions_camera = (camera_matrix_inv @ pixel_coords_homogeneous.T)[:3, :]

    # Transform the light ray direction to the world frame
    R_camera_to_world = R_world_to_camera.T
    t_camera_to_world = -R_camera_to_world @ t_world_to_camera
    ray_directions_world = R_camera_to_world @ ray_directions_camera
    
    # Normalize the rays to have unit length
    ray_directions_world /= np.linalg.norm(ray_directions_world, axis=0)
    
    # Compute the intersection between the light rays and the plane
    ray_origin_world = t_camera_to_world.reshape(3)
    world_coords = []
    for ray_direction_world in ray_directions_world.T:
        intersection = ray_plane_intersection(ray_origin_world, ray_direction_world, plane_normal_world, plane_distance_world)
        if intersection is not None:
            world_coords.append(intersection)
    world_coords = np.array(world_coords)

    return world_coords

• 所以我尝试通过将输入像素设置为 (w/2, h/2) 来调试它。它导致 ray_directions_camera = near [0,0,1]
的预期值

但不知何故，我在世界坐标系中光线方向的旋转是错误的。

• 我还绘制了结果，相机的变换和旋转。所以平移和旋转一定是对的

中间向下的红线是像素 (w/2, h/2) 的结果光线

我在这里遗漏了什么吗？已经一个星期了，我试图用许多不同的方法来解决这个问题。我真的需要完成这个。谢谢大家