• documentation
main content

stereo visual slam for uav navigation in 3d simulation -凯发k8网页登录

this example uses:

visual slam is the process of calculating the position and orientation of a camera with respect to its surroundings while simultaneously mapping the environment. developing a visual slam algorithm and evaluating its performance in varying conditions is a challenging task. one of the biggest challenges is generating the ground truth of the camera sensor, especially in outdoor environments. the use of simulation enables testing under a variety of scenarios and camera configurations while providing precise ground truth.

this example demonstrates the use of unreal engine® simulation to develop a visual slam algorithm for a uav equipped with a stereo camera in a city block scenario. for more information about the implementation of the visual slam pipeline for a stereo camera , see the (computer vision toolbox) example.

set up simulation environment

first, set up a scenario in the simulation environment that can be used to test the visual slam algorithm. use a scene depicting a typical city block with a uav as the vehicle under test.

next, select a trajectory for the uav to follow in the scene. you can follow the (automated driving toolbox) example to interactively select a sequence of waypoints and then use the helperselectscenewaypoints function to generate a reference trajectory for the uav. this example uses a recorded reference trajectory as shown below:

% load reference path
data        = load("uavstereoslamdata.mat");
pos         = data.pos;          % position
orienteuler = data.orienteuler;  % orientation

uavtrajectoryvslam.png

the uavvisualslamin3dsimulation simulink® model is configured with the us city block scene using the block. the model places a uav on the scene using the block. a stereo camera consisting of two blocks is attached to the uav. in the dialog box of the block, use the mounting tab to adjust the placement of the camera. use the parameters tab to configure properties of the camera to simulate different cameras. to estimate the intrinsics of the stereo camera that you want to simulate, use the (computer vision toolbox) app.

% stereo camera parameters
focallength    = [1109, 1109]; % in pixels
principalpoint = [640, 360];   % in pixels [x, y]
imagesize      = [720, 1280];  % in pixels [mrows, ncols]
baseline       = 0.5;          % in meters
% open the model
modelname = 'uavvisualslamin3dsimulation';
open_system(modelname);

vslammodeluav.png

implement the stereo visual slam algorithm

the helper stereo visual slam system block implements the stereo visual slam pipeline, consisting of the following steps:

  • map initialization: the pipeline starts by initializing the map of 3-d points from a pair of images generated from the stereo camera using the disparity map. the left image is stored as the first key frame.

  • tracking: once a map is initialized, for each new stereo pair, the pose of the camera is estimated by matching features in the left image to features in the last key frame. the estimated camera pose is refined by tracking the local map.

  • local mapping: if the current left image is identified as a key frame, new 3-d map points are computed from the disparity of the stereo pair. at this stage, bundle adjustment is used to minimize reprojection errors by adjusting the camera pose and 3-d points.

  • loop closure: loops are detected for each key frame by comparing it against all previous key frames using the bag-of-features approach. once a loop closure is detected, the pose graph is optimized to refine the camera poses of all the key frames.

for the implementation details of the algorithm, see the (computer vision toolbox) example.

run stereo visual slam simulation

simulate the model and visualize the results. the video viewer block displays the stereo image output. the point cloud player displays the reconstructed 3-d map with the estimated camera trajectory.

if ~ispc
    error("unreal engine simulation is supported only on microsoft"   char(174)   " windows"   char(174)   ".");
end
% set the random seed to get consistent results
rng(0);
% run simulation
sim(modelname);

figure point cloud player contains an axes object. the axes object contains 2 objects of type scatter, line. these objects represent map points, estimated trajectory.

loop edge added between keyframe: 2 and 91

stereoimageuav.png

close the model.

close_system(modelname);

references

[1] mur-artal, raul, and juan d. tardós. "orb-slam2: an open-source slam system for monocular, stereo, and rgb-d cameras." ieee transactions on robotics 33, no. 5 (2017): 1255-1262.

网站地图