run_slam keeps losing tracking, keyframes not being added to the graph

[john-maidbot]

Hello,

I am working on running stella-vslam on a ROS2 rosbag created from a simulated stereo camera.

The process runs without any errors or warnings, and I can see the SLAM process happening in the SocketViewer. However, the tracking just keeps failing within 5 seconds of initialization.

I am not sure if this is the problem or just another symptom but It seems that the keyframes are not being stored in the graph. i.e. As the camera moves around in the viewer, previous keyframes just disappear (this happens before the tracking failure), rather than being connected in a graph. the result is that there is no local mapping (which seems like a reasonable explanation for why tracking keeps failing?)

I have tried to tweak my config.yaml (https://stella-cv.readthedocs.io/en/latest/parameters.html) but no luck.

Does anyone know of things I can try? I have also posted my config below in case maybe there is something obviously wrong with it 😅

Thank you for your time!

This is the command:

ros2 run stella_vslam_ros run_slam -v ./Inputs/orb_vocab.fbow -c ./Inputs/stereo_config.yaml --frame-skip 1 --ros-args -r /camera/left/image_raw:=/d435/infra1/image_rect_raw --ros-args -r /camera/right/image_raw:=/d435/infra2/image_rect_raw --ros-args -p use_exact_time:=True

This is the config file

# Stereo config

#==============#
# Camera Model #
#==============#

Camera:
  name: "Sim Realsense d435 stereo"
  setup: "stereo"
  model: "perspective"

  fx: 454.68
  fy: 454.68
  cx: 424.0
  cy: 240.0

  k1: 0.0
  k2: 0.0
  p1: 0.0
  p2: 0.0
  k3: 0.0

  fps: 30.0
  cols: 848
  rows: 480
  focal_x_baseline: 22.6 # 0.05 m baseline * fx
  depth_threshold: 40

  color_order: "Gray"

#=====================#
# Stereo Parameters #
#=====================#

# original intrinsic parameters (K, D) and stereo-recitification parameters (R)
# matrices (K, R) are written in row-major order

StereoRectifier: # Images are already rectified
  K_left: [454.68, 0.0, 424, 0.0, 454.68, 240.0, 0.0, 0.0, 1.0]
  D_left: [0.0, 0.0, 0.0, 0.0, 0.0]
  R_left:
    [
      1.0,
      0.0,
      0.0,
      0.0,
      1.0,
      0.0,
      0.0,
      0.0,
      1.0,
    ]
  K_right: [454.68, 0.0, 424, 0.0, 454.68, 240.0, 0.0, 0.0, 1.0]
  D_right: [0.0, 0.0, 0.0, 0.0, 0.0]
  R_right:
    [
      1.0,
      0.0,
      0.0,
      0.0,
      1.0,
      0.0,
      0.0,
      0.0,
      1.0,
    ]

#=====================#
# Tracking Parameters #
#=====================#

Preprocessing:
  min_size: 800
  depthmap_factor: 1000.0

KeyframeInserter:
  wait_for_local_bundle_adjustment: false

#================#
# ORB Parameters #
#================#

Feature:
  name: "default ORB feature extraction setting"
  scale_factor: 1.2
  num_levels: 8
  ini_fast_threshold: 20
  min_fast_threshold: 7

#====================#
# Mapping Parameters #
#====================#

Mapping:
  baseline_dist_thr_ratio: 0.02
  redundant_obs_ratio_thr: 0.6

#========================#
# Initializer Parameters #
#========================#

Initializer:
  min_num_triangulated_pts: 200

screenshot of the viewer

[john-maidbot]

interesting, when i try with a realsense d435 in the real world and a very similar config file, run_slam does track and the graph is generated

maybe its a lack of features in the simulation environment or some artificial image noise 🤔

[john-maidbot]

I am so sorry for the noise. I found the problem, my messages were not stamped properly 🤦
looks great now 🎉

closing