Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(nebula_decoders_hesai) unify pointcloud order and filtering of all sensors #56

Closed
wants to merge 11 commits into from

Conversation

mojomex
Copy link
Collaborator

@mojomex mojomex commented Aug 21, 2023

Unified output and filtering behavior for all Hesai decoders, fixes #48 (duplicate points in output), fixes #49 (incoherent distance checks).

PR Type

  • Fix
  • Improvement

Related Links

#48 -- duplicate points being output
#49 -- range checks not being consistent

Description

This PR unifies the behavior of all Hesai decoders to produce outputs with

  • no duplicate points
  • the same ordering return_group > channel > block
  • the same filtering rules
    • point.distance != 0
    • point.distance in [min_range, max_range]
    • dist(point1, point2) >= dual_return_distance_threshold for dual return
  • the same precision regarding point distance

The unit tests are regenerated to match the new conventions.

Detailed changes

Below are the detailed changes made, grouped by sensor.
For details on which comparison operators have been changed, please refer to the table in #49.
For details on how this PR affects output point counts (in the unit test data), see the table in #48.

XT32M

  • Point output order is changed to return_group > channel > block from block > channel to match the other sensors
  • The check whether a scan is complete is adapted to match those of the other sensors (dropped support for time-based scan slicing)
  • Dual return distance threshold checks are added

40P

  • Channels are now processed in ascending index order instead of chronological firing order to match other sensors
  • The LASER_RETURN_TO_DISTANCE_RATE is changed to double to match the precision of the calculations of other sensors
  • min_range and max_range distance checks are added

XT32

  • All dual return types are now correctly interpreted as dual-return (previously only Last+Strongest was recognized)
  • Dual return distance threshold checks are added

AT128

  • All dual return types are now correctly interpreted as dual-return (previously only Last+Strongest was recognized)
  • Dual return distance threshold checks are added

Review Procedure

Remarks

Pre-Review Checklist for the PR Author

PR Author should check the checkboxes below when creating the PR.

  • Assign PR to reviewer

Checklist for the PR Reviewer

Reviewers should check the checkboxes below before approval.

  • Commits are properly organized and messages are according to the guideline
  • (Optional) Unit tests have been written for new behavior
  • PR title describes the changes

Post-Review Checklist for the PR Author

PR Author should check the checkboxes below before merging.

  • All open points are addressed and tracked via issues or tickets

CI Checks

  • Build and test for PR: Required to pass before the merge.

The XT32 decoder, in dual return mode, employs a check whether the
two returned points are identical. The way the point for comparison was
indexed was erroneous and ended up always being the 1st or 2nd block of
the packet, although there are a total of 8 blocks.
This field was missing earlier, leading to avoidable troubleshooting.
The XT32 decoder did not classify First+Last, First+Strongest as dual
return in its check.

Added the return mode constants to the header and to the check
The Pandar40P decoder decoded points in firing time order. Since all
other sensors use logical ordering (by ascending index), Pandar40P is
adapted to this convention.
LASER_RETURN_DISTANCE was defined as float, is now double. This brings
Pandar40P distance accuracy up to the level of all other sensors,
which calculate distance on the fly in higher precision.
Other sensors output in `return_group > channel > block` order, the XT32M
decoder was outputting `block > channel`.

The XT32M scan completion check involved timing checks. These are
currently unused and complicate the code.
Further, they prevent a generic implementation.
Thus, the check has been streamlined.
Fixes tier4#49, fixes tier4#48.

Multiple decoders (QT64, QT128, 64, XT32, 128) had distance checks using
comparison operators different to the other sensors
(e.g. `>` instead of `>=`).

Pandar40P had no min_range / max_range checks.

AT128, XT32, XT32M had no dual_return_threshold checks.

Identical points were not removed when
dual_return_distance_threshold = 0.

All of these checks are corrected/added with this commit.
The sin/cos azimuth lookup tables did not include the channel-wise
azimuth offset in their calculation, leading to up to 0.166 deg of
azimuth error in the output X/Y values.

The tables have been replaced with sinf/cosf calls.
Unit tests for Hesai decoders were not setting
`dual_return_distance_threshold` at all, causing it to be `0`
implicitly. This caused a large number of duplicate points to be
output by the decoders.

The distance is set to `0.1 [m]` for all Hesai sensors now, which is the
default in the Hesai launch file.
@codecov-commenter
Copy link

Codecov Report

Patch coverage: 49.20% and project coverage change: +2.81% 🎉

Comparison is base (5d4b39a) 13.35% compared to head (4ee86a5) 16.17%.

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #56      +/-   ##
==========================================
+ Coverage   13.35%   16.17%   +2.81%     
==========================================
  Files         111       54      -57     
  Lines       10989     8706    -2283     
  Branches     1725     1735      +10     
==========================================
- Hits         1468     1408      -60     
+ Misses       8345     6104    -2241     
- Partials     1176     1194      +18     
Flag Coverage Δ
differential 16.17% <49.20%> (?)
total ?

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed Coverage Δ
...decoders_hesai/decoders/pandar_128_e4x_decoder.cpp 0.00% <0.00%> (ø)
...bula_decoders_hesai/decoders/pandar_64_decoder.cpp 75.59% <0.00%> (+0.78%) ⬆️
..._decoders_hesai/decoders/pandar_qt_128_decoder.cpp 0.00% <0.00%> (ø)
...a_decoders_hesai/decoders/pandar_qt_64_decoder.cpp 77.60% <0.00%> (+0.79%) ⬆️
nebula_tests/hesai/hesai_ros_decoder_test_40p.cpp 26.79% <0.00%> (-0.13%) ⬇️
nebula_tests/hesai/hesai_ros_decoder_test_64.cpp 26.79% <0.00%> (-0.13%) ⬇️
nebula_tests/hesai/hesai_ros_decoder_test_xt32.cpp 26.92% <0.00%> (ø)
...bula_decoders_hesai/decoders/pandar_40_decoder.cpp 74.40% <33.33%> (-0.60%) ⬇️
nebula_tests/hesai/hesai_ros_decoder_test_qt64.cpp 26.79% <40.00%> (+0.66%) ⬆️
...ebula_tests/hesai/hesai_ros_decoder_test_xt32m.cpp 26.79% <40.00%> (+0.66%) ⬆️
... and 4 more

... and 57 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@mojomex
Copy link
Collaborator Author

mojomex commented Aug 22, 2023

This PR will be made obsolete by #42 but it still serves as documentation on how unit test reference data was changed and how the new, generic decoder handles range checks etc.

@mojomex mojomex marked this pull request as ready for review August 22, 2023 04:54
@amc-nu
Copy link
Contributor

amc-nu commented Sep 11, 2023

Superseeded by #38

@amc-nu amc-nu closed this Sep 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants