feat(autoware_tensorrt_bevdet): add new 3d object detection method

[cyn-liu]

Description

Integrating BEVDet into Autoware for 3D object detection based on multi-view images.

Related links

Issue Link

How was this PR tested?

This PR has been tested on local environment.
env1:

RTX 3080
docker image: `ghcr.io/autowarefoundation/autoware-universe:humble-20231001-cuda`
CUDA11.6
TensorRT8.4.2

env2:

RTXA1000 `Dell notebook computer`
docker image: ghcr.io/autowarefoundation/autoware:20240612-devel-cuda-amd64
CUDA==12.3
libnvinfer==8.6.1.6

Notes for reviewers

Test using a bag containing Nuscenes data.
Note: The frame_id of /lidar_top in this bag is map
onnx model file
bag

Interface changes

Add a new perception_mode, If set perception_mode = camera, the detector will launch tesnsorrt_bevdet node.

Topic changes

Additions and removals

Change type	Topic Type	Topic Name	Message Type	Description
Added	Sub	~/input/topic_cloud	sensor_msgs::msg::PointCloud2	used for time alignment and display
Added	Sub	~/input/topic_img_fl	sensor_msgs::msg::Image	input front_left camera image
Added	Sub	~/input/topic_img_f	sensor_msgs::msg::Image	input front camera image
Added	Sub	~/input/topic_img_fr	sensor_msgs::msg::Image	input front_right camera image
Added	Sub	~/input/topic_img_bl	sensor_msgs::msg::Image	input back_left camera image
Added	Sub	~/input/topic_img_b	sensor_msgs::msg::Image	input back camera image
Added	Sub	~/input/topic_img_br	sensor_msgs::msg::Image	input back_right camera image
Added	Pub	~/output/painting_cloud	sensor_msgs::msg::PointCloud2	output pointcloud just used for display
Added	Pub	~/output/boxes	autoware_perception_msgs::msg::DetectedObjects	detected objects

ROS Parameter Changes

Additions and removals

Change type	Parameter Name	Type	Default Value	Description
Added	bev_detection_model	string	bevdet	Set BEV detection model type

Modifications

Version	Parameter Name	Type	Default Value	Description
Old	perception_mode	string	lidar	Set perception mode
New	perception_mode	string	lidar	Add a new perception mode of camera

Effects on system behavior

None.

[github-actions]

Thank you for contributing to the Autoware project!

🚧 If your pull request is in progress, switch it to draft mode.

Please ensure:

• You've checked our contribution guidelines.
• Your PR follows our pull request guidelines.
• All required CI checks pass before marking the PR ready for review.

[kminoda]

Cool PR 🚀

Let me provide some minor comments first. Also, please make sure that all the CIs are passing before opening the PR 🙏

[kminoda]

@cyn-liu Please make sure that the node follows coding guildline: https://autowarefoundation.github.io/autoware-documentation/main/contributing/coding-guidelines/ros-nodes/class-design/

such as

• https://autowarefoundation.github.io/autoware-documentation/main/contributing/coding-guidelines/ros-nodes/class-design/#component-registration
• https://autowarefoundation.github.io/autoware-documentation/main/contributing/coding-guidelines/ros-nodes/directory-structure/#directory-structure

[kminoda]

(Friendly ping 🙏 ) Would you mention us once all the CIs are fixed?

[technolojin]

I could see two required checks are not passed.

1. cppcheck-differential > Details > Show cppcheck-report result

Run cat cppcheck-report.txt
Error: /home/runner/work/autoware.universe/autoware.universe/perception/tensorrt_bevdet/src/bevdet.cpp:167:13: performance: Range variable 'task_factors' should be declared as const reference. [iterateByValue]
  for (auto task_factors : nms_factor_temp) {
            ^
Error: /home/runner/work/autoware.universe/autoware.universe/perception/tensorrt_bevdet/src/data.cpp:37:20: performance: Range variable 'name' should be declared as const reference. [iterateByValue]
  for (std::string name : cams_name) {
                   ^

2. pre-commit.ci -pr > Details

fix include guard........................................................Failed
- hook id: ros-include-guard
- exit code: 1

No include guard in perception/tensorrt_bevdet/include/gatherbev_plugin.hpp
No include guard in perception/tensorrt_bevdet/include/bevpool_plugin.hpp
No include guard in perception/tensorrt_bevdet/include/preprocess_plugin.hpp
No include guard in perception/tensorrt_bevdet/include/bevdet_node.hpp
No include guard in perception/tensorrt_bevdet/include/alignbev_plugin.hpp

sort package.xml.........................................................Passed
shellcheck...............................................................Passed
shfmt....................................................................Passed
isort....................................................................Passed
black....................................................................Passed
clang-format.............................................................Passed
cpplint..................................................................Failed
- hook id: cpplint
- exit code: 1

perception/tensorrt_bevdet/include/bevdet_node.hpp:0:  No copyright message found.  You should have a line: "Copyright [year] <Copyright Owner>"  [legal/copyright] [5]
perception/tensorrt_bevdet/include/bevdet_node.hpp:86:  Add #include <vector> for vector<>  [build/include_what_you_use] [4]
perception/tensorrt_bevdet/include/bevdet_node.hpp:113:  Add #include <memory> for shared_ptr<>  [build/include_what_you_use] [4]
Done processing perception/tensorrt_bevdet/include/bevdet_node.hpp
perception/tensorrt_bevdet/include/cpu_jpegdecoder.hpp:0:  No copyright message found.  You should have a line: "Copyright [year] <Copyright Owner>"  [legal/copyright] [5]
Done processing perception/tensorrt_bevdet/include/cpu_jpegdecoder.hpp
perception/tensorrt_bevdet/include/gatherbev_plugin.hpp:45:  Single-parameter constructors should be marked explicit.  [runtime/explicit] [5]
Done processing perception/tensorrt_bevdet/include/gatherbev_plugin.hpp
perception/tensorrt_bevdet/include/preprocess.hpp:0:  No copyright message found.  You should have a line: "Copyright [year] <Copyright Owner>"  [legal/copyright] [5]
Done processing perception/tensorrt_bevdet/include/preprocess.hpp
perception/tensorrt_bevdet/src/bevdet.cpp:0:  No copyright message found.  You should have a line: "Copyright [year] <Copyright Owner>"  [legal/copyright] [5]
perception/tensorrt_bevdet/src/bevdet.cpp:190:  Using C-style cast.  Use static_cast<float>(...) instead  [readability/casting] [4]

...

[kminoda]

@cyn-liu As this PR is stale for a week, from the maintenance perspective, let me make this PR a draft for now. Feel free to re-open the PR once all the CIs are fixed and ready to be reviewed. Thank you for your understanding, and we are looking forward to reviewing your PR.

[kminoda]

As a general comment, please follow the coding guideline in Autoware
https://autowarefoundation.github.io/autoware-documentation/main/contributing/coding-guidelines/languages/cpp/

[Shin-kyoto]

@cyn-liu

Please apply the changes not only to the point we pointed out, but to all points related to the comment.

As we said in #7956 (comment), don't use new.
Please check our coding guidelines:

https://autowarefoundation.github.io/autoware-documentation/main/contributing/coding-guidelines/languages/cpp/
Our coding guideline follows CppCoreGuidelines

[cyn-liu]

Refactoring the entire open source project to fully meet the specifications is a significant workload, but I have to do so.
I found that the refactored code often encounters unknown problems during launch, and the cause of this problem has not been found. After restarting the node multiple times, it can be resolved.
Although the previous code did not meet your coding standards, it was simple clear, and can ran stably.

[Shin-kyoto]

@cyn-liu
Is this what you meant in your comment?

• autoware_tensorrt_bevdet/src/bevdet.cpp was copied from another open source project.
• The content written in the other open source project is being used as is.

If that's the case, instead of modifying the code, the following actions are also possible:

• Clearly indicate where the code was copied from, like in this example.
• Store all copied code in a lib directory.

For code outside the lib, you have to follow Autoware's coding guidelines. For example, Autoware’s coding guidelines recommend avoiding the use of new.

[mojomex]

@cyn-liu (cc. @Shin-kyoto)
With all due respect, but

Although the previous code did not meet your coding standards

All PR authors are asked to read and accept the coding standards in the contribution guidelines, so this is a requirement that was clear even before the work on this PR started, so we expect this to be followed. A large open source project cannot be managed otherwise and this is not a concept unique to Autoware.

it was simple clear

I disagree, and so do our coding standards and the C++ core guidelines. Also, this PR is almost 6000 lines which makes even perfectly simple and clear code hard to review thoroughly.

and can ran stably.

As a reviewer, this was impossible to verify due to all the non-standard and almost C-style C++. There is a reason that core guidelines and tools like Clang-Tidy are enforcing the use of smart pointers, disallowing reinterpret_cast etc.
If it does not run stably after the refactoring, that hints at race conditions and memory problems that still need to be addressed. Consider using AddressSanitizer and ThreadSanitizer to pinpoint the issues.

As this review is taking quite some hours to review for our employees, I would strongly suggest that the coding style issues are addressed, and that the PR is split up before occupying people with the same discussion over and over again.

If you do not want to comply with our coding standards, I am afraid we cannot merge this PR.