feat(autoware_cuda_pointcloud_preprocessor): a cuda-accelerated pointcloud preprocessor

[knzo25]

Description

This PR is part of a series of PRs that aim to accelerate the Sensing/Perception pipeline through an appropriate use of CUDA.

List of PRs:

• feat(autoware_cuda_pointcloud_preprocessor): a cuda-accelerated pointcloud preprocessor #9454 (pointcloud preprocessing)
• feat(autoware_pointcloud_preprocessor): cuda-accelerated pointcloud concatenation #9455 (concatenation)
• feat(autoware_lidar_centerpoint): added the cuda_blackboard to centerpoint #9453 (centerpoint)
• transfusion (TODO)
• feat(autoware_probabilistic_occupancy_grid_map): cuda accelerated implementation #9542 (OGM - the first implementation will be independent of the blackboard to ease the transition)
• feat: acceleration and transport layer tier4/aip_launcher#348 (aip_launcher)
• feat: acceleration and transport layer sample_sensor_kit_launch#111 (sample_sensor_kit_launch)

To use these branches, the following additions to the autoware.repos are necessary:

  vendor/cuda_blackboard:
    type: git
    url: git@github.com:knzo25/cuda_blackboard.git
    version: main
  vendor/negotiated:
    type: git
    url: https://github.com/osrf/negotiated.git
    version: master

Depending on your machine and how many nodes are in a container, the following branch may also be required:
https://github.com/knzo25/launch_ros/tree/fix/load_composable_node
There seems to be a but in ROS where if you send too many services at once some will be lost and ros_launch can not handle that.

Related links

Parent Issue:

• Link

How was this PR tested?

The sensing/perception pipeline was tested until centerpoint for TIER IV's taxi using the logging simulator.
The following tests were executed in a laptop equipped with a RTX 4060 (laptop) GPU and a Intel(R) Core(TM) Ultra 7 165H (22 cores)

Node / processing time [ms]	Current	PR
/sensing/lidar/top/crop_box_filter_self/debug/processing_time_ms	5.81	N/A
/sensing/lidar/top/crop_box_filter_mirror/debug/processing_time_ms	4.59	N/A
/sensing/lidar/top/distortion_corrector/debug/processing_time_ms	10.96	N/A
/sensing/lidar/top/ring_outlier_filter/debug/processing_time_ms	10.69	N/A
/sensing/lidar/top/cuda_pointcloud_preprocessor/debug/processing_time_ms	N/A	3.08 (2.03 are H->D copies)
/sensing/lidar/concatenate_data_synchronizer/debug/processing_time_ms	7.83	0.70
Total	38.8	3.78

10.26 speedup!

Notes for reviewers

The main branch that I used for development is feat/cuda_acceleration_and_transport_layer.
However, the changes were too big so I split the PRs. That being said, development, if any will still be on that branch (and then cherrypicked to the respective PRs), and the review changes will be cherrypicked into the development branch.

Interface changes

An additional topic is added to perform type negotiation:
Example: input/pointcloud -> input/pointcloud and input/pointcloud/cuda

Effects on system behavior

Enabling this preprocessing in the launchers should provide a much reduced latency and cpu usage (at the cost of a higher GPU usage)

[github-actions]

Thank you for contributing to the Autoware project!

🚧 If your pull request is in progress, switch it to draft mode.

Please ensure:

• You've checked our contribution guidelines.
• Your PR follows our pull request guidelines.
• All required CI checks pass before marking the PR ready for review.

[mojomex]

Thank you for the amazing PR, these performance improvements are desperately needed.

I haven't checked the PR for functionality yet, but I'll leave my first round of comments here.

The main points I'd like to address are

• memory safety and idiomatic C++ (there is currently a lot of raw-pointer code which should be avoided whenever possible)
• modulatiry: currently the pipeline is hard-coded and all in one place. This makes the module hard to adapt to different projects, and hard to maintain individual modules in the pipeline.

Thank you for your time!

[manato]

@knzo25
Thank you very much for proposing a fantastic PR, and I'm sorry for taking a long time for the review. From a viewpoint of CUDA usage, I left some comments. I'd appreciate it if you could consider them.

[manato]

Suggested change

	const InputPointType * input_points, InputPointType * output_points, int num_points,
	const InputPointType * __restrict__ input_points, InputPointType * output_points, int num_points,

To enable "read-only data cache", I would suggest using __restrict__ for read-only input array. This suggestion can be applied to the other kernel arguments.

[amadeuszsz]

@knzo25 could you please double check all input arrays across kernels? I think for some of them __restrict__ keyword might be also applicable. I don't know if you missed it or skipped on purpose.
I unresolved this conversation, please resolve it again after reading this comment 🙏🏻

[knzo25]

@amadeuszsz
Ahh now I can reply here. Don't know why but before I could not

Answer: the extract kernel could indeed use restrict. The kernel alone, strictly speaking can not, but due to how the indexes are computed there is no problem

[knzo25]

Addressed in 68d1e42

[knzo25]

@amadeuszsz

@knzo25 could you please double check all input arrays across kernels? I think for some of them restrict keyword might be also applicable. I don't know if you missed it or skipped on purpose.
I unresolved this conversation, please resolve it again after reading this comment 🙏🏻

Could not answer this one, but the extract kernel could indeed use restrict. The kernel alone, strictly speaking can not, but due to how the indexes are computed there is no problem

[knzo25]

@amadeuszsz
Regarding CI/CD, I have not executed in ob purpose since the blackboard is not yet added to autoware.
I will investigate the loop error tomorrow

[knzo25]

terminate called after throwing an instance of 'std::system_error'
what(): Invalid argument
Aborted (core dumped)
It happens when timer is restarted (--loop) and only if output pointcloud from autoware_cuda_pointcloud_preprocessor is visualized. Could you please check if it happens to you too? If so, I would appreciate if you can investigate this issue 🙏🏻

Sorry, I only experienced it once in several experiments, and the pointcloud itself at the time was valid (checked with ros2 topic echo)

[amadeuszsz]

LGTM!

[codecov]

Codecov Report

Attention: Patch coverage is 0% with 469 lines in your changes missing coverage. Please review.

Project coverage is 26.23%. Comparing base (7686e5a) to head (2a1ff8c).
Report is 4 commits behind head on main.

Files with missing lines	Patch %	Lines
...cloud_preprocessor/cuda_pointcloud_preprocessor.cu	0.00%	165 Missing ⚠️
...preprocessor/cuda_pointcloud_preprocessor_node.cpp	0.00%	124 Missing ⚠️
.../cuda_pointcloud_preprocessor/undistort_kernels.cu	0.00%	89 Missing ⚠️
...e/autoware/cuda_pointcloud_preprocessor/memory.hpp	0.00%	43 Missing ⚠️
...src/cuda_pointcloud_preprocessor/common_kernels.cu	0.00%	24 Missing ⚠️
...c/cuda_pointcloud_preprocessor/organize_kernels.cu	0.00%	17 Missing ⚠️
...rc/cuda_pointcloud_preprocessor/outlier_kernels.cu	0.00%	6 Missing ⚠️
...loud_preprocessor/cuda_pointcloud_preprocessor.hpp	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #9454      +/-   ##
==========================================
+ Coverage   26.05%   26.23%   +0.18%     
==========================================
  Files        1374     1387      +13     
  Lines      106351   107189     +838     
  Branches    40877    41227     +350     
==========================================
+ Hits        27709    28124     +415     
- Misses      75940    75996      +56     
- Partials     2702     3069     +367     

Flag	Coverage Δ		*Carryforward flag
differential-cuda	0.00% <0.00%> (?)
total	26.38% <ø> (+0.33%)	⬆️	Carriedforward from 368009e

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Changes had already been addressed