feat(autoware_scene_to_image_projector): init package #7951

palas21 · 2024-07-10T14:55:44Z

Description

The autoware_scene_to_image_projector package facilitates visualizing 3D data such as bounding boxes of detected objects, ego vehicle trajectory, and road boundaries onto the 2D camera image plane. It uses transformations and projection algorithms to achieve this visualization, ensuring accurate placement of elements relative to the camera's perspective.

How was this PR tested?

Tested functionality using a sample rosbag containing necessary topics:
- Camera image
- Camera information
- Detected objects
- Trajectory
- Path
Utilized rviz2 for visualizing the processed camera image data.

Notes for Reviewers

You can use this rosbag for testing.
Note that the camera image topic is in compressed format. Ensure to decompress it before processing within the package.

Interface changes

None.

Effects on system behavior

None.

github-actions · 2024-07-10T14:56:02Z

Thank you for contributing to the Autoware project!

🚧 If your pull request is in progress, switch it to draft mode.

Please ensure:

You've checked our contribution guidelines.
Your PR follows our pull request guidelines.
All required CI checks pass before marking the PR ready for review.

…autoware_auto. Signed-off-by: palas21 <palas21@itu.edu.tr>

… are fixed. Signed-off-by: palas21 <palas21@itu.edu.tr>

Signed-off-by: palas21 <palas21@itu.edu.tr>

…according to reviewer suggestions. Signed-off-by: palas21 <palas21@itu.edu.tr>

…s fixed. Signed-off-by: palas21 <palas21@itu.edu.tr>

…-and-useful-informations-onto-the-2d-image

codecov · 2024-07-27T10:49:23Z

Codecov Report

Attention: Patch coverage is 0% with 254 lines in your changes missing coverage. Please review.

Project coverage is 29.18%. Comparing base (95aef0d) to head (9100ba1).
Report is 1119 commits behind head on main.

Files with missing lines	Patch %	Lines
...tor/src/autoware_scene_to_image_projector_node.cpp	0.00%	215 Missing ⚠️
...ils/include/object_recognition_utils/transform.hpp	0.00%	39 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #7951      +/-   ##
==========================================
- Coverage   29.25%   29.18%   -0.08%     
==========================================
  Files        1601     1607       +6     
  Lines      117738   118024     +286     
  Branches    50709    50766      +57     
==========================================
+ Hits        34450    34451       +1     
- Misses      74060    74345     +285     
  Partials     9228     9228

Flag	Coverage Δ		*Carryforward flag
differential	`18.55% <0.00%> (?)`
total	`29.25% <ø> (ø)`		Carriedforward from 95aef0d

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

a-maumau · 2024-08-05T07:48:21Z

@palas21 Hi, I am checking your package.
Your implementation is visually more synchronized than the one with the RViz camera plugin, looks good.

Is it possible to combine the visualization of the plugin versions?

In my opinion, the plugin version better shows the visualization which lines have transparency in their color and the line (polygons) get disappear in the distance.
For example:

Additionally, the plugin version can indicate the path information based on the planning by changing the color of the path.

a-maumau · 2024-08-05T07:51:16Z

perception/autoware_scene_to_image_projector/src/autoware_scene_to_image_projector_node.cpp

+  image_pub_ = this->create_publisher<sensor_msgs::msg::Image>("~/output/image", 10);
+
+  image_sub_ = this->create_subscription<sensor_msgs::msg::Image>(
+    "~/input/image", 10,


I need to change the QoS to rclcpp::QoS{10}.best_effort() for the ~/input/image.
Can you check this?

a-maumau · 2024-08-05T07:54:47Z

perception/autoware_scene_to_image_projector/src/autoware_scene_to_image_projector_node.cpp

+      std::vector<std::vector<cv::Point2f>> previous_polygons;
+
+      for (const auto & object : objects) {
+        if (should_skip_object(object.classification.front().label)) {


Sometimes object does not have classification, so I needed to check object.classification.empty().
This happens when I was checking a local rosbag with logging simulator.
Could you check this too?

Shin-kyoto · 2024-08-13T07:28:12Z

perception/autoware_scene_to_image_projector/src/autoware_scene_to_image_projector_node.cpp

+        latest_trajectory_->header.stamp);
+
+      if (!self_transform) {
+        RCLCPP_WARN(this->get_logger(), "Transform is not possible!");


@palas21

Thank you for your PR!
Can you use RCLCPP_WARN_THROTTLE ?

Shin-kyoto · 2024-08-13T07:29:05Z

perception/autoware_scene_to_image_projector/README.md

+
+## Future extensions / Unimplemented parts
+
+- The algorithm works for syncronized data but if the data is unsyncronized another method should be obtained.


@palas21

Please fix typos.

knzo25 · 2024-08-19T23:22:21Z

@Shin-kyoto @palas21
Since this package is for visualization purposes, would it not be more suited for https://github.com/autowarefoundation/autoware_tools ?

(shamelessly attempting to avoid compiling packages that we do not always use)

Shin-kyoto · 2024-08-19T23:32:46Z

@Shin-kyoto @palas21

Since this package is for visualization purposes, would it not be more suited for https://github.com/autowarefoundation/autoware_tools ?

Thank you for your comment!!
@knzo25, I agree with you. This package is more suited for autoware_tools.

@palas21
What do you think about creating the PR to autoware_tools?

Shin-kyoto · 2024-08-27T03:37:55Z

@palas21

friendly ping

palas21 · 2024-08-30T19:36:16Z

@Shin-kyoto sorry for the late response. I will move this PR to autoware_tools. I will open the PR after i make the suggested changes. Thanks all for the reviews.

stale · 2024-12-04T04:00:09Z

This pull request has been automatically marked as stale because it has not had recent activity.

palas21 requested a review from kaancolak July 10, 2024 14:55

palas21 self-assigned this Jul 10, 2024

palas21 requested review from scepter914, miursh, takayuki5168 and YoshiRi as code owners July 10, 2024 14:55

palas21 marked this pull request as draft July 10, 2024 14:58

palas21 force-pushed the feat/visualize-3d-objects-and-useful-informations-onto-the-2d-image branch from 3667345 to 99c909a Compare July 12, 2024 08:27

palas21 force-pushed the feat/visualize-3d-objects-and-useful-informations-onto-the-2d-image branch from 99c909a to 1c0b7b9 Compare July 12, 2024 09:41

palas21 and others added 6 commits July 16, 2024 13:31

fix(autoware_scene_to_image_projector): message type is changed from …

4fd8c02

…autoware_auto. Signed-off-by: palas21 <palas21@itu.edu.tr>

fix(autoware_scene_to_image_projector): file structure and namespaces…

c31800e

… are fixed. Signed-off-by: palas21 <palas21@itu.edu.tr>

fix(autoware_scene_to_image_projector): readme is updated.

4f2edda

Signed-off-by: palas21 <palas21@itu.edu.tr>

style(pre-commit): autofix

8760f46

fix(autoware_scene_to_image_projector): some missing headers are added.

2cd9931

Signed-off-by: palas21 <palas21@itu.edu.tr>

fix(autoware_scene_to_image_projector): fix typo and change the code …

2c8c45d

…according to reviewer suggestions. Signed-off-by: palas21 <palas21@itu.edu.tr>

palas21 force-pushed the feat/visualize-3d-objects-and-useful-informations-onto-the-2d-image branch from acc05f3 to 2c8c45d Compare July 16, 2024 10:32

pre-commit-ci bot and others added 3 commits July 16, 2024 10:35

style(pre-commit): autofix

e14a5bb

fix(autoware_scene_to_image_projector): should_skip_object function i…

385f3d9

…s fixed. Signed-off-by: palas21 <palas21@itu.edu.tr>

style(pre-commit): autofix

b42385e

xmfcx added the run:build-and-test-differential Mark to enable build-and-test-differential workflow. (used-by-ci) label Jul 24, 2024

xmfcx mentioned this pull request Jul 24, 2024

ci(pr-agent): disable newly added default review labels #8177

Merged

xmfcx removed the Review effort [1-5]: 4 label Jul 24, 2024

Merge branch 'autowarefoundation:main' into feat/visualize-3d-objects…

9100ba1

…-and-useful-informations-onto-the-2d-image

a-maumau reviewed Aug 5, 2024

View reviewed changes

Shin-kyoto requested changes Aug 13, 2024

View reviewed changes

palas21 closed this Aug 30, 2024

xmfcx reopened this Oct 4, 2024

xmfcx marked this pull request as draft October 4, 2024 14:16

xmfcx unassigned palas21 Oct 4, 2024

stale bot added the status:stale Inactive or outdated issues. (auto-assigned) label Dec 4, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(autoware_scene_to_image_projector): init package #7951

feat(autoware_scene_to_image_projector): init package #7951

palas21 commented Jul 10, 2024 •

edited

Loading

github-actions bot commented Jul 10, 2024 •

edited

Loading

codecov bot commented Jul 27, 2024 •

edited

Loading

a-maumau commented Aug 5, 2024 •

edited

Loading

a-maumau Aug 5, 2024 •

edited

Loading

a-maumau Aug 5, 2024

Shin-kyoto Aug 13, 2024

Shin-kyoto Aug 13, 2024

knzo25 commented Aug 19, 2024 •

edited

Loading

Shin-kyoto commented Aug 19, 2024 •

edited

Loading

Shin-kyoto commented Aug 27, 2024

palas21 commented Aug 30, 2024

stale bot commented Dec 4, 2024


		## Future extensions / Unimplemented parts

		- The algorithm works for syncronized data but if the data is unsyncronized another method should be obtained.

feat(autoware_scene_to_image_projector): init package #7951

Are you sure you want to change the base?

feat(autoware_scene_to_image_projector): init package #7951

Conversation

palas21 commented Jul 10, 2024 • edited Loading

Description

Related links

How was this PR tested?

Notes for Reviewers

Interface changes

Effects on system behavior

github-actions bot commented Jul 10, 2024 • edited Loading

codecov bot commented Jul 27, 2024 • edited Loading

Codecov Report

a-maumau commented Aug 5, 2024 • edited Loading

a-maumau Aug 5, 2024 • edited Loading

Choose a reason for hiding this comment

a-maumau Aug 5, 2024

Choose a reason for hiding this comment

Shin-kyoto Aug 13, 2024

Choose a reason for hiding this comment

Shin-kyoto Aug 13, 2024

Choose a reason for hiding this comment

knzo25 commented Aug 19, 2024 • edited Loading

Shin-kyoto commented Aug 19, 2024 • edited Loading

Shin-kyoto commented Aug 27, 2024

palas21 commented Aug 30, 2024

stale bot commented Dec 4, 2024

palas21 commented Jul 10, 2024 •

edited

Loading

github-actions bot commented Jul 10, 2024 •

edited

Loading

codecov bot commented Jul 27, 2024 •

edited

Loading

a-maumau commented Aug 5, 2024 •

edited

Loading

a-maumau Aug 5, 2024 •

edited

Loading

knzo25 commented Aug 19, 2024 •

edited

Loading

Shin-kyoto commented Aug 19, 2024 •

edited

Loading