feat(tensorrt yolox): inference and publish mask image from yolox model with semantic segmentation header #5553

badai-nguyen · 2023-11-10T12:55:30Z

Description

This PR is to add semantic segmentation mask output topic to solve the issue #5594

Tests performed

Test tensorrt_yolox output on internal rosbag:
ros2 launch tensorrt_yolox yolox_s_plus_opt.launch.xml
Screencast from 2024年05月30日 09時02分23秒.webm
Note that, adding semantic segmentation header causes increasing processing_time
- Tested reults on Embedded Robotic Controller Powered by NVIDIA® Jetson AGX Xavier ™ Module RQX-58G with preprocess_on_gpu: true:
- OS version:
- cat /etc/nv_tegra_release
- -> R32 (release), REVISION: 6.1, GCID: 27863751, BOARD: t186ref, EABI: aarch64, DATE: Mon Jul 26 19:36:31 UTC 2021

ros2 launch tensorrt_yolox yolox_s_plus_opt.launch.xml model_name:=yolox-sPlus-T4-960x960-pseudo-finetune	ros2 launch tensorrt_yolox yolox_s_plus_opt.launch.xml model_name:=yolox-sPlus-opt-pseudoV2-T4-960x960-T4-seg16cls
Average 18.9983671 STD 0.3530808684	Average 51.28096922 STD ：0.6388894661

Notes for reviewers

Interface changes

Effects on system behavior

Pre-review checklist for the PR author

The PR author must check the checkboxes below when creating the PR.

I've confirmed the contribution guidelines.
The PR follows the pull request guidelines.

In-review checklist for the PR reviewers

The PR reviewers must check the checkboxes below before approval.

The PR follows the pull request guidelines.
The PR has been properly tested.
The PR has been reviewed by the code owners.

Post-review checklist for the PR author

The PR author must check the checkboxes below before merging.

There are no open discussions or they are tracked via tickets.
The PR is ready for merge.

After all checkboxes are checked, anyone who has write access can merge the PR.

github-actions · 2023-11-21T00:42:39Z

Documentation URL: https://autowarefoundation.github.io/autoware.universe/pr-5553/
Modified URLs:

https://autowarefoundation.github.io/autoware.universe/pr-5553/perception/tensorrt_yolox/

badai-nguyen · 2023-12-08T02:08:30Z

Since @dan-dnn is still working on update the model so I convert this to draft and reopen when the new version of yolox is ready.

stale · 2024-02-06T03:35:32Z

This pull request has been automatically marked as stale because it has not had recent activity.

badai-nguyen · 2024-02-26T16:08:04Z

Since the utilized model is reconsidered so I close this for now and reopen it after the model is fixed.

Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> fix: add multitask for segment Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp>

Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp>

Signed-off-by: Manato HIRABAYASHI <manato.hirabayashi@tier4.jp>

perception/tensorrt_yolox/src/tensorrt_yolox.cpp

manato · 2024-06-21T10:38:05Z

@badai-nguyen
Thank you very much for your contribution. And I'm sorry for the delay in the review of this PR.
Overall, it looks good to me, but I added some suggestions to reduce unnecessary Device to Host memory copy. I'd appreciate it if you could consider them. Thank you!

dan-dnn · 2024-06-21T12:55:23Z

@badai-nguyen
Thank you for your contribution.
I apologize for the delay in reviewing this PR.

Overall, I think it looks good, butI have one suggestion regarding performance.
From the perspective of data transfer speed, it is better to publish the mask data before resizing and let the subscriber resize it.

Also, could you please add the following repository related to trt-YOLOXP in the README?

Co-authored-by: Manato Hirabayashi <3022416+manato@users.noreply.github.com>

Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp>

badai-nguyen · 2024-06-24T00:21:26Z

@badai-nguyen Thank you very much for your contribution. And I'm sorry for the delay in the review of this PR. Overall, it looks good to me, but I added some suggestions to reduce unnecessary Device to Host memory copy. I'd appreciate it if you could consider them. Thank you!

@manato Thank you for your suggestion, I followed all of them.

badai-nguyen · 2024-06-24T00:24:17Z

@badai-nguyen Thank you for your contribution. I apologize for the delay in reviewing this PR.

Overall, I think it looks good, butI have one suggestion regarding performance. From the perspective of data transfer speed, it is better to publish the mask data before resizing and let the subscriber resize it.

Also, could you please add the following repository related to trt-YOLOXP in the README?

@dan-dnn

I updated README here d800691
I keep the small size of mask when publishing here a15ac04#diff-c07e477020c17e860704e7e466d42f493fc51a02ea27e8134c203b240079f80fR192-R193

Could you review this again?

codecov · 2024-06-24T00:26:39Z

Codecov Report

Attention: Patch coverage is 0% with 204 lines in your changes missing coverage. Please review.

Project coverage is 1.89%. Comparing base (507e3f4) to head (a15ac04).
Report is 109 commits behind head on main.

Files	Patch %	Lines
perception/tensorrt_yolox/src/tensorrt_yolox.cpp	0.00%	139 Missing ⚠️
...ception/tensorrt_yolox/src/tensorrt_yolox_node.cpp	0.00%	57 Missing ⚠️
...lox/include/tensorrt_yolox/tensorrt_yolox_node.hpp	0.00%	5 Missing ⚠️
...rt_yolox/include/tensorrt_yolox/tensorrt_yolox.hpp	0.00%	2 Missing ⚠️
...rt_yolox/src/yolox_single_image_inference_node.cpp	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #5553       +/-   ##
==========================================
- Coverage   14.84%   1.89%   -12.96%     
==========================================
  Files        1999     230     -1769     
  Lines      139163   14120   -125043     
  Branches    43716    2209    -41507     
==========================================
- Hits        20661     267    -20394     
+ Misses      95731   13797    -81934     
+ Partials    22771      56    -22715

Flag	Coverage Δ
differential	`1.89% <0.00%> (?)`
total	`?`

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

dan-dnn · 2024-06-24T00:35:28Z

@badai-nguyen

Thank you for the update.
It looks good for me.

manato

@badai-nguyen
Thank you very much for your consideration. LGMT!

…el with semantic segmentation header (autowarefoundation#5553) * add segmentation model Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> fix: add multitask for segment Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: publish mask Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * feat: publish colorized mask Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: resize yolox mask Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: add memory allocate operations Signed-off-by: Manato HIRABAYASHI <manato.hirabayashi@tier4.jp> * refactor: remove underscore for a local variable Signed-off-by: Manato HIRABAYASHI <manato.hirabayashi@tier4.jp> * chore: add condition to check the number of subscriber for newly added topics Signed-off-by: Manato HIRABAYASHI <manato.hirabayashi@tier4.jp> * chore: pre-commit Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: add roi overlapping segment Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: roi overlay segment Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: refactor Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * docs: update readme Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: update model name Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: add utils into tensorrt_yolox Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: launch file Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: remove unnecessary depend Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: fix yaml file Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: remove duplicated param in launch Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: semantic class Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * docs: update readme Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: update default param Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: add processing time topic Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: typo Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * style(pre-commit): autofix * chore: fix cspell error Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: yolox default param Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: rename debug topics Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: rename debug topics Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * docs: update model description Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * style(pre-commit): autofix * fix: launch Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * refactor: unpublish mask for single task Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * Update perception/tensorrt_yolox/src/tensorrt_yolox.cpp Co-authored-by: Manato Hirabayashi <3022416+manato@users.noreply.github.com> * Update perception/tensorrt_yolox/src/tensorrt_yolox.cpp Co-authored-by: Manato Hirabayashi <3022416+manato@users.noreply.github.com> * Update perception/tensorrt_yolox/src/tensorrt_yolox.cpp Co-authored-by: Manato Hirabayashi <3022416+manato@users.noreply.github.com> * style(pre-commit): autofix * docs: update reamde Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * style(pre-commit): autofix * fix: skip mask size as yolox output Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> --------- Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> Signed-off-by: Manato HIRABAYASHI <manato.hirabayashi@tier4.jp> Co-authored-by: Manato HIRABAYASHI <manato.hirabayashi@tier4.jp> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Manato Hirabayashi <3022416+manato@users.noreply.github.com> Signed-off-by: Simon Eisenmann <simon.eisenmann@driveblocks.ai>

…el with semantic segmentation header (autowarefoundation#5553) * add segmentation model Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> fix: add multitask for segment Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: publish mask Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * feat: publish colorized mask Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: resize yolox mask Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: add memory allocate operations Signed-off-by: Manato HIRABAYASHI <manato.hirabayashi@tier4.jp> * refactor: remove underscore for a local variable Signed-off-by: Manato HIRABAYASHI <manato.hirabayashi@tier4.jp> * chore: add condition to check the number of subscriber for newly added topics Signed-off-by: Manato HIRABAYASHI <manato.hirabayashi@tier4.jp> * chore: pre-commit Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: add roi overlapping segment Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: roi overlay segment Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: refactor Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * docs: update readme Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: update model name Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: add utils into tensorrt_yolox Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: launch file Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: remove unnecessary depend Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: fix yaml file Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: remove duplicated param in launch Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: semantic class Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * docs: update readme Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: update default param Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: add processing time topic Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: typo Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * style(pre-commit): autofix * chore: fix cspell error Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: yolox default param Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: rename debug topics Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: rename debug topics Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * docs: update model description Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * style(pre-commit): autofix * fix: launch Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * refactor: unpublish mask for single task Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * Update perception/tensorrt_yolox/src/tensorrt_yolox.cpp Co-authored-by: Manato Hirabayashi <3022416+manato@users.noreply.github.com> * Update perception/tensorrt_yolox/src/tensorrt_yolox.cpp Co-authored-by: Manato Hirabayashi <3022416+manato@users.noreply.github.com> * Update perception/tensorrt_yolox/src/tensorrt_yolox.cpp Co-authored-by: Manato Hirabayashi <3022416+manato@users.noreply.github.com> * style(pre-commit): autofix * docs: update reamde Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * style(pre-commit): autofix * fix: skip mask size as yolox output Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> --------- Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> Signed-off-by: Manato HIRABAYASHI <manato.hirabayashi@tier4.jp> Co-authored-by: Manato HIRABAYASHI <manato.hirabayashi@tier4.jp> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Manato Hirabayashi <3022416+manato@users.noreply.github.com>

…el with semantic segmentation header (#5553) * add segmentation model Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> fix: add multitask for segment Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: publish mask Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * feat: publish colorized mask Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: resize yolox mask Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: add memory allocate operations Signed-off-by: Manato HIRABAYASHI <manato.hirabayashi@tier4.jp> * refactor: remove underscore for a local variable Signed-off-by: Manato HIRABAYASHI <manato.hirabayashi@tier4.jp> * chore: add condition to check the number of subscriber for newly added topics Signed-off-by: Manato HIRABAYASHI <manato.hirabayashi@tier4.jp> * chore: pre-commit Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: add roi overlapping segment Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: roi overlay segment Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: refactor Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * docs: update readme Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: update model name Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: add utils into tensorrt_yolox Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: launch file Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: remove unnecessary depend Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: fix yaml file Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: remove duplicated param in launch Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: semantic class Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * docs: update readme Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: update default param Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: add processing time topic Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: typo Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * style(pre-commit): autofix * chore: fix cspell error Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: yolox default param Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: rename debug topics Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: rename debug topics Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * docs: update model description Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * style(pre-commit): autofix * fix: launch Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * refactor: unpublish mask for single task Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * Update perception/tensorrt_yolox/src/tensorrt_yolox.cpp Co-authored-by: Manato Hirabayashi <3022416+manato@users.noreply.github.com> * Update perception/tensorrt_yolox/src/tensorrt_yolox.cpp Co-authored-by: Manato Hirabayashi <3022416+manato@users.noreply.github.com> * Update perception/tensorrt_yolox/src/tensorrt_yolox.cpp Co-authored-by: Manato Hirabayashi <3022416+manato@users.noreply.github.com> * style(pre-commit): autofix * docs: update reamde Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * style(pre-commit): autofix * fix: skip mask size as yolox output Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> --------- Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> Signed-off-by: Manato HIRABAYASHI <manato.hirabayashi@tier4.jp> Co-authored-by: Manato HIRABAYASHI <manato.hirabayashi@tier4.jp> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Manato Hirabayashi <3022416+manato@users.noreply.github.com>

…el with semantic segmentation header (autowarefoundation#5553) * add segmentation model Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> fix: add multitask for segment Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: publish mask Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * feat: publish colorized mask Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: resize yolox mask Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: add memory allocate operations Signed-off-by: Manato HIRABAYASHI <manato.hirabayashi@tier4.jp> * refactor: remove underscore for a local variable Signed-off-by: Manato HIRABAYASHI <manato.hirabayashi@tier4.jp> * chore: add condition to check the number of subscriber for newly added topics Signed-off-by: Manato HIRABAYASHI <manato.hirabayashi@tier4.jp> * chore: pre-commit Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: add roi overlapping segment Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: roi overlay segment Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: refactor Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * docs: update readme Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: update model name Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: add utils into tensorrt_yolox Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: launch file Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: remove unnecessary depend Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: fix yaml file Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: remove duplicated param in launch Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: semantic class Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * docs: update readme Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: update default param Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: add processing time topic Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: typo Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * style(pre-commit): autofix * chore: fix cspell error Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: yolox default param Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: rename debug topics Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: rename debug topics Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * docs: update model description Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * style(pre-commit): autofix * fix: launch Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * refactor: unpublish mask for single task Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * Update perception/tensorrt_yolox/src/tensorrt_yolox.cpp Co-authored-by: Manato Hirabayashi <3022416+manato@users.noreply.github.com> * Update perception/tensorrt_yolox/src/tensorrt_yolox.cpp Co-authored-by: Manato Hirabayashi <3022416+manato@users.noreply.github.com> * Update perception/tensorrt_yolox/src/tensorrt_yolox.cpp Co-authored-by: Manato Hirabayashi <3022416+manato@users.noreply.github.com> * style(pre-commit): autofix * docs: update reamde Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * style(pre-commit): autofix * fix: skip mask size as yolox output Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> --------- Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> Signed-off-by: Manato HIRABAYASHI <manato.hirabayashi@tier4.jp> Co-authored-by: Manato HIRABAYASHI <manato.hirabayashi@tier4.jp> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Manato Hirabayashi <3022416+manato@users.noreply.github.com>

….29.0 (#1526) * feat(tensorrt yolox): inference and publish mask image from yolox model with semantic segmentation header (autowarefoundation#5553) * add segmentation model Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> fix: add multitask for segment Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: publish mask Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * feat: publish colorized mask Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: resize yolox mask Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: add memory allocate operations Signed-off-by: Manato HIRABAYASHI <manato.hirabayashi@tier4.jp> * refactor: remove underscore for a local variable Signed-off-by: Manato HIRABAYASHI <manato.hirabayashi@tier4.jp> * chore: add condition to check the number of subscriber for newly added topics Signed-off-by: Manato HIRABAYASHI <manato.hirabayashi@tier4.jp> * chore: pre-commit Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: add roi overlapping segment Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: roi overlay segment Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: refactor Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * docs: update readme Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: update model name Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: add utils into tensorrt_yolox Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: launch file Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: remove unnecessary depend Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: fix yaml file Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: remove duplicated param in launch Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: semantic class Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * docs: update readme Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: update default param Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: add processing time topic Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: typo Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * style(pre-commit): autofix * chore: fix cspell error Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: yolox default param Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: rename debug topics Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: rename debug topics Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * docs: update model description Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * style(pre-commit): autofix * fix: launch Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * refactor: unpublish mask for single task Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * Update perception/tensorrt_yolox/src/tensorrt_yolox.cpp Co-authored-by: Manato Hirabayashi <3022416+manato@users.noreply.github.com> * Update perception/tensorrt_yolox/src/tensorrt_yolox.cpp Co-authored-by: Manato Hirabayashi <3022416+manato@users.noreply.github.com> * Update perception/tensorrt_yolox/src/tensorrt_yolox.cpp Co-authored-by: Manato Hirabayashi <3022416+manato@users.noreply.github.com> * style(pre-commit): autofix * docs: update reamde Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * style(pre-commit): autofix * fix: skip mask size as yolox output Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> --------- Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> Signed-off-by: Manato HIRABAYASHI <manato.hirabayashi@tier4.jp> Co-authored-by: Manato HIRABAYASHI <manato.hirabayashi@tier4.jp> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Manato Hirabayashi <3022416+manato@users.noreply.github.com> * fix(tensorrt_yolox): add run length encoding for sematic segmentation mask (autowarefoundation#7905) * fix: add rle compress Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: rle compress Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: move rle into utils Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: pre-commit Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * Update perception/autoware_tensorrt_yolox/src/utils.cpp Co-authored-by: Yukihiro Saito <yukky.saito@gmail.com> * fix: remove unused variable Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * Update perception/autoware_tensorrt_yolox/src/utils.cpp Co-authored-by: Manato Hirabayashi <3022416+manato@users.noreply.github.com> * style(pre-commit): autofix * feat: add unit test for utils Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * style(pre-commit): autofix * fix: unit test Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: change to explicit index Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * style(pre-commit): autofix * fix: cuda cmake Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: separate unit test into different PR Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> --------- Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> Co-authored-by: Yukihiro Saito <yukky.saito@gmail.com> Co-authored-by: Manato Hirabayashi <3022416+manato@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * refactor(tensorrt_yolox): move utils into perception_utils (autowarefoundation#8435) * chore(tensorrt_yolo): refactor utils Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * style(pre-commit): autofix * fix: tensorrt_yolox Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> --------- Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * fix(perception_utils): install library after build (autowarefoundation#8501) Signed-off-by: Manato HIRABAYASHI <manato.hirabayashi@tier4.jp> * fix(image_projection_based_fusion): resize sematic segmentation mask as input image size (autowarefoundation#7635) Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix(image_projection_based_fusion): segmentation pointcloud fusion param update (autowarefoundation#7858) * fix(image_projection_based_fusion): add run length decoding for segmentation_pointcloud_fusion (autowarefoundation#7909) * fix: add rle decompress Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * style(pre-commit): autofix * fix: use rld in tensorrt utils Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: rebase error Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: dependency Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: skip publish debug mask Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * Update perception/autoware_image_projection_based_fusion/src/segmentation_pointcloud_fusion/node.cpp Co-authored-by: kminoda <44218668+kminoda@users.noreply.github.com> * style(pre-commit): autofix * Revert "fix: skip publish debug mask" This reverts commit 30fa9ae. --------- Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: kminoda <44218668+kminoda@users.noreply.github.com> * feat(tier4_perception_launch): add image segmentation based pointcloud filter (autowarefoundation#7225) * feat(tier4_perception_launch): add image segmentation based pointcloud filter Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: typo Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * fix: detection launch Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: add maintainer Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * Revert "chore: add maintainer" This reverts commit 5adfef6. --------- Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * style(pre-commit): autofix * fix(autoware_image_projection_based_fusion): resolve issue with segmentation pointcloud fusion node failing with multiple mask inputs (autowarefoundation#8769) --------- Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> Signed-off-by: Manato HIRABAYASHI <manato.hirabayashi@tier4.jp> Co-authored-by: Manato HIRABAYASHI <manato.hirabayashi@tier4.jp> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Manato Hirabayashi <3022416+manato@users.noreply.github.com> Co-authored-by: Yukihiro Saito <yukky.saito@gmail.com> Co-authored-by: kminoda <44218668+kminoda@users.noreply.github.com> Co-authored-by: Yi-Hsiang Fang (Vivid) <146902905+vividf@users.noreply.github.com>

github-actions bot added component:perception Advanced sensor data processing and environment understanding. (auto-assigned) component:common Common packages from the autoware-common repository. (auto-assigned) labels Nov 10, 2023

badai-nguyen mentioned this pull request Nov 13, 2023

feat(image_projection_based_fusion): add image segmentation_pointcloud_fusion for pointcloud filter #5562

Merged

7 tasks

badai-nguyen force-pushed the feat/tensorrt_yolox/add_semantic_segmentation branch 3 times, most recently from 16e5aa1 to e301a20 Compare November 20, 2023 12:49

badai-nguyen mentioned this pull request Nov 21, 2023

fix: add download url of yolox with semantic segmentation header autowarefoundation/autoware#4012

Merged

4 tasks

badai-nguyen requested review from miursh and yukkysaito November 21, 2023 00:36

badai-nguyen force-pushed the feat/tensorrt_yolox/add_semantic_segmentation branch from e301a20 to 8b787e3 Compare November 21, 2023 00:37

github-actions bot added the type:documentation Creating or refining documentation. (auto-assigned) label Nov 21, 2023

badai-nguyen added the run:deploy-docs Mark for deploy-docs action generation. (used-by-ci) label Nov 21, 2023

badai-nguyen requested a review from YoshiRi November 21, 2023 00:41

badai-nguyen marked this pull request as ready for review November 21, 2023 02:36

badai-nguyen requested review from wep21, dan-dnn and manato as code owners November 21, 2023 02:36

badai-nguyen marked this pull request as draft December 8, 2023 02:08

stale bot added the status:stale Inactive or outdated issues. (auto-assigned) label Feb 6, 2024

badai-nguyen closed this Feb 26, 2024

badai-nguyen and others added 6 commits May 22, 2024 19:24

add segmentation model

767a936

Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> fix: add multitask for segment Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp>

fix: publish mask

8356196

Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp>

feat: publish colorized mask

297528e

Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp>

fix: resize yolox mask

6dbbe77

Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp>

fix: add memory allocate operations

f648980

Signed-off-by: Manato HIRABAYASHI <manato.hirabayashi@tier4.jp>

refactor: remove underscore for a local variable

3ab80e9

Signed-off-by: Manato HIRABAYASHI <manato.hirabayashi@tier4.jp>

manato reviewed Jun 21, 2024

View reviewed changes

perception/tensorrt_yolox/src/tensorrt_yolox.cpp Outdated Show resolved Hide resolved

manato reviewed Jun 21, 2024

View reviewed changes

perception/tensorrt_yolox/src/tensorrt_yolox.cpp Outdated Show resolved Hide resolved

manato reviewed Jun 21, 2024

View reviewed changes

perception/tensorrt_yolox/src/tensorrt_yolox.cpp Show resolved Hide resolved

badai-nguyen and others added 7 commits June 24, 2024 08:18

Update perception/tensorrt_yolox/src/tensorrt_yolox.cpp

ff14e52

Co-authored-by: Manato Hirabayashi <3022416+manato@users.noreply.github.com>

Update perception/tensorrt_yolox/src/tensorrt_yolox.cpp

9df6103

Co-authored-by: Manato Hirabayashi <3022416+manato@users.noreply.github.com>

Update perception/tensorrt_yolox/src/tensorrt_yolox.cpp

f305df2

Co-authored-by: Manato Hirabayashi <3022416+manato@users.noreply.github.com>

style(pre-commit): autofix

174010e

docs: update reamde

d800691

Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp>

style(pre-commit): autofix

8832a3a

fix: skip mask size as yolox output

a15ac04

Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp>

dan-dnn approved these changes Jun 24, 2024

View reviewed changes

manato approved these changes Jun 24, 2024

View reviewed changes

badai-nguyen merged commit 2ca276f into autowarefoundation:main Jun 24, 2024
27 of 30 checks passed

badai-nguyen mentioned this pull request Jun 24, 2024

fix(image_projection_based_fusion): resize sematic segmentation mask as input image size #7635

Merged

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(tensorrt yolox): inference and publish mask image from yolox model with semantic segmentation header #5553

feat(tensorrt yolox): inference and publish mask image from yolox model with semantic segmentation header #5553

badai-nguyen commented Nov 10, 2023 •

edited

Loading

github-actions bot commented Nov 21, 2023

badai-nguyen commented Dec 8, 2023

stale bot commented Feb 6, 2024

badai-nguyen commented Feb 26, 2024

manato commented Jun 21, 2024

dan-dnn commented Jun 21, 2024

badai-nguyen commented Jun 24, 2024

badai-nguyen commented Jun 24, 2024 •

edited

Loading

codecov bot commented Jun 24, 2024 •

edited

Loading

dan-dnn commented Jun 24, 2024

manato left a comment

feat(tensorrt yolox): inference and publish mask image from yolox model with semantic segmentation header #5553

feat(tensorrt yolox): inference and publish mask image from yolox model with semantic segmentation header #5553

Conversation

badai-nguyen commented Nov 10, 2023 • edited Loading

Description

Related links

Tests performed

Notes for reviewers

Interface changes

Effects on system behavior

Pre-review checklist for the PR author

In-review checklist for the PR reviewers

Post-review checklist for the PR author

github-actions bot commented Nov 21, 2023

badai-nguyen commented Dec 8, 2023

stale bot commented Feb 6, 2024

badai-nguyen commented Feb 26, 2024

manato commented Jun 21, 2024

dan-dnn commented Jun 21, 2024

badai-nguyen commented Jun 24, 2024

badai-nguyen commented Jun 24, 2024 • edited Loading

codecov bot commented Jun 24, 2024 • edited Loading

Codecov Report

dan-dnn commented Jun 24, 2024

manato left a comment

Choose a reason for hiding this comment

badai-nguyen commented Nov 10, 2023 •

edited

Loading

badai-nguyen commented Jun 24, 2024 •

edited

Loading

codecov bot commented Jun 24, 2024 •

edited

Loading