Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(multi_object_tracker): object size becomes zero, risk of numeric error and overflow on IoU calculation #6597

Conversation

technolojin
Copy link
Contributor

Description

When IoU is calculated in sanity check process, the following error was shown in a low probability.

[multi_object_tracker-56] terminate called after throwing an instance of 'boost::numeric::positive_overflow'
[multi_object_tracker-56]   what():  bad numeric conversion: positive overflow
[multi_object_tracker-56] *** Aborted at 1709866773 (unix time) try "date -d @1709866773" if you are using GNU date ***
[multi_object_tracker-56] PC: @                0x0 (unknown)
[multi_object_tracker-56] *** SIGABRT (@0x3e80003cd17) received by PID 249111 (TID 0x7f9198c53f40) from PID 249111; stack trace: ***
[multi_object_tracker-56]     @     0x7f9197903046 (unknown)
[multi_object_tracker-56]     @     0x7f9198f33520 (unknown)
[multi_object_tracker-56]     @     0x7f9198f879fc pthread_kill
[multi_object_tracker-56]     @     0x7f9198f33476 raise
[multi_object_tracker-56]     @     0x7f9198f197f3 abort
[multi_object_tracker-56]     @     0x7f91991dcb9e (unknown)
[multi_object_tracker-56]     @     0x7f91991e820c (unknown)
[multi_object_tracker-56]     @     0x7f91991e8277 std::terminate()
[multi_object_tracker-56]     @     0x7f91991e84d8 __cxa_throw
[multi_object_tracker-56]     @     0x7f9197b5dd57 boost::geometry::detail::recalculate::recalculate_point<>::apply<>()
[multi_object_tracker-56]     @     0x7f9197b73796 boost::geometry::detail::get_turns::get_turns_generic<>::apply<>()
[multi_object_tracker-56]     @     0x7f9197b7a026 boost::geometry::detail::overlay::overlay<>::apply<>()
[multi_object_tracker-56]     @     0x7f9197b7d454 object_recognition_utils::get2dIoU<>()
[multi_object_tracker-56]     @     0x7f9197b2915f MultiObjectTracker::sanitizeTracker()
[multi_object_tracker-56]     @     0x7f9197b2ad21 MultiObjectTracker::onMeasurement()
[multi_object_tracker-56]     @     0x7f9197b42397 std::_Function_handler<>::_M_invoke()
[multi_object_tracker-56]     @     0x7f9197b422f2 _ZNSt8__detail9__variant17__gen_vtable_implINS0_12_Multi_arrayIPFNS0_21__deduce_visit_resultIvEEOZN6rclcpp23AnySubscriptionCallbackIN29autoware_auto_perception_msgs3msg16DetectedObjects_ISaIvEEESA_E8dispatchESt10shared_ptrISB_ERKNS5_11MessageInfoEEUlOT_E_RSt7variantIJSt8functionIFvRKSB_EESN_IFvSP_SH_EESN_IFvRKNS5_17SerializedMessageEEESN_IFvSW_SH_EESN_IFvSt10unique_ptrISB_St14default_deleteISB_EEEESN_IFvS14_SH_EESN_IFvS11_ISU_S12_ISU_EEEESN_IFvS1A_SH_EESN_IFvSD_ISO_EEESN_IFvS1F_SH_EESN_IFvSD_ISV_EEESN_IFvS1K_SH_EESN_IFvRKS1F_EESN_IFvS1Q_SH_EESN_IFvRKS1K_EESN_IFvS1W_SH_EESN_IFvSE_EESN_IFvSE_SH_EESN_IFvSD_ISU_EEESN_IFvS25_SH_EEEEEJEEESt16integer_sequenceImJLm8EEEE14__visit_invokeESL_S2B_
[multi_object_tracker-56]     @     0x7f9197b89b4a rclcpp::Subscription<>::handle_message()
[multi_object_tracker-56]     @     0x7f919951fba0 rclcpp::Executor::execute_subscription()
[multi_object_tracker-56]     @     0x7f919952110e rclcpp::Executor::execute_any_executable()
[multi_object_tracker-56]     @     0x7f9199527c90 rclcpp::executors::SingleThreadedExecutor::spin()
[multi_object_tracker-56]     @     0x563d5521143e main
[multi_object_tracker-56]     @     0x7f9198f1ad90 (unknown)
[multi_object_tracker-56]     @     0x7f9198f1ae40 __libc_start_main
[multi_object_tracker-56]     @     0x563d55211df5 _start
[multi_object_tracker-56]     @                0x0 (unknown)
[component_container_mt-39] [WARN] [1709866773.359701203] [sensing.lidar.concatenate_data]: Empty sensor points!
[ERROR] [multi_object_tracker-56]: process has died......................
  • Found reason of error
    Bicycle object sizes were zero (x=0, y=0, z=0). When just before the error is occur, one of the value was a huge value (probably a maximum value of double type).

  • Unintended operation
    In the pedestrian_and_bicycle tracker, two trackers are combined and updated simultaneously. There is chance to update cylinder shape (pedestrian) to the bicycle tracker, but the cylinder shape update in the bicycle tracker is not considered.
    In the current bicycle tracker, it always updates its size as bounding box, therefore the bounding size can became unintended value (default value of zero, or any arbitrary value).

  • Countermeasure

  1. In the bicycle tracker, the shape will be converted to bounding box.
  2. In all of the trackers treats object size, minimum object size is set and the size will be limited to have at least of the minimum size.

Tests performed

Tested by data-relay simulation under local PC environment, the same data that occurred the error and confirmed that the error is not happening anymore.

Effects on system behavior

The error probability is eliminated.

Pre-review checklist for the PR author

The PR author must check the checkboxes below when creating the PR.

In-review checklist for the PR reviewers

The PR reviewers must check the checkboxes below before approval.

Post-review checklist for the PR author

The PR author must check the checkboxes below before merging.

  • There are no open discussions or they are tracked via tickets.

After all checkboxes are checked, anyone who has write access can merge the PR.

@github-actions github-actions bot added the component:perception Advanced sensor data processing and environment understanding. (auto-assigned) label Mar 12, 2024
@technolojin technolojin changed the title bugfix(multi_object_tracker): object size becomes zero, risk of numeric error and overflow on IoU calculation fix(multi_object_tracker): object size becomes zero, risk of numeric error and overflow on IoU calculation Mar 12, 2024
Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>
Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>
@technolojin technolojin force-pushed the bugfix/tracker-object-size-zero-iou-overflow branch from 41e8713 to 87dbd8e Compare March 12, 2024 05:25
@technolojin technolojin marked this pull request as ready for review March 12, 2024 05:29
Copy link
Contributor

@YoshiRi YoshiRi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! There is just a minor comment. (Sorry, just my mistake ignore this comment)

@YoshiRi YoshiRi dismissed their stale review March 12, 2024 05:35

Wrong review comments

Copy link
Contributor

@yukkysaito yukkysaito left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@technolojin technolojin added the run:build-and-test-differential Mark to enable build-and-test-differential workflow. (used-by-ci) label Mar 12, 2024
Copy link

codecov bot commented Mar 12, 2024

Codecov Report

Attention: Patch coverage is 0% with 34 lines in your changes are missing coverage. Please review.

Project coverage is 14.79%. Comparing base (1de37a3) to head (79bddce).
Report is 2 commits behind head on main.

Files Patch % Lines
...ject_tracker/src/tracker/model/bicycle_tracker.cpp 0.00% 12 Missing ⚠️
...t_tracker/src/tracker/model/pedestrian_tracker.cpp 0.00% 10 Missing ⚠️
..._tracker/src/tracker/model/big_vehicle_tracker.cpp 0.00% 6 Missing ⚠️
...acker/src/tracker/model/normal_vehicle_tracker.cpp 0.00% 6 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #6597      +/-   ##
==========================================
- Coverage   14.80%   14.79%   -0.01%     
==========================================
  Files        1915     1915              
  Lines      132301   132329      +28     
  Branches    39320    39316       -4     
==========================================
  Hits        19581    19581              
- Misses      90888    90916      +28     
  Partials    21832    21832              
Flag Coverage Δ *Carryforward flag
differential 0.00% <0.00%> (?)
total 14.80% <ø> (+<0.01%) ⬆️ Carriedforward from c48e7d5

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

The minimum size is unified to a small number (0.3 m) to avoid side-effect of tracking.

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>
@technolojin technolojin enabled auto-merge (squash) March 12, 2024 07:00
@technolojin technolojin requested a review from YoshiRi March 12, 2024 07:03
Copy link
Contributor

@YoshiRi YoshiRi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@technolojin technolojin merged commit b7fdccc into autowarefoundation:main Mar 12, 2024
22 of 25 checks passed
technolojin added a commit to tier4/autoware.universe that referenced this pull request Mar 19, 2024
…error and overflow on IoU calculation (autowarefoundation#6597)

* fix: set object minimum size limits

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>

* fix: object shape conversion, from cylinder to bbox

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>

* fix: adjust minimum sizes to 0.3 m

The minimum size is unified to a small number (0.3 m) to avoid side-effect of tracking.

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>

---------

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>
kaigohirao pushed a commit to kaigohirao/autoware.universe that referenced this pull request Mar 22, 2024
…error and overflow on IoU calculation (autowarefoundation#6597)

* fix: set object minimum size limits

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>

* fix: object shape conversion, from cylinder to bbox

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>

* fix: adjust minimum sizes to 0.3 m

The minimum size is unified to a small number (0.3 m) to avoid side-effect of tracking.

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>

---------

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>
Signed-off-by: kaigohirao <kaigo.hirao@proxima-ai-tech.com>
karishma1911 pushed a commit to Interplai/autoware.universe that referenced this pull request Jun 3, 2024
…error and overflow on IoU calculation (autowarefoundation#6597)

* fix: set object minimum size limits

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>

* fix: object shape conversion, from cylinder to bbox

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>

* fix: adjust minimum sizes to 0.3 m

The minimum size is unified to a small number (0.3 m) to avoid side-effect of tracking.

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>

---------

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>
@technolojin technolojin deleted the bugfix/tracker-object-size-zero-iou-overflow branch July 26, 2024 02:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component:perception Advanced sensor data processing and environment understanding. (auto-assigned) run:build-and-test-differential Mark to enable build-and-test-differential workflow. (used-by-ci)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants