Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(multi_object_tracker): mot multi-step prediction is not work as intended #6611

Conversation

technolojin
Copy link
Contributor

@technolojin technolojin commented Mar 13, 2024

Description

The extended kalman filter update process was done in multiple-step when the given time-step is too large.
However, the implementation was not work as intended.

  • multi-step prediction
// if dt is too large, shorten dt and repeat prediction
const uint32_t repeat = std::ceil(dt / dt_max_);
const double dt_ = dt / repeat;
for (uint32_t i = 0; i < repeat; ++i) {
  if (!predictStateStep(dt_, tmp_ekf_for_no_update)) {
    return false;
  }
}

The motion model prediction is linealization of non-linear motion model. The prediction step assumes the time difference is small enough. By keeping the prediction time step small, a maximum time step was set and the prediction is to be conducted in multiple times.

  • Found symptom: the multi-step prediction is not working as intended.
    Until the reason is found, bring back to the one-step prediction.

FOUND BUG and FIX

A bug was found on predictStateStep methods.

  • intention: predict the future state of the variable Kalman filter ekf
  • bug: using the member Kalman filter ekf_
  • fix: get the state vector from the variable Kalman filter ekf

Tests performed

Tested on recorded data replay recomputation.

Effects on system behavior

Not applicable.

Pre-review checklist for the PR author

The PR author must check the checkboxes below when creating the PR.

In-review checklist for the PR reviewers

The PR reviewers must check the checkboxes below before approval.

Post-review checklist for the PR author

The PR author must check the checkboxes below before merging.

  • There are no open discussions or they are tracked via tickets.

After all checkboxes are checked, anyone who has write access can merge the PR.

…e-step prediction.

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>
@technolojin technolojin self-assigned this Mar 13, 2024
@github-actions github-actions bot added the component:perception Advanced sensor data processing and environment understanding. (auto-assigned) label Mar 13, 2024
@YoshiRi
Copy link
Contributor

YoshiRi commented Mar 13, 2024

@technolojin
Successfully built on my laptop and it seems to work, but could you describe more detail so that we can check what was wrong with past PR and how do you think fixed it?

@technolojin technolojin changed the title bugfix(multi_object_tracker): mot multi-step prediction is not work as intended fix(multi_object_tracker): mot multi-step prediction is not work as intended Mar 13, 2024
Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>
@technolojin technolojin added the run:build-and-test-differential Mark to enable build-and-test-differential workflow. (used-by-ci) label Mar 13, 2024
@technolojin technolojin enabled auto-merge (squash) March 13, 2024 23:47
Copy link

codecov bot commented Mar 14, 2024

Codecov Report

Attention: Patch coverage is 0% with 16 lines in your changes are missing coverage. Please review.

Project coverage is 14.78%. Comparing base (fcebec6) to head (c753924).
Report is 10 commits behind head on main.

Files Patch % Lines
...ker/src/tracker/motion_model/motion_model_base.cpp 0.00% 12 Missing ⚠️
...i_object_tracker/src/multi_object_tracker_core.cpp 0.00% 1 Missing ⚠️
.../src/tracker/motion_model/bicycle_motion_model.cpp 0.00% 1 Missing ⚠️
...ker/src/tracker/motion_model/ctrv_motion_model.cpp 0.00% 1 Missing ⚠️
...acker/src/tracker/motion_model/cv_motion_model.cpp 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #6611      +/-   ##
==========================================
- Coverage   14.79%   14.78%   -0.01%     
==========================================
  Files        1920     1920              
  Lines      132349   132430      +81     
  Branches    39339    39339              
==========================================
  Hits        19578    19578              
- Misses      90935    91016      +81     
  Partials    21836    21836              
Flag Coverage Δ *Carryforward flag
differential 0.00% <0.00%> (?)
total 14.79% <0.00%> (+<0.01%) ⬆️ Carriedforward from d28afea

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@technolojin
Copy link
Contributor Author

@YoshiRi
I described definition of multi-step prediction and current issue.
The reason of the issue is under investigation. Until then, I would like to roll-back the part to the previous status, and this PR is for that.

@technolojin technolojin requested a review from yukkysaito March 14, 2024 00:32
@technolojin technolojin disabled auto-merge March 14, 2024 02:19
@technolojin technolojin marked this pull request as draft March 14, 2024 02:19
@technolojin
Copy link
Contributor Author

The bug solution is found. Please wait for the fix.

a bug was found on `predictStateStep` methods

* intention: predict the future state of the variable kalman filter `ekf`
* bug: using the member kalman filter `ekf_`
* fix: get the state vector from the variable kalman filter `ekf`

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>
@technolojin technolojin marked this pull request as ready for review March 14, 2024 02:46
@technolojin
Copy link
Contributor Author

the bug is found, the solution is applied, and the explanation is added to the description.

Copy link
Contributor

@yukkysaito yukkysaito left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@YoshiRi YoshiRi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

The delay compensation worked with my rosbag. (blue: detection, yellow: prediction/tracking)
image

@YoshiRi YoshiRi merged commit ca46a8f into autowarefoundation:main Mar 14, 2024
24 of 27 checks passed
yhisaki pushed a commit to yhisaki/autoware.universe that referenced this pull request Mar 15, 2024
…ntended (autowarefoundation#6611)

* bugfix: mot multi-step prediction is not work as intended. back to one-step prediction.

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>

* fix: remove comment-out codes

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>

* fix: prediction to use variable kalman filter

a bug was found on `predictStateStep` methods

* intention: predict the future state of the variable kalman filter `ekf`
* bug: using the member kalman filter `ekf_`
* fix: get the state vector from the variable kalman filter `ekf`

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>

---------

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>
technolojin added a commit to tier4/autoware.universe that referenced this pull request Mar 18, 2024
…ntended (autowarefoundation#6611)

* bugfix: mot multi-step prediction is not work as intended. back to one-step prediction.

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>

* fix: remove comment-out codes

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>

* fix: prediction to use variable kalman filter

a bug was found on `predictStateStep` methods

* intention: predict the future state of the variable kalman filter `ekf`
* bug: using the member kalman filter `ekf_`
* fix: get the state vector from the variable kalman filter `ekf`

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>

---------

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>
technolojin added a commit to tier4/autoware.universe that referenced this pull request Mar 19, 2024
…ntended (autowarefoundation#6611)

* bugfix: mot multi-step prediction is not work as intended. back to one-step prediction.

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>

* fix: remove comment-out codes

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>

* fix: prediction to use variable kalman filter

a bug was found on `predictStateStep` methods

* intention: predict the future state of the variable kalman filter `ekf`
* bug: using the member kalman filter `ekf_`
* fix: get the state vector from the variable kalman filter `ekf`

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>

---------

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>
kaigohirao pushed a commit to kaigohirao/autoware.universe that referenced this pull request Mar 22, 2024
…ntended (autowarefoundation#6611)

* bugfix: mot multi-step prediction is not work as intended. back to one-step prediction.

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>

* fix: remove comment-out codes

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>

* fix: prediction to use variable kalman filter

a bug was found on `predictStateStep` methods

* intention: predict the future state of the variable kalman filter `ekf`
* bug: using the member kalman filter `ekf_`
* fix: get the state vector from the variable kalman filter `ekf`

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>

---------

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>
Signed-off-by: kaigohirao <kaigo.hirao@proxima-ai-tech.com>
karishma1911 pushed a commit to Interplai/autoware.universe that referenced this pull request Jun 3, 2024
…ntended (autowarefoundation#6611)

* bugfix: mot multi-step prediction is not work as intended. back to one-step prediction.

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>

* fix: remove comment-out codes

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>

* fix: prediction to use variable kalman filter

a bug was found on `predictStateStep` methods

* intention: predict the future state of the variable kalman filter `ekf`
* bug: using the member kalman filter `ekf_`
* fix: get the state vector from the variable kalman filter `ekf`

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>

---------

Signed-off-by: Taekjin LEE <taekjin.lee@tier4.jp>
@technolojin technolojin deleted the bugfix/mot-multi-step-prediction-bug branch July 26, 2024 02:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component:perception Advanced sensor data processing and environment understanding. (auto-assigned) run:build-and-test-differential Mark to enable build-and-test-differential workflow. (used-by-ci)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants