Skip to content

Commit ae73d48

Browse files
YoheiMishinawep21kmiyatkimura4yukkysaito
authored
fate: add object merger package (autowarefoundation#107)
* Ros2 v0.8.0 object merger (autowarefoundation#302) * Fix typo in perception module (autowarefoundation#440) * add use_sim-time option (autowarefoundation#454) * Unify Apache-2.0 license name (autowarefoundation#1242) * Remove use_sim_time for set_parameter (autowarefoundation#1260) Signed-off-by: wep21 <border_goldenmarket@yahoo.co.jp> * Perception components (autowarefoundation#1368) * [bev_optical_flow]: component node Signed-off-by: wep21 <border_goldenmarket@yahoo.co.jp> * [object_merger]: component node Signed-off-by: wep21 <border_goldenmarket@yahoo.co.jp> * [object_range_splitter]: component node Signed-off-by: wep21 <border_goldenmarket@yahoo.co.jp> * [shape_estimation]: component node Signed-off-by: wep21 <border_goldenmarket@yahoo.co.jp> * [map_based_prediction]: component node Signed-off-by: wep21 <border_goldenmarket@yahoo.co.jp> * [naive_path_prediction]: component node Signed-off-by: wep21 <border_goldenmarket@yahoo.co.jp> * [roi_image_saver]: component node Signed-off-by: wep21 <border_goldenmarket@yahoo.co.jp> * [lidar_apollo_instance_segmentation]: component node Signed-off-by: wep21 <border_goldenmarket@yahoo.co.jp> * [object_flow_fusion]: component node Signed-off-by: wep21 <border_goldenmarket@yahoo.co.jp> * [traffic_light_map_based_detector]: component node Signed-off-by: wep21 <border_goldenmarket@yahoo.co.jp> * [dynamic_object_visualization]: component node Signed-off-by: wep21 <border_goldenmarket@yahoo.co.jp> * Fix typo Signed-off-by: wep21 <border_goldenmarket@yahoo.co.jp> * Feature/porting mussp with vendor (autowarefoundation#1323) * change to mussp (autowarefoundation#1180) * change to mussp * add license * add license * change to solver plugin * fix bug * cosmetic change * modified to follow ROS2 coding style * Use mussp vendor Signed-off-by: wep21 <border_goldenmarket@yahoo.co.jp> * Fix dependencies in package.xml Signed-off-by: wep21 <border_goldenmarket@yahoo.co.jp> * Add mussp_vendor into build_depends.repos Signed-off-by: wep21 <border_goldenmarket@yahoo.co.jp> * Fix lint Signed-off-by: wep21 <border_goldenmarket@yahoo.co.jp> * Fix typo Signed-off-by: wep21 <border_goldenmarket@yahoo.co.jp> * Fix typo Signed-off-by: wep21 <border_goldenmarket@yahoo.co.jp> * Fix lint Signed-off-by: wep21 <border_goldenmarket@yahoo.co.jp> * Use fork instead of vendor package Signed-off-by: wep21 <border_goldenmarket@yahoo.co.jp> * Cleanup CMakeLists.txt Signed-off-by: wep21 <border_goldenmarket@yahoo.co.jp> * Remove comment Signed-off-by: wep21 <border_goldenmarket@yahoo.co.jp> * Sort package dependencies in alphabetical order Signed-off-by: wep21 <border_goldenmarket@yahoo.co.jp> Co-authored-by: Yukihiro Saito <yukky.saito@gmail.com> Co-authored-by: Keisuke Shima <keisuke.shima@tier4.jp> * Fix -Wunused-parameter (autowarefoundation#1836) * Fix -Wunused-parameter Signed-off-by: Kenji Miyake <kenji.miyake@tier4.jp> * Fix mistake Signed-off-by: Kenji Miyake <kenji.miyake@tier4.jp> * fix spell * Fix lint issues Signed-off-by: Kenji Miyake <kenji.miyake@tier4.jp> * Ignore flake8 warnings Signed-off-by: Kenji Miyake <kenji.miyake@tier4.jp> Co-authored-by: Hiroki OTA <hiroki.ota@tier4.jp> * add sort-package-xml hook in pre-commit (autowarefoundation#1881) * add sort xml hook in pre-commit * change retval to exit_status * rename * add prettier plugin-xml * use early return * add license note * add tier4 license * restore prettier * change license order * move local hooks to public repo * move prettier-xml to pre-commit-hooks-ros * update version for bug-fix * apply pre-commit * Detection by tracker (autowarefoundation#1910) * initial commit * backup * apply format * cosmetic change * implement divided under segmenterd clusters * cosmetic change * bug fix * bug fix * bug fix * modify launch * add debug and bug fix * bug fix * bug fix * add no found tracked object * modify parameters and cmake * bug fix * remove debug info * add readme * modify clustering launch * run pre-commit * cosmetic change * cosmetic change * cosmetic change * apply markdownlint * modify launch * modify for cpplint * modify qos * change int to size_T * bug fix * change perception qos * Update perception/object_recognition/detection/detection_by_tracker/package.xml Co-authored-by: Daisuke Nishimatsu <42202095+wep21@users.noreply.github.com> * cosmetic change * cosmetic change * fix launch * Update perception/object_recognition/detection/detection_by_tracker/src/utils.cpp Co-authored-by: Daisuke Nishimatsu <42202095+wep21@users.noreply.github.com> * modify header include order * change include order * Update perception/object_recognition/detection/detection_by_tracker/src/detection_by_tracker_core.cpp Co-authored-by: Daisuke Nishimatsu <42202095+wep21@users.noreply.github.com> * change to std::optional * cosmetic change * Update perception/object_recognition/detection/detection_by_tracker/src/detection_by_tracker_core.cpp Co-authored-by: Daisuke Nishimatsu <42202095+wep21@users.noreply.github.com> * Update perception/object_recognition/detection/detection_by_tracker/src/detection_by_tracker_core.cpp Co-authored-by: Daisuke Nishimatsu <42202095+wep21@users.noreply.github.com> * bug fix * modify readme Co-authored-by: Daisuke Nishimatsu <42202095+wep21@users.noreply.github.com> * Change formatter to clang-format and black (autowarefoundation#2332) * Revert "Temporarily comment out pre-commit hooks" This reverts commit 748e9cdb145ce12f8b520bcbd97f5ff899fc28a3. * Replace ament_lint_common with autoware_lint_common Signed-off-by: Kenji Miyake <kenji.miyake@tier4.jp> * Remove ament_cmake_uncrustify and ament_clang_format Signed-off-by: Kenji Miyake <kenji.miyake@tier4.jp> * Apply Black Signed-off-by: Kenji Miyake <kenji.miyake@tier4.jp> * Apply clang-format Signed-off-by: Kenji Miyake <kenji.miyake@tier4.jp> * Fix build errors Signed-off-by: Kenji Miyake <kenji.miyake@tier4.jp> * Fix for cpplint * Fix include double quotes to angle brackets Signed-off-by: Kenji Miyake <kenji.miyake@tier4.jp> * Apply clang-format Signed-off-by: Kenji Miyake <kenji.miyake@tier4.jp> * Fix build errors Signed-off-by: Kenji Miyake <kenji.miyake@tier4.jp> * Add COLCON_IGNORE (autowarefoundation#500) Signed-off-by: Kenji Miyake <kenji.miyake@tier4.jp> * port object merger (autowarefoundation#524) * support for autoware_auto_perception_msgs * use existence_probability * add README of object_merger (autowarefoundation#621) * add README of object_merger * add comment about score * fix typo * Update README.md Co-authored-by: Daisuke Nishimatsu <42202095+wep21@users.noreply.github.com> Co-authored-by: Kazuki Miyahara <kmiya@outlook.com> Co-authored-by: tkimura4 <tomoya.kimura@tier4.jp> Co-authored-by: Yukihiro Saito <yukky.saito@gmail.com> Co-authored-by: Keisuke Shima <keisuke.shima@tier4.jp> Co-authored-by: Kenji Miyake <31987104+kenji-miyake@users.noreply.github.com> Co-authored-by: Hiroki OTA <hiroki.ota@tier4.jp> Co-authored-by: Keisuke Shima <19993104+KeisukeShima@users.noreply.github.com> Co-authored-by: Satoshi OTA <44889564+satoshi-ota@users.noreply.github.com> Co-authored-by: Yusuke Muramatsu <yukke42@users.noreply.github.com>
1 parent 1f9486e commit ae73d48

File tree

12 files changed

+1247
-0
lines changed

12 files changed

+1247
-0
lines changed
+47
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
cmake_minimum_required(VERSION 3.5)
2+
project(object_merger)
3+
4+
find_package(PCL REQUIRED COMPONENTS common filters)
5+
find_package(ament_cmake_auto REQUIRED)
6+
ament_auto_find_build_dependencies()
7+
8+
### Compile options
9+
if(NOT CMAKE_CXX_STANDARD)
10+
set(CMAKE_CXX_STANDARD 14)
11+
set(CMAKE_CXX_STANDARD_REQUIRED ON)
12+
set(CMAKE_CXX_EXTENSIONS OFF)
13+
endif()
14+
if(CMAKE_COMPILER_IS_GNUCXX OR CMAKE_CXX_COMPILER_ID MATCHES "Clang")
15+
add_compile_options(-Wall -Wextra -Wpedantic)
16+
endif()
17+
18+
include_directories(
19+
include
20+
)
21+
22+
ament_auto_add_library(object_association_merger SHARED
23+
src/object_association_merger/utils/utils.cpp
24+
src/object_association_merger/data_association/data_association.cpp
25+
src/object_association_merger/data_association/successive_shortest_path/successive_shortest_path.cpp
26+
src/object_association_merger/node.cpp
27+
)
28+
29+
target_link_libraries(object_association_merger ${PCL_LIBRARIES})
30+
31+
rclcpp_components_register_node(object_association_merger
32+
PLUGIN "object_association::ObjectAssociationMergerNode"
33+
EXECUTABLE object_association_merger_node
34+
)
35+
36+
if(BUILD_TESTING)
37+
find_package(ament_lint_auto REQUIRED)
38+
ament_lint_auto_find_test_dependencies()
39+
endif()
40+
41+
#############
42+
## Install ##
43+
#############
44+
45+
ament_auto_package(INSTALL_TO_SHARE
46+
launch
47+
)

perception/object_merger/README.md

+72
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
# object_merger
2+
3+
## Purpose
4+
5+
object_merger is a package for merging detected objects from two methods by data association.
6+
7+
## Inner-workings / Algorithms
8+
9+
The successive shortest path algorithm is used to solve the data association problem (the minimum-cost flow problem). The cost is calculated by the distance between two objects and gate functions are applied to reset cost, s.t. the maximum distance, the maximum area and the minimum area.
10+
11+
## Inputs / Outputs
12+
13+
### Input
14+
15+
| Name | Type | Description |
16+
| --------------- | ----------------------------------------------------- | ----------------- |
17+
| `input/object0` | `autoware_auto_perception_msgs::msg::DetectedObjects` | detection objects |
18+
| `input/object1` | `autoware_auto_perception_msgs::msg::DetectedObjects` | detection objects |
19+
20+
### Output
21+
22+
| Name | Type | Description |
23+
| --------------- | ----------------------------------------------------- | ---------------- |
24+
| `output/object` | `autoware_auto_perception_msgs::msg::DetectedObjects` | modified Objects |
25+
26+
## Parameters
27+
28+
No Parameters.
29+
30+
## Assumptions / Known limits
31+
32+
<!-- Write assumptions and limitations of your implementation.
33+
34+
Example:
35+
This algorithm assumes obstacles are not moving, so if they rapidly move after the vehicle started to avoid them, it might collide with them.
36+
Also, this algorithm doesn't care about blind spots. In general, since too close obstacles aren't visible due to the sensing performance limit, please take enough margin to obstacles.
37+
-->
38+
39+
## (Optional) Error detection and handling
40+
41+
<!-- Write how to detect errors and how to recover from them.
42+
43+
Example:
44+
This package can handle up to 20 obstacles. If more obstacles found, this node will give up and raise diagnostic errors.
45+
-->
46+
47+
## (Optional) Performance characterization
48+
49+
<!-- Write performance information like complexity. If it wouldn't be the bottleneck, not necessary.
50+
51+
Example:
52+
### Complexity
53+
54+
This algorithm is O(N).
55+
56+
### Processing time
57+
58+
...
59+
-->
60+
61+
## (Optional) References/External links
62+
63+
<!-- Write links you referred to when you implemented.
64+
65+
Example:
66+
[1] {link_to_a_thesis}
67+
[2] {link_to_an_issue}
68+
-->
69+
70+
## (Optional) Future extensions / Unimplemented parts
71+
72+
Data association algorithm was the same as that of multi_object_tracker, but the algorithm of multi_object_tracker was already updated.
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
// Copyright 2020 Tier IV, Inc.
2+
//
3+
// Licensed under the Apache License, Version 2.0 (the "License");
4+
// you may not use this file except in compliance with the License.
5+
// You may obtain a copy of the License at
6+
//
7+
// http://www.apache.org/licenses/LICENSE-2.0
8+
//
9+
// Unless required by applicable law or agreed to in writing, software
10+
// distributed under the License is distributed on an "AS IS" BASIS,
11+
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
// See the License for the specific language governing permissions and
13+
// limitations under the License.
14+
15+
#ifndef OBJECT_ASSOCIATION_MERGER__DATA_ASSOCIATION_HPP_
16+
#define OBJECT_ASSOCIATION_MERGER__DATA_ASSOCIATION_HPP_
17+
18+
#include <list>
19+
#include <unordered_map>
20+
#include <vector>
21+
22+
#define EIGEN_MPL2_ONLY
23+
#include <Eigen/Core>
24+
#include <Eigen/Geometry>
25+
26+
#include <autoware_auto_perception_msgs/msg/detected_objects.hpp>
27+
#include <geometry_msgs/msg/point.hpp>
28+
#include <sensor_msgs/msg/point_cloud2.hpp>
29+
class DataAssociation
30+
{
31+
private:
32+
double getDistance(
33+
const geometry_msgs::msg::Point & point0, const geometry_msgs::msg::Point & point1);
34+
geometry_msgs::msg::Point getCentroid(const sensor_msgs::msg::PointCloud2 & pointcloud);
35+
Eigen::MatrixXi can_assign_matrix_;
36+
Eigen::MatrixXd max_dist_matrix_;
37+
Eigen::MatrixXd max_area_matrix_;
38+
Eigen::MatrixXd min_area_matrix_;
39+
const double score_threshold_;
40+
41+
public:
42+
DataAssociation();
43+
bool assign(
44+
const Eigen::MatrixXd & src, std::unordered_map<int, int> & direct_assignment,
45+
std::unordered_map<int, int> & reverse_assignment);
46+
Eigen::MatrixXd calcScoreMatrix(
47+
const autoware_auto_perception_msgs::msg::DetectedObjects & object0,
48+
const autoware_auto_perception_msgs::msg::DetectedObjects & object1);
49+
virtual ~DataAssociation() {}
50+
};
51+
52+
#endif // OBJECT_ASSOCIATION_MERGER__DATA_ASSOCIATION_HPP_
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
// Copyright 2020 Tier IV, Inc.
2+
//
3+
// Licensed under the Apache License, Version 2.0 (the "License");
4+
// you may not use this file except in compliance with the License.
5+
// You may obtain a copy of the License at
6+
//
7+
// http://www.apache.org/licenses/LICENSE-2.0
8+
//
9+
// Unless required by applicable law or agreed to in writing, software
10+
// distributed under the License is distributed on an "AS IS" BASIS,
11+
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
// See the License for the specific language governing permissions and
13+
// limitations under the License.
14+
15+
#ifndef OBJECT_ASSOCIATION_MERGER__NODE_HPP_
16+
#define OBJECT_ASSOCIATION_MERGER__NODE_HPP_
17+
18+
#include <object_association_merger/data_association.hpp>
19+
#include <rclcpp/rclcpp.hpp>
20+
21+
#include <autoware_auto_perception_msgs/msg/detected_objects.hpp>
22+
23+
#include <message_filters/subscriber.h>
24+
#include <message_filters/sync_policies/approximate_time.h>
25+
#include <message_filters/synchronizer.h>
26+
#include <pcl_conversions/pcl_conversions.h>
27+
#include <tf2_ros/buffer.h>
28+
#include <tf2_ros/transform_listener.h>
29+
30+
#include <memory>
31+
32+
namespace object_association
33+
{
34+
class ObjectAssociationMergerNode : public rclcpp::Node
35+
{
36+
public:
37+
explicit ObjectAssociationMergerNode(const rclcpp::NodeOptions & node_options);
38+
39+
private:
40+
void objectsCallback(
41+
const autoware_auto_perception_msgs::msg::DetectedObjects::ConstSharedPtr & input_object0_msg,
42+
const autoware_auto_perception_msgs::msg::DetectedObjects::ConstSharedPtr & input_object1_msg);
43+
44+
tf2_ros::Buffer tf_buffer_;
45+
tf2_ros::TransformListener tf_listener_;
46+
rclcpp::Publisher<autoware_auto_perception_msgs::msg::DetectedObjects>::SharedPtr
47+
merged_object_pub_;
48+
message_filters::Subscriber<autoware_auto_perception_msgs::msg::DetectedObjects> object0_sub_;
49+
message_filters::Subscriber<autoware_auto_perception_msgs::msg::DetectedObjects> object1_sub_;
50+
typedef message_filters::sync_policies::ApproximateTime<
51+
autoware_auto_perception_msgs::msg::DetectedObjects,
52+
autoware_auto_perception_msgs::msg::DetectedObjects>
53+
SyncPolicy;
54+
typedef message_filters::Synchronizer<SyncPolicy> Sync;
55+
Sync sync_;
56+
DataAssociation data_association_;
57+
};
58+
} // namespace object_association
59+
60+
#endif // OBJECT_ASSOCIATION_MERGER__NODE_HPP_
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
// Copyright 2020 Tier IV, Inc.
2+
//
3+
// Licensed under the Apache License, Version 2.0 (the "License");
4+
// you may not use this file except in compliance with the License.
5+
// You may obtain a copy of the License at
6+
//
7+
// http://www.apache.org/licenses/LICENSE-2.0
8+
//
9+
// Unless required by applicable law or agreed to in writing, software
10+
// distributed under the License is distributed on an "AS IS" BASIS,
11+
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
// See the License for the specific language governing permissions and
13+
// limitations under the License.
14+
15+
#ifndef OBJECT_ASSOCIATION_MERGER__SUCCESSIVE_SHORTEST_PATH_HPP_
16+
#define OBJECT_ASSOCIATION_MERGER__SUCCESSIVE_SHORTEST_PATH_HPP_
17+
18+
#include <unordered_map>
19+
#include <vector>
20+
21+
namespace assignment_problem
22+
{
23+
// See IMPORTANT NOTE at the top of the file.
24+
void MaximizeLinearAssignment(
25+
const std::vector<std::vector<double>> & cost, std::unordered_map<int, int> * direct_assignment,
26+
std::unordered_map<int, int> * reverse_assignment);
27+
} // namespace assignment_problem
28+
29+
#endif // OBJECT_ASSOCIATION_MERGER__SUCCESSIVE_SHORTEST_PATH_HPP_
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
// Copyright 2020 Tier IV, Inc.
2+
//
3+
// Licensed under the Apache License, Version 2.0 (the "License");
4+
// you may not use this file except in compliance with the License.
5+
// You may obtain a copy of the License at
6+
//
7+
// http://www.apache.org/licenses/LICENSE-2.0
8+
//
9+
// Unless required by applicable law or agreed to in writing, software
10+
// distributed under the License is distributed on an "AS IS" BASIS,
11+
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
// See the License for the specific language governing permissions and
13+
// limitations under the License.
14+
15+
#ifndef OBJECT_ASSOCIATION_MERGER__UTILS__UTILS_HPP_
16+
#define OBJECT_ASSOCIATION_MERGER__UTILS__UTILS_HPP_
17+
18+
#include <autoware_auto_perception_msgs/msg/shape.hpp>
19+
#include <geometry_msgs/msg/polygon.hpp>
20+
#include <geometry_msgs/msg/vector3.hpp>
21+
22+
#include <cmath>
23+
24+
namespace utils
25+
{
26+
double getPolygonArea(const geometry_msgs::msg::Polygon & footprint);
27+
double getRectangleArea(const geometry_msgs::msg::Vector3 & dimensions);
28+
double getCircleArea(const geometry_msgs::msg::Vector3 & dimensions);
29+
double getArea(const autoware_auto_perception_msgs::msg::Shape & shape);
30+
} // namespace utils
31+
32+
#endif // OBJECT_ASSOCIATION_MERGER__UTILS__UTILS_HPP_
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
<?xml version="1.0"?>
2+
3+
<launch>
4+
<arg name="input/object0" default="object0"/>
5+
<arg name="input/object1" default="object1"/>
6+
<arg name="output/object" default="merged_object"/>
7+
8+
<node pkg="object_merger" exec="object_association_merger_node" name="object_association_merger" output="screen">
9+
<remap from="input/object0" to="$(var input/object0)"/>
10+
<remap from="input/object1" to="$(var input/object1)"/>
11+
<remap from="output/object" to="$(var output/object)"/>
12+
</node>
13+
14+
</launch>

perception/object_merger/package.xml

+28
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
<?xml version="1.0"?>
2+
<package format="3">
3+
<name>object_merger</name>
4+
<version>0.1.0</version>
5+
<description>The object_merger package</description>
6+
7+
<maintainer email="yukihiro.saito@tier4.jp">Yukihiro Saito</maintainer>
8+
<license>Apache License 2.0</license>
9+
10+
<buildtool_depend>ament_cmake_auto</buildtool_depend>
11+
<depend>autoware_auto_perception_msgs</depend>
12+
<depend>libpcl-all-dev</depend>
13+
<depend>message_filters</depend>
14+
<depend>pcl_conversions</depend>
15+
<depend>rclcpp</depend>
16+
<depend>rclcpp_components</depend>
17+
<depend>sensor_msgs</depend>
18+
<depend>tf2</depend>
19+
<depend>tf2_ros</depend>
20+
<depend>tf2_sensor_msgs</depend>
21+
22+
<test_depend>ament_lint_auto</test_depend>
23+
<test_depend>autoware_lint_common</test_depend>
24+
25+
<export>
26+
<build_type>ament_cmake</build_type>
27+
</export>
28+
</package>

0 commit comments

Comments
 (0)