Skip to content

Commit 8b787e3

Browse files
badai-nguyenbadai-nguyen
committed
docs: update readme
Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp>
1 parent 9148eb4 commit 8b787e3

File tree

1 file changed

+60
-21
lines changed

1 file changed

+60
-21
lines changed

‎perception/tensorrt_yolox/README.md

+60-21
Original file line numberDiff line numberDiff line change
@@ -2,13 +2,13 @@
22

33
## Purpose
44

5-
This package detects target objects e.g., cars, trucks, bicycles, and pedestrians on a image based on [YOLOX](https://github.com/Megvii-BaseDetection/YOLOX) model.
5+
This package detects target objects e.g., cars, trucks, bicycles, and pedestrians with semantic segmentation header including vehicle such as cars, trucks, buses and pedestrian, building, vegetation, road, sidewalk on a image based on [YOLOX](https://github.com/Megvii-BaseDetection/YOLOX) model.
66

77
## Inner-workings / Algorithms
88

99
### Cite
1010

11-
<!-- cspell: ignore Zheng, Songtao, Feng, Zeming, Jian -->
11+
<!-- cspell: ignore Zheng, Songtao, Feng, Zeming, Jian, semseg -->
1212

1313
Zheng Ge, Songtao Liu, Feng Wang, Zeming Li, Jian Sun, "YOLOX: Exceeding YOLO Series in 2021", arXiv preprint arXiv:2107.08430, 2021 [[ref](https://arxiv.org/abs/2107.08430)]
1414

@@ -22,10 +22,12 @@ Zheng Ge, Songtao Liu, Feng Wang, Zeming Li, Jian Sun, "YOLOX: Exceeding YOLO Se
2222

2323
### Output
2424

25-
| Name | Type | Description |
26-
| ------------- | -------------------------------------------------- | -------------------------------------------------- |
27-
| `out/objects` | `tier4_perception_msgs/DetectedObjectsWithFeature` | The detected objects with 2D bounding boxes |
28-
| `out/image` | `sensor_msgs/Image` | The image with 2D bounding boxes for visualization |
25+
| Name | Type | Description |
26+
| ---------------- | -------------------------------------------------- | ------------------------------------------------------------------- |
27+
| `out/objects` | `tier4_perception_msgs/DetectedObjectsWithFeature` | The detected objects with 2D bounding boxes |
28+
| `out/image` | `sensor_msgs/Image` | The image with 2D bounding boxes for visualization |
29+
| `out/mask` | `sensor_msgs/Image` | The semantic segmentation mask |
30+
| `out/color_mask` | `sensor_msgs/Image` | The colorized image of semantic segmentation mask for visualization |
2931

3032
## Parameters
3133

@@ -40,20 +42,32 @@ Zheng Ge, Songtao Liu, Feng Wang, Zeming Li, Jian Sun, "YOLOX: Exceeding YOLO Se
4042

4143
### Node Parameters
4244

43-
| Name | Type | Default Value | Description |
44-
| ----------------------------- | ------ | ------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
45-
| `model_path` | string | "" | The onnx file name for yolox model |
46-
| `label_path` | string | "" | The label file with label names for detected objects written on it |
47-
| `precision` | string | "fp16" | The inference mode: "fp32", "fp16", "int8" |
48-
| `build_only` | bool | false | shutdown node after TensorRT engine file is built |
49-
| `calibration_algorithm` | string | "MinMax" | Calibration algorithm to be used for quantization when precision==int8. Valid value is one of: Entropy",("Legacy" \| "Percentile"), "MinMax"] |
50-
| `dla_core_id` | int | -1 | If positive ID value is specified, the node assign inference task to the DLA core |
51-
| `quantize_first_layer` | bool | false | If true, set the operating precision for the first (input) layer to be fp16. This option is valid only when precision==int8 |
52-
| `quantize_last_layer` | bool | false | If true, set the operating precision for the last (output) layer to be fp16. This option is valid only when precision==int8 |
53-
| `profile_per_layer` | bool | false | If true, profiler function will be enabled. Since the profile function may affect execution speed, it is recommended to set this flag true only for development purpose. |
54-
| `clip_value` | double | 0.0 | If positive value is specified, the value of each layer output will be clipped between [0.0, clip_value]. This option is valid only when precision==int8 and used to manually specify the dynamic range instead of using any calibration |
55-
| `preprocess_on_gpu` | bool | true | If true, pre-processing is performed on GPU |
56-
| `calibration_image_list_path` | string | "" | Path to a file which contains path to images. Those images will be used for int8 quantization. |
45+
| Name | Type | Default Value | Description |
46+
| -------------------------------------- | ------ | ------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
47+
| `model_path` | string | "" | The onnx file name for yolox model |
48+
| `label_path` | string | "" | The label file with label names for detected objects written on it |
49+
| `precision` | string | "fp16" | The inference mode: "fp32", "fp16", "int8" |
50+
| `build_only` | bool | false | shutdown node after TensorRT engine file is built |
51+
| `calibration_algorithm` | string | "MinMax" | Calibration algorithm to be used for quantization when precision==int8. Valid value is one of: Entropy",("Legacy" \| "Percentile"), "MinMax"] |
52+
| `dla_core_id` | int | -1 | If positive ID value is specified, the node assign inference task to the DLA core |
53+
| `quantize_first_layer` | bool | false | If true, set the operating precision for the first (input) layer to be fp16. This option is valid only when precision==int8 |
54+
| `quantize_last_layer` | bool | false | If true, set the operating precision for the last (output) layer to be fp16. This option is valid only when precision==int8 |
55+
| `profile_per_layer` | bool | false | If true, profiler function will be enabled. Since the profile function may affect execution speed, it is recommended to set this flag true only for development purpose. |
56+
| `clip_value` | double | 0.0 | If positive value is specified, the value of each layer output will be clipped between [0.0, clip_value]. This option is valid only when precision==int8 and used to manually specify the dynamic range instead of using any calibration |
57+
| `preprocess_on_gpu` | bool | true | If true, pre-processing is performed on GPU |
58+
| `calibration_image_list_path` | string | "" | Path to a file which contains path to images. Those images will be used for int8 quantization. |
59+
| `yolox_s_plus_opt_param_path` | string | "" | Path to parameter file |
60+
| `is_publish_color_mask` | bool | false | If true, publish color mask for result visualization |
61+
| `is_roi_overlap_segment` | bool | true | If true, overlay detected object roi onto semantic segmentation as roi higher priority |
62+
| `overlap_roi_score_threshold` | float | 0.3 | minimum existence_probability of detected roi considered to replace segmentation |
63+
| `roi_overlay_segment_label.UNKNOWN` | bool | true | If true, unknown objects roi will be overlay onto sematic segmentation mask. |
64+
| `roi_overlay_segment_label.CAR` | bool | true | If true, car objects roi will be overlay onto sematic segmentation mask. |
65+
| `roi_overlay_segment_label.TRUCK` | bool | true | If true, truck objects roi will be overlay onto sematic segmentation mask. |
66+
| `roi_overlay_segment_label.BUS` | bool | true | If true, bus objects roi will be overlay onto sematic segmentation mask. |
67+
| `roi_overlay_segment_label.TRAILER` | bool | true | If true, trailer objects roi will be overlay onto sematic segmentation mask. |
68+
| `roi_overlay_segment_label.MOTORCYCLE` | bool | true | If true, motorcycle objects roi will be overlay onto sematic segmentation mask. |
69+
| `roi_overlay_segment_label.BICYCLE` | bool | true | If true, bicycle objects roi will be overlay onto sematic segmentation mask. |
70+
| `roi_overlay_segment_label.PEDESTRIAN` | bool | true | If true, pedestrian objects roi will be overlay onto sematic segmentation mask. |
5771

5872
## Assumptions / Known limits
5973

@@ -69,6 +83,31 @@ The label contained in detected 2D bounding boxes (i.e., `out/objects`) will be
6983
If other labels (case insensitive) are contained in the file specified via the `label_file` parameter,
7084
those are labeled as `UNKNOWN`, while detected rectangles are drawn in the visualization result (`out/image`).
7185

86+
The semantic segmentation mask are gray image whose pixel is index of one of the followings:
87+
88+
| index | semantic name |
89+
| ----- | ------------- |
90+
| 0 | road |
91+
| 1 | sidewalk |
92+
| 2 | building |
93+
| 3 | wall |
94+
| 4 | fence |
95+
| 5 | pole |
96+
| 6 | traffic_light |
97+
| 7 | traffic_sign |
98+
| 8 | vegetation |
99+
| 9 | terrain |
100+
| 10 | sky |
101+
| 11 | person |
102+
| 12 | ride |
103+
| 13 | car |
104+
| 14 | truck |
105+
| 15 | bus |
106+
| 16 | train |
107+
| 17 | motorcycle |
108+
| 18 | bicycle |
109+
| 19 | others |
110+
72111
## Onnx model
73112

74113
A sample model (named `yolox-tiny.onnx`) is downloaded by ansible script on env preparation stage, if not, please, follow [Manual downloading of artifacts](https://github.com/autowarefoundation/autoware/tree/main/ansible/roles/artifacts).
@@ -146,7 +185,7 @@ Please refer [the official document](https://github.com/Megvii-BaseDetection/YOL
146185

147186
## Label file
148187

149-
A sample label file (named `label.txt`)is also downloaded automatically during env preparation process
188+
A sample label file (named `label.txt`) and semantic segmentation color map file (name `semseg_color_map.csv`) are also downloaded automatically during env preparation process
150189
(**NOTE:** This file is incompatible with models that output labels for the COCO dataset (e.g., models from the official YOLOX repository)).
151190

152191
This file represents the correspondence between class index (integer outputted from YOLOX network) and

0 commit comments

Comments
 (0)