Stixel World Generator from LiDAR data

This repo is designed to generate custom Stixel training data based on different datasets. And is related to:

Toward Monocular Low-Weight Perception for Object Segmentation and Free Space Detection. IV 2024.
Marcel Vosshans, Omar Ait-Aider, Youcef Mezouar and Markus Enzweiler
University of Esslingen, UCA Sigma Clermont
[Xplore]

StixelGENerator

This repo provides the basic toolset to generate a Stixel World from LiDAR. It is used as Ground Truth for the StixelNExT 2D estimator as well as for the 3D approach: StixelNExT++.

Usage with Waymo or KITTI

1. Clone the repo to your local machine
2. Set up a virtual environment with python -m venv venv (we tested on Python 3.10) or Anaconda respectively conda create -n StixelGEN python=3.10. Activate with source venv/bin/activate/ conda activate StixelGEN
3. Install requirements with pip install -r requirements.txt/ conda install --file requirements.txt
4. Configure the project: adapt your paths in the config.yaml-file and select the dataset with the import in /generate.py like:

from dataloader import WaymoDataLoader as Dataset   # or KittiDataLoader

After that you can test the functionalities with utility/explore.py or run /generate.py to generate Stixel Worlds.

Output

The output is a .stx1 file which is a Protobuf for Stixel Worlds. It includes the Stixel as well as the image and is ready to serve as input data for StixelNExT++. The corresponding library can be installed with pip install pyStixel-lib and is public available here.

KITTI Training Data

We also provide an already generated dataset, based on the public available KITTI dataset. It can be downloaded here (35.48 GB)

Adaption to other Datasets

The repo is designed to work with adaptive dataloader, which can be handled by the import commands. For your own usage its necessary to write a new Dataloader for what the dataloader/BaseLoader and dataloader/BaseData can be used. Needed synchronised data/information are:

• The camera image
• LiDAR data
• Camera calibration data:
  • Camera Matrix
  • Projection Matrix
  • Rectify Matrix
  • Transformation Matrix
• Context information (image size, dataset name, ...)
• OPTIONAL: Stereo Images (right camera)
• OPTIONAL: LiDAR Calibration (in case of global T Matrices)

Fine tuning

You can heavily increase the results with the parameters from libraries/pcl-config.yaml. Documentation for the functions are provided by the code. The los (line of sight) parameter can cause huge holes!

Utilities

• explore: A simple workspace script to use and inspect the derived data, step by step.