Skip to content

Commit 81b7694

Browse files
xuankuzcrxuankuzcr
committed
Camvox
1 parent 219dee5 commit 81b7694

File tree

724 files changed

+56805
-1
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

724 files changed

+56805
-1
lines changed

‎README.md

+125-1
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,126 @@
11
# CamVox
2-
A low-cost SLAM system based on camera and Livox lidar.
2+
## A Low-cost and Accurate Lidar-assisted Visual SLAM System
3+
4+
<img src="./pics/CamVox.gif" alt="show" />
5+
6+
We propose **CamVox** by adapting Livox lidars into visual SLAM (ORB-SLAM2) by exploring the lidars’ unique features. Based on the non-repeating nature of Livox lidars, we propose an automatic lidar-camera calibration method that will work in uncontrolled scenes. The long depth detection range also benefit a more efficient mapping. Comparison of CamVox with visual SLAM (VINS-mono) and lidar SLAM (livox_horizon_loam) are evaluated on the same dataset to demonstrate the performance.
7+
8+
<div align="center">
9+
<img src="pics/compare.png" width = 100% >
10+
</div>
11+
12+
**Developer:** [Yuewen Zhu](https://github.com/zywok), [Chunran Zheng](https://github.com/xuankuzcr), [Chongjian Yuan](https://github.com/ChongjianYUAN)
13+
14+
**Our related video**: our related videos are now available on [[YouTube Video](https://www.youtube.com/watch?v=AUnZNBB-uUE)] [[bilibili Video](https://www.bilibili.com/video/BV1fZ4y1V795/)].
15+
16+
## 1. Prerequisites
17+
### 1.1 Ubuntu and ROS
18+
Ubuntu 64-bit 16.04 or 18.04.
19+
ROS Kinetic or Melodic. Follow [ROS Installation](http://wiki.ros.org/ROS/Installation).
20+
21+
### 1.2 Pangolin
22+
We use [Pangolin](https://github.com/stevenlovegrove/Pangolin) for visualization and user interface. Follow [Pangolin Installation](https://github.com/stevenlovegrove/Pangolin).
23+
24+
### 1.3 OpenCV
25+
We use OpenCV to manipulate images and features. Follow [Opencv Installation](http://opencv.org). **Required at leat 2.4.3. Tested with OpenCV 2.4.11 and OpenCV 3.2**.
26+
27+
### 1.4 Eigen3
28+
Follow [Eigen Installation](http://eigen.tuxfamily.org). **Required at least 3.1.0**.
29+
30+
### 1.5 Ceres Solver
31+
Follow [Ceres Installation](http://ceres-solver.org/installation.html).
32+
33+
## 2. Build CamVox
34+
Clone the repository and catkin_make:
35+
36+
```bash
37+
cd ~/catkin_ws/src
38+
git clone https://github.com/ISEE-Technology/CamVox && cd ..
39+
catkin_make
40+
cd src/CamVox/isee-camvox && chmod +x build_camvox.sh
41+
./build_camvox.sh
42+
source ~/catkin_ws/devel/setup.bash
43+
```
44+
## 3. Run with Hardware
45+
### 3.1 Hardware
46+
47+
<table>
48+
<tr>
49+
<th>Platform</th>
50+
<th>Item</th>
51+
<th>Pics</th>
52+
<th>Shopping Link</th>
53+
</tr >
54+
<tr >
55+
<td rowspan="5"><img src="./pics/platform.png" /></td>
56+
<td>Livox Horizon </td>
57+
<td align="center" valign="middle"><img src= "./pics/horizon.jpg" width=25% /></td>
58+
<td align="center" valign="middle"> <a href ="https://www.livoxtech.com/horizon"> Lidar </a> </td>
59+
</tr>
60+
<tr>
61+
<td> MV-CE060-10UC</td>
62+
<td align="center" valign="middle"><img src="./pics/camera.png" width=19% /></td>
63+
<td align="center" valign="middle"> <a href ="https://en.hikrobotics.com/vision/visioninfo.htm?type=42&oid=2451"> Camera </a> </td>
64+
</tr>
65+
<tr>
66+
<td> Inertial Sense uINS </td>
67+
<td align="center" valign="middle"> <img src="./pics/Inertial_Sense_uINS.png" width=22% /> </td>
68+
<td align="center" valign="middle"> <a href ="https://inertialsense.com/product/rugged-µins/"> RTK </a> </td>
69+
</tr>
70+
<tr>
71+
<td>Manifold2C</td>
72+
<td align="center" valign="middle"><img src="./pics/Manifold2C.jpg" width=22% /></td>
73+
<td align="center" valign="middle"> <a href ="https://www.dji.com/cn/manifold-2"> Onboard-Computer </a> </td>
74+
</tr>
75+
<tr>
76+
<td> Scout-mini </td>
77+
<td align="center" valign="middle"><img src="./pics/Scout-mini.jpg" width=28% /></td>
78+
<td align="center" valign="middle"> <a href ="http://www.agilex.ai/index/product/id/3?lang=zh-cn"> Robot Chassis </a> </td>
79+
</tr>
80+
</table>
81+
82+
### 3.2 Hard Synchronization
83+
84+
Hard synchronization is performed with all of these sensors by a trigger signal of 10 Hz. The camera output at each trigger signal(10 Hz). The lidar keeps a clock (synced with GPS-RTK) and continuously outputs the scanned point with an accurate timestamp. In the meantime, the IMU outputs at a frequency of 200 Hz synced with the trigger. The Hardware Synchronization diagram is as follows.
85+
86+
<img src="./pics/synchronization.jpg" width=100% />
87+
88+
### 3.3 Running
89+
90+
Connect to your PC to Livox Horizon lidar by following [Livox-ros-driver installation](https://github.com/Livox-SDK/livox_ros_driver).
91+
92+
```bash
93+
chmod +x run.sh
94+
./run.sh
95+
```
96+
97+
98+
## 4. Run with Rosbag Example
99+
100+
### 4.1 SUSTech Dataset (Loop Closure)
101+
102+
We open sourced our dataset in SUSTech campus with loop closure. [Download here](https://drive.google.com/file/).
103+
104+
### 4.2 Rosbag Example with static scenes (Automatic Calibration trigger)
105+
106+
We provide a rosbag file with static scenes to test the automatic calibration thread. [Download here](https://drive.google.com/file/). <br/>
107+
When the car detects more than **20 frames of still images (about 2 seconds)**, the automatic calibration thread starts to work. The thread will be interrupted to enter the SLAM mode if the car starts to move before the end of calibration.
108+
The effects of automatic calibration is shown as follows.
109+
110+
<img src="./pics/runyang.gif" alt="show" width = 49% /> <img src="./pics/flat.gif" alt="show" width = 49% />
111+
112+
### 4.3 Running
113+
```bash
114+
chmod +x run.sh
115+
./run.sh
116+
rosbag play YOUR_DOWNLOADED.bag
117+
```
118+
119+
## 5. Acknowledgements
120+
The authors thank colleagues at Livox Technology for helpful discussion and support. <br/>
121+
The repository is from [**ISEE**](https://isee.technology/).
122+
123+
## 6. License
124+
125+
The source code is released under [GPLv2](http://www.gnu.org/licenses/) license.
126+

‎inertial-sense-ros

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Subproject commit 9b687f53bc9e2dd17fe4606da86d443327bb9975

‎isee-camvox

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Subproject commit 557bdc5ddad6214a18c245f503b5a7464c01f30b

‎isee-preprocessing/.vscode/c_cpp_properties.json

+17
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
{
2+
"configurations": [
3+
{
4+
"name": "Linux",
5+
"includePath": [
6+
"${workspaceFolder}/**"
7+
],
8+
"defines": [],
9+
"compilerPath": "/usr/bin/gcc",
10+
"cStandard": "gnu11",
11+
"cppStandard": "c++17",
12+
"intelliSenseMode": "gcc-x64",
13+
"compileCommands": "${workspaceFolder}/build/compile_commands.json"
14+
}
15+
],
16+
"version": 4
17+
}

‎isee-preprocessing/.vscode/settings.json

+65
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
{
2+
"files.associations": {
3+
"ostream": "cpp",
4+
"cctype": "cpp",
5+
"clocale": "cpp",
6+
"cmath": "cpp",
7+
"csignal": "cpp",
8+
"cstdarg": "cpp",
9+
"cstddef": "cpp",
10+
"cstdio": "cpp",
11+
"cstdlib": "cpp",
12+
"cstring": "cpp",
13+
"ctime": "cpp",
14+
"cwchar": "cpp",
15+
"cwctype": "cpp",
16+
"array": "cpp",
17+
"atomic": "cpp",
18+
"strstream": "cpp",
19+
"*.tcc": "cpp",
20+
"bitset": "cpp",
21+
"chrono": "cpp",
22+
"complex": "cpp",
23+
"cstdint": "cpp",
24+
"deque": "cpp",
25+
"list": "cpp",
26+
"unordered_map": "cpp",
27+
"vector": "cpp",
28+
"exception": "cpp",
29+
"algorithm": "cpp",
30+
"functional": "cpp",
31+
"ratio": "cpp",
32+
"system_error": "cpp",
33+
"tuple": "cpp",
34+
"type_traits": "cpp",
35+
"fstream": "cpp",
36+
"initializer_list": "cpp",
37+
"iomanip": "cpp",
38+
"iosfwd": "cpp",
39+
"iostream": "cpp",
40+
"istream": "cpp",
41+
"limits": "cpp",
42+
"memory": "cpp",
43+
"new": "cpp",
44+
"numeric": "cpp",
45+
"sstream": "cpp",
46+
"stdexcept": "cpp",
47+
"streambuf": "cpp",
48+
"thread": "cpp",
49+
"cfenv": "cpp",
50+
"cinttypes": "cpp",
51+
"utility": "cpp",
52+
"typeindex": "cpp",
53+
"typeinfo": "cpp",
54+
"string": "cpp",
55+
"codecvt": "cpp",
56+
"condition_variable": "cpp",
57+
"unordered_set": "cpp",
58+
"hash_map": "cpp",
59+
"hash_set": "cpp",
60+
"future": "cpp",
61+
"mutex": "cpp",
62+
"valarray": "cpp",
63+
"*.ipp": "cpp"
64+
}
65+
}

‎isee-preprocessing/CMakeLists.txt

+73
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,73 @@
1+
cmake_minimum_required(VERSION 2.8.3)
2+
project(isee_preprocessing)
3+
set(CMAKE_BUILD_TYPE "Release")
4+
set(CMAKE_CXX_FLAGS "-std=c++11 -fext-numeric-literals")
5+
set(CMAKE_CXX_FLAGS_RELEASE "-O2 -Wall -g")
6+
find_package(catkin REQUIRED COMPONENTS
7+
message_generation
8+
roscpp
9+
rospy
10+
std_msgs
11+
pcl_conversions
12+
pcl_ros
13+
cv_bridge
14+
image_transport
15+
sensor_msgs
16+
geometry_msgs
17+
camera_info_manager
18+
)
19+
20+
find_package( OpenCV 3.2.0 REQUIRED )
21+
22+
## Generate messages in the 'msg' folder
23+
add_message_files(
24+
FILES
25+
CustomPoint.msg
26+
CustomMsg.msg
27+
GPS.msg
28+
)
29+
add_service_files(
30+
FILES
31+
FirmwareUpdate.srv
32+
refLLAUpdate.srv
33+
)
34+
35+
## Generate added messages and services with any dependencies listed here
36+
generate_messages(
37+
DEPENDENCIES
38+
std_msgs # Or other packages containing msgs
39+
geometry_msgs
40+
)
41+
42+
43+
44+
include_directories(include ${catkin_INCLUDE_DIRS} ${OpenCV_INCLUDE_DIRS} include /opt/MVS/include/)
45+
LINK_DIRECTORIES( /opt/MVS/lib/64/ )
46+
47+
add_executable(isee_livox_repub src/livox_repub.cpp)
48+
target_link_libraries(isee_livox_repub ${catkin_LIBRARIES} ${PCL_LIBRARIES} ${OpenCV_LIBS})
49+
50+
add_executable(imu_processing src/data_process_node.cpp src/data_process.cpp src/gyr_int.cpp )
51+
target_link_libraries(imu_processing ${catkin_LIBRARIES} ${PCL_LIBRARIES} ${OpenCV_LIBS})
52+
53+
add_executable(isee_synchronize src/isee_synchronize.cpp src/camera.cc)
54+
add_dependencies(isee_synchronize ${PROJECT_NAME}_generate_messages_cpp)
55+
target_link_libraries(isee_synchronize ${catkin_LIBRARIES} ${OpenCV_LIBRARIES} ${PCL_LIBRARIES}
56+
GCBase_gcc421_v3_0
57+
MvCameraControl
58+
GenApi_gcc421_v3_0
59+
MVGigEVisionSDK
60+
log4cpp_gcc421_v3_0
61+
MVRender
62+
Log_gcc421_v3_0
63+
MvUsb3vTL
64+
MathParser_gcc421_v3_0
65+
NodeMapData_gcc421_v3_0
66+
MediaProcess
67+
XmlParser_gcc421_v3_0
68+
X11
69+
)
70+
71+
72+
73+

‎isee-preprocessing/build/CATKIN_IGNORE

Whitespace-only changes.

0 commit comments

Comments
 (0)