SphKpt

SphKpt is a uniform framework for detecting keypoints on both omnidirectional and perspective images.

Dependencies

• numpy
• scipy
• matplotlib
• opencv-python
• opencv-contrib-python
• pillow
• tensorflow >= 2.0
• tensorflow_addons

Train the framework

• Run pretrain/vp_pretrain.py to pre-train the perspective module.
• Run pretrain/erp_pretrain.py to pre-train the spherical module.
• run train/train.py to train the two modules jointly.

Test

• Run test/test.py to detect the keypoints.

Datasets

• PanoContext Dataset

Please download the PanoContext dataset from this link.

• Multimodal Panoramic 3D Outdoor Dataset

Please download the Multimodal Panoramic 3D Outdoor Dataset from this link.

Acknowledgements

We thank the reviewers for their valuable feedback that has helped us improve our paper.

We use the public implementation of fast_soft_sort and GCNN, we thank the authors.