How to get a surface normal image aligned with color and depth images?

[Boatsure]

I want to get an aligned 3-channel normal map while getting the aligned color and depth images. Each pixel position on this normal map stores the xyz normal direction of the object surface relative to the camera position.

Is there an officially recommended way to achieve this in the current version and time point? Better in python, thank you！

[MartyG-RealSense]

Hi @Boatsure A RealSense user at #3321 shared Python code for the method they used to obtain the surface normal of a depth image.

I was not able to find an example of a script that used both surface normals and depth-color alignment though.

[Boatsure]

@MartyG-RealSense Thanks for your reply！The script you mentioned is a kind of numerical calculation method based entirely on depth image, and its accuracy is worrying.

The viewer can export .ply file in the 3D mode, which has an option of normals output. May I ask how do the normals calculated here?

[MartyG-RealSense]

The save_to_ply() .ply file export function has a setting called OPTION_PLY_NORMALS that exports normals when set to true. The code for controlling this function is in the librealsense SDK file 'rs_export.hpp'

https://github.com/IntelRealSense/librealsense/blob/e1688cc318457f7dd57abcdbedd3398062db3009/include/librealsense2/hpp/rs_export.hpp

If you do a web browser 'find' operation on this page for the word normals then it will highlight the sections of code related to normals calculation.

The easiest way to export a ply with normals included is to use the Export option in the 3D pointcloud mode of the RealSense Viewer tool. You can also use Python code though. The SDK's export_ply_example.py ply export Python example program uses save_to_ply() and sets normals export to true.

https://github.com/IntelRealSense/librealsense/blob/master/wrappers/python/examples/export_ply_example.py