Simulation kinect in gazebo, Calibration validator error: TypeError: Conversion is only valid for arrays with 1 or 2 dimensions. Argument has 3 dimensions

[TouchDeeper]

Hello,

I calibrate the simulation kinect in gazebo with Kalibr, and the result is approximate to the true value. Then I want to try the Calibration validator, but an error occurred, all the output is below:

Initializing calibration target:
  Type: aprilgrid
  Tags: 
    Rows: 6
    Cols: 6
    Size: 0.088 [m]
    Spacing 0.0264 [m]
initializing camera geometry
Camera /camera/rgb/image_raw:
  Camera model: pinhole
  Focal length: [555.8460397102378, 556.7250965913527]
  Principal point: [318.05807762813083, 238.7783975622265]
  Distortion model: radtan
  Distortion coefficients: [0.006253042842637349, -0.006556783605854435, -0.0017719856280921736, -0.0003725808415176169]
rosout: bad callback: <bound method Subscriber.callback of <message_filters.Subscriber object at 0x7f0d5755c350>>
Traceback (most recent call last):
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/rospy/topics.py", line 750, in _invoke_callback
    cb(msg)
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/message_filters/__init__.py", line 75, in callback
    self.signalMessage(msg)
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/message_filters/__init__.py", line 57, in signalMessage
    cb(*(msg + args))
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/message_filters/__init__.py", line 224, in add
    self.signalMessage(*msgs)
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/message_filters/__init__.py", line 57, in signalMessage
    cb(*(msg + args))
  File "/home/wang/imta_project/kalibr_workspace/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_camera_validator", line 136, in synchronizedCallback
    success, observation = validator.target.detector.findTarget(timestamp, np_image)
TypeError: Conversion is only valid for arrays with 1 or 2 dimensions. Argument has 3 dimensions

I try to modify the ImageDatasetReader.py according to this issue, the raw version code is here, and the modified version is below:

    elif data.encoding == "16UC1" or data.encoding == "mono16":
      image_16u = np.squeeze(np.array(self.CVB.imgmsg_to_cv2(data,"mono16")))
      img_data = (image_16u / 256).astype("uint8")
    elif data.encoding == "8UC1" or data.encoding == "mono8":
      img_data = np.squeeze(np.array(self.CVB.imgmsg_to_cv2(data,"mono8")))
      
    elif data.encoding == "8UC3" or data.encoding == "bgr8":
      img_data = np.squeeze(np.array(self.CVB.imgmsg_to_cv2(data,"bgr8")))
      img_data = cv2.cvtColor(img_data, cv2.COLOR_BGR2GRAY)
    elif data.encoding == "rgb8":
      img_data = np.squeeze(np.array(self.CVB.imgmsg_to_cv2(data,"rgb8")))
      img_data = cv2.cvtColor(img_data, cv2.COLOR_RGB2GRAY)
    elif data.encoding == "8UC4" or data.encoding == "bgra8":
      img_data = np.squeeze(np.array(self.CVB.imgmsg_to_cv2(data,"bgra8")))
      img_data = cv2.cvtColor(img_data, cv2.COLOR_BGRA2GRAY)
    elif data.encoding == "bayer_rggb8":
      img_data = np.squeeze(np.array(self.CVB.imgmsg_to_cv2(data,"bayer_rggb8")))
      img_data = cv2.cvtColor(img_data, cv2.COLOR_BAYER_BG2GRAY)

However, the error still exists, Could you please provide some advice to fix this issue, thanks a lot.

[875353581]

in this case you use Calibration validator, you should modify kalibr_camera_validator.py instead of mageDatasetReader.py :
line 127
np_image = np.array(cv_image)
->
np_image = np.array(cv_image)
np_image = cv2.cvtColor(np_image, cv2.COLOR_BGR2GRAY)
what i've done is just add "np_image = cv2.cvtColor(np_image, cv2.COLOR_BGR2GRAY)", conver rgb image into gray image

[goktugyildirim]

Hello,

I calibrate the simulation kinect in gazebo with Kalibr, and the result is approximate to the true value. Then I want to try the Calibration validator, but an error occurred, all the output is below:

Initializing calibration target:
  Type: aprilgrid
  Tags: 
    Rows: 6
    Cols: 6
    Size: 0.088 [m]
    Spacing 0.0264 [m]
initializing camera geometry
Camera /camera/rgb/image_raw:
  Camera model: pinhole
  Focal length: [555.8460397102378, 556.7250965913527]
  Principal point: [318.05807762813083, 238.7783975622265]
  Distortion model: radtan
  Distortion coefficients: [0.006253042842637349, -0.006556783605854435, -0.0017719856280921736, -0.0003725808415176169]
rosout: bad callback: <bound method Subscriber.callback of <message_filters.Subscriber object at 0x7f0d5755c350>>
Traceback (most recent call last):
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/rospy/topics.py", line 750, in _invoke_callback
    cb(msg)
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/message_filters/__init__.py", line 75, in callback
    self.signalMessage(msg)
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/message_filters/__init__.py", line 57, in signalMessage
    cb(*(msg + args))
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/message_filters/__init__.py", line 224, in add
    self.signalMessage(*msgs)
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/message_filters/__init__.py", line 57, in signalMessage
    cb(*(msg + args))
  File "/home/wang/imta_project/kalibr_workspace/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_camera_validator", line 136, in synchronizedCallback
    success, observation = validator.target.detector.findTarget(timestamp, np_image)
TypeError: Conversion is only valid for arrays with 1 or 2 dimensions. Argument has 3 dimensions

I try to modify the ImageDatasetReader.py according to this issue, the raw version code is here, and the modified version is below:

    elif data.encoding == "16UC1" or data.encoding == "mono16":
      image_16u = np.squeeze(np.array(self.CVB.imgmsg_to_cv2(data,"mono16")))
      img_data = (image_16u / 256).astype("uint8")
    elif data.encoding == "8UC1" or data.encoding == "mono8":
      img_data = np.squeeze(np.array(self.CVB.imgmsg_to_cv2(data,"mono8")))
      
    elif data.encoding == "8UC3" or data.encoding == "bgr8":
      img_data = np.squeeze(np.array(self.CVB.imgmsg_to_cv2(data,"bgr8")))
      img_data = cv2.cvtColor(img_data, cv2.COLOR_BGR2GRAY)
    elif data.encoding == "rgb8":
      img_data = np.squeeze(np.array(self.CVB.imgmsg_to_cv2(data,"rgb8")))
      img_data = cv2.cvtColor(img_data, cv2.COLOR_RGB2GRAY)
    elif data.encoding == "8UC4" or data.encoding == "bgra8":
      img_data = np.squeeze(np.array(self.CVB.imgmsg_to_cv2(data,"bgra8")))
      img_data = cv2.cvtColor(img_data, cv2.COLOR_BGRA2GRAY)
    elif data.encoding == "bayer_rggb8":
      img_data = np.squeeze(np.array(self.CVB.imgmsg_to_cv2(data,"bayer_rggb8")))
      img_data = cv2.cvtColor(img_data, cv2.COLOR_BAYER_BG2GRAY)

However, the error still exists, Could you please provide some advice to fix this issue, thanks a lot.

you can try to use a grayscale image

[goldbattle]

Can you provide a bag where this fails?

[goldbattle]

See PR #15

[brawner]

Ran into a similar issue with the validator. It appears the only color encodings the validator special cases in this 'if' are "rgb8" and "bgra8" (https://github.com/ethz-asl/kalibr/blob/master/aslam_offline_calibration/kalibr/python/kalibr_camera_validator#L137). However ROS supports many more color encodings (https://docs.ros.org/en/noetic/api/sensor_msgs/html/image__encodings_8h_source.html).

In my case my images are "bgr8" as well. Perhaps, in this section it might be better to use cv_bridge's "passthrough" encoding and then use opencv to convert to 8-bit grayscale? Or just use mono8 on all images (I haven't verified that solution).

My workaround was to add the "bgr8" encoding to the special cases.
elif msg.encoding in ("rgb8", "bgra8", "bgr8"):

[brawner]

Actually after reading the CvBridge docs, it might be simpler to just convert all uncompressed message types to mono8:

                if msg._type == 'sensor_msgs/CompressedImage':
                    np_image = np.array(self.bridge.compressed_imgmsg_to_cv2(msg))
                    if len(np_image.shape) > 2 and np_image.shape[2] == 3:
                        np_image = cv2.cvtColor(np_image, cv2.COLOR_BGR2GRAY)
                else:
                    np_image = np.array(self.bridge.imgmsg_to_cv2(msg, desired_encoding="mono8"))