danielpollithy / pypcd Goto Github PK

i have a code that i am using to convert a pcd file to bin. i am using your pypcd. i am getting an error in my code and wanted to know if you had an idea of how i could fix it.
this is line line the error happens at:

This is my error overall:

my pcd file has the fields of x y z r(reflectance) i (index)

import cStringIO as sio
ModuleNotFoundError: No module named 'cStringIO'

I'm try to use 'pypcd' from ROS2 (colcon build passes with small modifications)

However I got the following error in 'numpy_pc2.py' (see below for details)
when trying to run a simple ROS2 python file 'xxx.py' (included below)

 TypeError: fromstring() argument 1 must be read-only bytes-like object, not array.array

I'm using the ascii/binary (not compressed) pcd data from here :
https://drive.google.com/drive/folders/11PkiTw2ZXh1KaXzBlsjYgT4chPIcsOgJ

Any ideas what the problem could be? thanks in advance for any/all help.

run

root@jupiter:~/rosbag_to_pcd# python3 xxx.py
<class 'sensor_msgs.msg._point_cloud2.PointCloud2'>
Traceback (most recent call last):
  File "xxx.py", line 20, in <module>
    npc = pypcd.PointCloud.from_msg(msg)
  File "/root/rosbag_to_pcd/ros2_ws/install/pypcd/lib/python3.8/site-packages/pypcd/pypcd.py", line 827, in from_msg
    pc_array = numpy_pc2.pointcloud2_to_array(msg)
  File "/root/rosbag_to_pcd/ros2_ws/install/pypcd/lib/python3.8/site-packages/pypcd/numpy_pc2.py", line 134, in pointcloud2_to_array
    cloud_arr = np.fromstring(cloud_msg.data, dtype_list)
TypeError: fromstring() argument 1 must be read-only bytes-like object, not array.array
root@jupiter:~/rosbag_to_pcd#

file: xxx.py

import pypcd
from pypcd import pypcd
import rclpy
from rclpy.node import Node
from rclpy.qos import QoSProfile
from sensor_msgs.msg import PointCloud2

rclpy.init()
qos_profile = QoSProfile(depth=10)
node = Node("msg_publisher")
pub = node.create_publisher(PointCloud2, "/to_pcd", qos_profile=qos_profile)

fn = "foo.pcd"
fn = "x.data/pcd_files/0000000040_ascii.pcd"
out_fn = f"{fn}.msg.pcd"

pc =pypcd.PointCloud.from_path(fn)
msg = pc.to_msg()
print(type(msg))
npc = pypcd.PointCloud.from_msg(msg)
npc.save(out_fn)
print(type(npc))
if 0 :
    pub.publish(msg)
    rclpy.spin(node)
node.destroy_node()
rclpy.shutdown()

I'm using python 3.8 and want to read binary or binary compressed pcd files.
The pcd files seems to be ok, because other tools like pcl_viewer visualize them correctly.

ASCII ==> OK
{'version': '0.7', 'fields': ['x', 'y', 'z', 'intensity', 't', 'reflectivity', 'ring', 'ambient', 'range'], 'size': [4, 4, 4, 4, 4, 2, 1, 2, 4], 'type': ['F', 'F', 'F', 'F', 'U', 'U', 'U', 'U', 'U'], 'count': [1, 1, 1, 1, 1, 1, 1, 1, 1], 'width': 2048, 'height': 128, 'viewpoint': [0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0], 'points': 262144, 'data': 'ascii'}
[(-0.000000000000, 0.000000000000, 0.000000000000, 4.000000000000, 0, 0, 0, 229, 0)
(-0.313022673130, 0.022687347606, 0.153115406632, 52.000000000000, 43776, 117, 0, 154, 336)
(-0.306455463171, 0.023132724687, 0.150558993220, 60.000000000000, 95232, 119, 0, 165, 329)
(-0.306383043528, 0.024072807282, 0.150558993220, 70.000000000000, 149760, 122, 0, 159, 329)
(-0.000000000000, 0.000000000000, 0.000000000000, 3.000000000000, 197632, 0, 0, 217, 0)
(-0.000000000000, 0.000000000000, 0.000000000000, 4.000000000000, 239872, 0, 0, 211, 0)
(-0.301513284445, 0.026466993615, 0.148732990026, 53.000000000000, 289024, 117, 0, 152, 324)
(-0.000000000000, 0.000000000000, 0.000000000000, 4.000000000000, 345856, 0, 0, 211, 0)
(-0.290227115154, 0.027229102328, 0.144350573421, 62.000000000000, 392960, 120, 0, 160, 312)
(-0.288289755583, 0.027932453901, 0.143620163202, 48.000000000000, 435456, 115, 0, 164, 310)]

Binary ==> invalid values
{'version': '0.7', 'fields': ['x', 'y', 'z', 'intensity', 't', 'reflectivity', 'ring', 'ambient', 'range'], 'size': [4, 4, 4, 4, 4, 2, 1, 2, 4], 'type': ['F', 'F', 'F', 'F', 'U', 'U', 'U', 'U', 'U'], 'count': [1, 1, 1, 1, 1, 1, 1, 1, 1], 'width': 2048, 'height': 128, 'viewpoint': [0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0], 'points': 262144, 'data': 'binary'}
[(-0.322375327349, 0.022405456752, 0.156767427921, 1.000000000000, 1116340224, 0, 0, 31232, 3070230528)
( 9007199254740992.000000000000, 0.000000000000, 0.000000000000, 0.000000000000, 2181038080, 40663, 190, 7677, 2870295736)
( 0.000000000000, 0.000000000000, -0.031313404441, 0.000000000000, 10616832, 0, 77, 1, 0)
( 0.000000000000, 0.000000000000, -0.000000000000, -23.072139739990, 483019068, 62, 0, 16256, 1115947008)
( 0.000000000000, 0.000000000000, 0.000000000000, 0.000000000000, 0, 0, 0, 0, 0)
( 0.000000000000, 0.000000000000, 0.000000000000, 0.000000000000, 4194304, 595, 0, 0, 12451840)
( 0.000000000000, 0.000000000000, 0.000000000000, 0.000000000000, 1092157440, 48798, 235, 52953, 457922620)
(-0.000000000000, 33554684.000000000000, 0.000000000000, 0.000000000000, 39168, 19456, 1, 0, 0)
( 0.000000000000, 0.000000000000, -0.319211810827, 0.027101533487, 1042245743, 0, 128, 63, 4363264)
( 2658757058587385950290469587571965952.000000000000, -0.000000000000, 140737488355328.000000000000, 0.000000000000, 0, 0, 0, 0, 0)]

Binary Compressed ==> crash
File "/datasets/utils/src/pypcd/pypcd/pypcd.py", line 755, in from_path
return point_cloud_from_path(fname)
File "/datasets/utils/src/pypcd/pypcd/pypcd.py", line 312, in point_cloud_from_path
pc = point_cloud_from_fileobj(f)
File "/datasets/utils/src/pypcd/pypcd/pypcd.py", line 305, in point_cloud_from_fileobj
return PointCloud(metadata, pc_data)
File "/datasets/utils/src/pypcd/pypcd/pypcd.py", line 690, in init
self.check_sanity()
File "/datasets/utils/src/pypcd/pypcd/pypcd.py", line 703, in check_sanity
assert(self.pc_data.size == self.points)
AssertionError

Any ideas what's going wrong?

Hi @DanielPollithy
I am using your python pypcd package as per your suggestion in the tutorial.

But I am getting this error:

[ERROR] [1655964681.574969]: bad callback: <bound method Subscriber.callback of <message_filters.Subscriber object at 0x7f16ddc51760>>
Traceback (most recent call last):
  File "/opt/ros/noetic/lib/python3/dist-packages/rospy/topics.py", line 750, in _invoke_callback
    cb(msg)
  File "/opt/ros/noetic/lib/python3/dist-packages/message_filters/__init__.py", line 76, in callback
    self.signalMessage(msg)
  File "/opt/ros/noetic/lib/python3/dist-packages/message_filters/__init__.py", line 58, in signalMessage
    cb(*(msg + args))
  File "/opt/ros/noetic/lib/python3/dist-packages/message_filters/__init__.py", line 330, in add
    self.signalMessage(*msgs)
  File "/opt/ros/noetic/lib/python3/dist-packages/message_filters/__init__.py", line 58, in signalMessage
    cb(*(msg + args))
  File "/home/sxv1kor/rospy_tutorial_ws/src/message_synchrozier/src/rosBagMessageSynchronizer.py", line 57, in callback
    pc = pypcd.PointCloud.from_msg(pointcloud)
  File "/home/sxv1kor/.local/lib/python3.8/site-packages/pypcd/pypcd.py", line 807, in from_msg
    raise NotImplementedError('ROS sensor_msgs not found')
NotImplementedError: ROS sensor_msgs not found

This should not happen as the numpy_pc2 is alredy there:

 ll
total 84
drwxrwxr-x  3 sxv1kor sxv1kor  4096 Jun 23 10:47 ./
drwx------ 74 sxv1kor sxv1kor  4096 Jun 23 10:47 ../
-rw-rw-r--  1 sxv1kor sxv1kor    43 Jun 23 10:47 __init__.py
-rw-rw-r--  1 sxv1kor sxv1kor 10113 Jun 23 10:47 nea_pc_format.py
-rw-rw-r--  1 sxv1kor sxv1kor 13802 Jun 23 10:47 numpy_pc2.py
-rw-rw-r--  1 sxv1kor sxv1kor  1133 Jun 23 10:47 pdutil.py
drwxrwxr-x  2 sxv1kor sxv1kor  4096 Jun 23 10:47 __pycache__/
-rw-rw-r--  1 sxv1kor sxv1kor 28920 Jun 23 10:47 pypcd.py
-rw-rw-r--  1 sxv1kor sxv1kor  2506 Jun 23 10:47 sautil.py

When I try to do:

pip install git+https://github.com/DanielPollithy/pypcd@d77a6561dbe64bb61b5e5c941364116d00877783

I get:

Collecting git+https://github.com/DanielPollithy/pypcd@d77a6561dbe64bb61b5e5c941364116d00877783
  Cloning https://github.com/DanielPollithy/pypcd (to revision d77a6561dbe64bb61b5e5c941364116d00877783) to /tmp/pip-req-build-729s050a
ERROR: python_lzf-0.2.4-cp36-cp36m-linux_x86_64.whl is not a supported wheel on this platform.

Using the last commit before the prebuilt wheel was hardcoded as a dependency (7732b2c) works for me.

Am I doing something wrong?

Hi,

I'm currently trying to write a pcd file like this:

def make_xyz_label_layers_point_cloud(points, labels, layers):
    assert(len(points) == len(labels) and len(points) == len(layers))
    md = {'version': .7,
          'fields': ['x', 'y', 'z', 'label', 'layers'],
          'type': ['F', 'F', 'F', 'I', 'F'],
          'size': [4, 4, 4, 4, 4],
          'count': [1, 1, 1, 1, len(layers[0])],
          'width': len(points),
          'height': 1,
          'viewpoint': [0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0],
          'points': len(points),
          'data': 'ASCII'}

    dt = np.dtype([('x', np.float32), ('y', np.float32), ('z', np.float32),
                   ('label', np.int32), ('layers', np.float32, (len(layers[0]),))]) # len(layers[0]) == 20
    pc_data = np.rec.fromarrays([points[:, 0], points[:, 1], points[:, 2],
                                 labels, layers], dtype=dt)

    pc = pypcd.PointCloud(md, pc_data)
    return pc

Writing this to ascii I get an error as the package tries to write it as 24 consecutive values but gets 4 consecutive values and an array with 20 values.

Do you happen to know if the format of wanting to write 24 consecutive values is correct (I think so)?

Am I using this right? If so would you be interested in a pull request fixing the saving problem?