#StackBounty: #javascript #ros Decoding ROS messages from ROSBrdidge

Bounty: 200

How can I decode rosbridge data in the browser?

So far ive been able to decode the following types:

  • Uncompressed raw RGB
  • Uncompressed raw Depth
  • JPEG compressed RGB

My Problem now is decoding compressed depth and PointCloud2 data. As far as my understanding goes, the data is encoded as base64. The depth image has been compressed to a mono16 PNG. I have tried many different approaches, but none seem to work. The depth image is supposed to contain 307200 depth values of 16 bits each.

I do not want to display this data in something like ros3djs or webviz (cloud not figure out how they do the decoding). I want to decode the data and use it in my own analysis.

Steps to reproduce:

Here is a sample file. It contains the data field of the JSON message: https://drive.google.com/file/d/18ZPpWrH9TKtPBbevfGdceZVpmmkiP4bh/view?usp=sharing

OR

  1. Make sure you have a device publising or a rosbag playing
  2. roslaunch rosbridge_server rosbridge_websocket.launch
  3. Launch your web page and make sure you added roslibjs in your script tag

The JS of my web page is simplified to this:

var ros = new ROSLIB.Ros({
    url: 'ws://127.0.0.1:9090'
});


var depthListener = new ROSLIB.Topic({
    ros: ros,
    name: '/camera/color/image_raw/compressedDepth',
    messageType: 'sensor_msgs/CompressedImage'
});

var pointCloudListener = new ROSLIB.Topic({
    ros: ros,
    name: '/camera/depth/color/points',
    messageType: 'sensor_msgs/PointCloud2'
});



depthListener.subscribe(function (message) {
    console.log(message);
    depthListener.unsubscribe();
});

pointCloudListener.subscribe(function (message) {
    console.log(message);
    pointCloudListener.unsubscribe();
});

I have set the two topics to unsubscribe after the first message so that my console does net get flooded.

Provided a screenshot of the console logs for depth image

Screenshot of depth image output

and for pointcloud

screenshot of pointcloud console log

This is what I have so far, but the onload function is never triggered.

image = new Image();
    image.src = "data:image/png;base64, " + message.data

    image.onload = function(){
        image.decode().then(() =>{
            if(image.width != 0 && image.height != 0){
                canvas.width = image.width;
                canvas.height = image.height;
                ctx = canvas.getContext('2d');
                ctx.drawImage(image, 0, 0);
                image_data = canvas.getContext('2d').getImageData(0,0, 640,480).data;
            }
        });
    }

I think this opencv code was used to compress the image.
The message can essentially be thought of as a 16 bit gray scale image.

Please comment if I can update the question with specific information

Thanks in adance


Get this bounty!!!

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.