How to plot a simple 3D point cloud without any map and update it regularly?

[SM1991CODES]

Summary

I am trying to use streamlit with ROS where I get PointCloud2 messages which I can convert to numpy arrays - Nx4

I want to be able to display these in real-time - something like in RVIZ in ROS.
How may I do that?
Basically, I would like to plot the 3D points, be able to zoom, pan, rotate and also update the points every frame.

Please advise.

Steps to reproduce

Code snippet:

add code here

If applicable, please provide the steps we should take to reproduce the error or specified behavior.

Expected behavior:

Explain what you expect to happen when you run the code above.

Actual behavior:

Explain the undesired behavior or error you see when you run the code above.
If you’re seeing an error message, share the full contents of the error message here.

Debug info

• Streamlit version: (get it with $ streamlit version)
• Python version: (get it with $ python --version)
• Using Conda? PipEnv? PyEnv? Pex?
• OS version:
• Browser version:

Requirements file

Using Conda? PipEnv? PyEnv? Pex? Share the contents of your requirements file here.
Not sure what a requirements file is? Check out this doc and add a requirements file to your app.

Links

• Link to your GitHub repo:
• Link to your deployed app:

Additional information

If needed, add any other context about the problem here.

[dataprofessor]

Hi @SM1991CODES

This seems to be possible, I’d recommend to look into the rospy library in Python to retrieve the data.

Next, use plotly to render the data points from ROS as a 3D scatter plot.

Then, use st.plotly_chart - Streamlit Docs to display the plot in a Streamlit app.

Best regards,
Chanin

[SM1991CODES]

Hi,
Thank you for the reply.
Getting data from ros is not the main problem, I already have code for that.
Problem is, plotly scatter_3d or matplotlib 3d produce very bad 3D visualizations.
I use mayavi generally or rviz if withing ros.

So, is it somehow possible to use mayavi or PyVista or something that produces nice 3D point cloud visualizations?
It’s a case of automotive dataset - point clouds are generally 120000x4

Best Regards
Sambit

[dataprofessor]

Hi @SM1991CODES

@edsaac had created a Streamlit component stpyvista that allows you to use pyvista in a Streamlit app.

Also check out his demo app https://stpyvista.streamlit.app/

Best regards,
Chanin

[SM1991CODES]

Thanks for the link, looks promising. I will check it out.
However, based on my use case, there is another problem - how do I make certain widgets update automatically at intervals?

The idea is - click and button (draw point clouds) and then the Pyvista widget should update with new point clouds from the ROS backend.

I know we can use a for loop or call the st.experimental_rerun(), but I just don’t feel this is a very nice way to update just 1 widget. This will basically re-run the entire script.

Any better solutions?

Best Regards
Sambit

[dataprofessor]

Hi,

@kmcgrady has created a custom component called Streamlit Autorefresh that may possibly be what you’re looking for.

Best regard,
Chanin

[SM1991CODES]

I am aware of this. I have a somewhat similar way of refreshing. But I am looking for something more robust and proper.
Nevertheless, thanks a lot. I will try out the 3D point cloud plot and update in a day or two.

But - you have created a great framework, I really like it and would love to see it grow and be more stable for multiple use cases.
I am fairly new to streamlit and learning it while working on my case. Some nice things I imagine (may be already present):

1. Download button that opens up a certain public directory allowing user to select file to download
2. A modal dialog that allows for user input - usable for password authentication when logging into the app
3. Upload button that allows user to select where to upload a file to on server - easier workflow with binary and image files etc.
4. Coordinate based layout system for better usage of screen space - user get’s to select
5. More stability in multi-page apps - I encountered st.session_state loses parameters when page is changed in multi-page apps unless we do a st.update_session(st.session_state).
6. Push button
7. Status (like a LED symbol etc that autorefreshes)

Best Regards
Sambit

[mathcatsand]

To your point 5, you may be interested in the discussion about changing keyed widget behavior. Session state “losing” parameters is tied to the widget cleanup process and I’m guessing that’s what you came across. Namely, when a widget is not rendered on the screen, it’s data gets deleted (including its value in session state). This is entirely expected, and you mentioned a solution that “interrupts” the cleanup process thereby keeping the keys. (My usual preferred solution is to treat all data assigned to widget keys as transient and have separate keys for keeping data between pages. This also keeps the solution on the page with the affected key and doesn’t require a copy-paste script at the top of every page in the app. )

Keyed widget state persistence - discussion, possible fixes

Hi all! TL;DR: Take a look at the example app and let us know how it impacts you, and what you think about how to fix this issue! Discussion on GitHub here: #6074 One of the current Streamlit behaviors that causes confusion for many of us is how widgets with key='foo' lose their session_state value when they are not rendered in a given run. It seems to come up most often with multipage apps that have some state dependency between pages, but also possible in a single page app with conditional …