Did anyone in the community tried using Vaex with Streamlit

Guys,

Did anyone try using Vaex. Vaex is pretty fast compared to Pandas. Iโ€™m working on an app with 10 million records currently. This will increase in the future. I will be doing some aggregations and plots on the data to start with. I was wondering if I could somehow display the vaex dataframe. Below is the vaex df type. When I tried displaying the vaex df, it was pretty much displayed as string. Any inputs on the topic are welcome :slight_smile: By the way, jupyter knows how to display a vaex dataframe.

It works in Jupyter, because theyโ€™ve defined a Jupyter representation. You can use vaex right now with Streamlit using the following:

import streamlit as st
import streamlit.components.v1 as components
import vaex

df = vaex.example()

# option 1: html is unstyled
components.html(df._repr_html_())

# option 2: be careful, could transfer lots of data
st.dataframe(df.to_pandas_df())

2 Likes

Thank you @randyzwitch. I somehow wasnโ€™t able to figure out the to_pandas_df. And yes, what you said is true regarding the lots of data. Iโ€™ll be showing a minimum number of rows(mostly value_counts) using this and let the user download the full data if he wants to.