New library: stlite, a port of Streamlit to Wasm, powered by Pyodide

This is genuinely awesome! Well done!

1 Like

Many thanks for the great work. I tried to open a 鈥渓ocal鈥 csv file using Pandas and it could not find it. Is there some specific protocol I need to follow? Here is the code:

import io
import streamlit as st
import numpy as np
import pyodide.http
import pandas as pd
import plotly.express as px

df = pd.read_csv('./sg-words.csv')

st.table(df)

, and the csf file is in the same folder as index.html.

Agian, thanks for the work.

That is not supported now.

In stlite, the Streamlit server is running inside the browser environment from which it cannot access the local file system. The browser runtime is sandboxed.
It is a reasonable restriction for the security sake.

When you use the file accessing methods such as open() on stlite, it accesses another file system, Emscripten File System, which acts like a normal file system but is a virtual one isolated from the file system on your host environment.
Imagine something like a docker container, which has its own directory structure and it is isolated from the host system and cannot be accessed from the host and vice versa.

FYI,

is a related topic, though it鈥檚 not a solution.


Thank you for trying it out! :slight_smile:

Oh that鈥檚 an interesting one. Would notably unlock multi-page apps in stlite by scanning pages/ directory!

@aghasemi I think you鈥檙e left with at least these two workarounds:

  1. Host the .csv somewhere and use its URL in pd.read_csv("https://whatever.url/to/file.csv"). Kinda cumbersome but you probably want to memoize this using st.experimental_memo to avoid loading the file after every rerun!
  2. Use a st.file_uploader() to let the app visitor input the .csv
1 Like

Thanks. The first option doesn鈥檛 work either :rofl: That was indeed my next question to Yuichiro.
I haven鈥檛 tried pyodide.http yet. Hope that one works.

@arnaud Thank you!

multi-page apps

This may be included in Multiple scripts 路 Issue #4 路 whitphx/stlite 路 GitHub (and maybe some additional adjustments specifically for mutli-page apps) :+1:
Technically, the current version of stlite writes the Python script specified via mainScriptData option into the Emscripten File System so that the Streamlit server can read it as if it is a local file. So I plan to simply extend this mechanism to transmit multiple files :slight_smile:


@aghasemi

You are correct.
pd.read_csv() does not work when an external URL is given because it tries to establish HTTP connection to the external server but it is not permitted in the Pyodide environment. See Roadmap 鈥 Version 0.24.1 for the details.

So pyodide.httpis the solution although I鈥檓 not sure if it works well in a Streamlit script, which I think does not support async code well. If so, this ticket might help in the future.

Or wait for Multiple scripts 路 Issue #4 路 whitphx/stlite 路 GitHub as described above.


FYI,
https://pyodide.org/en/stable/usage/faq.html#how-can-i-load-external-files-in-pyodide
is also a related topic although this cannot be used within the current stlite API.

Custom components (experimental) support finally came out! I think one of the biggest advantage of Streamlit is the custom components maintained by the community, so they must be available on stlite too :slight_smile:

Along with it, I added the 鈥渞equirements鈥 button on the playground app so that users can install additional packages.

Now the initial sample code of the playground app has been updated where a custom component, hi-plot is installed and used for demonstration (and Matplotlib too)!

Please note that this is still experimental and not all the custom components are working, for example because

  • The custom component package has C-extension. Such non-pure Python packages must be specially built for Wasm/Pyodide runtime. This is not stlite鈥檚 but Pyodide/Wasm restriction. In this case, installing the package fails (see the devtool for now).
    • For example, stmol, streamlit-echarts, or streamlit-webrtc cannot be installed due to this reason.
  • The custom component frontend loads local resources dynamically after initialization.
    • For example, streamlit-ace can run and it works, but its syntax highlight does not, probably, though I haven鈥檛 investigated this issue so much.
1 Like

I remember that I had created a Wasm-based real-time video processing custom component, streamlit-fesion in the past as a prototype,
so I tried it on stlite as now the custom component can be installed:

demo
It amazingly worked without any modifications!
It鈥檚 completely running on the web browser with a local camera input.

What I did was just

  • On the playground app,
  • Install streamlit-fesion package,
  • Then copy and paste this app.py to the code pane.

Disclaimer:
streamlit-fesion was just a kind of PoC package that I made in a Streamlit hackathon and is not well maintained.
There are many things to improve, for example, its image processing blocks the main thread which leads to bad UX.

I feel that it鈥檚 time to come back to this library, as a replacement of streamlit-webrtc for stlite :slight_smile: .

I can confirm that calling HTTP works with js.XMLHTTPRequest, but not with pyodide.http.pyfetch, due to being async.

By the way, do you have any idea why I cannot see js.document or JavaScript window object from within stlite?

do you have any idea why I cannot see js.document or JavaScript window object from within stlite?

I think it鈥檚 because in the case of stlite, Pyodide is running on a WebWorker which is separated from the main thread and has a different global context where, for example, window is not available.

1 Like

Oh I see. Thanks. Is there no way to get it to work then?

If not, getting streamlit.components.v1, to work will unlock many of the potential benefits, and ability to run a basic, only-html component such as the one in Code snippet: create Components without any frontend tooling (no React, Babel, Webpack, etc) will do the rest.

But I have a feeling that accessing the underlying DOM and communicating objects with JS is a more native approach here. No?

Shuny by the way has apparently this tags UI elements that can run custom JS code (see their Wordle example). So it might be technically doable.

Thanks,

getting streamlit.components.v1 , to work will unlock many of the potential benefits

streamlit.components.v1 is already supported (with some exceptions).
Supporting custom components is meaning it.

The technique introduced in that topic is currently not possible on stlite, but it鈥檚 not due to the lack of streamlit.components.v1, but of the way to mount multiple files.
If you want to do something like that, please wait for Multiple scripts 路 Issue #4 路 whitphx/stlite 路 GitHub.

But I have a feeling that accessing the underlying DOM and communicating objects with JS is a more native approach here. No?

No, for this case [1].

As I have written, Pyodide is running on a web worker, so it鈥檚 not possible anyway.

And even if possible, Streamlit of course does not have such Pyodide-native APIs, so we should not use such APIs on stlite apps too.
stlite is a 鈥渢ransparent鈥 layer that bridges Streamlit to Wasm/Pyodide, so I don鈥檛 have a plan to provide its original API including the way to manipulate DOM that will probably conflict or break the original Streamlit design.

For example, especially about this topic, directly manipulating DOM breaks the Streamlit frontend because it is well-managed by its React app and must not be controlled from outside.
As Streamlit already provides a way for devs to do it safely, which is called as 鈥渃ustom components鈥, where the third-party code is sandboxed in an iframe and guaranteed not to break the other parts of the frontend, so we should follow that way.

Shuny by the way has apparently this tags UI elements that can run custom JS code (see their Wordle example). So it might be technically doable.

From my understanding, Shiny鈥檚 ability to inject JS code into the frontend is not such a 鈥渘ative鈥 API.
Shiny being possible to inject frontend code is at the same level to Streamlit being able to load any code into the frontend as a custom component.
Of course the Python versions of both do not have the 鈥渘ative鈥 way to control the frontend JS from the Python code, and both have some APIs to 鈥渂ridge鈥 them which are tags$script for Shiny and custom components for Streamlit.
The situation does not change even when they are ported to Pyodide environment; stlite for Streamlit and Shinylive for Shiny.
Both stlite and Shinylive simply run Streamlit or Shiny on the web worker with no (or little) modifications and try to keep the original APIs available. Then, they still do not provide the Pyodide-native way to control DOM from Python.


  1. Maybe, PyScript have a similar philosophy. From my understanding, PyScript is a wrapper of the Pyodide DOM APIs (with some utils). 鈫╋笌

I understand your point and it is of course a very strong defence in favour of staying with custom components.

On the other hand, however, I can argue that, Isn鈥檛 avoiding such a big work of providing seamless interoperability between JS and Python, where you can even run JS methods as Python functions and just have the output without even noticing it鈥檚 a different language, a waste of resources? Custom components can of cour still be there and be preferred as the 鈥渃ross-platform鈥 solution.

Take this code for example: Isn鈥檛 it nice? :smiley: Why should we need two (or at least one) files in two/three programming languages, just to know the width of the screen?

P.S. This Shiny issue is also interesting. Apparently, you can switch between webworker and main thread with a URL parameter. Is it difficult for STLite to provide the same config parameter?

Can you create a new GitHub issue for this topic separately and move the discussion there?

1 Like

Hi, I started versioned releases and publishing an NPM package, so now the script can be loaded from the jsDelivr CDN with a version-pinned URL as below.
So I have removed the warning from the README saying stlite might be broken or the API might change without any notice. It never happens now as long as we use the pinned URL.

<!DOCTYPE html>
<html>
<head>
  <meta charset="UTF-8">
  <meta http-equiv="X-UA-Compatible" content="IE=edge">
  <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
  <title>stlite app</title>
</head>
<body>
  <div id="root"></div>
  <script src="https://cdn.jsdelivr.net/npm/@stlite/mountable@0.1.0/build/stlite.js"></script>
  <!-- 馃憜馃憜馃憜 New URL! 馃憜馃憜馃憜 -->
  <script>
    stlite.mount(`
import streamlit as st

name = st.text_input('Your name')
st.write("Hello,", name or "world")
`,
    document.getElementById("root"))
  </script>
</body>
</html>

The URL starting with https://whitphx.github.io/stlite/ that has been used in some old examples is now recommended not to use.

Please check out the updated README :smiley:

(BTW, I created a logo 鈫 :v:)

1 Like

Multipage apps (MPA) are supported!

The playground app has been updated to be an MPA too.

You can mount multiple files including pages/*.py as below.

stlite.mount(
  {
    entrypoint: "馃憢_Hello.py",
    files: {
      "馃憢_Hello.py": `
import streamlit as st

st.set_page_config(page_title="Hello")
st.title("Main page")
`,
      "pages/1_猸愶笍_Page1.py": `
import streamlit as st

st.set_page_config(page_title="Page1")
st.title("Page 1")
`,
      "pages/2_馃巿_Page2.py": `
import streamlit as st

st.set_page_config(page_title="Page2")
st.title("Page 2")
`,
    },
  },
  document.getElementById("root")
);

I am developing the online code editor for stlite.

editor screenshot

I created the dedicated post for it :slight_smile:

1 Like

I created a desktop app toolkit for stlite.
Here is the thread about it :slight_smile:


A sample app repository
Distributable files (the macOS app file is not signed, so the security alert may be shown)

1 Like

This is a great tool! Do you have a list of packages that are compatible with stlite? I鈥檝e tried a few like millify, sklearn and pdfkit but they didn鈥檛 work.

Thank you!

In short, please see Packages built in Pyodide 鈥 Version 0.21.3


The installable packages are

  1. Pure Python packages (packages that do not include C code).
  2. C extensions compiled for the Pyodide runtime.

The linked page is the list for 2.

There is no complete list for 1.
You can only check the availability of each package by checking if the distributed package file name ends with 鈥減y3-none-any.whl鈥.
For example, semver package is installable because its file name is semver-2.13.0-py2.py3-none-any.whl as found at the PyPI package that ends with 鈥減y3-none-any.whl鈥.