Using PyInstaller (or similar) to create an executable

[Alchemyst-Trader]

Hi everyone,

I got this run app.py from the pynsist github repository, I got it to work with pynsist, but sadly I need it to work with pyinstaller instead. I have no idea what I am doing but I tried combining the run app.py from pynsist and hmadevs method (excluding --clean), and almost got a working exe file. The only problem is when I open the exe it creates web browser on an infinite loop. Based on what I have gathered it the problem is from the subproces. So can anyone help me figure this out?

from subprocess import Popen, PIPE, STDOUT
import sys
import time
import webbrowser

def main():

# Getting path to python executable (full path of deployed python on Windows)
executable = sys.executable

path_to_main = os.path.join(os.path.dirname(__file__), "app.py")

# Running streamlit server in a subprocess and writing to log file
proc = Popen(
    [
        executable,
        "-m",
        "streamlit",
        "run",
        path_to_main,
        # The following option appears to be necessary to correctly start the streamlit server,
        # but it should start without it. More investigations should be carried out.
        "--server.headless=true",
        "--global.developmentMode=false",
    ],
    stdin=PIPE,
    stdout=PIPE,
    stderr=STDOUT,
    text=True,
)
proc.stdin.close()

# Force the opening (does not open automatically) of the browser tab after a brief delay to let
# the streamlit server start.
time.sleep(3)
webbrowser.open("http://localhost:8501")

while True:
    s = proc.stdout.read()
    if not s:
        break
    print(s, end="")

proc.wait()

if name == “main”:
main()

[SimonBiggs]

For some, the following approach may work well:

New library: stlite, a port of Streamlit to Wasm, powered by Pyodide

Hello everyone , I have started a new project, stlite ( GitHub - whitphx/stlite: A port of Streamlit to Wasm, powered by Pyodide. ) which aims to make Streamlit completely run on browsers by using Wasm, with no servers. It’s like JupyterLite for Jupyter, which inspired this project actually so much. To try it out, I created a playground app with a code editor: (~50MB will be downloaded including the Pyodide runtime and some Python packages) Please visit the app and see Streamlit i…

It can make sharing a streamlit application as simple as emailing a html file.

[OneYoungLion]

Hi @SimonBiggs! Do you have any availability for that video call that was discussed a while ago? To use your method for making an executable?

[SimonBiggs]

Yup, for sure. Would both you @OneYoungLion, and @AntonisCSt be able to meet together first, compare notes, detail on here what you have tried, and where you have struggled, as well as formulate some clear questions. Then once you both have helped each other as much as you can, let us three (and any others who are interested) jump on a video call to see how I can help.

Cheers,
Simon

[OneYoungLion]

I believe the question has been discussed enough already. Here, on Github, and in our email correspondence a few months ago. In my opinion, this forum, and this thread, would make a great place for you to share your thoughts and the potential solution with the community. Otherwise, I’ll work around the issue or seek the solution elsewhere. This is for a hobby project and I don’t think I should keep the final step hostage anymore. Thanks.

[SimonBiggs]

Fair enough .

[HHest]

The time goes to unpacking the files, since the exe was created with --onefile.

[SimonBiggs]

Hi @AntonisCSt, @OneYoungLion, and others,

As of last week I have successfully bundled a Streamlit application with PyOxidizer and Electron. I have written up the following draft document that details how I achieved it:

https://docs.pymedphys.com/en/continue-binary/contrib/dive/create-streamlit-exe.html

If you were interested in taking the approach for a spin I’d be keen for your feedback on the document. Also, for any questions about it, it might be best to create a post over on the PyMedPhys discourse forum:

PyMedPhys

Cheers,
Simon

[the-dharma-bum]

Hi @SimonBiggs ,

Thanks for writing and sharing this guide (and also for PyMedPhys, neat tool).
However I feel like your guide is for now indeed a draft: the PyMedPhys project is quite big, and following the guide by looking at the codebase while trying to replicate a mininal working example for a small standalone streamlit app is almost impossible. Would you be willing to provide a smaller, self-contained, example app ? I’m willing to help on such a project.

Cheers,
Luc

[SimonBiggs]

Hi @the-dharma-bum,

Yes, I’d be happy to set up a minimal example, and your help would be much appreciated. I was just about to make a standalone repo for one of my products. I’ll set up the repo, send through the link, and get started.

Once we’ve made the minimal example I’ll continue to build on top of the repo, but we can make a branch/release tag that hangs around and is referenced within the docs for those who want to go back to the beginning and see the minimal example.

What do you think about that?

Cheers,
Simon

[SimonBiggs]

Hi @the-dharma-bum,

So, I have created a “bare bones” example over at:

GitHub - Enopios/Enopios at v0.2.0

Let me know if that meets yours and others needs .

Cheers,
Simon

[the-dharma-bum]

Hi @SimonBiggs,

Wow ! Thank you so much for your quick reply and your work ! It’s exactly what I had in mind.
I think the coolest part of your work is the Electron embedding and the use of Github Actions, allowing for deployment on any OS, so that would be a good thing to add to this minimal example, while still keeping things simple.
I submitted a PR with the code to build the Electron app (which I tested successfully on Debian) which is basically your PyMedPhys code.
I guess there’s only the Github Actions to set up. I think we can continue this discussion on the github repo ?
Anyway, thanks again !

Luc

[SimonBiggs]

@the-dharma-bum

Anyway, thanks again !

My pleasure, I hope it is helpful . Let’s make sure to post a link back on this forum to the final pinned branch that we think will be most useful for everyone .

Cheers,
Simon

[CBrown]

Hey Simon and @the-dharma-bum,

Mind if I join in on the fun? I’m trying to replicate what you have done Simon, but am running into issues. It probably stems from lack of understanding of pyoxidizer, poetry, node, yarn, essentially everything you have used to create your app.

I guess to start from the top, installation:

• pyoxidizer
  • Instructions.
  • Ran the following in CMD prompt:
    pip install pyoxidizer
• poetry
  • Instructions
  • Ran the following in Windows (Powershell):
    (Invoke-WebRequest -Uri https://install.python-poetry.org -UseBasicParsing).Content | py -
• Node
  • Instructions
  • Downloaded and installed Node.js 16.17.0 LTS.
  • Funny Story… This also deleted Python from my computer. I clicked the button which said “Automatically install the necessary tools”, which I guess also installs Python. It tried to install Python 3.10, said “Install Fail” and once it finished doing everything else, I tried to use pip and was told told “No Python”.
• Yarn
  • Instructions
  • Ran the following in CMD prompt:
    • npm install --global yarn
    • npm install -g npm@8.18.0

Once I had everything installed, I began to run down your instructions.

Just as an FYI, I’m not working on github, just my local desktop.

First, it seems like I need to configure my pyoxidizer.bzl config file. I’m looking at the example you provided here, and this is where I start to get lost. Specifically, what do I need to change in here to point to my streamlit application? I’m looking back and forth between what was written for Enopios and the pymedphys .bzl files and am making the best guess and am changing the following to match my .py files:

#python_config.run_module = "pymedphys"
python_config.run_module = "_Introduction_And_Setup"

    #exe = dist.to_python_executable(
        #name = "pymedphys", etc.,

    exe = dist.to_python_executable(
        name = "_Introduction_And_Setup",
        packaging_policy = policy,
        config = python_config,
    )

That’s the only thing I can see to update.

Next, looking at poetry, I’m familiar with virtual environments, but normally use pipenv to manage mine, but I’m willing to give poetry a try. Launching CMD Prompt from the project folder, I used, poetry init and went through the steps to set up the dependencies I use in this project. Looking at the repo posted for Enopios, the readme states I should use the following to create my requirements.txt:
poetry export -f requirements.txt --output requirements.txt

Then I use:

poetry run pyoxidizer build

However, I kept getting the following error:

In --require-hashes mode, all requirements must have their versions pinned with ==. These do not:
setuptools>=18.5 from https://files.pythonhosted.org/packages/d9/5f/2daccd14278b6b780ae6799f85998377c06019354982391245f4b58a927d/setuptools-65.3.0-py3-none-any.whl#sha256=2e24e0bec025f035a2e72cdd1961119f557d78ad331bb00ff82efb2ab8da8e82 (from ipython==8.4.0->-r requirements.txt (line 275))

researching the error led me to adding --without-hashes to the end of the command like so:

poetry export -f requirements.txt --output requirements.txt --without-hashes

and then reran the pyoxidizer command. Which is now erroring out on “exe.add_python_resources(exe.pip_install([”.“]))”. and after working on this some this past weekend and all day today, I’ve ran out of ways to troubleshoot this myself and am now asking for help. Thank you!

[SimonBiggs]

@CBrown

Mind if I join in on the fun? I’m trying to replicate what you have done Simon, but am running into issues.

Most certainly . As a first step in helping you here, would you be able to confirm that the issues you are detailing occur when building off of the minimal example produced over at:

GitHub - Enopios/Enopios at v0.2.0

Cheers ,
Simon

[the-dharma-bum]

Hi @CBrown,

This seems to be an issue with pip itself.
Can you share your Python version, your pip version, your poetry version, and your pyproject.toml file ?

As for the NodeJS issue, I think it would be best to install it through Node Version Manager.

I’m making a guide of the whole process for Linux & Windows that will appear on the Enopios repo within the week.

[CBrown]

None of the issues occur when I use the minimal example build. Is this correct, the total file size is 178mb?

[SimonBiggs]

@CBrown

Is this correct, the total file size is 178mb?

Hehe yup. Other bundler methods (like pyinstaller) attempt to strip out unused components and just deploy that which is utilised. However, this “stripping out” process can often cause weird edge cases that don’t align between normal python and the distributed version. And this… costs a lot of developer time.

…so… here I have made the trade off of developer time being more important than the final distribution size, and, the entire Python distribution, unfettered, is distributed. (Depending on your use case, a different trade-off might be appropriate)

…so… 178mb is just what you get when streamlit installs all of its dependencies + a complete batteries included Python distribution.

[sss163ii]

Hi Simon,
Thanks for sharing.

After I downloaded your barebone example and tried to execute it in powershell, I got an error like that.

Since I haven’t used poetry before and it’s my first time trying to create an exe file. I have no idea what’s going on there and I guess something is missing (in my computer)? Would you have time to give a glance at this pic and help me out?

[SimonBiggs]

Hi @sss163ii,

Welcome to the streamlit community .

I’m not immediately sure what might be causing your issue. I might get a chance to have a look later on this week. Might you be in a position to raise an issue on the GitHub repo?