DLL load failed when using Streamlit (with pyqgis)

tambetvali · April 19, 2023, 9:48am

Summary

I’m trying to use pyqgis with Streamlit. For this, initially I have to import * from qgis.core - this fails with dll error. I’m running streamlit with QGIS’s included “python-qgis.bat”, using “-m streamlit run myapp”. Path and Pythonpath seem to be unchanged by streamlit.

Steps to reproduce

Code snippet:

import streamlit as st

from qgis.core import *

Expected behavior:

I excpect qgis core modules to be loaded with from qgis.core import *.

Actual behavior:

I get the following error message:
ImportError: DLL load failed while importing _core: No se encontró el proceso especificado.

Traceback:

File "C:\OSGeo4W\apps\Python39\lib\site-packages\streamlit\runtime\scriptrunner\script_runner.py", line 565, in _run_script
    exec(code, module.__dict__)File "C:\Users\Tambet\eclipse-workspace\qgis test\strl1.py", line 15, in <module>
    from qgis.core import *File "C:\OSGeo4W\apps\qgis\python\qgis\core\__init__.py", line 25, in <module>
    from qgis._core import *

Debug info

Streamlit version: 1.21.0
Python version: 3.9.5
OS version: Windows 10
Browser version: Microsoft Edge versión 112.0.1722.48 (Compilación oficial) (64 bits)

Requirements file

I use the environment included with QGIS advanced network installation, where you have to check to install python and python-tools (and pip to install streamlit). Then, use python-qgis.bat in osgeo4w bin directory, as a default python installation.

Caroline · April 28, 2023, 4:59pm

Hey @tambetvali,

Thanks for sharing this question! Can you also share the full contents of your requirements file so we can try to reproduce this issue?

tambetvali · May 3, 2023, 9:18am

@Caroline, I think there are no more requirements than importing qgis.core.

For background information, you need to install qgis cartographic library by getting it here - Download QGIS and selecting network install. A Quick Guide to Getting Started with PyQGIS on Windows - Spatial Galaxy is the installation guide for Python with qgis, instead of qt4-devel you need to pick qt5-devel. Additionally, you need to select pip to install streamlit library.

Now, when installed, you should have, in your osgeo4w/bin folder, python-qgis.bat, which should be used as python interpreter. You first need to run “python-qgis.bat -m pip install streamlit”, and if it complains some missing exe file, run it twice. Then, “python-qgis.bat -m streamlit run yourcode.py” should run yourcode.py. Try with python-qgis.bat the python interpreter is not complaining about “from qgis.core import *”, which imports all the qgis.core libraries. Now try this within your yourcode.py to be ran with streamlit - if you are as lucky as me, it will complain that some dll files are missing.

In streamlit, I did the following:

st.write(sys.path)

try:
    user_paths = os.environ['PYTHONPATH'].split(os.pathsep)
except KeyError:
    user_paths = []

This shows the PYTHONPATH and PATH variables. I did it in pyqgis environment as well (the one which comes directly from the bat file) and it shows that the paths are identical - in the version, where I could import qgis.core properly and in the version, where I could not. Windows has also os.add_dll_directory command for adding dll directories and I tried this one also to add the most obvious folder - “C:\OSGeo4W\apps\qgis\bin”, which contains the qgis_core.dll file (the one qgis.core should be missing). It does not show properly, which dll file it’s missing, because Windows does not share information about this. So I tried some more folders, but unluckily.

My whole first requirement is to get import qgis.core to work, to get the initial environment. The next requirements do not matter much here - I need to use qgis with streamlit.

My current solution that I call qgis part as external program, but it’s not really a complete solution - it makes code slower and more complicated, and I cannot use qgis directly from Streamlit (but I want to learn this combination).

tambetvali · May 3, 2023, 9:31am

By the way, Caroline, it’s good to see that in the last years, more women are professional programmers - catched up the men finally in this new technology. I think the next best thing to catch up are artificial intelligences, ChatGPT makes me funny feelings about how the programming work could change. But I think it does not go away - in old times, when there were only non-artificial humans in the field, it made a lot to express your mathematical ideas with clear clarity, I think the uttermost where artificial intelligence can turn is that we once again need to write down the requirements with very clear clarity, which is close to programming. TensorFlow seems to be too complicated with deep mathematics requirements, although it’s one of the best, but scikit-learn seems to be like normal programming with some artificial intelligence capabilities. Build A Beautiful Machine Learning Web App With Streamlit And Scikit-learn - Python Engineer (python-engineer.com) is one example, how to install scikit-learn with Streamlit. It allows user to do some artificial intelligence operations, for example to automatically classify data. I think after a few years you need to be able to say you have done something with artificial intelligence.

I saw this, also - Home | Women Who Code