TA-Lib Streamlit Deploy Error

Building wheels for collected packages: TA-Lib
  Building wheel for TA-Lib (setup.py): started
  Building wheel for TA-Lib (setup.py): finished with status 'error'
  ERROR: Command errored out with exit status 1:
   command: /usr/local/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-a4r42tsy/ta-lib/setup.py'"'"'; __file__='"'"'/tmp/pip-install-a4r42tsy/ta-lib/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-nzhdy2sb
       cwd: /tmp/pip-install-a4r42tsy/ta-lib/
  Complete output (27 lines):
  /tmp/pip-install-a4r42tsy/ta-lib/setup.py:71: UserWarning: Cannot find ta-lib library, installation may fail.
    warnings.warn('Cannot find ta-lib library, installation may fail.')
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-3.7
  creating build/lib.linux-x86_64-3.7/talib
  copying talib/test_pandas.py -> build/lib.linux-x86_64-3.7/talib
  copying talib/test_stream.py -> build/lib.linux-x86_64-3.7/talib
  copying talib/test_data.py -> build/lib.linux-x86_64-3.7/talib
  copying talib/stream.py -> build/lib.linux-x86_64-3.7/talib
  copying talib/test_func.py -> build/lib.linux-x86_64-3.7/talib
  copying talib/__init__.py -> build/lib.linux-x86_64-3.7/talib
  copying talib/test_abstract.py -> build/lib.linux-x86_64-3.7/talib
  copying talib/abstract.py -> build/lib.linux-x86_64-3.7/talib
  copying talib/deprecated.py -> build/lib.linux-x86_64-3.7/talib
  running build_ext
  building 'talib._ta_lib' extension
  creating build/temp.linux-x86_64-3.7
  creating build/temp.linux-x86_64-3.7/talib
  gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/usr/include -I/usr/local/include -I/opt/include -I/opt/local/include -I/usr/local/lib/python3.7/site-packages/numpy/core/include -I/usr/local/include/python3.7m -c talib/_ta_lib.c -o build/temp.linux-x86_64-3.7/talib/_ta_lib.o
  talib/_ta_lib.c:611:10: fatal error: ta-lib/ta_defs.h: No such file or directory
   #include "ta-lib/ta_defs.h"
            ^~~~~~~~~~~~~~~~~~
  compilation terminated.
  error: command 'gcc' failed with exit status 1
  ----------------------------------------
  ERROR: Failed building wheel for TA-Lib
  Running setup.py clean for TA-Lib
Failed to build TA-Lib

I need help with the error above. Thank you!

Hi @juwimana, welcome to the Streamlit community.

I’ve never used this specific library, but per their troubleshooting, do you have the underlying TA-Lib library installed? It appears to be a C library of some sort.

Yes, I have it installed on my local machine. I only have issues when deploying to share.streamlit.io

If there is a Debian package of the library, you can create a packages.txt file, which will install system dependencies:

If you need to compile the library yourself to install it, we don’t currently have functionality for that, but I believe it’s on our roadmap.

Best,
Randy

I can’t find any Debian package for TA-Lib, probably not currently available.

1 Like

Exactly, there is no debian package. I’ll come up with some sort of “solution” and more info soon!

Hi @randyzwitch and @juwimana

I’d call it a hack instead of a solution, I hope they’ll find a better way to support it soon. i.e. Heroku has a buildpack option which is a solution for this.

So first of all some information:

  • It’s a docker container where we’re not root (they did a nice job, it’s simple and work quite nice)
  • ta-lib doesn’t have a debian package on the official repo
  • Since we’re not root we can’t simply add a repo, or install a package on the system

In the end we need to follow these steps:

  • Add gcc and make to packages.txt
  • Remove ta-lib from requirements.txt
  • Add requests to requirements.txt
  • Change the code:
    Download ta-lib
    Build it and install to your home directory
    Install python package, pointing to home directory
    Add the library to your code, since python is already running and will not get new library dir

NOTE: The first time will take a few minutes, bc it needs to build the lib

Code:

 import requests
 import os
 import sys
# check if the library folder already exists, to avoid building everytime you load the pahe
if not os.path.isdir("/tmp/ta-lib"):

    # Download ta-lib to disk
    with open("/tmp/ta-lib-0.4.0-src.tar.gz", "wb") as file:
        response = requests.get(
            "http://prdownloads.sourceforge.net/ta-lib/ta-lib-0.4.0-src.tar.gz"
        )
        file.write(response.content)
    # get our current dir, to configure it back again. Just house keeping
    default_cwd = os.getcwd()
    os.chdir("/tmp")
    # untar
    os.system("tar -zxvf ta-lib-0.4.0-src.tar.gz")
    os.chdir("/tmp/ta-lib")
    # build
    os.system("./configure --prefix=/home/appuser")
    os.system("make")
    # install
    os.system("make install")
    # install python package
    os.system(
        'pip3 install --global-option=build_ext --global-option="-L/home/appuser/lib/" --global-option="-I/home/appuser/include/" ta-lib'
    )
    # back to the cwd
    os.chdir(default_cwd)
    print(os.getcwd())
    sys.stdout.flush()

# add the library to our current environment
from ctypes import *

lib = CDLL("/home/appuser/lib/libta_lib.so.0")
# import library
import talib

# here goes your code

I hope that helps.

Cheers

4 Likes

Hi @franciscoed,

Thank You so much for coming up with this.

I tried the suggested method and I am getting the following in my log while deploying on share streamlit.

Installing collected packages: numpy, ta-lib
    Running setup.py install for ta-lib: started
    Running setup.py install for ta-lib: finished with status 'done'
Successfully installed numpy-1.21.1 ta-lib-0.4.21
/home/appuser/.local/lib/python3.7/site-packages/pip/_internal/commands/install.py:232: UserWarning: Disabling all use of wheels due to the use of --build-option / --global-option / --install-option.
  cmdoptions.check_install_build_global(options)
WARNING: You are using pip version 21.1.2; however, version 21.1.3 is available.
You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.
/app/stock-app-streamlit
2021-07-24 09:13:36.827 Uncaught app exception
Traceback (most recent call last):
  File "/home/appuser/venv/lib/python3.7/site-packages/streamlit/script_runner.py", line 350, in _run_script
    exec(code, module.__dict__)
  File "/app/stock-app-streamlit/trade.py", line 38, in <module>
    import talib
ModuleNotFoundError: No module named 'talib'

Please kindly guide me on how to overcome this.

Thank You so much!

1 Like

Hi, It works on local docker (debian buster-slim), but on streamlit I can confirm it’s not working anymore.
I’ll take a look on that.

Cheers

@pitanjal

Try this block:

import streamlit as st
import requests
import os
import sys
import subprocess

# check if the library folder already exists, to avoid building everytime you load the pahe
if not os.path.isdir("/tmp/ta-lib"):

    # Download ta-lib to disk
    with open("/tmp/ta-lib-0.4.0-src.tar.gz", "wb") as file:
        response = requests.get(
            "http://prdownloads.sourceforge.net/ta-lib/ta-lib-0.4.0-src.tar.gz"
        )
        file.write(response.content)
    # get our current dir, to configure it back again. Just house keeping
    default_cwd = os.getcwd()
    os.chdir("/tmp")
    # untar
    os.system("tar -zxvf ta-lib-0.4.0-src.tar.gz")
    os.chdir("/tmp/ta-lib")
    os.system("ls -la /app/equity/")
    # build
    os.system("./configure --prefix=/home/appuser")
    os.system("make")
    # install
    os.system("make install")
    # back to the cwd
    os.chdir(default_cwd)
    sys.stdout.flush()

# add the library to our current environment
from ctypes import *

lib = CDLL("/home/appuser/lib/libta_lib.so.0.0.0")
# import library
try:
    import talib
except ImportError:
    subprocess.check_call([sys.executable, "-m", "pip", "install", "--global-option=build_ext", "--global-option=-L/home/appuser/lib/", "--global-option=-I/home/appuser/include/", "ta-lib"])
finally:
    import talib
2 Likes