I can’t find any Debian package for TA-Lib, probably not currently available.
Exactly, there is no debian package. I’ll come up with some sort of “solution” and more info soon!
Hi @randyzwitch and @juwimana
I’d call it a hack instead of a solution, I hope they’ll find a better way to support it soon. i.e. Heroku has a buildpack option which is a solution for this.
So first of all some information:
- It’s a docker container where we’re not root (they did a nice job, it’s simple and work quite nice)
- ta-lib doesn’t have a debian package on the official repo
- Since we’re not root we can’t simply add a repo, or install a package on the system
In the end we need to follow these steps:
- Add gcc and make to packages.txt
- Remove ta-lib from requirements.txt
- Add requests to requirements.txt
- Change the code:
Download ta-lib
Build it and install to your home directory
Install python package, pointing to home directory
Add the library to your code, since python is already running and will not get new library dir
NOTE: The first time will take a few minutes, bc it needs to build the lib
Code:
import requests
import os
import sys
# check if the library folder already exists, to avoid building everytime you load the pahe
if not os.path.isdir("/tmp/ta-lib"):
# Download ta-lib to disk
with open("/tmp/ta-lib-0.4.0-src.tar.gz", "wb") as file:
response = requests.get(
"http://prdownloads.sourceforge.net/ta-lib/ta-lib-0.4.0-src.tar.gz"
)
file.write(response.content)
# get our current dir, to configure it back again. Just house keeping
default_cwd = os.getcwd()
os.chdir("/tmp")
# untar
os.system("tar -zxvf ta-lib-0.4.0-src.tar.gz")
os.chdir("/tmp/ta-lib")
# build
os.system("./configure --prefix=/home/appuser")
os.system("make")
# install
os.system("make install")
# install python package
os.system(
'pip3 install --global-option=build_ext --global-option="-L/home/appuser/lib/" --global-option="-I/home/appuser/include/" ta-lib'
)
# back to the cwd
os.chdir(default_cwd)
print(os.getcwd())
sys.stdout.flush()
# add the library to our current environment
from ctypes import *
lib = CDLL("/home/appuser/lib/libta_lib.so.0")
# import library
import talib
# here goes your code
I hope that helps.
Cheers
Hi @franciscoed,
Thank You so much for coming up with this.
I tried the suggested method and I am getting the following in my log while deploying on share streamlit.
Installing collected packages: numpy, ta-lib
Running setup.py install for ta-lib: started
Running setup.py install for ta-lib: finished with status 'done'
Successfully installed numpy-1.21.1 ta-lib-0.4.21
/home/appuser/.local/lib/python3.7/site-packages/pip/_internal/commands/install.py:232: UserWarning: Disabling all use of wheels due to the use of --build-option / --global-option / --install-option.
cmdoptions.check_install_build_global(options)
WARNING: You are using pip version 21.1.2; however, version 21.1.3 is available.
You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.
/app/stock-app-streamlit
2021-07-24 09:13:36.827 Uncaught app exception
Traceback (most recent call last):
File "/home/appuser/venv/lib/python3.7/site-packages/streamlit/script_runner.py", line 350, in _run_script
exec(code, module.__dict__)
File "/app/stock-app-streamlit/trade.py", line 38, in <module>
import talib
ModuleNotFoundError: No module named 'talib'
Please kindly guide me on how to overcome this.
Thank You so much!
Hi, It works on local docker (debian buster-slim), but on streamlit I can confirm it’s not working anymore.
I’ll take a look on that.
Cheers
Try this block:
import streamlit as st
import requests
import os
import sys
import subprocess
# check if the library folder already exists, to avoid building everytime you load the pahe
if not os.path.isdir("/tmp/ta-lib"):
# Download ta-lib to disk
with open("/tmp/ta-lib-0.4.0-src.tar.gz", "wb") as file:
response = requests.get(
"http://prdownloads.sourceforge.net/ta-lib/ta-lib-0.4.0-src.tar.gz"
)
file.write(response.content)
# get our current dir, to configure it back again. Just house keeping
default_cwd = os.getcwd()
os.chdir("/tmp")
# untar
os.system("tar -zxvf ta-lib-0.4.0-src.tar.gz")
os.chdir("/tmp/ta-lib")
os.system("ls -la /app/equity/")
# build
os.system("./configure --prefix=/home/appuser")
os.system("make")
# install
os.system("make install")
# back to the cwd
os.chdir(default_cwd)
sys.stdout.flush()
# add the library to our current environment
from ctypes import *
lib = CDLL("/home/appuser/lib/libta_lib.so.0.0.0")
# import library
try:
import talib
except ImportError:
subprocess.check_call([sys.executable, "-m", "pip", "install", "--global-option=build_ext", "--global-option=-L/home/appuser/lib/", "--global-option=-I/home/appuser/include/", "ta-lib"])
finally:
import talib
Hi @franciscoed, thanks for your grate help on this.
I got the same error on the above line and I has checked the file is under /home/appuser/include/ta-lib by running subprocess.run(['ls', '-al', '/home/appuser/include/ta-lib/'])
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/usr/include -I/usr/local/include -I/opt/include -I/opt/local/include -I/opt/homebrew/include -I/opt/homebrew/opt/ta-lib/include -I/tmp/pip-build-env-uj4o5ojt/normal/lib/python3.10/site-packages/numpy/core/include -I/home/appuser/venv/include -I/usr/local/include/python3.10 -c talib/_ta_lib.c -o build/temp.linux-x86_64-cpython-310/talib/_ta_lib.o
talib/_ta_lib.c:747:10: fatal error: ta-lib/ta_defs.h: No such file or directory
747 | #include “ta-lib/ta_defs.h”
| ^~~~~~~~~~~~~~~~~~
compilation terminated.
error: command ‘/usr/bin/gcc’ failed with exit code 1
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for ta-lib
Building wheel for numpy (pyproject.toml): started
Thanks for help
Yes, you’re right.
It turn’s out we have two issues here:
First one: the global option is not being respected, it looks like it’s an expected behavior on the newer versions
Second one: current TA Lib version (0.4.25) has some issue with deprecated numpy functions
subprocess.check_call([sys.executable, "-m", "pip", "install", "--global-option=build_ext", "--global-option=-L/home/appuser/lib/", "--global-option=-I/home/appuser/include/", "ta-lib"])
So we have to change the include and lib directory and also fix the ta-lib version to 0.4.24.
Follow below the new code, please let me know if that works well for you
import streamlit as st
import requests
import os
import sys
import subprocess
# check if the library folder already exists, to avoid building everytime you load the pahe
if not os.path.isdir("/tmp/ta-lib"):
# Download ta-lib to disk
with open("/tmp/ta-lib-0.4.0-src.tar.gz", "wb") as file:
response = requests.get(
"http://prdownloads.sourceforge.net/ta-lib/ta-lib-0.4.0-src.tar.gz"
)
file.write(response.content)
# get our current dir, to configure it back again. Just house keeping
default_cwd = os.getcwd()
os.chdir("/tmp")
# untar
os.system("tar -zxvf ta-lib-0.4.0-src.tar.gz")
os.chdir("/tmp/ta-lib")
# build
os.system("./configure --prefix=/home/appuser/venv/")
os.system("make")
# install
os.system("mkdir -p /home/appuser/venv/")
os.system("make install")
os.system("ls -la /home/appuser/venv/")
# back to the cwd
os.chdir(default_cwd)
sys.stdout.flush()
# add the library to our current environment
from ctypes import *
lib = CDLL("/home/appuser/venv/lib/libta_lib.so.0.0.0")
# import library
try:
import talib
except ImportError:
subprocess.check_call([sys.executable, "-m", "pip", "install", "--global-option=build_ext", "--global-option=-L/home/appuser/venv/lib/", "--global-option=-I/home/appuser/venv/include/", "ta-lib==0.4.24"])
finally:
import talib
Cheers
Francisco
I am having issues as well with TA-lib. Is it accurate that the code shown is to be added at the top of main Python app? Is that only applicable when running in Streamlit Cloud or should that work for local development as well?
In reference to the code above, do the various paths such as ‘/home/appuser/venv/’ need to be updated based on local config?
Note:
I’ve seen several posts about integration errors with TA-lib. Once a consistent solution is available, it might be helpful to consolidate the various posts so there is a helpful guide located within one thread. Just a thought…
Thanks in advance!
Have you tried the pre-built binary?
$ pip install ta-lib-bin
I’ve stopped using TA-Lib due to similar deployment issues, and since I’m not using the candle patterns in ta-lib
, I started using pandas-ta
instead.
Arvindra
Thanks Arvindra,
I did and I received the following errors:
ERROR: Could not find a version that satisfies the requirement ta-lib-bin (from versions: none)
ERROR: No matching distribution found for ta-lib-bin
I don’t plan to use TA-lib for the long run, so I appreciate your suggestion on pandas-ta.
I imagine others will try TA-lib so I thought it would be worthwhile to see if there could be a relatively easy solution going forward.
There’s also a talib-binary
you could try to pip install.
I’ll have a look, thanks.
I also have this problem. I put a call to the code franciscoed provided right after the streamlit line of my driver script. I’m getting the following error:
Collecting TA-Lib
Downloading TA-Lib-0.4.28.tar.gz (357 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 357.1/357.1 KB 235.2 MB/s eta 0:00:00[2023-08-24 17:44:05.135903]
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Installing backend dependencies: started
Installing backend dependencies: finished with status 'done'
Preparing metadata (pyproject.toml): started
Preparing metadata (pyproject.toml): finished with status 'done'
Building wheels for collected packages: pyngrok, validators, TA-Lib
Building wheel for pyngrok (setup.py): started
Building wheel for pyngrok (setup.py): finished with status 'done'
Created wheel for pyngrok: filename=pyngrok-6.0.0-py3-none-any.whl size=19867 sha256=58cabd5d7d2e2e0db3d05b2219e0ecc715a7b11c246ff8bc19b6c0f9870c7e26
Stored in directory: /tmp/pip-ephem-wheel-cache-ofmemp71/wheels/b6/2f/3d/4957f583fdfd1c359843d2e57981c634692c579788b2dfc088
Building wheel for validators (setup.py): started
Building wheel for validators (setup.py): finished with status 'done'
Created wheel for validators: filename=validators-0.20.0-py3-none-any.whl size=19575 sha256=6555b359c039468ac4f273250b1a7ca703772cb3deea7505ab0a910c5c84a345
Stored in directory: /tmp/pip-ephem-wheel-cache-ofmemp71/wheels/82/35/dc/f88ec71edf2a5596bd72a8fa1b697277e0fcd3cde83048b8bf
Building wheel for TA-Lib (pyproject.toml): started
Building wheel for TA-Lib (pyproject.toml): finished with status 'error'
error: subprocess-exited-with-error
× Building wheel for TA-Lib (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [21 lines of output]
<string>:77: UserWarning: Cannot find ta-lib library, installation may fail.
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-cpython-311
creating build/lib.linux-x86_64-cpython-311/talib
copying talib/__init__.py -> build/lib.linux-x86_64-cpython-311/talib
copying talib/stream.py -> build/lib.linux-x86_64-cpython-311/talib
copying talib/abstract.py -> build/lib.linux-x86_64-cpython-311/talib
copying talib/deprecated.py -> build/lib.linux-x86_64-cpython-311/talib
running build_ext
building 'talib._ta_lib' extension
creating build/temp.linux-x86_64-cpython-311
creating build/temp.linux-x86_64-cpython-311/talib
gcc -pthread -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/usr/include -I/usr/local/include -I/opt/include -I/opt/local/include -I/opt/homebrew/include -I/opt/homebrew/opt/ta-lib/include -I/tmp/pip-build-env-t5e634sm/normal/lib/python3.11/site-packages/numpy/core/include -I/home/adminuser/venv/include -I/usr/local/include/python3.11 -c talib/_ta_lib.c -o build/temp.linux-x86_64-cpython-311/talib/_ta_lib.o
talib/_ta_lib.c:1082:10: fatal error: ta-lib/ta_defs.h: No such file or directory
1082 | #include "ta-lib/ta_defs.h"
| ^~~~~~~~~~~~~~~~~~
compilation terminated.
error: command '/usr/bin/gcc' failed with exit code 1
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for TA-Lib
Blockquote
How do I work around this? Thanks.
Use ta-lib-bin
instead
I have the following at the top of my requirements.txt:
ta-lib-bin
TA-Lib==0.4.26
(I also tried commenting out TA-Lib and even ta-lib-bin too)
It doesn’t matter, I still get:
gcc -pthread -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/usr/include -I/usr/local/include -I/opt/include -I/opt/local/include -I/opt/homebrew/include -I/opt/homebrew/opt/ta-lib/include -I/tmp/pip-build-env-f3t5prjq/normal/lib/python3.11/site-packages/numpy/core/include -I/home/adminuser/venv/include -I/usr/local/include/python3.11 -c talib/_ta_lib.c -o build/temp.linux-x86_64-cpython-311/talib/_ta_lib.o
talib/_ta_lib.c:747:10: fatal error: ta-lib/ta_defs.h: No such file or directory
747 | #include "ta-lib/ta_defs.h"
| ^~~~~~~~~~~~~~~~~~
compilation terminated.
error: command '/usr/bin/gcc' failed with exit code 1
[end of output]
Remove TA-Lib
and leave only ta-lib-bin
And if this does not help, show us your public github repo, otherwise we are poking around in the dark.
I tried what you suggested, and it failed. My repo is private, and I’m not comfortable making it public. I guess I have to give up.