Force NumExpr to use a specific number of cores

[thayne.edwards]

Mostly likely everyone that has run a Streamlit app has seen the warning:

numexpr.utils: Note: NumExpr detected XX cores but “NUMEXPR_MAX_THREADS” not set, so enforcing safe limit of 8.

I have 88 cores on my system and would like to use most of them for some applications. For others, I would only like to use a few.

I propose a Streamlit run parameter be added to set the number of cores used.

[randyzwitch]

Hi @thayne.edwards -

This isn’t a Streamlit issue, but rather a NumExpr one (via NumPy via conda if I’m not mistaken). The error message is telling you to set an environment variable NUMEXPR_MAX_THREADS prior to starting Streamlit if you don’t like the default:

https://numexpr.readthedocs.io/projects/NumExpr3/en/latest/user_guide.html#threadpool-configuration

How you set this environment variable depends on the operating system where you are running. If you are running on linux, you could do something like:

(base) rzwitch@FQW652Q602 ~ % export NUMEXPR_MAX_THREADS=64;
(base) rzwitch@FQW652Q602 ~ % echo $NUMEXPR_MAX_THREADS
64
(base) rzwitch@FQW652Q602 ~ % streamlit run app.py

Best,
Randy