Wss/https terribly slow!

[Jacob_Lukose]

Hello Community!

I recently built a streamlit app and deployed it using Nginx in the front for handling SSL.

From my observations, its very quick to establish a connection when running over http. But when trying to access the site using https + wss, its taking several seconds ( 10s +) sometimes to setup the intital websocket connection. It is working, but its very slow to establish the websocket connection over https.

Does anyone have any pointers or learnings on this ?

Thankyou! in advance.

[Jacob_Lukose]

Bumping this up ^^

[Jacob_Lukose]

@andfanilo @thiago @randyzwitch sorry to tag you folks in here. But I would need some help in understanding the importance of this hard coded timeout value streamlit/WebsocketConnection.tsx at 96f17464a13969ecdfe31043ab8e875718eb0d10 · streamlit/streamlit · GitHub.

[Jacob_Lukose]

Attaching screenshot of the same

[thiago]

Hi @Jacob_Lukose

But when trying to access the site using https + wss, its taking several seconds ( 10s +) sometimes to setup the intital websocket connection.

That’s very odd. Can you provide a little more detail about your setup for serving HTTPS/WSS?

Also: can you repro the bug when running locally? (with nginx)

But I would need some help in understanding the importance of this hard coded timeout value

That’s used in Streamlit’s web client’s state machine, which is used to manage the websocket connection.

If you want to get into the weeds, the state machine works like this:

    INITIAL
      │
      │               on conn succeed
      v               :
    CONNECTING ───────────────> CONNECTED
      │  ^                          │
      │  │:on ping succeed          │
      │:on timeout/error/closed     │
      v  │                          │:on error/closed
    PINGING_SERVER <────────────────┘

In words: the client tries to connect to the server, and if there’s a connection error, or if 1s goes by without a connection (due to that timeout you linked), the client goes into a mode where it makes HTTP requests every so often to see if the server is alive (PINGING_SERVER). If an HTTP request succeeds, the client attempts to connect the websocket again.

The reason we do this is because websocket connections tend to take a long time to let you know they have failed. So we use normal HTTP requests to detect if the server is up, and then we use a timeout in the websocket connection to guess that the connection failed.

[Jacob_Lukose]

Thanks for responding back.

Setup:

I have a nginx server at front with below configuration, there are python wrappers used to initialise the configs.

NGINX_WORKER_CONNECTION_COUNT = 8192
events = Section( ‘events’, worker_connections=NGINX_WORKER_CONNECTION_COUNT )
http = Section( ‘http’, access_log=[‘logs/nginx.access.log’, ‘combined’], include=os.path.join(self.env.root_path, ‘conf’, ‘mime.types’), ssl_certificate=os.path.join(self.env.var_path, ‘identity.cert’), ssl_certificate_key=os.path.join(self.env.var_path, ‘identity.key’) )
http = Section(
‘http’,
access_log=[‘logs/nginx.access.log’, ‘combined’],
include=os.path.join(self.env.root_path, ‘conf’, ‘mime.types’),
ssl_certificate=os.path.join(self.env.var_path, ‘cert’),
ssl_certificate_key=os.path.join(self.env.var_path, ‘key’)
)
http.sections.add(
Section(
‘server’,
duplicate_options(‘listen’, ([‘{0} ipv6only=off ssl’.format(scheduler_healthcheck_port)])),
Location(
‘/admin’,
KeyMultiValueOption(‘return’, value=[200, ‘GOOD’])
),
Location(
‘/’,
KeyValueOption(‘proxy_pass’, ‘http://127.0.0.1:{}/’.format(scheduler_health_port))
),
Location(
‘^~ /healthz’,
KeyValueOption(‘proxy_pass’, ‘http://127.0.0.1:{}/healthz’.format(scheduler_health_port))
),
Location(
‘^~ /vendor’,
KeyValueOption(‘proxy_pass’, ‘http://127.0.0.1:{}/vendor’.format(scheduler_health_port))
),
Location(
‘^~ /static’,
KeyValueOption(‘proxy_pass’, ‘http://127.0.0.1:{}/static’.format(scheduler_health_port))
),
Location(
‘/stream’,
KeyValueOption(‘proxy_pass’, ‘http://127.0.0.1:{}/stream’.format(scheduler_health_port)),
KeyValueOption(‘proxy_http_version’, ‘1.1’),
KeyValuesMultilines(‘proxy_set_header’,
values=[[‘Host’, ‘$host’], [‘Upgrade’, ‘$http_upgrade’],
[‘Connection’, ‘upgrade’], [‘X-Forwarded-For’, ‘$proxy_add_x_forwarded_for’]]),
KeyValueOption(‘proxy_read_timeout’, 86400)
)
)
)
return Config(
events,
http,
worker_processes=1,
worker_rlimit_nofile=20248,
error_log=‘logs/nginx_error.log’,
pid=LOGS_NGINX_PID_FILE,
daemon=‘on’
)

So when trying to load the app on HTTP its perfectly fine, over HTTPs I am seeing the issue in connection esthablishment, most cases it connects after several seconds. I cannot reproduce it locally, because when I run it locally I use without https.

[Jacob_Lukose]

@thiago I am facing this only on slow networks using https, is there a way I can override the 1s hard timeout so as to see if it helps.

[thiago]

It’s not possible to override that number, but I built a version of Streamlit for you that has that timeout set to 10s.

Can you try downloading the file above, installing it as shown below, and then see if you can repro the bug?

pip uninstall streamlit
pip install streamlit-0.82.0-py2.py3-none-any.whl

[Jacob_Lukose]

@thiago is there a way this can be published to pypi ? I am not sure the environment I am working on, allows for random installation like this, we have a pipeline which pulls sdist/wheels from pypi and does a bunch of security checks before making it available for consumption.

[thiago]

Can you first try installing from the URL directly?

You can add the URL directly to a requirements.txt file, for example:

https://drive.google.com/file/d/13GLuVTwvnimOMfV-7A6j4Uh2Wz94gblH/view

Otherwise, I’ll have to create a throw-away PyPI project, then do some trial and error iteration loops changing some strings in setup.py to make it work (because I’m pretty sure pip does some filename+string checks), so that would take a little bit of effort. Not particularly hard, but would require some contiguous debugging time, which is hard to find nowadays

[Jacob_Lukose]

I will see if the installation from URL can be done. To give you more context, only when the pipeline pulls in the package from PYPI, it gets listed in the artifactory setup we use. And only when the package gets in artifactory, I can use it in my project build and deploy it to staging/prod to run on https. Locally installed package cannot be deployed and https test cannot be done, because https can only be enabled when I am able to deploy to staging or prod environment.

[Aleexxen]

I would like to check if it will also work for me, could you please give me an access?