Websocket connection keeps closing and reopening automatically, deployed on gke gcp

Deployment
,
1

Hello Streamlit Community,
I have deployed 2 apps on Google Kubernetes Engine with Cloud Build.

My problem is that Every 10 seconds or so there is a websocket upgrade request coming through, I can connect to the socket but for some unknown reason the websocket connection is lost after 10 seconds and another request to /_stcore/stream is made. This happens non-stop and causes every variable to be deleted. I want to underline that this happens even if I do not click anywhere, do any reruns or refresh the page, this happens automatically without any kind of user input.

I also have deployed apps on aws without kubernetes and docker and it works fine, the websocket connection is made once and it is stable.

You can find the logs of the pod below, it is clear that the connection breaks every 15 seconds and reconnects for some reason I still don’t know:

2023-04-08 11:02:12.805 Script run finished successfully; removing expired entries from MessageCache (max_age=2)
2023-04-08 11:02:42.542 Runtime state: RuntimeState.ONE_OR_MORE_SESSIONS_CONNECTED -> RuntimeState.NO_SESSIONS_CONNECTED
2023-04-08 11:02:43.050 in-event <InotifyEvent: src_path=b'/app/main.py', wd=1, mask=IN_OPEN, cookie=0, name='main.py'>
2023-04-08 11:02:43.050 Watcher created for /app/main.py
2023-04-08 11:02:43.050 Runtime state: RuntimeState.NO_SESSIONS_CONNECTED -> RuntimeState.ONE_OR_MORE_SESSIONS_CONNECTED
2023-04-08 11:02:43.112 Received the following back message:
rerun_script {
  widget_states {
    widgets {
      id: "$$GENERATED_WIDGET_ID-03ad0d756f891356fcbde6a33ea79fe3-None"
      int_value: 0
    }
    widgets {
      id: "$$GENERATED_WIDGET_ID-7fb2e9aed3162f53d160852436451362-None"
      file_uploader_state_value {
      }
    }
  }
  page_script_hash: "5bc02cefb3ea9e27f1a6776eabd1935d"
}

2023-04-08 11:02:43.113 Beginning script thread
2023-04-08 11:02:43.113 Running script RerunData(query_string='', widget_states=widgets {
  id: "$$GENERATED_WIDGET_ID-03ad0d756f891356fcbde6a33ea79fe3-None"
  int_value: 0
}
widgets {
  id: "$$GENERATED_WIDGET_ID-7fb2e9aed3162f53d160852436451362-None"
  file_uploader_state_value {
  }
}
, page_script_hash='5bc02cefb3ea9e27f1a6776eabd1935d', page_name='')
2023-04-08 11:02:43.113 Disconnecting files for session with ID 1d33344d-ba21-4ed6-97d7-e93f1cf9fe80
2023-04-08 11:02:43.113 Sessions still active: dict_keys([])
2023-04-08 11:02:43.113 Files: 0; Sessions with files: 0
2023-04-08 11:02:43.114 in-event <InotifyEvent: src_path=b'/app/main.py', wd=1, mask=IN_OPEN, cookie=0, name='main.py'>
2023-04-08 11:02:43.148 Removing orphaned files...
2023-04-08 11:02:43.195 in-event <InotifyEvent: src_path=b'/app/.env', wd=1, mask=IN_OPEN, cookie=0, name='.env'>
2023-04-08 11:02:43.309 Script run finished successfully; removing expired entries from MessageCache (max_age=2)
2023-04-08 11:03:13.051 Runtime state: RuntimeState.ONE_OR_MORE_SESSIONS_CONNECTED -> RuntimeState.NO_SESSIONS_CONNECTED
2023-04-08 11:03:13.515 in-event <InotifyEvent: src_path=b'/app/main.py', wd=1, mask=IN_OPEN, cookie=0, name='main.py'>
2023-04-08 11:03:13.516 Watcher created for /app/main.py
2023-04-08 11:03:13.516 Runtime state: RuntimeState.NO_SESSIONS_CONNECTED -> RuntimeState.ONE_OR_MORE_SESSIONS_CONNECTED
2023-04-08 11:03:13.585 Received the following back message:
rerun_script {
  widget_states {
    widgets {
      id: "$$GENERATED_WIDGET_ID-03ad0d756f891356fcbde6a33ea79fe3-None"
      int_value: 0
    }
    widgets {
      id: "$$GENERATED_WIDGET_ID-7fb2e9aed3162f53d160852436451362-None"
      file_uploader_state_value {
      }
    }
  }
  page_script_hash: "5bc02cefb3ea9e27f1a6776eabd1935d"
}

2023-04-08 11:03:13.585 Beginning script thread
2023-04-08 11:03:13.585 Running script RerunData(query_string='', widget_states=widgets {
  id: "$$GENERATED_WIDGET_ID-03ad0d756f891356fcbde6a33ea79fe3-None"
  int_value: 0
}
widgets {
  id: "$$GENERATED_WIDGET_ID-7fb2e9aed3162f53d160852436451362-None"
  file_uploader_state_value {
  }
}
, page_script_hash='5bc02cefb3ea9e27f1a6776eabd1935d', page_name='')
2023-04-08 11:03:13.585 Disconnecting files for session with ID 1d33344d-ba21-4ed6-97d7-e93f1cf9fe80
2023-04-08 11:03:13.585 Sessions still active: dict_keys([])
2023-04-08 11:03:13.585 Files: 0; Sessions with files: 0
2023-04-08 11:03:13.586 in-event <InotifyEvent: src_path=b'/app/main.py', wd=1, mask=IN_OPEN, cookie=0, name='main.py'>
2023-04-08 11:03:13.620 Removing orphaned files...
2023-04-08 11:03:13.669 in-event <InotifyEvent: src_path=b'/app/.env', wd=1, mask=IN_OPEN, cookie=0, name='.env'>
2023-04-08 11:03:13.784 Script run finished successfully; removing expired entries from MessageCache (max_age=2)
2023-04-08 11:03:43.517 Runtime state: RuntimeState.ONE_OR_MORE_SESSIONS_CONNECTED -> RuntimeState.NO_SESSIONS_CONNECTED
2023-04-08 11:03:44.005 in-event <InotifyEvent: src_path=b'/app/main.py', wd=1, mask=IN_OPEN, cookie=0, name='main.py'>
2023-04-08 11:03:44.005 Watcher created for /app/main.py
2023-04-08 11:03:44.005 Runtime state: RuntimeState.NO_SESSIONS_CONNECTED -> RuntimeState.ONE_OR_MORE_SESSIONS_CONNECTED
2023-04-08 11:03:44.067 Received the following back message:
rerun_script {
  widget_states {
    widgets {
      id: "$$GENERATED_WIDGET_ID-03ad0d756f891356fcbde6a33ea79fe3-None"
      int_value: 0
    }
    widgets {
      id: "$$GENERATED_WIDGET_ID-7fb2e9aed3162f53d160852436451362-None"
      file_uploader_state_value {
      }
    }
  }
  page_script_hash: "5bc02cefb3ea9e27f1a6776eabd1935d"
}

2023-04-08 11:03:44.068 Beginning script thread
2023-04-08 11:03:44.068 Running script RerunData(query_string='', widget_states=widgets {
  id: "$$GENERATED_WIDGET_ID-03ad0d756f891356fcbde6a33ea79fe3-None"
  int_value: 0
}
widgets {
  id: "$$GENERATED_WIDGET_ID-7fb2e9aed3162f53d160852436451362-None"
  file_uploader_state_value {
  }
}
, page_script_hash='5bc02cefb3ea9e27f1a6776eabd1935d', page_name='')
2023-04-08 11:03:44.068 Disconnecting files for session with ID 1d33344d-ba21-4ed6-97d7-e93f1cf9fe80
2023-04-08 11:03:44.068 Sessions still active: dict_keys([])
2023-04-08 11:03:44.068 Files: 0; Sessions with files: 0
2023-04-08 11:03:44.069 in-event <InotifyEvent: src_path=b'/app/main.py', wd=1, mask=IN_OPEN, cookie=0, name='main.py'>
2023-04-08 11:03:44.102 Removing orphaned files...
2023-04-08 11:03:44.155 in-event <InotifyEvent: src_path=b'/app/.env', wd=1, mask=IN_OPEN, cookie=0, name='.env'>
2023-04-08 11:03:44.269 Script run finished successfully; removing expired entries from MessageCache (max_age=2)
2023-04-08 11:04:14.006 Runtime state: RuntimeState.ONE_OR_MORE_SESSIONS_CONNECTED -> RuntimeState.NO_SESSIONS_CONNECTED
2023-04-08 11:04:14.500 in-event <InotifyEvent: src_path=b'/app/main.py', wd=1, mask=IN_OPEN, cookie=0, name='main.py'>

These are the automatically repeating logs, without any user interaction whatsoever.

2

UPDATE

I have solved the problem, turns out it was a timeout issue of my services, simply setting the timeout on my services to a desired number fixed the issue.

1 Like
3

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.

4

Thank you for sharing the solution! :heart: