When I fetch data around 200M from S3 caused connection error:
“The service has encountered an error while checking the health of the Streamlit app: Get “http://localhost:8501/healthz”: read tcp 127.0.0.1:46878->127.0.0.1:8501: read: connection reset by peer”
If I run the app locally with the 200M data fetching from S3, it works.
If I only fetch 50M sized data from S3, the app works fine on the cloud.
Here is my code snippet for fetching data from S3:
““conn =st.connection(“s3”,type = FilesConnection,max_entries= 1)
swiftly_file = conn.read(“miamicoaanalysis/otp/all_months.parquet”, input_format= “parquet”)””
Is there anyway to query data before fetching?