Hello.
I’m testing the behaviour of Langchain WebBaseLoader in Streamlit. In the following code (no Streamlit), the webpage from the specified URL is read in 0.15 seconds.
from langchain_community.document_loaders import WebBaseLoader
import time
tic = time.perf_counter()
loader = WebBaseLoader("https://joint-research-centre.ec.europa.eu/welcome-jec-website/reference-regulatory-framework/renewable-energy-recast-2030-red-ii_en")
tac = time.perf_counter()
data = loader.load()
toc = time.perf_counter()
print(data)
print(f"Loader: {tac - tic:0.3f}, load: {toc - tac:0.3f}")
But, when I just initialize the Streamlit app, the loader.load() takes 10 minutes!
import streamlit as st
from langchain_community.document_loaders import WebBaseLoader
import time
tic = time.perf_counter()
loader = WebBaseLoader("https://joint-research-centre.ec.europa.eu/welcome-jec-website/reference-regulatory-framework/renewable-energy-recast-2030-red-ii_en")
tac = time.perf_counter()
data = loader.load()
toc = time.perf_counter()
st.set_page_config(page_title="Test loader.load() & Streamlit", page_icon="spider")
st.title("Here's where st.title goes.")
print(data)
print(f"Loader: {tac - tic:0.1f}, load: {toc - tac:0.1f}")
The app is being hosted in Firefox, iMac, osx 10.15. Does anyone have an idea what might be going on here?
Cheers