Hi everyone! I’m working on deploying a demo app for my research lab’s fine-tuned models. Essentially, it’s like a domain specific collection of agents which have access to external tools; it’d also be using python interpreters, a VectorStore, access to the web, arxiv search, etc.
I’m trying to understand how to build this. Please bear with me, this is the first time I’m working on a streamlit app big enough for hosting.
So, when I’m coming to hosting, I’m thinking there are 2 options for me:
- Host the entire app on AWS EC2 Instance, and have everything run on it with the IP address public.
- Host the streamlit app on the community cloud, and make a FastAPI based backend which runs on an AWS server (where the vectorstore will also be) which gets all the requests and puts it into task queues to ensure responses for multiple users.
- Host the streamlit app on the community cloud, have the vector store run on AWS, such that the app itself just has it’s own vector store client and other tools.
My questions mainly are:
- Which is the better option?
- Is there actually need for a REST API here?
Could you also help me pitch this to my lab mentor? I think he’s expecting all the data ingestion and everything to happen on the community cloud itself, so I’m confused on how to explain it to him.