Ask Questions, Get Answers: An approach to conversational data analysis using Streamlit

[NickPacione]

https://youtu.be/msGshEEzUek

[VIDEO FOR YOUTUBE DEMO]

This streamlit application was built by myself and my colleague and it allows you to connect to your database and use natural language querying to retrieve data from it.

The general workflow for using the application can be seen in the video and is as follows:

1. Insert your database connection details in the panel on the left to connect.
2. You can view the database schema to see available tables and columns.
3. Create individual agents to separate your workflow, enter your question in plain english and the application will translate it to SQL and retrieve the relevant information for you.
4. You can also use the visualize tab to interact with your data in a tableau-like application, with many customizable features.

This application has been tested with many different postgres databases, with various schemas, and performed well, even with much more complex database structures.

Unfortunately, I am not providing a hosted application to avoid a large OpenAI usage bill. I would encourage anyone that would like to test the application with a database of their own (or our provided database) to visit our project repo:

GitHub - Convergent-Software/nlq_database: Natural Language Querying...

All feedback is welcome, hope you enjoy!

[Samad_Ahmed]

Had fun building this! If anyone has questions, feel free to ask us.

[IrfanRocks24]

Is data sent to the LLM for SQL generation?

[NickPacione]

Hey InfraRocks, that is definitely one approach you can take, however, this can lead to very high token costs per query.

An approach we took, which can be explored in the repo, was to instead send the database schema to the LLM. The, the LLM, depending on your question, will use its own logic on the schema to determine the SQL code that is needed to retrieve the data.

Hope this helps!