We know that coding AIs can be amazing coders but they hallucinate when coding Streamlit. Here’s an example of a request to code an text_input() callback. It hallucinates that callbacks have a parameter and doesn’t show how to access the new input. Perhaps Streamlit is under represented in training data or were callbacks not originally supported? Results are so much better with other apis such as SQL or Pandas.
I would not expect these kinds of hallucinations if trained on API Reference manuals, programming language manuals or Swagger files. I wonder if these kinds of documents are given priority for training?
Can Streamlit do something about getting better results?
Here’s the problematic response from Bing, ChatGPT and Perplexity (Code Pilot?).
import streamlit as st
def text_input_callback(value):
st.write("Text input value changed to:", value)
st.write("Enter some text below:")
text_input_value = st.text_input("Text Input", value="", on_change=text_input_callback)
Claude 2 was just released. I reran my tests. Claude 2, Bing and ChatGPT 3.5 still give the same wrong answer. GPT 4 also gave the wrong answer using default model but the correct answer using “Code Interpreter” model.
So it’s best to use GPT4 Code Interpreter but it’s currently limited to 25 prompts every 3 hours.
Coding AIs are not yet good at coding Streamlit because Streamlit requires a deep understanding of user experience and interactivity, which is difficult for AIs to grasp. Additionally, Streamlit uses a number of unique features, such as its declarative syntax and its ability to render rich visualizations, which are not yet well-supported by AI coding tools.
Thanks for stopping by! We use cookies to help us understand how you interact with our website.
By clicking “Accept all”, you consent to our use of cookies. For more information, please see our privacy policy.
Cookie settings
Strictly necessary cookies
These cookies are necessary for the website to function and cannot be switched off. They are usually only set in response to actions made by you which amount to a request for services, such as setting your privacy preferences, logging in or filling in forms.
Performance cookies
These cookies allow us to count visits and traffic sources so we can measure and improve the performance of our site. They help us understand how visitors move around the site and which pages are most frequently visited.
Functional cookies
These cookies are used to record your choices and settings, maintain your preferences over time and recognize you when you return to our website. These cookies help us to personalize our content for you and remember your preferences.
Targeting cookies
These cookies may be deployed to our site by our advertising partners to build a profile of your interest and provide you with content that is relevant to you, including showing you relevant ads on other websites.