Better FastAPI lnagchain with_structured_output streaming (without jitter)

Using Streamlit
,
1

Debug answers:

  1. I am running my app locally
  3. Code at GitHub - iamishalkin/streaming-langchain-structured-output-via-fastapi-on-streamlit: Small example of streaming with_structured_output langchain model via FastAPI on streamlit.
  5. Streamlit version 1.36.0 Python version 3.10.11.

I want to do something like in this BAML example BAML Examples but in streamlit and with FastAPI.
The code in the attached repo works, and it does what I want, but the code looks to me like antipattern. Moreover, the already rendered text jitters a bit, presumably due to rerender.

I also could not use write_stream, because I get correctly formatted json from FastAPI and not just one next word from iterator.

So what is the correct way to display the streaming json from FastAPI?

2

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.