Any LLM developers with recommendations for testing libraries?

I am looking to put together some automated tests for my LLM RAG chatbot. I would be interested to hear from anyone already doing this.

1 Like

Hi @shawngiese,

Thanks for sharing this question!

Are you looking for tests for the app side or the RAG logic?

1 Like

I eventually will want to look for tests on the app side but for now I am mainly looking for ways to test RAG logic so I can combine that with the human tests : )

1 Like

That’s an interesting idea. I may suggest you look into some of the observability platforms like LangSmith or Nomic that help you evaluate your RAG pipelines.

Thanks tonykip. I liked the looks of giskard RAG Evaluation Toolkit where I can set my own ground truths to match either the role of the LLM or the embeddings. Have not tried it yet but it’s docs were speaking along the lines of what I was thinking.

1 Like

Cool! I haven’t tried them before, so thanks for sharing!