Any LLM developers with recommendations for testing libraries?

Community Cloud
,
1

I am looking to put together some automated tests for my LLM RAG chatbot. I would be interested to hear from anyone already doing this.

1 Like
2

Hi @shawngiese,

Thanks for sharing this question!

Are you looking for tests for the app side or the RAG logic?

1 Like
3

I eventually will want to look for tests on the app side but for now I am mainly looking for ways to test RAG logic so I can combine that with the human tests : )

1 Like
4

That’s an interesting idea. I may suggest you look into some of the observability platforms like LangSmith or Nomic that help you evaluate your RAG pipelines.

5

Thanks tonykip. I liked the looks of giskard RAG Evaluation Toolkit where I can set my own ground truths to match either the role of the LLM or the embeddings. Have not tried it yet but it’s docs were speaking along the lines of what I was thinking.

1 Like
6

Cool! I haven’t tried them before, so thanks for sharing!

7

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.