This is nice. Interestingly, caching the LLM responses makes a lot of sense because the response wont ever go out of date (due to LLMs having s cut-off date themselves)
Only exception is when you cache the response of agents that use LLMs. In that case we need to be intelligent enough to not cache things that are dynamic (very much like non-llm apps)
Interesting, thanks for sharing. How it handles multi-turn conversational context when a cached output is returned (even for chatbot usecase, chatbot will be sending context like user info, or product he is discussing about etc). ? Also, is there any case studies or validation data are available?
This would be a great improvement for every Agentic / LLM application. I'll have to give Redis 8 a try.
Thanks for this breakdown, Raul!
For sure, once you see the hit rates in action, it’s hard to imagine running without it.
this is a great one Raul. I don’t know how I missed it the other day
Great article but biased as if redis was super intelligent
CHATGPT would surely be using LLM CACHE.
I am pretty sure
Pls challenge this observation
This is nice. Interestingly, caching the LLM responses makes a lot of sense because the response wont ever go out of date (due to LLMs having s cut-off date themselves)
Only exception is when you cache the response of agents that use LLMs. In that case we need to be intelligent enough to not cache things that are dynamic (very much like non-llm apps)
Good one
Interesting, thanks for sharing. How it handles multi-turn conversational context when a cached output is returned (even for chatbot usecase, chatbot will be sending context like user info, or product he is discussing about etc). ? Also, is there any case studies or validation data are available?
Wow...pretty interesting stuff.
Didn't know about these Redis features. Thanks for sharing, Raul!
Great post. Going to see where I can potentially use this.