Improvement of AI Assitent Integration
1 vote
Hello my fellow RDM Users and Adminstrators :)
I came across the AI Assistent integration with some problems. It is quite not flexible at all when it comes to Local LLM Hosting. And by that i dont mean a localhost Ollama. I mean bigger appliances for companies. We are hosting an Ollama behind a OpenwebUI that uses API Tokens but they cant be used in the "Ollama" configuration context. When i try to bind it via the OpenAI config context there are enforced models, so i also cant use them here. It would be great to get an universal useable configuration context for more complex Local AI Hosting systems.
Thank you in advance
All Comments (1)
Hi @tobiasroeder,
I'm Maxime, AI Software Architect at Devolutions.
I had a similar request for another software that proxied OpenAI endpoints. OpenWebUI appears to do something similar with Ollama models.
I think the existing code would support it. The only missing piece in the RDM and AI Assistant configuration seems to be the ability to pass the Authorization header, which is required for OpenWebUI’s Ollama API proxy support: https://docs.openwebui.com/reference/api-endpoints#-ollama-api-proxy-support. I don’t currently support this for any of the AI Assistant providers in RDM.
I have one or two ideas in mind. One of them is a custom provider that could have a custom name, header configuration, such as Authorization in your case, a URL or endpoint, and the provider it proxies, such as Ollama, Anthropic, OpenAI, etc. This way, I could support any product that proxies providers.
I’ll create a feature request on my end and keep you posted.
Best regards,
Maxime Forest