Running Your Own LLM UI

This is a review of Open WebUI, an extensible and user-friendly self-hosted WebUI for LLMs.

I recently decided to run my own UI layer for LLMs, as I have some exciting ideas. For those who are not familiar with chatbots, their architecture basically looks like this:

While I’m pretty good at customizing the middleware, I’m not as skilled at customizing the UI layer.

I tried writing my own UI at first, but I quickly gave up when I realized it was beyond my skills as a frontend developer. The next thing I did was look into open-source solutions. There were a lot of choices, but I narrowed them down to these three:

  • Hugging Face Chat UI - Well-known brand, simple interface, and support for multiple middlewares and backends. Many authentication options.
  • Lobe Chat - Very pretty. Feature-rich. You can try out its features through their live demo site.
  • Open WebUI - Simple interface, and support for multiple middlewares and backends. Supports file attachments.

Hugging Face Chat UI

I was mostly pleased with Hugging Face Chat UI. I ran into errors when installing the dependencies. The very verbose error log was not helpful in diagnosing the cause. After a couple of hours of trying different solutions, I finally got past the problem by upgrading to the latest version of Node.

Things improved after getting through the installation. It was easy to connect both OpenAI and Gemini to the UI, which was a big plus. I enabled authentication through OAuth2 Proxy, which was tedious but not hard. The major flaw with Chat UI was that it didn’t support file uploads. NOTE: As I am writing this review, I noticed their documentation says file upload is supported, but there is also an open issue asking for the file upload feature. I assume it’s a work in progress.

Lobe Chat

I tried Lobe Chat next. It is very easy to try out their UI as long as you are willing to give them your OpenAPI access token.

BE SMART: CREATE A NEW TOKEN FOR THEM AND DELETE IT AFTERWARD.

Lobe Chat is very pretty. That’s all the nice things I can say about it.

The installation guide only described how to run Lobe using Docker. This wasn’t ideal because my goal was to customize code. On the other hand, it does mean there is less room for me (the user) to make mistakes.

Lobe Chat supports two authentication modes: Clerk and Next Auth. Clerk appears to be an authentication-as-a-service provider, which is really nifty, and I was glad to learn about it. The problem is, Clerk-based authentication didn’t work inside Docker. They tell users about this in their GitHub issues. Let’s think about this: Lobe’s installation guide only supported running Lobe via Docker and they were aware that their solution did not work through Docker. They did not take security seriously!

Lobe’s second form of authentication is Next Auth. Next Auth is a JavaScript framework that integrates various OAuth providers, including Auth0. It didn’t work. I followed the instructions for configuring Lobe to use Auth0 and received an error message when I tried to log in. Maybe it will work better with a different provider? I honestly didn’t know. At this point, I was done trying to get authentication to work.

Software security is kind of a big deal, and it’s a red flag that it doesn’t work out-of-the-box.

Open WebUI

I was very happy with Open WebUI. Setting it up was straightforward. I followed the self-hosting instructions and was able to run the code in a few minutes. Setting up authentication was easy:

  1. I already had a project in Google Cloud Platform (GCP).
  2. I went into my project in GCP, created new credentials for OAuth authentication, and configured the credentials to expect connections from my server.
  3. Using the new credentials, I defined ENABLE_OAUTH_SIGNUP, GOOGLE_CLIENT_ID, and GOOGLE_CLIENT_SECRET in Open WebUI’s start.sh file.
  4. I restarted the server, and as the first logged-in user, I was granted the administrator role.
  5. I added additional users by having them log in via Google Sign-In, then approved them as the administrator.

I configured Open WebUI with my OpenAPI access token, enabling it to replace ChatGPT. At of this writing, I have been using it for all my AI conversations, and it consistently performed well. Open WebUI serves as a fully capable replacement for ChatGPT.

Open WebUI supports the ability to itercept and modify the response from an LLM. I was very excited to see this because I want to chain two LLMs together to get a second opinion on the output of the first LLM. I hope to write more about this in the future.