Ever since ChatGPT's inception, I've been fascinated by its text-streaming experience. The text streams word by word, making an HTTP request to the server and delivering the response incrementally. This approach makes you feel like you're receiving information continuously, rather than waiting for a complete response. Unlike Bard’s approach, I find streaming responses more effective, especially since large language models (LLMs) have not yet achieved instant responses.

Google eventually recognized the user experience issue, implementing streaming responses for their Gemini model. While I appreciate their fading animation, it still lacks some fluidity. Ultimately, I prefer the ChatGPT experience.

The Project Request

The real challenge began when my manager asked me to implement a ChatGPT-like interface using OpenAI's Assistant API on FlutterFlow, with Supabase as the backend. The task was clear but required some exploration to finalize the design.

Key Components of a Simple ChatGPT UI

- Conversations List: This saves messages between the LLM and the user in a database (we'll use Supabase Postgres for this).

- Streaming LLM Response: Each chat requires the backend to connect the client app with OpenAI and save the conversation.

From this, we identified three main areas to address:

1.  Understanding the OpenAI Assistant API.
2.  Investigating Supabase edge functions and their support for streaming responses.
3.  Designing the backend.

We also needed to implement streaming responses in FlutterFlow and connect them with widgets.

Overcoming Challenges

One major challenge was that at the time of making this small project FlutterFlow did not support streaming responses in API calls. To prove that FlutterFlow could deliver a fluid experience like ChatGPT, I decided to go custom with Flutter.

Step 1: Understanding the OpenAI Assistant API

First, I learned how the OpenAI Assistant API works and tested it using Postman until I had a solid understanding. Here’s a brief explanation:

OpenAI Assistant API Overview

The OpenAI Assistant API allows you to interact with the Large Language Model (LLM) by sending a series of messages as input. It uses a RESTful approach where you send an HTTP POST request with your input messages, and the model responds with a continuation of the conversation. Here’s a simple example:

Request:

Response:

Step 2: Creating the Supabase Edge Function

I drafted a simple implementation in Deno TypeScript. The function creates a thread, opens a streaming connection with OpenAI, and ensures it delivers the response to the client. This was the first step in my prototype to verify that data streaming could be redirected to the client app without issues.

Edge Function:

Example Request and Response:

Request:

Response:

Step 3: Implementing in FlutterFlow

With the function tested and streaming correctly, I moved to FlutterFlow. I created a simple chat view widget that accepts parameters like thread ID (acting as the conversation ID in the OpenAI Assistant API). If the thread ID is null, it indicates the creation of a new conversation or thread.

I developed the UI and integrated the API calls, utilizing HTTPS streaming calls to enhance the streaming UI/UX experience.

Demonstration of the Widget:

Implementing the Streaming Call

To implement the streaming call, I created a method called fetchStreamResponse that handles the interaction with the Supabase edge function and streams the response back to the chat widget in real-time.

Example Code:

Explanation

• Request Setup: We initiate an HTTP POST request to the Supabase edge function, including the user’s message in the request body.
• Streaming Response Handling: The response stream is processed line by line. For each valid line of data, we decode it from JSON format and yield the message content.
• UI Update: The `_sendMessage` method updates the chat interface in real-time with each part of the assistant’s response as it is received.

This method ensures a smooth and interactive chat experience by streaming responses incrementally, much like ChatGPT.

Results

The final result was a fluid, ChatGPT-like streaming experience. You can see a demo of the application.

Conclusion

Leveraging FlutterFlow and Supabase to create applications as complex as ChatGPT demonstrates the powerful capabilities of these platforms. The integration of these tools allowed us to seamlessly build a sophisticated, real-time chat application with streaming responses, showcasing their potential for developing advanced applications.

Ease of Implementation

• Supabase Edge Function: Implementing the edge function in Supabase was straightforward. By writing a simple function in Deno TypeScript, we were able to handle HTTP requests, interact with the OpenAI API, and stream responses back to the client. Supabase's robust backend services made it easy to set up and deploy this functionality.

• FlutterFlow Widget: Creating the chat view widget in FlutterFlow was equally straightforward. The platform's intuitive interface and powerful capabilities enabled us to design the UI and handle user interactions efficiently. Even though FlutterFlow didn't natively support streaming API calls, it allowed us to incorporate custom Flutter code, ensuring we could implement the required functionality.

Next-Level Power

• Flexibility and Customization: One of the standout features of FlutterFlow is its flexibility. While it provides a user-friendly interface for building applications, it also allows developers to dive into custom code when necessary. This capability was crucial for our project, as it enabled us to implement the streaming API call and achieve the desired real-time response experience.
• Powerful Backend Integration: Supabase's powerful backend services, including its real-time database and edge functions, provided the necessary infrastructure to support our application's complex requirements. This integration enabled us to handle user interactions, manage conversations, and stream responses effectively.

By combining FlutterFlow's ease of use and flexibility with Supabase's powerful backend capabilities, we were able to build a sophisticated, real-time chat application with relative ease. These platforms empower developers to create complex applications without sacrificing simplicity or performance, truly taking app development to the next level.