Bobur UmurzokovOpenAI has recently launched a new version of ChatGPT which now allows plugins inside ChatGPT. These plugins can be added directly to the chatbot, providing it with access to a wide range of knowledge and information from its third-party partners through the APIs. ChatGPT plugins can extend its functionality and enhance its capabilities to access up-to-date information such as research travel costs, find out discount information, or help you book flights and order food. You can also build your own plugin that allows ChatGPT to call your API data intelligently.
Yes, that’s right! To make your data accessible through a ChatGPT custom plugin, ChatGTP requires you to build a new API or use an existing one that can be used to query it and receive its responses. Then it generates a user-friendly answer by combining the API data and its natural language capabilities. In this case, API Gateway can help with improving security, usability, and efficiency. You can learn more about the API Gateway usage in building enterprise search APIs with Large Language Models (LLMs) for production. This post explores how API Gateway can be beneficial for ChatGPT plugin developers to expose, secure, manage, and monitor their API endpoints.
According to OpenAI: The plugin developer writes a specification for an API using the OpenAPI standard that enables ChatGPT to interact with APIs defined by developers.
Learning objectives
You will learn the following throughout the article:
- Understand the role of API Gateway in building ChatGPT Plugin API.
- How to publish, secure, observe, and apply other cross-cutting concerns for plugin API.
- How to use Apache APISIX (an open-source API Gateway) with ChatGPT Plugins.
The Role of API Gateway
An API Gateway act as the bridge between ChatGPT Plugin and your API by providing a standardized interface for communication. APIs. It manages the API's access, security, and performance, and offers other cross-cutting features. Assume that you want to build a ChatGPT plugin for discount information from markets in your city, you might have different APIs to show new products, the nearest markets, and the latest deals. In this context, the API Gateway will be an additional layer between your API and the plugin.
For example, the API Gateway could combine ChatGPT with other APIs such as a natural language processing API other than OpenAI or a translation API from other providers, allowing clients to access multiple services with a single plugin through custom-defined URI paths and upstream services (Multiple backend API servers), route requests to the appropriate API and returning the response back to ChatGPT.
Enhanced security
One of the primary roles of an API Gateway in ChatGPT is to handle authentication and authorization. This involves verifying the identity of the user of a plugin and determining whether they have the necessary permissions to access the API from the plugin. The OpenAI may use a variety of authentication mechanisms, such as OAuth, API keys, or custom authentication protocols and it passes user credentials to the API Gateway. Then API Gateway can do auth verification out-of-box to check whether the user is authenticated or not. You don’t need to write any implementation code for this validation process for each API.
Essentially, API gateways serve as a security measure to safeguard against potential hacking attempts. The API Gateway restricts specific types of requests, such as blocking unauthorized POST requests to a particular Route unless the sender has appropriate privileges or includes a specific header in the request.
Let's say a company has an e-commerce API that allows customers to place orders via ChatGPT Plugin. The API Gateway is configured to block POST requests to the "place order" Route unless the request includes a valid API key in the request header, which acts as a privileged access token. This means that only authenticated and authorized requests with the correct API key will be allowed to create new orders, while unauthorized requests will be blocked, effectively protecting against potential malicious attempts to place fake orders or manipulate the system. See the summary of other security features below.
Efficient rate-limiting
Another important role of the API Gateway is to handle rate limiting. This ensures that the ChatGPT plugin is not overwhelmed the API with too many requests at once, which could impact its performance or cause it to crash. The API Gateway can limit the number of requests a client can make within a certain time frame and block requests that exceed the limit.
High-performance
Performance is another area where an API Gateway can help improve the ChatGPT Plugin's performance. For example, the API Gateway can implement caching to store frequently requested responses and return them quickly without needing to query the actual API. The API Gateway can also handle request/response transformation to convert incoming requests into a format (Like converting REST requests to GraphQL) that your API can understand and transform the responses into a format the plugin can consume.
Continuous monitoring
Even if OpenAI states that ChatGPT does not use data submitted by customers via our API to train or improve their models, it is important to enabling observability features for many other reasons. The API Gateway can provide insights into how the ChatGPT plugin is being used, what kind of data is shared, and identify any issues that need to be addressed by monitoring continuously the requests made by your plugin.
How to use Apache APISIX with ChatGPT Plugins
Once we understood why API Gateway plays a crucial role in building ChatGPT Plugins, let’s take a look at simple steps on how to use Apache APISIX API Gateway before you share a new plugin on ChatGPT. There are many other API Gateways in the market, this post can help you decide which one fits your need.
At the time of writing the current blog post, ChatGPT has restricted access and to gain alpha access to OpenAI, you need to sign up for a ChatGPT plugin waitlist. They will only be prioritizing developers and ChatGPT Plus users though, before releasing it to the wider public.
- Install and run Apache APISIX either locally in a development environment or on a remote server (in the cloud). If you run it locally, APISIX can be accessed via http://localhost:9080
- Configure a route and upstream for each of your API endpoints whether requesting Admin API (It is running on http://localhost:9180) or a user-friendly UI dashboard. You can also import existing OpenAPI specification to automatically registers routes and upstream.
- You enable some API Gateway features, authentication, rate-limiting, and observability using APISIX’s plugins.
- You export the updated OpenAPI specification to use in ChatGPT. Make necessary changes in the resulted in document
YAML/JSON. Place this file somewhere in the APISIX server via another route so that ChatGPT can find it in this path/openapi.yamllike[http://localhost:9080/openapi.yaml](http://localhost:9080/openapi.yaml).This specification is compiled into a prompt, which explains to ChatGPT how it may use the API to enhance its answers. Think of a detailed prompt, including a description of each endpoint that's available. - Other steps, like defining a manifest file, running a plugin, and writing descriptions are pretty much the same as it is already well-documented in the official OpenAI documentation. When you connect the plugin via the ChatGPT UI and run the plugin, make sure that the domain address point to APISIX API Gateway.
- Finally, the user asks new questions by enabling the plugin on ChatGPT UI. If ChatGPT decides it should grab information from the API, it will make the request to the API Gateway and add it to the context before attempting to respond.
API security for ChatGPT plugin summary
Look at this summary of API Gateway offerings to secure API for the ChatGPT Plugin:
- Authentication Protocols: With API Gateway, you can choose a robust and secure authentication protocol, such as OAuth 2.0 or JSON Web Tokens (JWT), to authenticate API requests.
- Two-Factor Authentication (2FA): You can implement 2FA through the integration with various identity providers which can add an extra layer of security to API authentication.
- Secure Token Management: You can store tokens securely by avoiding storing them in client-side applications or in insecure locations, such as client-side cookies or local storage.
- Role-Based Access Control (RBAC): You can enable RBAC to control the permissions and actions that different users or applications can perform within the ChatGPT Plugin API.
- Transport Layer Security (TLS): API has a TLS option to encrypt communication between clients and servers over the network.
- API Rate Limiting: API Gateway provides rate-limiting mechanisms to prevent abuse or misuse of the ChatGPT Plugin API.
- Logging and Auditing: You can use API Gateway with other observability platforms for comprehensive logging and auditing mechanisms to track and monitor API requests and responses.
Conclusion
Introducing plugin integration to ChatGPT is an upgrade for OpenAI. As well, it’s an important change in the field of user-facing AI for the GPT model. API Gateway provides a performant interface for communication to expose your API safely, along with security, rate-limiting policies, authentication methods, and monitoring. Without an API Gateway, ChatGPT would be much harder to integrate into other systems, and clients would need to manage authentication, rate limiting, and other features themselves.
