I’ve been heads-down with our engineering team for the past few weeks working on something that should make life a lot easier for developers in the OpenAI ecosystem. Today, we’re announcing our RAG API is OpenAI compatible.
This new feature is designed to give you a drop-in replacement for OpenAI’s completions endpoint—so you can bring RAG into the entire ecosystem of OpenAI compatible tools, frameworks, and services while leveraging CustomGPT.ai’s unique capabilities.
What does this mean? Simply swap out a few configuration lines and start using CustomGPT.ai with no fuss.
Check out this video. To demonstrate, I take an open source project, and swap two lines to Build a RAG Powered Voice AI in Under 2 Minutes!
Why OpenAI Compatibility? RAG + TOOLS
Our compatibility layer solves this by seamlessly integrating RAG with the OpenAI tool ecosystem. You get all of CustomGPT.ai’s specialized capabilities—like advanced accuracy and no-hallucinations— while accessing hundreds of existing tools and frameworks built with support for the OpenAI SDK.
Why choose between powerful RAG and a rich tool ecosystem when you can have both?
Try it out!
# 1. Just start by installing the OpenAI SDK.
pip install openai
#2. After that, just set your api_key and change your base_url to point to your CustomGPT.ai project endpoint, and you’re off to the races:
from openai import OpenAI
client = OpenAI(
api_key="CUSTOMGPT_API_KEY", # Your CustomGPT.ai API key
base_url="https://app.customgpt.ai/api/v1/projects/{project_id}/" # Replace with your project ID
)
response = client.chat.completions.create(
model="gpt-4", # This will be ignored; your project's model will be used
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Who are you?"}
],
)
print(response.choices[0].message.content)
How It Works
Endpoint & Parameters: Our RAG API is built to recognize the same chat completion parameters you’d send to OpenAI’s completion endpoint. We ignore or default others (like voice, temperature and model) —meaning you can keep them in your code, but we won’t actually apply them. If you call an endpoint we don’t support, we’ll return a 404 or 501. Get detailed info here.
-
- Same Developer Experience: If you’re used to the OpenAI Python SDK, you can just update the endpoint URL and your API key. Everything else should feel familiar. For advanced features that are unique to CustomGPT.ai, you’ll still need to use our main API.
What’s in It for You
-
- Single Endpoint, Multiple Tools: You can stay connected to the OpenAI ecosystem of tools and frameworks AND leverage our RAG API.
- Fast Integration: If you’ve been contemplating how to integrate RAG into your AI stack, it doesn’t get much simpler than this.
-
- Less Overhead: Nobody likes refactoring entire codebases. This approach lets you integrate us in minutes (2 lines of code change), not days.
Get full detailed information about what we support and how to implement this in our CustomGPT.ai OpenAI SDK Compatibility docs.
Known Limitations
Right now we support the completions endpoint. If you’re doing anything fancy with other OpenAI endpoints—like embeddings or audio processing—we’re not supporting those yet. You’ll get a 404 / 501 if you try.
Some optional parameters (like frequency_penalty, stop, or n) are simply ignored. We’ll give you a successful response, just without applying those extras.
Documentation & Feedback
We’d love to know if there’s anything else you’d like to see in this compatibility layer. Do you have a framework or feature you rely on that we haven’t covered yet? Let us know—you can reach out via our support channels or community forums, and we’ll do our best to get it on the roadmap.
Wrap-Up
Check out the docs, try updating your base URL, and let us know how it goes. We think this compatibility layer will make it simpler for you to explore CustomGPT.ai’s capabilities without tearing apart your existing integration. If you have feedback, ideas for future improvements, or run into issues, shoot them our way. We’re always listening.
FAQs
1. How does the new OpenAI-compatible RAG API work with existing OpenAI code?
Our new endpoint is designed to recognize the same parameters used in OpenAI’s chat.completions endpoint. You just swap out the base URL and API key in your code, and everything else—like conversation structure—remains the same.
2. Which OpenAI features are not supported yet?
We currently only support the chat.completions endpoint. Other endpoints (e.g., embeddings, audio processing) aren’t available yet, and attempts to call them will return a 404 or 501.
3. What happens to parameters like model, temperature, or frequency_penalty?
We ignore or default many of these (e.g., model, temperature, n, stop) when you send a request to our endpoint. You’ll still get a valid response, but we won’t apply those advanced OpenAI-specific parameters on the backend.
4. Do I still need to manage conversation history myself?
Yes, you need to manually handle any ongoing conversation context if you’re using our OpenAI-compatibility layer. Unlike our native CustomGPT.ai endpoints, we currently don’t store or manage conversation state for you.
5. Is this compatibility layer available on all subscription tiers?
Yes, it’s available to all CustomGPT.ai subscription tiers right now. Simply update your configuration to use our OpenAI-compatible endpoint to start exploring advanced RAG features with your existing OpenAI-based integrations.
