I’ve been heads-down with our engineering team for the past few weeks working on something that should make life a lot easier for developers in the OpenAI ecosystem. Today, we’re announcing our RAG API is OpenAI compatible.
This new feature is designed to give you a drop-in replacement for OpenAI’s completions endpoint—so you can bring RAG into the entire ecosystem of OpenAI compatible tools, frameworks, and services while leveraging CustomGPT.ai’s unique capabilities.
What does this mean? Simply swap out a few configuration lines and start using CustomGPT.ai with no fuss.
Check out this video. To demonstrate, I take an open source project, and swap two lines to Build a RAG Powered Voice AI in Under 2 Minutes!
Why OpenAI Compatibility? RAG + TOOLS
Our compatibility layer solves this by seamlessly integrating RAG with the OpenAI tool ecosystem. You get all of CustomGPT.ai’s specialized capabilities—like advanced accuracy and no-hallucinations— while accessing hundreds of existing tools and frameworks built with support for the OpenAI SDK.
Why choose between powerful RAG and a rich tool ecosystem when you can have both?
Try it out!
All you need is a CustomGPT.ai account (start a free trial) and an API Key. Below is a simple Python script showing how you can use to try it out.
# 1. Just start by installing the OpenAI SDK.
pip install openai
#2. After that, just set your api_key and change your base_url to point to your CustomGPT.ai project endpoint, and you’re off to the races:
from openai import OpenAI
client = OpenAI(
api_key="CUSTOMGPT_API_KEY", # Your CustomGPT.ai API key
base_url="https://app.customgpt.ai/api/v1/projects/{project_id}/" # Replace with your project ID
)
response = client.chat.completions.create(
model="gpt-4", # This will be ignored; your project's model will be used
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Who are you?"}
],
)
print(response.choices[0].message.content)
How It Works
Endpoint & Parameters: Our RAG API is built to recognize the same chat completion parameters you’d send to OpenAI’s completion endpoint. We ignore or default others (like voice, temperature and model) —meaning you can keep them in your code, but we won’t actually apply them. If you call an endpoint we don’t support, we’ll return a 404 or 501. Get detailed info here.
- Same Developer Experience: If you’re used to the OpenAI Python SDK, you can just update the endpoint URL and your API key. Everything else should feel familiar. For advanced features that are unique to CustomGPT.ai, you’ll still need to use our main API.
What’s in It for You
- Single Endpoint, Multiple Tools: You can stay connected to the OpenAI ecosystem of tools and frameworks AND leverage our RAG API.
- Fast Integration: If you’ve been contemplating how to integrate RAG into your AI stack, it doesn’t get much simpler than this.
- Less Overhead: Nobody likes refactoring entire codebases. This approach lets you integrate us in minutes (2 lines of code change), not days.
Get full detailed information about what we support and how to implement this in our CustomGPT.ai OpenAI SDK Compatibility docs.
Known Limitations
Right now we support the completions endpoint. If you’re doing anything fancy with other OpenAI endpoints—like embeddings or audio processing—we’re not supporting those yet. You’ll get a 404 / 501 if you try.
Some optional parameters (like frequency_penalty, stop, or n) are simply ignored. We’ll give you a successful response, just without applying those extras. We built this out based on community requests, with more features in the pipeline. Let us know if you run into any quirks. We plan on collecting developer feedback over the next couple of weeks, then smoothing out any bumps in a future release.
Documentation & Feedback
For full details, check out our complete OpenAI Compatible SDK documentation here: https://docs.customgpt.ai/reference/customgptai-openai-sdk-compatibility
We’d love to know if there’s anything else you’d like to see in this compatibility layer. Do you have a framework or feature you rely on that we haven’t covered yet? Let us know—you can reach out via our support channels or community forums, and we’ll do our best to get it on the roadmap.
Wrap-Up
Check out the docs, try updating your base URL, and let us know how it goes. We think this compatibility layer will make it simpler for you to explore CustomGPT.ai’s capabilities without tearing apart your existing integration. If you have feedback, ideas for future improvements, or run into issues, shoot them our way. We’re always listening.
FAQs
1. How does the new OpenAI-compatible RAG API work with existing OpenAI code?
chat.completions
endpoint. You just swap out the base URL and API key in your code, and everything else—like conversation structure—remains the same.2. Which OpenAI features are not supported yet?
chat.completions
endpoint. Other endpoints (e.g., embeddings, audio processing) aren’t available yet, and attempts to call them will return a 404
or 501
.3. What happens to parameters like model, temperature, or frequency_penalty?
n
, stop
) when you send a request to our endpoint. You’ll still get a valid response, but we won’t apply those advanced OpenAI-specific parameters on the backend.4. Do I still need to manage conversation history myself?
5. Is this compatibility layer available on all subscription tiers?
Priyansh is Developer Relations Advocate who loves technology, writer about them, creates deeply researched content about them.
5 Comments
The article is really useful, introducing CustomGPT.ai’s new RAG API, compatible with OpenAI, making it easy to integrate into tools like LangChain or Auto-GPT with just a few small changes – simple, convenient and time-saving.
Great overview! Having an OpenAI-compatible RAG API makes it much easier to integrate custom retrieval workflows without starting from scratch. I’m curious how it handles multilingual use cases—especially in French or Spanish. Looking forward to trying it out.
The new OpenAI-compatible RAG API is a game changer! It’s great that we can easily integrate it by just changing the base URL and API key. However, it’s important to note the limitations, like the lack of support for other endpoints and managing conversation history manually. Excited to explore its features!
I just tried integrating the RAG API into an old chatbot project, and it really was as simple as swapping two lines—less time coding, more time optimizing the experience.
This actually answered my drawback, thank you!