Introducing OpenAI compatibility – Integrate RAG with hundreds of OpenAI ecosystem tools

I’ve been heads-down with our engineering team for the past few weeks working on something that should make life a lot easier for developers in the OpenAI ecosystem. Today, we’re announcing our RAG API is OpenAI compatible.

This new feature is designed to give you a drop-in replacement for OpenAI’s completions endpoint—so you can bring RAG into the entire ecosystem of OpenAI compatible tools, frameworks, and services while leveraging CustomGPT.ai’s unique capabilities. 

What does this mean? Simply swap out a few configuration lines and start using CustomGPT.ai with no fuss. 

Check out this video. To demonstrate, I take an open source project, and swap two lines to Build a RAG Powered Voice AI in Under 2 Minutes!

Why OpenAI Compatibility? RAG + TOOLS

Our compatibility layer solves this by seamlessly integrating RAG with the OpenAI tool ecosystem. You get all of CustomGPT.ai’s specialized capabilities—like advanced accuracy and no-hallucinations— while accessing hundreds of existing tools and frameworks built with support for the OpenAI SDK.

Why choose between powerful RAG and a rich tool ecosystem when you can have both?

Try it out!

All you need is a CustomGPT.ai account (start a free trial) and an API Key. Below is a simple Python script showing how you can use to try it out. 

# 1. Just start by installing the OpenAI SDK.

pip install openai

#2.  After that, just set your  api_key and change your base_url to point to your CustomGPT.ai project endpoint, and you’re off to the races:

from openai import OpenAI
client = OpenAI(
    api_key="CUSTOMGPT_API_KEY",  # Your CustomGPT.ai API key
    base_url="https://app.customgpt.ai/api/v1/projects/{project_id}/"  # Replace with your project ID
)
response = client.chat.completions.create(
    model="gpt-4",  # This will be ignored; your project's model will be used
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Who are you?"}
    ],
)
print(response.choices[0].message.content)

How It Works

Endpoint & Parameters: Our RAG API is built to recognize the same chat completion parameters you’d send to OpenAI’s completion endpoint. We ignore or default others  (like voice, temperature and model) —meaning you can keep them in your code, but we won’t actually apply them. If you call an endpoint we don’t support, we’ll return a 404 or 501. Get detailed info here.

• Same Developer Experience: If you’re used to the OpenAI Python SDK, you can just update the endpoint URL and your API key. Everything else should feel familiar. For advanced features that are unique to CustomGPT.ai, you’ll still need to use our main API.

What’s in It for You

• Single Endpoint, Multiple Tools: You can stay connected to the OpenAI ecosystem of tools and frameworks AND leverage our RAG API.
• Fast Integration: If you’ve been contemplating how to integrate RAG into your AI stack, it doesn’t get much simpler than this.
• Less Overhead: Nobody likes refactoring entire codebases. This approach lets you integrate us in minutes (2 lines of code change), not days.

Get full detailed information about what we support and how to implement this in our CustomGPT.ai OpenAI SDK Compatibility docs.

Known Limitations

Right now we support the completions endpoint. If you’re doing anything fancy with other OpenAI endpoints—like embeddings or audio processing—we’re not supporting those yet. You’ll get a 404 / 501 if you try.

Some optional parameters (like frequency_penalty, stop, or n) are simply ignored. We’ll give you a successful response, just without applying those extras. We built this out based on community requests, with more features in the pipeline. Let us know if you run into any quirks. We plan on collecting developer feedback over the next couple of weeks, then smoothing out any bumps in a future release. 

Documentation & Feedback

For full details, check out our complete OpenAI Compatible SDK documentation here: https://docs.customgpt.ai/reference/customgptai-openai-sdk-compatibility

We’d love to know if there’s anything else you’d like to see in this compatibility layer. Do you have a framework or feature you rely on that we haven’t covered yet? Let us know—you can reach out via our support channels or community forums, and we’ll do our best to get it on the roadmap.

Wrap-Up

Check out the docs, try updating your base URL, and let us know how it goes. We think this compatibility layer will make it simpler for you to explore CustomGPT.ai’s capabilities without tearing apart your existing integration. If you have feedback, ideas for future improvements, or run into issues, shoot them our way. We’re always listening.

Frequently Asked Questions

Priyansh Khodiyar

Priyansh is Developer Relations Advocate who loves technology, writer about them, creates deeply researched content about them.