Create a Bot by Sitemap Using CustomGPT.ai RAG API’s Python SDK – A Comprehensive Developer Guide

Quickly create a bot by sitemap by spinning up a new chatbot, creating a project, and ingesting a sitemap — all via the Python SDK. We’ll:

1. Install and authenticate the CustomGPTs RAG APIs Python SDK
   
2. Create a project (agent) with a sitemap
   
3. Verify creation and monitor indexing
   
4. Start a test conversation
   
5. Troubleshoot and next steps

Before you dive in, don’t forget to Sign up for CustomGPT and check out the CustomGPT API documentation.

Notebook Link: https://github.com/Poll-The-People/customgpt-cookbook/blob/main/examples/SDK_Create_Bot_By_Sitemap.ipynb

Prerequisites

Make sure you have the following in place before proceeding:

• CustomGPT.ai Account & API Key
  
  • Create or sign in at CustomGPT.
    
  • Generate an API token under Settings → API Tokens.
    
• Python 3 Environment
  
  • Google Colab, Jupyter Notebook, or any local Python 3 setup.
    
• Basic Python Knowledge
  
  • Comfortable installing packages, importing modules, and working with JSON.
    
• Sitemap URL
  
  • A valid sitemap.xml URL for the site you want your bot to index.

Once you’ve checked these off, let’s install the SDK.

1. Install & Authenticate the SDK

First, install the customgpt-client package and configure your API key. This sets up the SDK so all calls are authenticated and ready to go.

# Install the CustomGPT Python SDK from PyPI

!pip install customgpt-client

# Import the main SDK class

from customgpt_client import CustomGPT

# Authenticate with your API token

CustomGPT.api_key = "YOUR_API_TOKEN"  # ← Replace with your actual token

What’s happening here?

• pip install customgpt-client fetches the SDK package.
  
• from customgpt_client import CustomGPT imports the SDK’s main interface.
  
• CustomGPT.api_key = … ensures every method you call includes your token in the header.

Get API keys

To get your API key, there are two ways:

Method 1 – Via Agent

1. Agent > All Agents.
2. Select your agent and go to deploy, click on the API key section and create an API. 

Method 2 – Via Profile section.

1. Go to profile (top right corner of your screen)
2. Click on My Profile
3. You will see the screen something like this (below screenshot). Here you can click on “Create API key”, give it a name and copy the key.

Please save this secret key somewhere safe and accessible. For security reasons, You won’t be able to view it again through your CustomGPT.ai account. If you lose this secret key, you’ll need to generate a new one.

With the SDK installed and authenticated, let’s move on to creating your agent.

2. Create the Project with Sitemap

Next, we create a new agent (formerly called a “project”) and point it at your sitemap. This single call both creates and starts indexing.

# Define your agent’s name and sitemap URL

project_name = "My New Bot from Sitemap"

sitemap_url   = "https://adorosario.github.io/small-sitemap.xml"

# Create the agent and ingest the sitemap

create_resp = CustomGPT.Project.create(

    project_name=project_name,  # Human‑readable bot name

    sitemap_path = sitemap_url   # Sitemap to crawl and index

)

# Extract the new agent’s unique ID for future calls

project_id = create_resp.parsed.data.id

print("Created Project ID:", project_id)

Explanation of key lines:

• Project.create(…)
  
  • Calls the /projects endpoint under the hood.
    
  • Starts crawling and indexing the pages listed in the sitemap.
    
• .parsed.data.id
  
  • Parses the JSON response and retrieves the id for subsequent SDK calls.

Now that your project is spinning up, let’s verify its creation and see initial stats.

3. Verify Creation & Initial Stats

Once the project is created, you’ll want to confirm its name and see how many pages it plans to index.

# 1) Confirm the project’s name

proj = CustomGPT.Project.get(project_id=project_id)

print("Project Name:", proj.parsed.data.project_name)

# 2) Retrieve initial indexing statistics

stats = CustomGPT.Project.stats(project_id=project_id)

print("Pages Found:", stats.parsed.data.pages_found)

What this does:

1. Project.get
   
   • Calls GET /projects/{id} to fetch metadata like project_name.
     
2. Project.stats
   
   • Calls GET /projects/{id}/stats to see metrics such as pages_found.

With your project confirmed and stats in hand, let’s wait for indexing to complete before chatting.

4. Wait for Indexing to Finish

Indexing large sitemaps can take time. Here’s a simple polling loop to monitor progress:

import time

# Poll until the number of indexed pages matches the total pages found

while True:

    stats = CustomGPT.Project.stats(project_id=project_id).parsed.data

    print(f"Indexed {stats.pages_indexed}/{stats.pages_found} pages")

    if stats.pages_indexed >= stats.pages_found:

        break

    time.sleep(5)  # Wait 5 seconds between checks

print("✅ Indexing complete!")

How it works:

• Repeatedly calls Project.stats every 5 seconds.
  
• Exits the loop once pages_indexed reaches pages_found.
  
• Avoids spamming the API by sleeping between requests.

Once indexing is done, your agent is ready to answer questions—let’s test it.

5. Start a Test Conversation

Finally, let’s create a conversation session and send a prompt to verify your bot is live.

# 1) Open a new conversation session

conv = CustomGPT.Conversation.create(project_id=project_id)

session_id = conv.parsed.data.session_id

print("Session ID:", session_id)

# 2) Send a test question

response = CustomGPT.Conversation.send(

    project_id=project_id,

    session_id=session_id,

    prompt="What topics does this site cover?"

)

# 3) Print the chatbot’s answer

print("Bot Response:", response.parsed.data.openai_response)

• Conversation.create
  
  • Calls POST /projects/{id}/conversations to start a chat session.
    
• Conversation.send
  
  • Calls POST /projects/{id}/conversations/{session_id}/messages.
    
  • Returns an openai_response and any associated citations.

Seeing a sensible response confirms your bot is working—congratulations!

FAQs & Next Steps

Related Posts

• Add a Sitemap to a Project
  

• Retrieve Messages of a Conversation

Priyansh Khodiyar

Priyansh is Developer Relations Advocate who loves technology, writer about them, creates deeply researched content about them.