Pagination Examples

This guide shows how to work with paginated API responses when listing prompts. Pagination allows you to efficiently retrieve large numbers of prompts without overwhelming your application or the API.

Prerequisites

pip install baytos-claro
export BAYT_API_KEY="your_api_key_here"

Understanding Pagination

The Claro API uses cursor-based pagination for list operations:

{
  "prompts": [...],      // Array of prompts
  "cursor": "abc123",    // Token for next page
  "hasMore": true        // Whether more pages exist
}

Cursor-based pagination is more reliable than offset-based pagination, especially when data is being added or removed during iteration.

Example 1: Basic Pagination

Fetch the first page of prompts:

import os
from baytos.claro import BaytClient

client = BaytClient(api_key=os.getenv("BAYT_API_KEY"))

# Get first page (default limit is 50)
result = client.list_prompts(limit=10)

print(f"Fetched {len(result['prompts'])} prompts")

for prompt in result['prompts']:
    print(f"- {prompt.title}")

if result['hasMore']:
    print(f"\nMore prompts available. Cursor: {result['cursor']}")
else:
    print("\nNo more prompts")

Expected output

Fetched 10 prompts
- Customer Support Assistant
- Code Review Helper
- Data Analysis Expert
- Content Writer
- Email Composer
- Meeting Summarizer
- SQL Query Generator
- Bug Report Analyzer
- Documentation Writer
- API Response Parser

More prompts available. Cursor: eyJpZCI6IjEyMyJ9

Example 2: Fetch Next Page

Use the cursor to fetch the next page:

import os
from baytos.claro import BaytClient

client = BaytClient(api_key=os.getenv("BAYT_API_KEY"))

# Get first page
result = client.list_prompts(limit=10)
print(f"Page 1: {len(result['prompts'])} prompts")

# Get second page using cursor
if result['hasMore']:
    result = client.list_prompts(limit=10, cursor=result['cursor'])
    print(f"Page 2: {len(result['prompts'])} prompts")

Example 3: Iterate Through All Pages

Loop through all pages to get all prompts:

import os
from baytos.claro import BaytClient

client = BaytClient(api_key=os.getenv("BAYT_API_KEY"))

all_prompts = []
cursor = None

while True:
    # Fetch a page
    result = client.list_prompts(limit=20, cursor=cursor)

    # Add prompts to our list
    all_prompts.extend(result['prompts'])

    print(f"Fetched {len(result['prompts'])} prompts (total: {len(all_prompts)})")

    # Check if there are more pages
    if not result['hasMore']:
        break

    # Update cursor for next page
    cursor = result['cursor']

print(f"\nTotal prompts: {len(all_prompts)}")

Example 4: Iterate with Limit

Fetch a specific maximum number of prompts:

import os
from baytos.claro import BaytClient

def get_prompts_limited(client, max_prompts=50):
    """Fetch up to max_prompts prompts"""
    all_prompts = []
    cursor = None

    while len(all_prompts) < max_prompts:
        # Calculate how many more we need
        remaining = max_prompts - len(all_prompts)
        limit = min(remaining, 50)  # API max is typically 50-100

        # Fetch a page
        result = client.list_prompts(limit=limit, cursor=cursor)

        # Add prompts
        all_prompts.extend(result['prompts'])

        # Check if we should continue
        if not result['hasMore']:
            break

        # Update cursor
        cursor = result['cursor']

    return all_prompts[:max_prompts]

client = BaytClient(api_key=os.getenv("BAYT_API_KEY"))
prompts = get_prompts_limited(client, max_prompts=25)
print(f"Fetched {len(prompts)} prompts")

Example 5: Process While Paginating

Process prompts as you fetch them (memory efficient):

import os
from baytos.claro import BaytClient

client = BaytClient(api_key=os.getenv("BAYT_API_KEY"))

def process_prompt(prompt):
    """Process a single prompt"""
    print(f"Processing: {prompt.title}")
    # Do something with the prompt
    return {
        'title': prompt.title,
        'package': prompt.package_name,
        'has_context': prompt.has_context()
    }

# Process prompts page by page
processed = []
cursor = None
page = 1

while True:
    print(f"\nFetching page {page}...")
    result = client.list_prompts(limit=20, cursor=cursor)

    # Process each prompt
    for prompt in result['prompts']:
        processed.append(process_prompt(prompt))

    # Check if done
    if not result['hasMore']:
        break

    cursor = result['cursor']
    page += 1

print(f"\nProcessed {len(processed)} prompts total")

Example 6: Generator Pattern

Use a generator for memory-efficient iteration:

import os
from baytos.claro import BaytClient

def iterate_all_prompts(client, page_size=50):
    """Generator that yields all prompts"""
    cursor = None

    while True:
        result = client.list_prompts(limit=page_size, cursor=cursor)

        # Yield each prompt
        for prompt in result['prompts']:
            yield prompt

        # Check if done
        if not result['hasMore']:
            break

        cursor = result['cursor']

# Usage
client = BaytClient(api_key=os.getenv("BAYT_API_KEY"))

for prompt in iterate_all_prompts(client):
    print(f"- {prompt.title}")

Generators are memory-efficient because they only load one page at a time, making them ideal for processing large numbers of prompts.

Example 7: Filter While Paginating

Filter prompts while iterating:

import os
from baytos.claro import BaytClient

client = BaytClient(api_key=os.getenv("BAYT_API_KEY"))

# Find all prompts with context files
prompts_with_files = []
cursor = None

while True:
    result = client.list_prompts(limit=50, cursor=cursor)

    # Filter prompts with files
    for prompt in result['prompts']:
        if prompt.has_context():
            files = prompt.get_file_contexts()
            if files:
                prompts_with_files.append(prompt)
                print(f"Found: {prompt.title} ({len(files)} files)")

    if not result['hasMore']:
        break

    cursor = result['cursor']

print(f"\nTotal prompts with files: {len(prompts_with_files)}")

Example 8: Paginate with Progress

Show progress while paginating:

import os
from baytos.claro import BaytClient

client = BaytClient(api_key=os.getenv("BAYT_API_KEY"))

# First, estimate total (get first page)
first_page = client.list_prompts(limit=1)
print("Fetching all prompts...")

all_prompts = []
cursor = None
page = 1

while True:
    result = client.list_prompts(limit=50, cursor=cursor)
    all_prompts.extend(result['prompts'])

    # Show progress
    print(f"Page {page}: {len(all_prompts)} prompts fetched", end='')
    if result['hasMore']:
        print(" (more available)")
    else:
        print(" (complete)")

    if not result['hasMore']:
        break

    cursor = result['cursor']
    page += 1

print(f"\nTotal: {len(all_prompts)} prompts")

Example 9: Parallel Processing

Process pages in parallel (advanced):

import os
from concurrent.futures import ThreadPoolExecutor
from baytos.claro import BaytClient

def process_page(prompts):
    """Process a page of prompts"""
    results = []
    for prompt in prompts:
        results.append({
            'title': prompt.title,
            'package': prompt.package_name
        })
    return results

client = BaytClient(api_key=os.getenv("BAYT_API_KEY"))

# First, fetch all pages
pages = []
cursor = None

while True:
    result = client.list_prompts(limit=50, cursor=cursor)
    pages.append(result['prompts'])

    if not result['hasMore']:
        break

    cursor = result['cursor']

print(f"Fetched {len(pages)} pages")

# Process pages in parallel
with ThreadPoolExecutor(max_workers=4) as executor:
    all_results = list(executor.map(process_page, pages))

# Flatten results
processed = [item for page_results in all_results for item in page_results]
print(f"Processed {len(processed)} prompts")

Be cautious with parallel processing to avoid hitting rate limits. Use this pattern only when processing is slow and API calls are already complete.

Example 10: Save and Resume

Save pagination state to resume later:

import os
import json
from baytos.claro import BaytClient

def save_cursor(cursor, filename='cursor.json'):
    """Save cursor to file"""
    with open(filename, 'w') as f:
        json.dump({'cursor': cursor}, f)

def load_cursor(filename='cursor.json'):
    """Load cursor from file"""
    try:
        with open(filename, 'r') as f:
            data = json.load(f)
            return data.get('cursor')
    except FileNotFoundError:
        return None

client = BaytClient(api_key=os.getenv("BAYT_API_KEY"))

# Load previous cursor if exists
cursor = load_cursor()

if cursor:
    print(f"Resuming from saved cursor...")
else:
    print("Starting from beginning...")

# Fetch next page
result = client.list_prompts(limit=50, cursor=cursor)

print(f"Fetched {len(result['prompts'])} prompts")

# Save cursor for next run
if result['hasMore']:
    save_cursor(result['cursor'])
    print("Cursor saved for next run")
else:
    print("All prompts fetched")

Example 11: Page Size Optimization

Find optimal page size for your use case:

import os
import time
from baytos.claro import BaytClient

client = BaytClient(api_key=os.getenv("BAYT_API_KEY"))

# Test different page sizes
page_sizes = [10, 25, 50, 100]

for page_size in page_sizes:
    start = time.time()

    # Fetch first page
    result = client.list_prompts(limit=page_size)

    elapsed = time.time() - start

    print(f"Page size {page_size}: {elapsed:.2f}s for {len(result['prompts'])} prompts")

Larger page sizes reduce the number of API calls but may increase response time. Find the balance that works for your application.

Example 12: Error Handling During Pagination

Robust pagination with error handling:

import os
import time
from baytos.claro import BaytClient, BaytRateLimitError, BaytAPIError

client = BaytClient(api_key=os.getenv("BAYT_API_KEY"))

all_prompts = []
cursor = None
max_retries = 3

while True:
    retry_count = 0

    while retry_count < max_retries:
        try:
            result = client.list_prompts(limit=50, cursor=cursor)
            all_prompts.extend(result['prompts'])

            print(f"Fetched {len(result['prompts'])} prompts (total: {len(all_prompts)})")

            if not result['hasMore']:
                print("All prompts fetched successfully")
                exit(0)

            cursor = result['cursor']
            break  # Success, exit retry loop

        except BaytRateLimitError as e:
            print(f"Rate limited. Waiting before retry...")
            time.sleep(60)
            retry_count += 1

        except BaytAPIError as e:
            print(f"API error: {e}")
            retry_count += 1
            time.sleep(2 ** retry_count)  # Exponential backoff

    if retry_count >= max_retries:
        print(f"Failed after {max_retries} retries")
        break

print(f"Fetched {len(all_prompts)} prompts before stopping")

Next Steps

Basic Usage

Learn basic SDK operations

File Downloads

Download context files

Error Handling

Handle errors gracefully

Advanced Patterns

Production-ready patterns

Getting Started

Python SDK

Examples

Guides

Prerequisites

Example 2: Fetch Next Page

Example 3: Iterate Through All Pages

Example 4: Iterate with Limit

Example 5: Process While Paginating

Example 6: Generator Pattern

Example 7: Filter While Paginating

Example 8: Paginate with Progress

Example 9: Parallel Processing

Example 10: Save and Resume

Example 11: Page Size Optimization

Next Steps

Basic Usage

File Downloads

Error Handling

Advanced Patterns

​Prerequisites

​Understanding Pagination

​Example 1: Basic Pagination

​Example 2: Fetch Next Page

​Example 3: Iterate Through All Pages

​Example 4: Iterate with Limit

​Example 5: Process While Paginating

​Example 6: Generator Pattern

​Example 7: Filter While Paginating

​Example 8: Paginate with Progress

​Example 9: Parallel Processing

​Example 10: Save and Resume

​Example 11: Page Size Optimization

​Example 12: Error Handling During Pagination

​Next Steps

Basic Usage

File Downloads

Error Handling

Advanced Patterns

Prerequisites

Understanding Pagination

Example 1: Basic Pagination

Example 2: Fetch Next Page

Example 3: Iterate Through All Pages

Example 4: Iterate with Limit

Example 5: Process While Paginating

Example 6: Generator Pattern

Example 7: Filter While Paginating

Example 8: Paginate with Progress

Example 9: Parallel Processing

Example 10: Save and Resume

Example 11: Page Size Optimization

Example 12: Error Handling During Pagination

Next Steps