OpenRouter Setup Guide

What You'll Learn

How to connect OpenRouter to RunForge for access to multiple AI models through one API, often at lower costs than going direct to providers.

Why Choose OpenRouter?

Access Multiple Models

• 100+ models: GPT, Claude, Llama, Mistral, and many more
• One API: Single integration for all models
• Easy switching: Change models without changing code
• Latest models: Quick access to new releases

Cost Benefits

• Competitive pricing: Often cheaper than direct provider APIs
• Transparent costs: Exact pricing shown upfront
• No monthly minimums: Pay only for what you use
• Bulk pricing: Better rates for high-volume usage

Developer Friendly

• OpenAI-compatible API: Drop-in replacement for OpenAI
• Detailed documentation: Easy integration guides
• Rate limit handling: Better reliability than some direct APIs
• Usage analytics: Built-in usage tracking

Before You Start

• ✅ OpenRouter account (sign up at openrouter.ai)
• ✅ Credit balance in your OpenRouter account
• ✅ RunForge project created (see Getting Started Guide)
• ✅ 5-10 minutes of time

Step 1: Create Your OpenRouter Account

Sign Up

1. Go to openrouter.ai
2. Click "Sign Up" in the top-right corner
3. Choose sign-up method: Email, GitHub, or Google
4. Complete verification if required
5. Sign in to your new account

Add Credits

1. Click "Credits" in the top navigation
2. Click "Add Credits"
3. Choose amount: Start with $10-20 for testing
4. Add payment method and complete purchase
5. Verify credits appear in your account balance

Step 2: Create an API Key

Generate Your Key

1. Click "Keys" in the top navigation
2. Click "Create Key"
3. Fill out the key details:
4. Name: "RunForge Production" or similar
5. Credit limit (optional): Set a spending limit for this key
6. Rate limit (optional): Requests per minute limit
7. Click "Create"
8. Copy the key immediately - save it somewhere secure

Key Security Best Practices

• Never share your API key publicly
• Use environment variables in your applications
• Set spending limits to prevent unexpected charges
• Rotate keys regularly (every 3-6 months)

Step 3: Add OpenRouter to RunForge

Navigate to API Keys

1. Open RunForge and sign in
2. Select your project from the project dropdown
3. Go to Settings → API Keys

Configure OpenRouter

1. Click "Add API Key" or "Add Provider Key"
2. Select "OpenRouter" from the provider dropdown (or "Custom" if not listed)
3. Fill out the configuration:
4. Key Name: "Production OpenRouter Key"
5. API Key: Paste your OpenRouter API key
6. Base URL (if using Custom): https://openrouter.ai/api/v1
7. Description: "Main OpenRouter key for multi-model access"
8. Test the connection - RunForge will verify it works
9. Save the configuration

Step 4: Configure Your Application

TypeScript/JavaScript Setup

Install the OpenAI SDK (OpenRouter is compatible):

npm install openai @runforge/sdk

Set up environment variables (.env file):

# OpenRouter Configuration
OPENROUTER_API_KEY=your-openrouter-key-here
OPENROUTER_BASE_URL=https://openrouter.ai/api/v1

# RunForge Configuration
RUNFORGE_API_KEY=your-runforge-ingest-key-here
RUNFORGE_PROJECT_ID=your-project-id-here
RUNFORGE_ENDPOINT=http://localhost:3000/api/ingest

Basic usage example:

import OpenAI from 'openai';
import { RunForge } from '@runforge/sdk';

// Configure for OpenRouter
const openrouter = new OpenAI({
  apiKey: process.env.OPENROUTER_API_KEY!,
  baseURL: 'https://openrouter.ai/api/v1',
  defaultHeaders: {
    'HTTP-Referer': 'https://yourapp.com', // Optional: for rankings
    'X-Title': 'Your App Name' // Optional: for rankings
  }
});

const runforge = new RunForge({
  apiKey: process.env.RUNFORGE_API_KEY!,
  projectId: process.env.RUNFORGE_PROJECT_ID!,
  endpoint: process.env.RUNFORGE_ENDPOINT!
});

// Make a tracked call through OpenRouter
async function generateResponse(userMessage: string) {
  const result = await runforge.track(
    { 
      experiment: 'openrouter-test',
      provider: 'openrouter'
    },
    () => openrouter.chat.completions.create({
      model: 'anthropic/claude-3-haiku', // Note the provider/model format
      messages: [
        { role: 'system', content: 'You are a helpful assistant.' },
        { role: 'user', content: userMessage }
      ],
      temperature: 0.7,
      max_tokens: 500
    })
  );

  return result.choices[0].message.content;
}

Python Setup

Install required packages:

pip install openai runforge

Set up environment variables:

export OPENROUTER_API_KEY=your-openrouter-key-here
export RUNFORGE_API_KEY=your-runforge-ingest-key-here
export RUNFORGE_PROJECT_ID=your-project-id-here  
export RUNFORGE_ENDPOINT=http://localhost:3000/api/ingest

Basic usage example:

import os
from openai import OpenAI
from runforge import RunForge

# Configure for OpenRouter
client = OpenAI(
    api_key=os.environ['OPENROUTER_API_KEY'],
    base_url='https://openrouter.ai/api/v1'
)

rf = RunForge(
    api_key=os.environ['RUNFORGE_API_KEY'],
    project_id=os.environ['RUNFORGE_PROJECT_ID'],
    endpoint=os.environ['RUNFORGE_ENDPOINT']
)

def generate_response(user_message):
    result = rf.track(
        {
            "experiment": "openrouter-test",
            "provider": "openrouter" 
        },
        lambda: client.chat.completions.create(
            model="anthropic/claude-3-haiku",  # provider/model format
            messages=[
                {"role": "system", "content": "You are a helpful assistant."},
                {"role": "user", "content": user_message}
            ],
            temperature=0.7,
            max_tokens=500
        )
    )

    return result.choices[0].message.content

Step 5: Understanding OpenRouter Models

Model Naming Convention

OpenRouter uses the format provider/model-name: - openai/gpt-4o-mini - OpenAI's GPT-4o-mini - anthropic/claude-3-haiku - Anthropic's Claude 3 Haiku - meta-llama/llama-3.1-8b-instruct - Meta's Llama model - google/gemini-pro - Google's Gemini Pro

Popular Models and Use Cases

Budget-Friendly Options

anthropic/claude-3-haiku - Cost: ~$0.25/1M input, ~$1.25/1M output tokens - Speed: Very fast (~1 second) - Use for: Simple tasks, high-volume applications - Quality: Good for most basic needs

openai/gpt-4o-mini
- Cost: ~$0.15/1M input, ~$0.60/1M output tokens - Speed: Very fast (~1 second) - Use for: Chatbots, content generation - Quality: Excellent value for money

Balanced Performance

anthropic/claude-3-sonnet - Cost: ~$3/1M input, ~$15/1M output tokens - Speed: Fast (~2 seconds)
- Use for: Complex analysis, creative tasks - Quality: High quality reasoning

openai/gpt-4o - Cost: ~$2.50/1M input, ~$10/1M output tokens - Speed: Fast (~1.5 seconds) - Use for: Most production applications - Quality: Excellent balance of cost and performance

Premium Options

anthropic/claude-3-opus - Cost: ~$15/1M input, ~$75/1M output tokens - Speed: Moderate (~3 seconds) - Use for: Complex reasoning, research, creative writing - Quality: Top tier, very expensive

openai/gpt-4-turbo - Cost: ~$10/1M input, ~$30/1M output tokens - Speed: Moderate (~2.5 seconds)
- Use for: Complex tasks where quality matters most - Quality: Excellent, but costly

Finding Available Models

Check openrouter.ai/models for: - Complete model list with current pricing - Model capabilities and context lengths
- Performance benchmarks - Real-time availability status

Step 6: Test Your Setup

Make Your First Call

Run your application code. You should see:

In your application: - Normal AI response from the chosen model - No errors about authentication or API access

In RunForge dashboard (within 30 seconds): - New request logged with correct model name - Accurate cost data (OpenRouter provides exact costs) - Response time recorded
- Success status showing

In OpenRouter dashboard: - Usage showing in your account - Credits deducted from your balance

Troubleshooting Common Issues

"Invalid API key" errors: 1. Double-check your OpenRouter API key 2. Ensure you're using the correct base URL 3. Verify your key has sufficient credits

Model not found errors: 1. Check the exact model name on openrouter.ai/models 2. Some models may be temporarily unavailable 3. Try a different model to test your setup

No data in RunForge: 1. Verify your RunForge configuration is correct 2. Check that you're using the right project ID 3. Make sure RunForge endpoint is accessible

OpenRouter Advantages

Exact Cost Tracking

Unlike other providers, OpenRouter returns the exact cost of each request in the response. This means: - Perfect cost accuracy in RunForge dashboards - No estimation errors from server-side pricing calculations - Real-time cost visibility as you make requests - Precise budget tracking for financial planning

Model Comparison Made Easy

Test different models with identical code:

// Test different models with same prompt
const models = [
  'anthropic/claude-3-haiku',
  'openai/gpt-4o-mini', 
  'meta-llama/llama-3.1-8b-instruct'
];

for (const model of models) {
  const result = await runforge.track(
    { experiment: 'model-comparison', variant: model },
    () => openrouter.chat.completions.create({
      model: model,
      messages: messages
    })
  );

  console.log(`${model}: $${result.cost}, ${result.latency}ms`);
}

Fallback Strategy

Handle model outages gracefully:

const fallbackModels = [
  'openai/gpt-4o-mini',     // Primary choice
  'anthropic/claude-3-haiku', // Backup if OpenAI is down
  'meta-llama/llama-3.1-8b-instruct' // Final fallback
];

async function robustGeneration(messages: any[]) {
  for (const model of fallbackModels) {
    try {
      return await runforge.track(
        { experiment: 'robust-generation', variant: model },
        () => openrouter.chat.completions.create({
          model: model,
          messages: messages
        })
      );
    } catch (error) {
      console.log(`${model} failed, trying next...`);
      continue;
    }
  }
  throw new Error('All models failed');
}

Cost Optimization Strategies

Smart Model Selection

• Start with cheapest: Try anthropic/claude-3-haiku for simple tasks
• Move up gradually: Test openai/gpt-4o-mini if quality isn't sufficient
• Use premium sparingly: Save claude-3-opus for only the most complex tasks

Bulk Operations

Process multiple items in one request when possible:

const batchPrompt = `Please process these 3 customer emails and respond to each:

Email 1: ${email1}
Email 2: ${email2}  
Email 3: ${email3}

Format: Email 1 Response: ..., Email 2 Response: ..., etc.`;

Token Optimization

• Shorter prompts: Be concise but clear
• Smart context: Only include relevant information
• Response limits: Set appropriate max_tokens

Monitoring and Alerts

Key Metrics for OpenRouter

• Credit balance: Don't let it hit zero
• Cost per successful request: Track efficiency
• Model availability: Some models have downtime
• Response quality: Cheaper models may need quality checks

Recommended Alerts

1. Low credit balance: Alert when under $10 remaining
2. High daily spend: More than 2x normal usage
3. Model failures: High error rates with specific models
4. Quality degradation: If using automated quality checks

Advanced Features

Custom Headers

Improve your experience with optional headers:

const openrouter = new OpenAI({
  apiKey: process.env.OPENROUTER_API_KEY!,
  baseURL: 'https://openrouter.ai/api/v1',
  defaultHeaders: {
    'HTTP-Referer': 'https://yourapp.com', // For model rankings
    'X-Title': 'Your App Name', // For model rankings  
    'X-Description': 'Brief description of your use case'
  }
});

Model Preferences

Some models have variants - specify preferences:

const result = await openrouter.chat.completions.create({
  model: 'anthropic/claude-3-haiku',
  messages: messages,
  // Additional OpenRouter-specific parameters
  transforms: ['middle-out'], // Optional: response transformations
  models: ['anthropic/claude-3-haiku', 'openai/gpt-4o-mini'], // Fallback list
  route: 'fallback' // Use fallback routing
});

Usage Analytics

OpenRouter provides detailed usage data: - Per-model costs and request counts
- Response time distributions - Error rates by model - Geographic routing information

Next Steps

• Set up alerts to monitor OpenRouter usage and credits
• Learn testing strategies to compare models efficiently
• Compare with other providers to optimize your model mix
• Explore use cases for specific OpenRouter optimization techniques