Getting Started with RunForge¶

What You'll Learn¶

By the end of this guide, you'll have your first RunForge project set up and tracking your AI usage, giving you clear insights into costs and performance.

What is RunForge?¶

RunForge helps you monitor and optimize your AI applications by tracking: - Costs - How much you're spending on different AI models - Performance - How fast your AI responses are - Usage patterns - Which models and features you use most - Reliability - Success rates and error tracking

Privacy First: RunForge never sees your actual prompts or AI responses - only usage statistics and performance data.

Before You Start¶

✅ A computer with internet access
✅ An AI provider account (OpenAI, OpenRouter, or similar)
✅ 10-15 minutes of time
✅ Basic comfort following step-by-step instructions

Step 1: Set Up Your RunForge Account¶

Create Your Account¶

Visit your RunForge installation (typically http://localhost:3000 for local setup)
Click Sign Up or Get Started
Choose to sign up with email or your preferred method
Complete the account verification process

Create Your Organization¶

After signing in, you'll be prompted to create an organization
Enter your organization name (this can be your company, team, or personal name)
Click Create Organization

Your organization helps keep your projects organized and separate from other users.

Step 2: Create Your First Project¶

Why Projects Matter¶

Projects help you organize different applications or experiments. For example: - Production Chatbot - Your live customer service bot - Content Generation - Your blog writing assistant
- Code Helper - Your programming assistant

Create a Project¶

Click the Settings tab in the top navigation
Select Projects from the sidebar
Click the Create New Project button
Fill out the project details:
Project Name: Give it a clear name like "My Chatbot" or "Content Assistant"
Description: Brief description of what this project does
Click Create Project

Step 3: Set Up Your API Keys¶

What Are API Keys?¶

API keys are like passwords that let RunForge connect to your AI provider (like OpenAI) securely. RunForge encrypts and protects these keys - they never leave your system unprotected.

Add Your First API Key¶

In your project settings, click API Keys in the sidebar
Click Add API Key
Select your provider (start with the one you have an account with):
OpenAI - If you use ChatGPT, GPT-4, etc.
OpenRouter - If you use multiple AI models through one service
Anthropic - If you use Claude
Enter your API key from your provider
Give it a name like "My OpenAI Key"
Click Save

Getting Your Provider API Key¶

For OpenAI: 1. Go to platform.openai.com 2. Sign in and go to API Keys 3. Click Create new secret key 4. Copy the key immediately (you won't see it again)

For OpenRouter: 1. Go to openrouter.ai
2. Sign in and go to Keys 3. Click Create Key 4. Copy the key

Step 4: Get Your RunForge Ingest Key¶

What is an Ingest Key?¶

This is a special key that your applications use to send data to RunForge. Think of it as your project's unique identifier.

Find Your Ingest Key¶

In your project settings, look for Ingest Keys or API Keys section
You should see an automatically generated key
Click Copy to copy it to your clipboard
Save this key somewhere safe - you'll need it for the next step

Step 5: Start Tracking Your First AI Call¶

Option A: Using TypeScript/JavaScript¶

If you're using Node.js or a web application:

Install the RunForge SDK:
```
npm install @runforge/sdk
```

Add this code to your application:

import OpenAI from 'openai'
import { RunForge } from '@runforge/sdk'

// Set up your AI provider
const openai = new OpenAI({ 
  apiKey: 'your-openai-api-key-here' 
})

// Set up RunForge tracking
const runforge = new RunForge({
  apiKey: 'your-runforge-ingest-key-here',
  endpoint: 'http://localhost:3000/api/ingest',  // Your RunForge URL
  projectId: 'your-project-id-here'
})

// Make a tracked AI call
const result = await runforge.track(
  { experiment: 'my-first-test' },
  () => openai.chat.completions.create({
    model: 'gpt-4o-mini',
    messages: [{ role: 'user', content: 'Hello, AI!' }]
  })
)

console.log('AI Response:', result.choices[0].message.content)

Option B: Using Python¶

If you're using Python:

Install the RunForge SDK:
```
pip install runforge
```

Add this code to your application:

from runforge import RunForge
from openai import OpenAI
import os

# Set up your AI provider
client = OpenAI(api_key="your-openai-api-key-here")

# Set up RunForge tracking
rf = RunForge(
    api_key="your-runforge-ingest-key-here",
    endpoint="http://localhost:3000/api/ingest",  # Your RunForge URL
    project_id="your-project-id-here"
)

# Make a tracked AI call
result = rf.track(
    {"experiment": "my-first-test"},
    lambda: client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": "Hello, AI!"}]
    )
)

print("AI Response:", result.choices[0].message.content)

Replace the Placeholder Values¶

your-openai-api-key-here: Your actual OpenAI API key
your-runforge-ingest-key-here: The ingest key you copied in Step 4
your-project-id-here: Your project ID (found in project settings)

Step 6: See Your Data Live¶

View Your Dashboard¶

Go back to your RunForge dashboard (click Dashboard in the navigation)
Run your code with the tracking enabled
Within seconds, you should see:
Total Cost: How much that AI call cost
Response Time: How long it took
Model Used: Which AI model processed your request
Success Rate: Whether it worked properly

Understanding Your Metrics¶

Cost: Exact amount spent on AI calls (in your currency)
Latency: Response time in milliseconds (lower is faster)
Tokens: Number of words/tokens processed (input + output)
Error Rate: Percentage of failed requests (lower is better)

What's Next?¶

Congratulations! You've successfully: - ✅ Created your RunForge account and project - ✅ Connected your AI provider - ✅ Made your first tracked AI call - ✅ Viewed real-time usage data

Explore More Features¶

Understanding Your Dashboard - Get the most from your metrics
Managing Multiple Projects - Organize different applications
Setting Up Alerts - Get notified of issues or cost spikes
Provider Setup Guides - Connect more AI services

Common Next Steps¶

Set up monitoring for your production applications
Create separate projects for development and production
Configure cost alerts to avoid surprises
Explore different AI models to optimize costs

Need Help?¶

Questions? Check our Troubleshooting Guide
Specific Use Cases? Browse our Use Case Examples
Technical Issues? See our detailed Provider Setup Guides

Privacy Reminder¶

RunForge only tracks usage statistics - never your actual conversations or prompts. Your AI interactions remain private between you and your AI provider.