Budget & Cost Management
Overview
The budget and cost management system provides comprehensive cost tracking, budget controls, and cost optimization for LLM usage and agent operations.
Key Features
- Cost Tracking: Real-time cost tracking for LLM usage with per-request, per-session, and per-agent aggregation
- Token Counting: Accurate token counting for input/output with model-specific tokenizers
- Cost Analytics: Detailed cost analytics with breakdowns by agent, session, model, and time period
- Budget Management: Per-agent and per-session budget controls with hard and soft limits
- Budget Alerts: Configurable alerts for budget thresholds via email and webhooks
- Cost Forecasting: Predictive cost forecasting based on usage patterns
- Cost Optimization: Recommendations for cost optimization based on usage patterns
Cost Tracking
Real-time cost tracking monitors LLM usage and calculates costs based on token usage and model pricing.
Cost Aggregation
- Per-Request: Cost per individual request
- Per-Session: Total cost per agent session
- Per-Agent: Total cost per agent across all sessions
- Per-Model: Cost breakdown by LLM model
- Time-Based: Cost aggregation by time period (hourly, daily, monthly)
Cost Calculation
Costs are calculated using model-specific pricing, token counts, and usage patterns, providing accurate cost tracking.
Token Counting
Accurate token counting uses model-specific tokenizers to count input and output tokens, enabling precise cost calculation.
Tokenizer Support
Token counting supports multiple tokenizers for different models, ensuring accurate token counts across all supported LLMs.
Token Tracking
Token usage is tracked for both input and output, providing detailed insights into token consumption patterns.
Budget Management
Budget management provides per-agent and per-session budget controls with hard and soft limits for cost control.
Budget Types
- Hard Limits: Strict budget limits that prevent further spending
- Soft Limits: Warning thresholds that trigger alerts but allow continued spending
- Per-Agent Budgets: Budget limits for individual agents
- Per-Session Budgets: Budget limits for individual sessions
Budget Enforcement
Budget limits are enforced at runtime, preventing agents from exceeding allocated budgets.
Cost Analytics
Cost analytics provide detailed insights into cost patterns, enabling informed decisions about cost optimization.
Analytics Dimensions
- By Agent: Cost breakdown by agent
- By Session: Cost breakdown by session
- By Model: Cost breakdown by LLM model
- By Time Period: Cost trends over time
- By Operation Type: Cost breakdown by operation type
Analytics Reports
Analytics reports provide visualizations and insights into cost patterns, trends, and optimization opportunities.
Budget Alerts
Budget alerts notify users when budget thresholds are reached, enabling proactive cost management.
Alert Types
- Threshold Alerts: Alerts when budget thresholds are reached
- Projection Alerts: Alerts based on projected spending
- Anomaly Alerts: Alerts for unusual spending patterns
Notification Channels
Alerts are sent via email and webhooks, enabling integration with external notification systems.
Cost Optimization
Cost optimization provides recommendations for reducing costs based on usage patterns and best practices.
Optimization Strategies
- Model Selection: Recommendations for more cost-effective models
- Token Optimization: Suggestions for reducing token usage
- Batch Processing: Recommendations for batch operations
- Caching: Suggestions for caching frequently used results
Optimization Reports
Optimization reports provide actionable recommendations with estimated cost savings.
API Reference
Budget and cost management is accessible through the NeuronAgent REST API for managing budgets and viewing cost analytics.
Budget Endpoints
GET /api/v1/costs- Get cost analyticsGET /api/v1/budgets- List budgetsPOST /api/v1/budgets- Create a budgetPUT /api/v1/budgets/:id- Update a budgetGET /api/v1/agents/:id/costs- Get agent costsGET /api/v1/sessions/:id/costs- Get session costs