diff --git a/runtime-analysis/AGENT_RUNTIME_ANALYSIS.md b/runtime-analysis/AGENT_RUNTIME_ANALYSIS.md new file mode 100644 index 0000000..a14d3af --- /dev/null +++ b/runtime-analysis/AGENT_RUNTIME_ANALYSIS.md @@ -0,0 +1,345 @@ +# Agent Runtime Analysis Report + +## Overview + +| Component | Description | Status | +|-----------|-------------|--------| +| MiniMax Connection | LLM Provider Integration | ✅ Implemented | +| Output Contracts | Response Schema Validation | ✅ Implemented | +| Retry Mechanism | Error Recovery Strategy | ✅ Implemented | +| Observation Points | Telemetry & Monitoring | ✅ Implemented | + +--- + +## 1. MiniMax Connection Analysis + +### Architecture + +``` +┌─────────────────────────────────────────────────────────────┐ +│ Agent Runtime │ +├─────────────────────────────────────────────────────────────┤ +│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ +│ │ Client │───▶│ Connector │───▶│ MiniMax │ │ +│ │ Layer │ │ Pool │ │ API │ │ +│ └─────────────┘ └─────────────┘ └─────────────┘ │ +│ │ │ │ +│ ▼ ▼ │ +│ ┌─────────────┐ ┌─────────────┐ │ +│ │ Request │ │ Connection │ │ +│ │ Builder │ │ Health │ │ +│ └─────────────┘ └─────────────┘ │ +└─────────────────────────────────────────────────────────────┘ +``` + +### Connection Configuration + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `api_endpoint` | String | `https://api.minimax.chat` | MiniMax API base URL | +| `timeout` | Integer | 30000 | Connection timeout (ms) | +| `max_connections` | Integer | 100 | Connection pool size | +| `keep_alive` | Boolean | true | Connection reuse | +| `retry_count` | Integer | 3 | Max retry attempts | + +### Request Flow + +1. **Request Building**: Construct API payload with model parameters +2. **Connection Acquisition**: Get connection from pool +3. **Request Execution**: Send HTTP POST to MiniMax API +4. **Response Handling**: Parse JSON response stream +5. **Connection Release**: Return connection to pool + +### Error Handling + +| Error Type | HTTP Code | Action | +|------------|-----------|--------| +| `ConnectionTimeout` | 408 | Retry with backoff | +| `RateLimitExceeded` | 429 | Wait and retry | +| `ServerError` | 5xx | Retry with backoff | +| `AuthenticationError` | 401 | Fail fast, alert | +| `InvalidRequest` | 400 | Fail fast, log details | + +--- + +## 2. Output Contracts Analysis + +### Contract Schema + +```json +{ + "output_contract": { + "version": "1.0", + "required_fields": [ + "content", + "model", + "usage", + "finish_reason" + ], + "optional_fields": [ + "tool_calls", + "citations", + "metadata" + ], + "validation_rules": { + "content": { + "type": "string", + "min_length": 0, + "max_length": 128000 + }, + "usage": { + "type": "object", + "fields": ["prompt_tokens", "completion_tokens", "total_tokens"] + }, + "finish_reason": { + "type": "enum", + "values": ["stop", "length", "content_filter", "tool_calls"] + } + } + } +} +``` + +### Validation Pipeline + +``` +Raw Response → Schema Validation → Type Coercion → Contract Emit + │ │ │ │ + ▼ ▼ ▼ ▼ + Parse JSON Check Required Normalize Types Emit Valid + or Stream Fields Exist to Contract Output +``` + +### Contract Enforcement Points + +| Stage | Validation | Action on Failure | +|-------|------------|-------------------| +| Pre-send | Request schema | Reject before API call | +| Post-receive | Response schema | Retry or fallback | +| Stream chunk | Partial schema | Buffer until complete | +| Final | Full contract | Return or error | + +--- + +## 3. Retry Mechanism Analysis + +### Retry Strategy + +``` +┌──────────────────────────────────────────────────────────────┐ +│ Retry Decision Flow │ +├──────────────────────────────────────────────────────────────┤ +│ │ +│ Request Failed │ +│ │ │ +│ ▼ │ +│ ┌─────────────┐ │ +│ │ Is Retryable│───No──▶ Return Error │ +│ └─────────────┘ │ +│ │Yes │ +│ ▼ │ +│ ┌─────────────┐ │ +│ │ Retry Count │───Max──▶ Return Error │ +│ │ < Max? │ │ +│ └─────────────┘ │ +│ │Yes │ +│ ▼ │ +│ ┌─────────────┐ │ +│ │ Calculate │ │ +│ │ Backoff │ │ +│ └─────────────┘ │ +│ │ │ +│ ▼ │ +│ ┌─────────────┐ │ +│ │ Wait & │ │ +│ │ Retry │ │ +│ └─────────────┘ │ +│ │ +└──────────────────────────────────────────────────────────────┘ +``` + +### Backoff Configuration + +| Parameter | Value | Formula | +|-----------|-------|---------| +| `initial_delay` | 1000ms | Base delay | +| `max_delay` | 30000ms | Cap | +| `multiplier` | 2.0 | Exponential factor | +| `jitter` | ±10% | Random variation | + +### Retryable Errors + +| Error Category | Examples | Retryable | +|----------------|----------|-----------| +| Network | Timeout, DNS failure | ✅ | +| Server 5xx | Internal error, overload | ✅ | +| Rate Limit | 429 Too Many Requests | ✅ | +| Client 4xx | Bad request, auth | ❌ | +| Validation | Schema mismatch | ❌ | + +### Circuit Breaker + +| Metric | Threshold | State | +|--------|-----------|-------| +| Failure Rate | >50% in 10s | Open | +| Success Rate | >90% in 10s | Closed | +| Half-Open Requests | 5 | Test | + +--- + +## 4. Observation Points Analysis + +### Telemetry Architecture + +``` +┌─────────────────────────────────────────────────────────────┐ +│ Observation Pipeline │ +├─────────────────────────────────────────────────────────────┤ +│ │ +│ Application Code │ +│ │ │ +│ ▼ │ +│ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ │ +│ │ Traces │───▶│ Metrics │───▶│ Logs │───▶│ Alerts │ │ +│ └─────────┘ └─────────┘ └─────────┘ └─────────┘ │ +│ │ │ │ │ │ +│ ▼ ▼ ▼ ▼ │ +│ Span Context Counters/Gauges Structured Threshold │ +│ Parent/Child Histograms Events Based │ +│ │ +└─────────────────────────────────────────────────────────────┘ +``` + +### Key Observation Points + +| Point | Metric | Labels | Interval | +|-------|--------|--------|----------| +| `request.start` | Counter | model, endpoint | Per request | +| `request.duration` | Histogram | model, status | Per request | +| `connection.acquired` | Gauge | pool_name | 10s | +| `connection.wait_time` | Histogram | pool_name | Per acquire | +| `retry.attempt` | Counter | attempt_num, error_type | Per retry | +| `contract.validation` | Counter | result (pass/fail) | Per validation | +| `circuit.state` | Gauge | circuit_name | 5s | + +### Span Structure + +``` +Parent Span: agent-runtime-request +├── Span: connection-acquire +├── Span: request-build +├── Span: api-call +│ ├── Span: retry-attempt-1 +│ ├── Span: retry-attempt-2 +│ └── Span: retry-attempt-3 +├── Span: response-parse +├── Span: contract-validate +└── Span: response-emit +``` + +### Log Events + +| Event | Level | Fields | +|-------|-------|--------| +| `request_initiated` | INFO | request_id, model, timestamp | +| `request_completed` | INFO | request_id, duration, tokens | +| `retry_triggered` | WARN | request_id, attempt, error | +| `circuit_opened` | ERROR | circuit, failure_rate | +| `contract_violation` | ERROR | request_id, violations[] | + +--- + +## 5. Integration Summary + +### Component Interaction Matrix + +| From \ To | MiniMax | Contracts | Retry | Observability | +|-----------|---------|-----------|-------|---------------| +| **MiniMax** | - | Provides raw response | Reports errors | Emits spans | +| **Contracts** | Validates input | - | Triggers retry on fail | Logs violations | +| **Retry** | Wraps calls | May re-validate | - | Counts retries | +| **Observability** | Instruments | Instruments | Instruments | - | + +### Data Flow + +``` +Client Request + │ + ▼ +┌─────────────────┐ +│ Observation: │ ← Span start, log request_initiated +│ Start │ +└─────────────────┘ + │ + ▼ +┌─────────────────┐ +│ Contract: │ ← Validate request schema +│ Pre-validate │ +└─────────────────┘ + │ + ▼ +┌─────────────────┐ +│ MiniMax: │ ← Acquire connection, send request +│ Execute │ +└─────────────────┘ + │ + ├──┐ Retry Loop + │ │ + │ ▼ + │ ┌─────────────────┐ + │ │ Observation: │ ← Span retry-attempt-N + │ │ Retry │ + │ └─────────────────┘ + │ │ + │ ▼ + │ ┌─────────────────┐ + │ │ MiniMax: │ + │ │ Retry Request │ + │ └─────────────────┘ + │ │ + └──◀┘ + │ + ▼ +┌─────────────────┐ +│ Contract: │ ← Validate response schema +│ Post-validate │ +└─────────────────┘ + │ + ▼ +┌─────────────────┐ +│ Observation: │ ← Span end, log request_completed +│ Complete │ +└─────────────────┘ + │ + ▼ +Client Response +``` + +--- + +## 6. Recommendations + +### MiniMax Connection +- [ ] Add connection pool health check endpoint +- [ ] Implement request batching for efficiency +- [ ] Add streaming response compression + +### Output Contracts +- [ ] Support contract versioning for backward compatibility +- [ ] Add partial validation mode for streaming +- [ ] Implement contract migration tooling + +### Retry Mechanism +- [ ] Add adaptive retry based on error patterns +- [ ] Implement request deduplication +- [ ] Add retry budget tracking + +### Observation Points +- [ ] Add P99 latency tracking +- [ ] Implement correlation ID propagation +- [ ] Add cost tracking per request + +--- + +*Generated: 2026-07-10* +*Version: 1.0*