{ "interval": { "intervalStart": "2025-08-17T00:00:00.000Z", "intervalEnd": "2025-08-24T00:00:00.000Z", "intervalType": "week" }, "repository": "elizaos/eliza", "overview": "From 2025-08-17 to 2025-08-24, elizaos/eliza had 17 new PRs (15 merged), 4 new issues, and 15 active contributors.", "topIssues": [ { "id": "I_kwDOMT5cIs7GTJvn", "title": "Implement Run Orchestration & Isolation", "author": "linear", "number": 5782, "repository": "elizaos/eliza", "body": "### **Ticket: Implement Run Orchestration & Isolation**\n\n**ID:** `FEAT-126` (Example ID)\n\n**Epic:** `Scenario Matrix Runner`\n\n**Tags:** `cli`, `scenario-testing`, `feature`, `orchestration`\n\n**Estimated Story Points:** `13`\n\n**Dependencies:** `FEAT-123` (Matrix Configuration Schema), `FEAT-124` (CLI Command), `FEAT-125` (Parameter Override System)\n\n#### **1. Title**\n\n`feat(cli): Implement matrix run orchestration with complete isolation and cleanup`\n\n#### **2. Description**\n\nThis ticket implements the core execution engine for the Scenario Matrix Runner. It orchestrates the execution of all matrix combinations, ensures complete isolation between runs, and manages cleanup of artifacts. This is the most complex ticket in Epic 1 as it ties together all previous work and handles the actual scenario execution.\n\nThe orchestration system must be robust, handling failures gracefully while maintaining data integrity and system cleanliness. Each run must be completely isolated to prevent interference between test executions.\n\n#### **3. Acceptance Criteria**\n\n1. **Matrix Execution Loop:**\n * Implement `executeMatrixRuns(config: MatrixConfig, combinations: MatrixCombination[]): Promise` that:\n * Iterates through all generated matrix combinations\n * Executes each combination the specified number of times (`runs_per_combination`)\n * Maintains execution order and provides progress feedback\n * Handles individual run failures without stopping the entire matrix execution\n2. **Run Isolation System:**\n * Each scenario run must be completely isolated with:\n * Separate temporary directories for each run (e.g., `./temp/matrix-run-001/`)\n * Independent database instances (using unique database paths)\n * Isolated log files and output artifacts\n * Clean environment variable state between runs\n * Implement `createIsolatedEnvironment(runId: string): Promise` that sets up the isolated environment\n * Implement `cleanupIsolatedEnvironment(context: IsolationContext): Promise` that removes all artifacts\n3. **Scenario Override Integration:**\n * Use the parameter override system from ticket 1.3 to generate unique scenario configurations\n * Write the modified scenario to the isolated environment as a temporary `.scenario.yaml` file\n * Pass the temporary scenario file path to the existing scenario runner\n4. **Progress Tracking and Logging:**\n * Real-time progress updates: `\"Executing run 15 of 36 (Combination 5/12, Run 3/3)\"`\n * Estimated time remaining based on average run duration\n * Clear indication of which parameter combination is currently running\n * Summary statistics after each combination completes\n5. **Data Collection and Storage:**\n * Capture comprehensive data for each run:\n * Execution start and end timestamps\n * Parameter combination used\n * Scenario execution results (pass/fail, evaluation details)\n * Performance metrics (execution time, memory usage)\n * Any error messages or exceptions\n * Store results in structured JSON format in the output directory\n * Implement `saveRunResult(runId: string, result: ScenarioRunResult, outputDir: string): Promise`\n6. **Error Handling and Recovery:**\n * Graceful handling of individual run failures without stopping the matrix\n * Timeout handling for runs that exceed reasonable execution time\n * Resource cleanup even when runs fail or are interrupted\n * Detailed error logging with context about which combination failed\n * Option to continue or abort on first failure\n7. **Resource Management:**\n * Monitor system resources (memory, disk space) during execution\n * Implement safeguards against resource exhaustion\n * Parallel execution limits to prevent system overload\n * Cleanup of temporary files and databases between runs\n8. **Output Directory Structure:**\n\n ```\n output/matrix-/\n ├── config.yaml # Copy of the matrix configuration\n ├── summary.json # High-level execution summary\n ├── runs/\n │ ├── run-001.json # Individual run results\n │ ├── run-002.json\n │ └── ...\n └── logs/\n ├── matrix-execution.log # Overall execution log\n ├── run-001.log # Individual run logs\n └── ...\n ```\n\n#### **4. Technical Implementation Details**\n\n**File Structure:**\n\n```\npackages/cli/src/commands/scenario/src/\n├── matrix-orchestrator.ts # Main orchestration logic\n├── run-isolation.ts # Environment isolation utilities\n├── progress-tracker.ts # Progress reporting and ETA calculation\n├── resource-monitor.ts # System resource monitoring\n└── __tests__/\n ├── matrix-orchestrator.test.ts\n ├── run-isolation.test.ts\n └── integration/\n └── full-matrix.test.ts\n```\n\n**Core Types:**\n\n```typescript\ninterface IsolationContext {\n runId: string;\n tempDir: string;\n dbPath: string;\n logPath: string;\n scenarioPath: string;\n cleanup: () => Promise;\n}\ninterface MatrixRunResult {\n runId: string;\n combinationId: string;\n parameters: Record;\n startTime: Date;\n endTime: Date;\n duration: number;\n success: boolean;\n scenarioResult?: any;\n error?: string;\n metrics: {\n memoryUsage: number;\n diskUsage: number;\n tokenCount?: number;\n };\n}\ninterface MatrixExecutionSummary {\n totalRuns: number;\n successfulRuns: number;\n failedRuns: number;\n totalDuration: number;\n averageRunTime: number;\n combinations: CombinationSummary[];\n}\n```\n\n#### **5. Integration with Existing Scenario Runner**\n\n* Reuse existing scenario execution logic from `packages/cli/src/commands/scenario/index.ts`\n* Modify the existing runner to accept an output directory parameter for isolated execution\n* Ensure the existing scenario runner can work with temporary scenario files\n* Maintain compatibility with all existing scenario features and evaluators\n\n#### **6. Performance Considerations**\n\n* Implement configurable parallel execution (default: 1, max: CPU cores)\n* Efficient cleanup that doesn't block subsequent runs\n* Memory-efficient handling of large numbers of runs\n* Disk space monitoring to prevent system exhaustion\n\n#### **7. Testing Requirements**\n\n**Unit Tests:**\n\n* Isolation context creation and cleanup\n* Progress tracking accuracy\n* Resource monitoring functionality\n* Error handling for various failure scenarios\n\n**Integration Tests:**\n\n* Full matrix execution with real scenario files\n* Cleanup verification (no leftover artifacts)\n* Resource exhaustion handling\n* Interruption and recovery testing\n\n**Load Tests:**\n\n* Large matrix configurations (100+ combinations)\n* Long-running scenarios\n* Memory and disk usage validation\n\n#### **8. Error Handling Examples**\n\n```\n✗ Run 15 failed: Timeout after 300 seconds\n Combination: model=gpt-4, prompt=variant-3\n Cleaning up isolated environment...\n⚠ System memory usage high (85%), reducing parallel execution\n Running 1 scenario at a time instead of 4\n✗ Matrix execution aborted: Insufficient disk space\n Required: 2.5 GB, Available: 1.2 GB\n Cleaned up 14 completed runs, 3 runs remaining\n```\n\n#### **9. Out of Scope**\n\n* Report generation functionality (Epic 3)\n* Advanced evaluation and data collection (Epic 2)\n* CLI improvements beyond basic progress reporting", "createdAt": "2025-08-16T05:34:34Z", "closedAt": "2025-08-19T15:59:43Z", "state": "CLOSED", "commentCount": 2 }, { "id": "I_kwDOMT5cIs7GTMUc", "title": "Implement Dynamic Report Rendering", "author": "linear", "number": 5789, "repository": "elizaos/eliza", "body": "### **Ticket: Implement Dynamic Report Rendering**\n\n**ID:** `FEAT-133` (Example ID)\n\n**Epic:** `Performance Reporting Dashboard`\n\n**Tags:** `cli`, `reporting`, `feature`, `javascript`, `visualization`\n\n**Estimated Story Points:** `8`\n\n**Dependencies:** `FEAT-131` (Data Aggregation), `FEAT-132` (HTML Template)\n\n#### **1. Title**\n\n`feat(cli): Implement dynamic data injection and rendering for the HTML Performance Report`\n\n#### **2. Description**\n\nThis ticket brings the Performance Report to life. It involves creating the logic to take the aggregated `report.json` data and inject it into the static `report_template.html`, producing a final, fully-rendered, and interactive HTML file.\n\nThe work will primarily be in modifying the `elizaos report generate` command to read both the data and the template, inject the data, and write the final output. This includes writing the client-side JavaScript (embedded within the template) that will be responsible for all DOM manipulation, chart generation, and rendering of dynamic tables and lists.\n\n#### **3. Acceptance Criteria**\n\n1. **File I/O in** `generate` command:\n * The `elizaos report generate` command is updated to:\n * Read the aggregated `report.json` file into memory.\n * Read the static `report_template.html` file into memory.\n * Inject the entire JSON data object into the `\n```\n\n#### **5. Testing Requirements**\n\n* **Integration Tests:**\n * Update the integration test for the `generate` command. After the command runs, the test should:\n * Read the output `performance_report.html`.\n * Use a tool like `jsdom` to parse the HTML and verify that the data has been correctly injected into the ``.\n5. **Self-Contained Asset:**\n * The final deliverable is a single `.html` file. All CSS and JavaScript must be embedded. No external network requests should be necessary to view the report's structure and styling (data will be injected later).\n\n#### **4. Technical Implementation Details**\n\n**File Structure:**\n\n```\npackages/cli/src/commands/report/src/\n└── assets/\n └── report_template.html # The new, self-contained HTML file\n```\n\n**Example HTML Placeholders:**\n\n```html\n\n

Summary

\n

Total Runs: [loading...]

\n\n

Capability Success Rates

\n\n\n

Detailed Run Results

\n\n \n \n \n \n \n \n \n \n \n \n \n
Run IDSuccessModelPrompt
\n\n\n```\n\n#### **5. Design Mockup / Wireframe**\n\n(A simple text-based wireframe should be included in the ticket to guide the developer)\n\n```\n+------------------------------------------------------+\n| Performance Report: GitHub Issue Analysis |\n| Generated: 2023-10-27 |\n+------------------------------------------------------+\n| SUMMARY |\n| Total Runs: 18 Success Rate: 66% Avg Time: 8.2s |\n+------------------------------------------------------+\n| CAPABILITY SUCCESS RATES |\n| [================\n| [=========== ] Formats Response (55%) |\n+------------------------------------------------------+\n| PERFORMANCE BY LLM MODEL |\n| gpt-4-turbo: 9/9 (100%) gpt-3.5-turbo: 3/9 (33%) |\n| [ Bar Chart comparing models on key metrics ] |\n+------------------------------------------------------+\n| COMMON ACTION TRAJECTORIES |\n| 1. THINK -> LIST_ISSUES -> REPLY (12 runs) |\n| 2. THINK -> SEARCH -> REPLY (6 runs) |\n+------------------------------------------------------+\n| DETAILED RUNS |\n| [ A filterable, sortable table of all 18 runs ] |\n+------------------------------------------------------+\n```\n\n#### **6. Testing Requirements**\n\n* **Manual Testing:** Open the final `report_template.html` file in multiple browsers (Chrome, Firefox, Safari) to ensure consistent rendering and responsiveness.\n* **Static Analysis:** Run an HTML validator and a linter on the file to ensure it is well-formed and follows best practices.\n\n#### **7. Out of Scope**\n\n* The logic for actually injecting the `ReportData` JSON into the template. This ticket is only about creating the static HTML shell.\n* The implementation of the `elizaos report generate` command itself.\n* The data aggregation logic.", "createdAt": "2025-08-16T05:50:44Z", "closedAt": "2025-08-18T05:15:20Z", "state": "CLOSED", "commentCount": 1 }, { "id": "I_kwDOMT5cIs7GTLk4", "title": "Implement `elizaos report generate` Command", "author": "linear", "number": 5787, "repository": "elizaos/eliza", "body": "### **Ticket: Implement** `elizaos report generate` Command\n\n**ID:** `FEAT-131` (Example ID)\n\n**Epic:** `Performance Reporting Dashboard`\n\n**Tags:** `cli`, `reporting`, `feature`, `data-analysis`\n\n**Estimated Story Points:** `8`\n\n**Dependencies:** `FEAT-130` (Centralize and Serialize Run Data)\n\n#### **1. Title**\n\n`feat(cli): Implement 'elizaos report generate' command for data aggregation and analysis`\n\n#### **2. Description**\n\nThis ticket introduces the user-facing entry point for the Performance Reporting Dashboard: the `elizaos report generate` command. The primary responsibility of this command is to ingest the raw JSON output from a Scenario Matrix run, process it, and perform the complex data aggregation required to generate an insightful report.\n\nThe command will read all individual `run-*.json` files, calculate high-level statistics, group results by the matrix parameters, and analyze agent trajectories. The final output of this command will be a single, structured `ReportData` object, which will serve as the complete data context for rendering the HTML report in a subsequent ticket.\n\nThis is a critical data processing step that transforms raw run data into meaningful, aggregated insights.\n\n#### **3. Acceptance Criteria**\n\n1. **CLI Command Registration:**\n * A new top-level command `report` is created.\n * Under `report`, a subcommand `generate` is registered.\n * The command accepts one required argument, ``, which is the path to the output directory of a matrix run.\n * It also accepts an optional `--output-path` flag to specify where to save the final `report.json` file. If not provided, it defaults to `/report.json`.\n2. **Data Ingestion and Validation:**\n * The command recursively finds and reads all `run-*.json` files within the ``.\n * It validates the structure of each JSON file against the `ScenarioRunResult` schema.\n * Malformed or incomplete files are gracefully skipped, and a warning is logged.\n * The command provides a clear error and exits if the input directory is not found or contains no valid run files.\n3. **Data Aggregation Logic:**\n * The core of the command is an `AnalysisEngine` that processes the array of `ScenarioRunResult` objects.\n * It must calculate **overall summary statistics**: total runs, average execution time, overall capability success rates, etc.\n * It must calculate **grouped statistics**: The engine must group the results by each matrix parameter and value (e.g., group by `character.llm.model`). For each group, it calculates the same summary statistics, allowing for direct comparison (e.g., success rate of `gpt-4` vs. `gpt-3.5`).\n * It must perform **trajectory analysis**: Aggregate all `trajectory` arrays to identify the most common action sequences and calculate the frequency of each path.\n4. **Structured** `ReportData` Output:\n * The `AnalysisEngine` produces a single, large `ReportData` object.\n * This object's schema is formally defined in TypeScript and contains all aggregated data in a clean, predictable structure, ready for a rendering engine.\n * The command serializes this `ReportData` object to a pretty-printed JSON file at the specified output path.\n\n#### **4. Technical Implementation Details**\n\n**File Structure:**\n\n```\npackages/cli/src/commands/report/\n├── index.ts # Registers the 'report' command\n├── generate.ts # Implements the 'generate' subcommand\n└── src/\n ├── analysis-engine.ts # Core data aggregation logic\n ├── report-schema.ts # Defines the 'ReportData' interface\n └── __tests__/\n └── analysis-engine.test.ts\n```\n\n`ReportData` Interface (High-Level Example):\n\n```typescript\ninterface ReportData {\n metadata: {\n report_generated_at: string;\n matrix_config: MatrixConfig; // From the original run\n };\n summary_stats: {\n total_runs: number;\n total_failed_runs: number;\n average_execution_time: number;\n capability_success_rates: Record; // { \"Formats Response\": 0.75 }\n };\n results_by_parameter: {\n [parameter_name: string]: { // e.g., \"character.llm.model\"\n [parameter_value: string]: ReportSummaryStats; // e.g., \"gpt-4-turbo\" has its own summary_stats\n };\n };\n common_trajectories: {\n sequence: string[]; // e.g., [\"THINK\", \"LIST_ISSUES\", \"REPLY\"]\n count: number;\n average_duration: number;\n }[];\n raw_results: ScenarioRunResult[]; // Include all original data for detailed drill-downs\n}\n```\n\n**Example CLI Usage:**\n\n```bash\n# Generate a report from a previous matrix run\nelizaos report generate ./output/matrix-20231027-1000/\n# Specify a different output location for the aggregated data\nelizaos report generate ./output/matrix-20231027-1000/ --output-path ./reports/latest-report.json\n```\n\n#### **5. Testing Requirements**\n\n* **Unit Tests:**\n * Extensive unit tests for the `AnalysisEngine` are critical. Provide it with a mock array of `ScenarioRunResult` objects and assert that the resulting `ReportData` object contains the correct aggregations, groupings, and statistical calculations.\n * Test edge cases like empty input, single-run input, and runs with missing data points.\n* **Integration Tests:**\n * Create a test that runs the `elizaos report generate` command on a pre-generated fixture directory containing a set of `run-*.json` files.\n * The test should verify that the command exits successfully and that the output `report.json` file is created and contains valid, non-empty data.\n\n#### **6. Out of Scope**\n\n* The design or implementation of the final HTML template. This ticket's final artifact is the `report.json` file, not a user-facing document.\n* The logic for rendering charts or any other UI components (handled in Ticket 3.2 and 3.3).\n* Any modification to the scenario running or data collection process; this command is strictly read-only on the matrix run output.", "createdAt": "2025-08-16T05:47:08Z", "closedAt": "2025-08-18T05:15:07Z", "state": "CLOSED", "commentCount": 1 }, { "id": "I_kwDOMT5cIs7GTLOh", "title": "Centralize and Serialize Run Data", "author": "linear", "number": 5786, "repository": "elizaos/eliza", "body": "### **Ticket: Centralize and Serialize Run Data**\n\n**ID:** `FEAT-130` (Example ID)\n\n**Epic:** `Advanced Evaluation & Data Collection`\n\n**Tags:** `cli`, `scenario-testing`, `feature`, `data-pipeline`\n\n**Estimated Story Points:** `5`\n\n**Dependencies:** `FEAT-126` (Run Orchestration), `FEAT-127` (Structured JSON Output), `FEAT-129` (Agent Trajectory Logging)\n\n#### **1. Title**\n\n`feat(cli): Centralize and serialize all scenario run data into a structured JSON output file`\n\n#### **2. Description**\n\nThis ticket focuses on the final step of the data collection phase: bringing together all the rich data gathered from a single scenario run and saving it to a well-defined, structured JSON file. This file will be the canonical source of truth for a single execution and will serve as the input for the Performance Reporting Dashboard (Epic 3).\n\nThe implementation will create a new service or utility responsible for aggregating data from various sources—the matrix configuration, the evaluation engine, the agent runtime's trajectory log, and performance timers—and ensuring it conforms to our master `ScenarioRunResult` schema before being written to disk.\n\n#### **3. Acceptance Criteria**\n\n1. `ScenarioRunResult` Schema:\n * A comprehensive `ScenarioRunResult` interface is finalized in `packages/cli/src/commands/scenario/src/schema.ts`.\n * This master interface must consolidate all data points from the previous tickets and include:\n * `run_id`: A unique identifier for the run.\n * `matrix_combination_id`: An identifier linking it to a specific set of parameters.\n * `parameters`: An object detailing the specific matrix configuration for this run (e.g., `{ \"character.llm.model\": \"gpt-4-turbo\" }`).\n * `metrics`: An object containing performance data (`execution_time_seconds`, `llm_calls`, `total_tokens`, etc.).\n * `evaluations`: The array of `EvaluationResult` objects from the `EvaluationEngine` (Ticket 2.1).\n * `trajectory`: The array of `TrajectoryStep` objects from the `AgentRuntime` (Ticket 2.3).\n * `final_agent_response`: The final text/object response from the agent to the user.\n * `error`: A field to store any fatal error message if the run failed unexpectedly.\n2. **Data Aggregation Utility:**\n * A new utility or class, e.g., `RunDataAggregator`, is created.\n * It will have methods to collect data from different parts of the system throughout a run's lifecycle.\n * It will have a final `buildResult()` method that assembles all the collected pieces into a single, validated `ScenarioRunResult` object.\n3. **File Serialization Logic:**\n * The `MatrixOrchestrator` (from Ticket 1.4) will use this new utility to generate the result for each run.\n * After a run completes (or fails), the orchestrator will call a function to serialize the `ScenarioRunResult` object to a JSON file.\n * The output file will be named according to a consistent pattern (e.g., `run-.json`) and saved within the unique output directory created for the matrix execution.\n * The JSON output must be cleanly formatted and human-readable (pretty-printed with an indentation of 2 spaces).\n4. **Integration with Matrix Orchestrator:**\n * The main execution loop in the `MatrixOrchestrator` is updated to wrap each scenario run in a `try...catch` block.\n * In both success and failure cases, the orchestrator ensures that the data aggregator is called and a result file is written, capturing the error details if the run failed.\n\n#### **4. Technical Implementation Details**\n\n**Files to Modify / Create:**\n\n* `packages/cli/src/commands/scenario/src/schema.ts`: Finalize the `ScenarioRunResult` master interface.\n* `packages/cli/src/commands/scenario/src/data-aggregator.ts`: New file for the `RunDataAggregator` class.\n* `packages/cli/src/commands/scenario/src/matrix-orchestrator.ts`: Integrate the aggregator and serialization logic into the main execution loop.\n\n**Final** `ScenarioRunResult` JSON Structure (Example):\n\n```json\n{\n \"run_id\": \"run-20231027-015\",\n \"matrix_combination_id\": \"combo-003\",\n \"parameters\": {\n \"character.llm.model\": \"gpt-4-turbo\",\n \"run[0].input\": \"Show me what's open in the elizaOS/eliza GitHub.\"\n },\n \"metrics\": {\n \"execution_time_seconds\": 14.7,\n \"llm_calls\": 2,\n \"total_tokens\": 2100\n },\n \"final_agent_response\": \"Here are the open issues for elizaOS/eliza: #123 Fix the login button, #124 Improve documentation...\",\n \"evaluations\": [\n {\n \"evaluator_type\": \"llm_judge\",\n \"success\": false,\n \"summary\": \"...\",\n \"details\": {\n \"qualitative_summary\": \"...\",\n \"capability_checklist\": [\n { \"capability\": \"Formats Final Response\", \"achieved\": false, \"reasoning\": \"...\" }\n ]\n }\n }\n ],\n \"trajectory\": [\n { \"type\": \"thought\", \"content\": \"...\" },\n { \"type\": \"action\", \"content\": { \"name\": \"LIST_GITHUB_ISSUES\", \"parameters\": { \"owner\": \"elizaOS\", \"repo\": \"eliza\" } } },\n { \"type\": \"observation\", \"content\": \"[...]\" },\n { \"type\": \"thought\", \"content\": \"...\" }\n ],\n \"error\": null\n}\n```\n\n#### **5. Testing Requirements**\n\n* **Unit Tests:**\n * Test the `RunDataAggregator` to ensure it correctly assembles the `ScenarioRunResult` object from mock inputs.\n * Test the file serialization logic, including pretty-printing and correct file pathing.\n * Test the error handling path, ensuring that a valid result file (with the `error` field populated) is created even when a run fails catastrophically.\n* **Integration Tests:**\n * Run a full matrix test with a small number of combinations.\n * After the test completes, manually inspect the generated JSON files in the output directory to verify their structure and content are correct and complete.\n\n#### **6. Out of Scope**\n\n* The creation of the final `summary.json` file for the entire matrix run. This ticket is only concerned with the individual `run-*.json` files.\n* Any form of report generation or data visualization (this is handled in Epic 3).\n* The implementation of any of the data sources themselves (e.g., trajectory logging); this ticket only consumes and aggregates the data.", "createdAt": "2025-08-16T05:44:32Z", "closedAt": "2025-08-18T05:14:54Z", "state": "CLOSED", "commentCount": 1 } ], "topPRs": [ { "id": "PR_kwDOMT5cIs6iWsk7", "title": "feat(scenarios): Add comprehensive scenario testing system", "author": "wtfsayo", "number": 5723, "body": "## Summary\n- Add ElizaOS scenario testing system with YAML-based test definitions\n- Support for both local and E2B sandboxed testing environments \n- Comprehensive evaluation engine with action tracking and LLM judges\n- Mock service support for deterministic testing\n- CLI command `elizaos scenario run` for running individual scenarios\n- Batch testing support with `bun run test:scenarios`\n\n## Key Features\n- **Environment Providers**: Local and E2B sandbox support with fallback\n- **Mock Engine**: Service call interception and response mocking\n- **Evaluation Engine**: Action tracking, response validation, trajectory analysis\n- **LLM Judgment**: AI-powered evaluation of test results\n- **Comprehensive Documentation**: Examples, specs, and usage guides\n\n## Files Added\n- Scenario CLI command implementation\n- Environment providers (Local, E2B, Mock)\n- Evaluation and reporting engines\n- 15+ example scenarios covering various test cases\n- Comprehensive documentation and guides\n\n## Testing\n- Adds `test:scenarios` script to CLI package\n- 15+ example scenarios with different complexity levels\n- E2B integration with graceful fallbacks\n- Mock service testing capabilities\n\n🤖 Generated with [Claude Code](https://claude.ai/code)", "repository": "elizaos/eliza", "createdAt": "2025-08-06T10:13:37Z", "mergedAt": "2025-08-22T17:05:10Z", "additions": 25130, "deletions": 255 }, { "id": "PR_kwDOMT5cIs6ePDWm", "title": "feat: training models on own data", "author": "lalalune", "number": 5510, "body": "The goal of this plugin-training is to insert as a custom reasoning module which self-trains on the agent's data and the runs online after training. It's very WIP, and demonstrates a few of these steps, such as training a custom DeepSeek distilled model on together.ai from data, and overriding the internal models with the DeepSeek models. Some of it is still vibe code trash.\r\n\r\nIt was developed on the 'next' branch and might need a little work to integrate into current develop, especially wrt how we integrate the custom reasoning. It's also a bit messy and needs some consistency, and we should make CustomReasoningService into a shared IReasoningService type.", "repository": "elizaos/eliza", "createdAt": "2025-07-10T05:58:47Z", "mergedAt": null, "additions": 18236, "deletions": 0 }, { "id": "PR_kwDOMT5cIs6k0-7e", "title": "feat: bun build, remove tsup", "author": "ChristopherTrimboli", "number": 5807, "body": "This pull request introduces a new standardized Bun-based build system for ElizaOS packages, replacing the previous use of `tsup` and related tooling. It adds reusable build utilities, custom build scripts for `@elizaos/api-client` and `@elizaos/cli`, and updates package scripts and dependencies to leverage these changes. Additionally, there are targeted fixes and improvements to messaging service payloads, environment setup, and process management.\r\n\r\n**Build System Modernization**\r\n\r\n* Added `build-utils.ts` with reusable Bun build utilities for cleaning, building, copying assets, and generating TypeScript declarations, enabling consistent builds across packages.\r\n* Replaced `tsup` with Bun-based scripts in `@elizaos/api-client` and `@elizaos/cli`, including custom build scripts (`build.ts`) and removal of `tsup.config.ts`. Updated package scripts to use Bun for build, watch, and clean operations. [[1]](diffhunk://#diff-c64f755bd238752518269ba933743007c8b7c2b3db7b2663c3cfd8eee3e66ee1R1-R59) [[2]](diffhunk://#diff-fd8bcdbf9ab496c42cc8f5a68cbb792d0cd44d138d9f35e6ef960ac6a7b97168L10-R16) [[3]](diffhunk://#diff-fd8bcdbf9ab496c42cc8f5a68cbb792d0cd44d138d9f35e6ef960ac6a7b97168L24) [[4]](diffhunk://#diff-1870665c82b8dca54d6e63c49dfa1077b64d4b37bc32f66610ff543ad7f00dcbL1-L18) [[5]](diffhunk://#diff-38a7bfc3de9135d0af986757713f0fc589a906dd9be2f715932e303ae8bc7e4cR1-R82) [[6]](diffhunk://#diff-6e2e2a1851648938b325ba84de634407a4e69a644ea61102df15ca4a8a7a9758L40-R44) [[7]](diffhunk://#diff-6e2e2a1851648938b325ba84de634407a4e69a644ea61102df15ca4a8a7a9758R59) [[8]](diffhunk://#diff-6e2e2a1851648938b325ba84de634407a4e69a644ea61102df15ca4a8a7a9758L71-L75)\r\n* Updated monorepo-level build and clean scripts in `package.json` to filter out certain packages, remove `tsup`, and improve cache handling for faster, more reliable builds. [[1]](diffhunk://#diff-7ae45ad102eab3b6d7e7896acd08c427a9b25b346470d7bc6507b6481575d519L11-R19) [[2]](diffhunk://#diff-7ae45ad102eab3b6d7e7896acd08c427a9b25b346470d7bc6507b6481575d519L52)\r\n\r\n**Messaging Service Improvements**\r\n\r\n* Refactored payload construction for channel creation methods in `MessagingService` to match server expectations, including proper handling of metadata, participant IDs, and DM channel parameters.\r\n* Fixed type handling in participant filtering logic for channel updates.\r\n\r\n**TypeScript Declaration Generation**\r\n\r\n* Enabled TypeScript declaration file (`.d.ts`) generation via Bun build scripts and updated `tsconfig.build.json` to include `\"declaration\": true`. [[1]](diffhunk://#diff-c64f755bd238752518269ba933743007c8b7c2b3db7b2663c3cfd8eee3e66ee1R1-R59) [[2]](diffhunk://#diff-38a7bfc3de9135d0af986757713f0fc589a906dd9be2f715932e303ae8bc7e4cR1-R82) [[3]](diffhunk://#diff-c4d3cf5a942b53c98c3bd2b84fbc766b044cfa190d26f1a249dd7357486a7a5aL4-R5)\r\n\r\n**Process and Environment Management**\r\n\r\n* Improved environment variable setup in CLI server manager to filter out undefined values and ensure proper module resolution.\r\n* Updated server process termination logic to ensure graceful shutdown and state cleanup.\r\n* Minor fix to plugin upgrade progress event handling for more robust logging.\r\n* Removed unused `writeFileSync` import in CLI plugin environment variable utilities.", "repository": "elizaos/eliza", "createdAt": "2025-08-22T07:00:52Z", "mergedAt": null, "additions": 11920, "deletions": 10128 }, { "id": "PR_kwDOMT5cIs6kzWrr", "title": "feat: Add AI Gateway plugin - Universal access to 100+ AI models", "author": "Dexploarer", "number": 5806, "body": "## Summary\nAdds a universal AI Gateway plugin that provides access to 100+ AI models through Vercel AI Gateway and other OpenAI-compatible gateways.\n\n## Features\n- 🚀 **100+ AI Models** - OpenAI, Anthropic, Google, Meta, Mistral, and more through unified gateways\n- 🔄 **Universal Gateway Support** - Works with Vercel AI Gateway, OpenRouter, and any OpenAI-compatible endpoint\n- 💾 **Response Caching** - Built-in LRU cache for cost optimization\n- 📊 **Built-in Telemetry** - Track usage and performance metrics\n- ⚡ **High Performance** - Automatic retry logic and connection pooling\n- 🎯 **Multiple Actions** - Text generation, image generation, embeddings, and model listing\n\n## Highlights\n\n### Vercel AI Gateway Integration\n- Optimized for Vercel's AI Gateway with proper URL format (`https://ai-gateway.vercel.sh/v1`)\n- Supports Vercel's colon-based model naming convention (e.g., `openai:gpt-4o`)\n- Compatible with Vercel's app attribution headers for analytics\n\n### Flexible Provider Support\n- Works with any OpenAI-compatible API endpoint\n- Automatic model format detection (colon vs slash separators)\n- Seamless switching between providers with minimal configuration\n\n### Enterprise Features\n- Response caching with configurable TTL\n- Automatic retry logic with exponential backoff\n- OIDC authentication support for enterprise deployments\n- Comprehensive error handling and fallback mechanisms\n\n## Installation\n\nThe plugin is already published to npm for immediate use:\n```bash\nnpm install @promptordie/plugin-aigateway\n```\n\nAfter merge, it will be available as:\n```bash\nnpm install @elizaos/plugin-aigateway\n```\n\n## Configuration\n\n```env\n# Required\nAIGATEWAY_API_KEY=your_api_key_here\n\n# Optional (defaults shown)\nAIGATEWAY_BASE_URL=https://ai-gateway.vercel.sh/v1\nAIGATEWAY_DEFAULT_MODEL=openai:gpt-4o-mini\nAIGATEWAY_LARGE_MODEL=openai:gpt-4o\nAIGATEWAY_EMBEDDING_MODEL=openai:text-embedding-3-small\nAIGATEWAY_CACHE_TTL=300\nAIGATEWAY_MAX_RETRIES=3\n```\n\n## Usage\n\n```typescript\nimport aiGatewayPlugin from '@elizaos/plugin-aigateway';\n\nconst character = {\n name: 'MyAgent',\n plugins: [aiGatewayPlugin],\n settings: {\n AIGATEWAY_API_KEY: 'your-api-key'\n }\n};\n```\n\n## Test Plan\n- [x] Built successfully with `bun run build`\n- [x] All TypeScript types validated\n- [x] Published to npm registry for testing\n- [x] Tested with Vercel AI Gateway\n- [x] Tested model provider registration\n- [ ] Community testing welcomed\n\n## Documentation\nFull documentation and examples included in packages/plugin-aigateway/README.md\n\n## Related Links\n- NPM Package: https://www.npmjs.com/package/@promptordie/plugin-aigateway\n- Vercel AI Gateway Docs: https://vercel.com/docs/ai-gateway", "repository": "elizaos/eliza", "createdAt": "2025-08-22T02:36:55Z", "mergedAt": null, "additions": 4985, "deletions": 423 }, { "id": "PR_kwDOMT5cIs6kWSnk", "title": "feat: Sessions API ++", "author": "ChristopherTrimboli", "number": 5799, "body": "## Enhanced Session Management with Advanced Timeout Configuration and Lifecycle Control\r\n\r\n### Overview\r\nThis PR significantly enhances the sessions API with comprehensive timeout management, auto-renewal capabilities, and robust error handling. The changes transform the basic session management into an enterprise-ready system with configurable lifecycles, warning states, and graceful cleanup mechanisms.\r\n\r\n### Key Improvements\r\n\r\n#### 🔧 **Advanced Timeout Configuration System**\r\n**Before:** Simple fixed timeout (`SESSION_TIMEOUT_MS`)\r\n**After:** Multi-layered timeout configuration with:\r\n- Configurable timeout minutes (5-1440 min range)\r\n- Auto-renewal capabilities\r\n- Maximum session duration limits\r\n- Warning threshold notifications\r\n- Agent-specific timeout settings\r\n- Session-specific overrides\r\n\r\n#### 🛡️ **Robust Error Handling**\r\n**Before:** Basic `errorResponse` function with generic error messages\r\n**After:** Comprehensive error system with:\r\n- 11 custom error classes (`SessionNotFoundError`, `SessionExpiredError`, etc.)\r\n- Detailed error context and metadata\r\n- Centralized error middleware\r\n- Type-safe error handling throughout\r\n\r\n#### ♻️ **Session Lifecycle Management**\r\n**New Features:**\r\n- **Auto-renewal**: Sessions automatically extend on activity\r\n- **Manual renewal**: `/sessions/:id/renew` endpoint\r\n- **Heartbeat support**: `/sessions/:id/heartbeat` for keep-alive\r\n- **Warning states**: Proactive expiration notifications\r\n- **Renewal tracking**: Monitor renewal counts and patterns\r\n- **Graceful expiration**: Proper cleanup of expired sessions\r\n\r\n#### 🎯 **Type Safety Enhancements**\r\n**Added:**\r\n- Type guards for all request/response objects\r\n- `isValidSession()`, `isCreateSessionRequest()`, `isSendMessageRequest()`\r\n- `isValidTimeoutConfig()` for configuration validation\r\n- Proper TypeScript interfaces for all data structures\r\n\r\n#### 🔌 **New API Endpoints**\r\n```\r\nPOST /sessions/:id/renew - Manually renew a session\r\nPOST /sessions/:id/heartbeat - Keep session alive\r\nPATCH /sessions/:id/timeout - Update timeout configuration\r\n```\r\n\r\n#### 📊 **Enhanced Health Monitoring**\r\n**Before:** Basic session count\r\n**After:** Detailed health metrics including:\r\n- Active vs expired sessions\r\n- Sessions expiring soon\r\n- Invalid session detection\r\n- Server uptime tracking\r\n\r\n#### 🧹 **Memory Leak Prevention**\r\n**New:**\r\n- `SessionRouter` interface with cleanup method\r\n- Proper interval management with `activeCleanupIntervals` Set\r\n- Process handler registration tracking\r\n- Resource cleanup on router destruction\r\n\r\n#### 📈 **Improved Pagination**\r\n**Enhanced:**\r\n- Better cursor-based pagination with proper `before`/`after` handling\r\n- Cursor information in response for easier client implementation\r\n- Proper handling of edge cases in date range queries\r\n\r\n### Breaking Changes\r\nNone - All existing endpoints maintain backward compatibility while adding optional new features.\r\n\r\n### Migration Guide\r\nExisting clients will continue to work without changes. To leverage new features:\r\n\r\n1. **Enable auto-renewal**: Include `timeoutConfig: { autoRenew: true }` in session creation\r\n2. **Set custom timeouts**: Add `timeoutConfig: { timeoutMinutes: 60 }` \r\n3. **Monitor expiration**: Check `isNearExpiration` in responses\r\n4. **Use heartbeats**: Send periodic POST to `/sessions/:id/heartbeat`\r\n\r\n### Configuration\r\nNew environment variables (all optional):\r\n```env\r\nSESSION_DEFAULT_TIMEOUT_MINUTES=30 # Default: 30\r\nSESSION_MIN_TIMEOUT_MINUTES=5 # Default: 5 \r\nSESSION_MAX_TIMEOUT_MINUTES=1440 # Default: 1440 (24h)\r\nSESSION_MAX_DURATION_MINUTES=720 # Default: 720 (12h)\r\nSESSION_WARNING_THRESHOLD_MINUTES=5 # Default: 5\r\nSESSION_CLEANUP_INTERVAL_MINUTES=5 # Default: 5\r\nCLEAR_SESSIONS_ON_SHUTDOWN=false # Default: false\r\n```\r\n\r\n### Technical Details\r\n\r\n#### Session State Structure\r\n```typescript\r\ninterface Session {\r\n // ... existing fields ...\r\n expiresAt: Date; // NEW: Calculated expiration time\r\n timeoutConfig: { // NEW: Comprehensive timeout settings\r\n timeoutMinutes?: number;\r\n autoRenew?: boolean;\r\n maxDurationMinutes?: number;\r\n warningThresholdMinutes?: number;\r\n };\r\n renewalCount: number; // NEW: Track renewal history\r\n warningState?: { // NEW: Warning notification state\r\n sent: boolean;\r\n sentAt: Date;\r\n };\r\n}\r\n```\r\n\r\n#### Error Classes Hierarchy\r\n- `SessionError` (base class)\r\n - `SessionNotFoundError`\r\n - `SessionExpiredError`\r\n - `SessionCreationError`\r\n - `SessionRenewalError`\r\n - `MessageSendError`\r\n - `InvalidUuidError`\r\n - `MissingFieldsError`\r\n - `InvalidContentError`\r\n - `InvalidMetadataError`\r\n - `InvalidPaginationError`\r\n - `InvalidTimeoutConfigError`\r\n - `AgentNotFoundError`\r\n\r\n### Testing\r\n- ✅ All error scenarios covered with proper error classes\r\n- ✅ Timeout and renewal logic validated\r\n- ✅ Memory leak prevention verified\r\n- ✅ Backward compatibility maintained\r\n\r\n### Performance Impact\r\n- Minimal overhead from timeout calculations\r\n- Efficient cleanup with configurable intervals\r\n- Optimized validation with early returns\r\n- No impact on existing session operations\r\n\r\n### Security Considerations\r\n- Session expiration enforced at API level\r\n- Automatic cleanup of stale sessions\r\n- Validation of all input parameters\r\n- No sensitive data in error responses (except in dev mode)", "repository": "elizaos/eliza", "createdAt": "2025-08-19T18:39:03Z", "mergedAt": "2025-08-20T14:10:28Z", "additions": 2377, "deletions": 559 } ], "codeChanges": { "additions": 27626, "deletions": 5707, "files": 237, "commitCount": 209 }, "completedItems": [ { "title": "feat(scenarios): Add comprehensive scenario testing system", "prNumber": 5723, "type": "feature", "body": "## Summary\n- Add ElizaOS scenario testing system with YAML-based test definitions\n- Support for both local and E2B sandboxed testing environments \n- Comprehensive evaluation engine with action tracking and LLM judges\n- Mock service support", "files": [ "bun.lock", "package.json", "packages/cli/package.json", "packages/cli/scripts/run-all-scenarios.ts", "packages/cli/src/commands/scenario/docs/README.md", "packages/cli/src/commands/scenario/docs/analyze-past-trade.md", "packages/cli/src/commands/scenario/docs/answer-roadmap-questions.md", "packages/cli/src/commands/scenario/docs/check-evm-balance.md", "packages/cli/src/commands/scenario/docs/example_scenarios/analyze-past-trade.scenario.yaml", "packages/cli/src/commands/scenario/docs/example_scenarios/answer-roadmap-questions.scenario.yaml", "packages/cli/src/commands/scenario/docs/example_scenarios/check-evm-balance.scenario.yaml", "packages/cli/src/commands/scenario/docs/scenario-runner-spec.md", "packages/cli/src/commands/scenario/docs/scenarios.md", "packages/cli/src/commands/scenario/examples/action-tracking-test.scenario.yaml", "packages/cli/src/commands/scenario/examples/e2b-fallback.scenario.yaml", "packages/cli/src/commands/scenario/examples/e2b-test.scenario.yaml", "packages/cli/src/commands/scenario/examples/evaluation-test.scenario.yaml", "packages/cli/src/commands/scenario/examples/failing-test.scenario.yaml", "packages/cli/src/commands/scenario/examples/fully-passing.scenario.yaml", "packages/cli/src/commands/scenario/examples/invalid-missing-fields.scenario.yaml", "packages/cli/src/commands/scenario/examples/llm-judge-failure-test.scenario.yaml", "packages/cli/src/commands/scenario/examples/llm-judge-test.scenario.yaml", "packages/cli/src/commands/scenario/examples/mixed-results.scenario.yaml", "packages/cli/src/commands/scenario/examples/mock-e2b-test.scenario.yaml", "packages/cli/src/commands/scenario/examples/mock-test.scenario.yaml", "packages/cli/src/commands/scenario/examples/multi-step.scenario.yaml", "packages/cli/src/commands/scenario/examples/simple-mock-test.scenario.yaml", "packages/cli/src/commands/scenario/examples/simple-test.scenario.yaml", "packages/cli/src/commands/scenario/examples/trajectory-test.scenario.yaml", "packages/cli/src/commands/scenario/examples/valid.scenario.yaml", "packages/cli/src/commands/scenario/index.ts", "packages/cli/src/index.ts", "packages/cli/src/scenarios/E2BEnvironmentProvider.ts", "packages/cli/src/scenarios/EvaluationEngine.ts", "packages/cli/src/scenarios/LocalEnvironmentProvider.ts", "packages/cli/src/scenarios/MockEngine.ts", "packages/cli/src/scenarios/Reporter.ts", "packages/cli/src/scenarios/env-loader.ts", "packages/cli/src/scenarios/providers.ts", "packages/cli/src/scenarios/runtime-factory.ts", "packages/cli/src/scenarios/schema.ts", "packages/cli/src/types/elizaos-modules.d.ts", "packages/cli/src/commands/scenario/examples/advanced-mocking-test.scenario.yaml", "packages/cli/src/commands/scenario/examples/enhanced-mock-test.scenario.yaml", "packages/cli/src/commands/scenario/examples/plugin-parsing-test.scenario.yaml", "packages/cli/src/commands/scenario/examples/simple-advanced-mock-test.scenario.yaml", "packages/cli/src/scenarios/plugin-parser.ts", "packages/cli/test-plugin-parsing.ts", "packages/cli/test-scenario-validation.ts", ".github/workflows/ci.yaml", ".github/workflows/pre-release.yml", ".github/workflows/release.yaml", ".github/workflows/update-news.yml", ".gitignore", "lerna.json", "llms.txt", "packages/api-client/README.md", "packages/api-client/docs/sessions-api.md", "packages/api-client/package.json", "packages/api-client/src/__tests__/base-client.test.ts", "packages/api-client/src/__tests__/services/sessions.test.ts", "packages/api-client/src/lib/base-client.ts", "packages/api-client/src/services/sessions.ts", "packages/api-client/src/types/sessions.ts", "packages/app/package.json", "packages/autodoc/package.json", "packages/cli/src/commands/create/index.ts", "packages/cli/src/commands/plugins/utils/env-vars.ts", "packages/cli/src/commands/scenario/examples/.gitignore", "packages/cli/src/commands/scenario/examples/analyze-past-trade.scenario.yaml", "packages/cli/src/commands/scenario/examples/answer-roadmap-questions.scenario.yaml", "packages/cli/src/commands/scenario/examples/check-evm-balance.scenario.yaml", "packages/cli/src/commands/scenario/examples/natural-language-test.scenario.yaml", "packages/cli/src/commands/scenario/examples/nl-smoke.scenario.yaml", "packages/cli/src/commands/scenario/examples/execution-time-test.scenario.yaml", "packages/cli/.gitignore", "packages/cli/src/commands/scenario/examples/check-coinbase-balance.scenario.yaml", "packages/cli/src/commands/scenario/examples/test-github-issues.scenario.yaml", "packages/cli/src/commands/scenario/src/E2BEnvironmentProvider.ts", "packages/cli/src/commands/scenario/src/EvaluationEngine.ts", "packages/cli/src/commands/scenario/src/LocalEnvironmentProvider.ts", "packages/cli/src/commands/scenario/src/MockEngine.ts", "packages/cli/src/commands/scenario/src/Reporter.ts", "packages/cli/src/commands/scenario/src/env-loader.ts", "packages/cli/src/commands/scenario/src/plugin-parser.ts", "packages/cli/src/commands/scenario/src/providers.ts", "packages/cli/src/commands/scenario/src/runtime-factory.ts", "packages/cli/src/commands/scenario/src/schema.ts", "packages/plugin-dummy-services/package.json", "packages/project-starter/src/__tests__/events.test.ts", "packages/server/package.json", "packages/cli/src/commands/scenario/docs/matrix-testing.md", "packages/cli/src/commands/scenario/examples/github-issue-analysis.matrix.yaml", "packages/cli/src/commands/scenario/src/__tests__/example-validation.test.ts", "packages/cli/src/commands/scenario/src/__tests__/matrix-command.test.ts", "packages/cli/src/commands/scenario/src/__tests__/matrix-runner.test.ts", "packages/cli/src/commands/scenario/src/__tests__/matrix-schema.test.ts", "packages/cli/src/commands/scenario/src/__tests__/parameter-override.test.ts", "packages/cli/src/commands/scenario/src/__tests__/validation-demo.test.ts", "packages/cli/src/commands/scenario/src/matrix-runner.ts", "packages/cli/src/commands/scenario/src/matrix-schema.ts", "packages/cli/src/commands/scenario/src/matrix-types.ts", "packages/cli/src/commands/scenario/src/parameter-override.ts", "packages/server/src/index.ts", "packages/cli/src/commands/report/demo-html-report.ts", "packages/cli/src/commands/report/generate.ts", "packages/cli/src/commands/report/index.ts", "packages/cli/src/commands/report/src/__tests__/analysis-engine.test.ts", "packages/cli/src/commands/report/src/__tests__/html-template.test.ts", "packages/cli/src/commands/report/src/__tests__/integration.test.ts", "packages/cli/src/commands/report/src/__tests__/pdf-export.test.ts", "packages/cli/src/commands/report/src/__tests__/pdf-generator.test.ts", "packages/cli/src/commands/report/src/__tests__/template-integration.test.ts", "packages/cli/src/commands/report/src/analysis-engine.ts", "packages/cli/src/commands/report/src/assets/report_template.html", "packages/cli/src/commands/report/src/pdf-generator.ts", "packages/cli/src/commands/report/src/report-schema.ts", "packages/cli/src/commands/scenario/examples/debug-llm-judge.scenario.yaml", "packages/cli/src/commands/scenario/examples/enhanced-demo.scenario.yaml", "packages/cli/src/commands/scenario/examples/llm-judge-with-capabilities.scenario.yaml", "packages/cli/src/commands/scenario/examples/simple-test.matrix.yaml", "packages/cli/src/commands/scenario/examples/trajectory-demo.scenario.yaml", "packages/cli/src/commands/scenario/src/EnhancedEvaluationEngine.ts", "packages/cli/src/commands/scenario/src/TrajectoryReconstructor.ts", "packages/cli/src/commands/scenario/src/__tests__/capabilities-evaluation.test.ts", "packages/cli/src/commands/scenario/src/__tests__/data-aggregator.test.ts", "packages/cli/src/commands/scenario/src/__tests__/deep-clone.test.ts", "packages/cli/src/commands/scenario/src/__tests__/e2e/centralized-data.test.ts", "packages/cli/src/commands/scenario/src/__tests__/enhanced-evaluation.test.ts", "packages/cli/src/commands/scenario/src/__tests__/evaluation-integration.test.ts", "packages/cli/src/commands/scenario/examples/test-basic.scenario.yaml", "packages/cli/src/commands/scenario/src/__tests__/LocalEnvironmentProvider.test.ts", "CLAUDE.md" ] }, { "title": "fix: correct comma placement when adding entries to registry index.json", "prNumber": 5774, "type": "bugfix", "body": "## Description\r\n\r\n### Problem\r\nThe `elizaos publish` command incorrectly handled commas when adding new plugin entries to the registry's `index.json` file:\r\n- Did not add a comma to the previously last entry\r\n- Incorrectly added a comma to ", "files": [ "packages/cli/src/utils/publisher.ts", "packages/cli/tests/unit/utils/publisher.test.ts" ] }, { "title": "fix: fix: phala CLI argument handling and tee starter docker build", "prNumber": 5773, "type": "bugfix", "body": "## Description\r\n\r\nThis PR fixes two minor issues preventing the tee command from working as intended:\r\n\r\n### 1. Phala CLI Wrapper Argument Handling\r\n\r\nThe ElizaOS wrapper for the Phala CLI was not correctly capturing arguments, causing comm", "files": [ "packages/cli/src/commands/tee/phala-wrapper.ts", "packages/project-tee-starter/src/index.ts", "packages/cli/src/commands/tee/index.ts", "packages/cli/tests/commands/create.test.ts", "packages/cli/tests/commands/tee.test.ts" ] }, { "title": "feat(bootstrap): async embedding generation via queue service", "prNumber": 5793, "type": "feature", "body": "# Relates to\r\n\r\nPerformance improvement for message processing latency - embeddings were blocking runtime for 500ms+ per message\r\n\r\n# Risks\r\n\r\n**Low risk** - The change is backward compatible and includes fallback behavior. Main risks:\r\n- M", "files": [ "packages/core/src/__tests__/runtime-embedding.test.ts", "packages/core/src/runtime.ts", "packages/core/src/types/events.ts", "packages/core/src/types/runtime.ts", "packages/plugin-bootstrap/src/__tests__/embedding-service.test.ts", "packages/plugin-bootstrap/src/__tests__/evaluators.test.ts", "packages/plugin-bootstrap/src/__tests__/test-utils.ts", "packages/plugin-bootstrap/src/evaluators/reflection.ts", "packages/plugin-bootstrap/src/index.ts", "packages/plugin-bootstrap/src/services/embedding.ts", "packages/plugin-quick-starter/src/__tests__/test-utils.ts", "packages/plugin-starter/src/__tests__/test-utils.ts", "packages/server/src/__tests__/test-utils/mocks.ts", "packages/test-utils/src/mocks/runtime.ts", "packages/plugin-bootstrap/src/__tests__/embedding-queue-management.test.ts" ] }, { "title": "fix: resolve test failures and enhance XML parsing reliability in CI environment", "prNumber": 5792, "type": "bugfix", "body": "## Problem\nMultiple GitHub Actions test failures were occurring across different packages, plus a critical plugin configuration bug:\n\n### Original Issues (https://github.com/elizaOS/eliza/actions/runs/17020769599/job/48249463787)\n- ❌ **10 e", "files": [ "bun.lock", "packages/plugin-bootstrap/src/__tests__/attachments.test.ts", "packages/plugin-bootstrap/src/__tests__/evaluators.test.ts", "packages/plugin-bootstrap/src/__tests__/services.test.ts", "packages/plugin-bootstrap/src/__tests__/test-utils.ts", "packages/plugin-bootstrap/src/index.ts", "packages/project-tee-starter/src/__tests__/config.test.ts", "packages/project-tee-starter/src/plugin.ts" ] }, { "title": "fix: resolve entity creation SQL parameter mismatch", "prNumber": 5791, "type": "bugfix", "body": "## Summary\n\nThis PR fixes a critical database error that was occurring during entity creation:\n\n```\n[ERROR] Error creating entity: Failed query: insert into \"entities\" values ($1, $2, default, default, default)\nparams: [only 2 parameters pr", "files": [ "packages/core/src/types/environment.ts", "packages/plugin-sql/src/base.ts" ] }, { "title": "feat: Cross-Environment Logger Support.", "prNumber": 5797, "type": "feature", "body": "## Logger Module Refactoring: Cross-Platform Support & Enhanced Architecture\r\n\r\n### Overview\r\nThis PR introduces a comprehensive refactoring of the logger module to support both browser and Node.js environments while maintaining backward co", "files": [ "packages/core/src/__tests__/logger-browser-node.test.ts", "packages/core/src/logger.ts" ] }, { "title": "fix: improve TypeScript types and error logging in publisher", "prNumber": 5796, "type": "bugfix", "body": "## Summary\n\nThis PR improves TypeScript type safety and error logging in the publisher module by:\n\n### Changes Made\n\n1. **Type Safety Improvements**:\n - Replaced all types with proper TypeScript types in \n - Added interface for better", "files": [ "packages/cli/src/utils/publisher.ts", "packages/cli/tests/unit/utils/publisher.test.ts" ] }, { "title": "fix: code formatting improvements and dependency updates", "prNumber": 5795, "type": "bugfix", "body": "## Summary\n\nThis PR contains code formatting improvements and dependency updates:\n\n### Changes Made:\n- **Test File Cleanup**: Removed unnecessary empty lines in `attachments.test.ts`\n- **Logger Formatting**: Fixed line wrapping for `logger.", "files": [ "bun.lock", "packages/plugin-bootstrap/src/__tests__/attachments.test.ts", "packages/plugin-bootstrap/src/index.ts", "packages/project-tee-starter/src/plugin.ts" ] }, { "title": "chore: 1.4.3", "prNumber": 5794, "type": "other", "body": "", "files": [ ".github/workflows/release.yaml", "bun.lock", "lerna.json", "packages/api-client/package.json", "packages/cli/package.json", "packages/cli/src/commands/publish/index.ts", "packages/cli/src/commands/publish/types.ts", "packages/cli/src/commands/publish/utils/metadata.ts", "packages/cli/src/utils/github.ts", "packages/cli/src/utils/publisher.ts", "packages/cli/tests/unit/utils/publisher.test.ts", "packages/client/package.json", "packages/core/src/types/environment.ts", "packages/plugin-bootstrap/package.json", "packages/plugin-bootstrap/src/__tests__/attachments.test.ts", "packages/plugin-bootstrap/src/__tests__/evaluators.test.ts", "packages/plugin-bootstrap/src/__tests__/services.test.ts", "packages/plugin-bootstrap/src/__tests__/test-utils.ts", "packages/plugin-bootstrap/src/index.ts", "packages/plugin-dummy-services/package.json", "packages/plugin-sql/package.json", "packages/plugin-sql/src/base.ts", "packages/project-starter/src/__tests__/config.test.ts", "packages/project-starter/src/__tests__/error-handling.test.ts", "packages/project-starter/src/__tests__/events.test.ts", "packages/project-tee-starter/src/__tests__/config.test.ts", "packages/project-tee-starter/src/plugin.ts", "packages/server/package.json", "packages/server/src/index.ts", "packages/test-utils/package.json" ] }, { "title": "feat: Sessions API ++", "prNumber": 5799, "type": "feature", "body": "## Enhanced Session Management with Advanced Timeout Configuration and Lifecycle Control\r\n\r\n### Overview\r\nThis PR significantly enhances the sessions API with comprehensive timeout management, auto-renewal capabilities, and robust error han", "files": [ "packages/server/src/api/messaging/__tests__/sessions.test.ts", "packages/server/src/api/messaging/channels.ts", "packages/server/src/api/messaging/errors/SessionErrors.ts", "packages/server/src/api/messaging/sessions.ts", "packages/server/src/index.ts", "packages/server/src/types/sessions.ts" ] }, { "title": "feat: getServiceLoadPromise", "prNumber": 5801, "type": "feature", "body": "# Risks\r\n\r\nLow\r\n\r\n# Background\r\n\r\n## What does this PR do?\r\n\r\n- add getServiceLoadPromise interface to runtime\r\n- fix component queries in plugin-sql (was too easy for dates to get invalid, this is a more flexible set up, allowing the inten", "files": [ "packages/core/src/runtime.ts", "packages/core/src/settings.ts", "packages/core/src/types/runtime.ts", "packages/plugin-bootstrap/src/index.ts", "packages/plugin-sql/src/base.ts", "packages/test-utils/src/mocks/runtime.ts" ] }, { "title": "fix: metadata in sessions", "prNumber": 5805, "type": "bugfix", "body": "## Session Metadata Propagation for Plugin Actions\n\n### Overview\nThis PR implements session metadata propagation throughout the ElizaOS message processing pipeline, enabling plugins and actions to access custom session metadata (like `ethAd", "files": [ "packages/server/src/__tests__/message-bus.test.ts", "packages/server/src/api/messaging/__tests__/sessions.test.ts", "packages/server/src/api/messaging/sessions.ts", "packages/server/src/services/message.ts" ] }, { "title": "feat: Convert packages/docs to git submodule from elizaos/docs", "prNumber": 5803, "type": "feature", "body": "## Summary\n\nThis PR converts the `packages/docs` directory from tracked files to a git submodule pointing to the external documentation repository at https://github.com/elizaos/docs.\n\n## Changes\n\n- Removed 171 documentation files that were ", "files": [ ".gitmodules", "packages/docs", "packages/docs/.github/workflows/check-dead-links.yml", "packages/docs/.github/workflows/check-documentation-quality.yml", "packages/docs/.github/workflows/claude-code-review.yml", "packages/docs/.github/workflows/claude.yml", "packages/docs/.gitignore", "packages/docs/CLAUDE.md", "packages/docs/README.md", "packages/docs/api-reference/agents/create-a-new-agent.mdx", "packages/docs/api-reference/agents/create-a-world-for-an-agent.mdx", "packages/docs/api-reference/agents/delete-an-agent.mdx", "packages/docs/api-reference/agents/get-agent-details.mdx", "packages/docs/api-reference/agents/get-agent-panels.mdx", "packages/docs/api-reference/agents/get-all-worlds.mdx", "packages/docs/api-reference/agents/list-all-agents.mdx", "packages/docs/api-reference/agents/start-an-agent.mdx", "packages/docs/api-reference/agents/stop-an-agent.mdx", "packages/docs/api-reference/agents/update-a-world.mdx", "packages/docs/api-reference/agents/update-agent.mdx", "packages/docs/api-reference/audio/convert-conversation-to-speech.mdx", "packages/docs/api-reference/audio/generate-speech-from-text.mdx", "packages/docs/api-reference/audio/process-audio-message.mdx", "packages/docs/api-reference/audio/synthesize-speech-from-text.mdx", "packages/docs/api-reference/audio/transcribe-audio.mdx", "packages/docs/api-reference/media/upload-media-for-agent.mdx", "packages/docs/api-reference/media/upload-media-to-channel.mdx", "packages/docs/api-reference/memory/create-a-room.mdx", "packages/docs/api-reference/memory/delete-all-agent-memories.mdx", "packages/docs/api-reference/memory/delete-all-memories-for-a-room.mdx", "packages/docs/api-reference/memory/get-agent-memories.mdx", "packages/docs/api-reference/memory/get-room-memories.mdx", "packages/docs/api-reference/memory/update-a-memory.mdx", "packages/docs/api-reference/messaging/add-agent-to-channel.mdx", "packages/docs/api-reference/messaging/add-agent-to-server.mdx", "packages/docs/api-reference/messaging/create-central-channel.mdx", "packages/docs/api-reference/messaging/create-channel.mdx", "packages/docs/api-reference/messaging/create-group-channel.mdx", "packages/docs/api-reference/messaging/create-server.mdx", "packages/docs/api-reference/messaging/delete-all-channel-messages-by-user.mdx", "packages/docs/api-reference/messaging/delete-all-channel-messages.mdx", "packages/docs/api-reference/messaging/delete-channel-message.mdx", "packages/docs/api-reference/messaging/delete-channel.mdx", "packages/docs/api-reference/messaging/get-central-server-channels.mdx", "packages/docs/api-reference/messaging/get-central-servers.mdx", "packages/docs/api-reference/messaging/get-channel-details.mdx", "packages/docs/api-reference/messaging/get-channel-info.mdx", "packages/docs/api-reference/messaging/get-channel-messages.mdx", "packages/docs/api-reference/messaging/get-channel-participants.mdx", "packages/docs/api-reference/messaging/get-or-create-dm-channel.mdx" ] }, { "title": "fix: plugin-sql test", "prNumber": 5802, "type": "bugfix", "body": "# Risks\r\n\r\nMedium, not sure this is what we want\r\n\r\n# Background\r\n\r\n## What does this PR do?\r\n\r\n- make plugin-sql tests pass for me from monorepo\r\n- mainly createdAt have to be a date object for w/e reason now for pglite (timestamps no long", "files": [ "packages/core/src/runtime.ts", "packages/plugin-sql/src/__tests__/e2e/postgres.test.ts", "packages/plugin-sql/src/__tests__/integration/base-adapter-methods.test.ts", "packages/plugin-sql/src/__tests__/integration/base-comprehensive.test.ts", "packages/plugin-sql/src/__tests__/integration/component.test.ts", "packages/plugin-sql/src/__tests__/unit/utils.test.ts", "packages/plugin-sql/src/base.ts", "packages/plugin-sql/src/utils.ts" ] } ], "topContributors": [ { "username": "wtfsayo", "avatarUrl": "https://avatars.githubusercontent.com/u/82053242?u=98209a1f10456f42d4d2fa71db4d5bf4a672cbc3&v=4", "totalScore": 306.8053766636691, "prScore": 286.0653766636691, "issueScore": 0, "reviewScore": 20, "commentScore": 0.74, "summary": null }, { "username": "ChristopherTrimboli", "avatarUrl": "https://avatars.githubusercontent.com/u/27584221?u=0d816ce1dcdea8f925aba18bb710153d4a87a719&v=4", "totalScore": 150.7893868808183, "prScore": 134.9133868808183, "issueScore": 0, "reviewScore": 15, "commentScore": 0.8759999999999999, "summary": null }, { "username": "Dexploarer", "avatarUrl": "https://avatars.githubusercontent.com/u/211557447?u=21a243d61cc1f87574328ae07fc64d7d7577b53d&v=4", "totalScore": 90.76725587834024, "prScore": 90.76725587834024, "issueScore": 0, "reviewScore": 0, "commentScore": 0, "summary": null }, { "username": "0xbbjoker", "avatarUrl": "https://avatars.githubusercontent.com/u/54844437?u=90fe1762420de6ad493a1c1582f1f70c0d87d8e2&v=4", "totalScore": 89.35202526023288, "prScore": 74.15202526023288, "issueScore": 0, "reviewScore": 15, "commentScore": 0.2, "summary": null }, { "username": "odilitime", "avatarUrl": "https://avatars.githubusercontent.com/u/16395496?u=c9bac48e632aae594a0d85aaf9e9c9c69b674d8b&v=4", "totalScore": 81.12918495803116, "prScore": 80.72918495803115, "issueScore": 0, "reviewScore": 0, "commentScore": 0.4, "summary": null }, { "username": "Sergey1997", "avatarUrl": "https://avatars.githubusercontent.com/u/22988415?u=5ac6b5778c46fd3783556a946cac05ba4d7bd2aa&v=4", "totalScore": 12.873759469228055, "prScore": 12.673759469228056, "issueScore": 0, "reviewScore": 0, "commentScore": 0.2, "summary": null }, { "username": "monilpat", "avatarUrl": "https://avatars.githubusercontent.com/u/15067321?v=4", "totalScore": 6.076, "prScore": 0, "issueScore": 0, "reviewScore": 5, "commentScore": 1.0759999999999998, "summary": null }, { "username": "standujar", "avatarUrl": "https://avatars.githubusercontent.com/u/16385918?u=718bdcd1585be8447bdfffb8c11ce249baa7532d&v=4", "totalScore": 4.5, "prScore": 0, "issueScore": 0, "reviewScore": 4.5, "commentScore": 0, "summary": null }, { "username": "linear", "avatarUrl": "https://avatars.githubusercontent.com/in/20150?v=4", "totalScore": 4, "prScore": 0, "issueScore": 4, "reviewScore": 0, "commentScore": 0, "summary": null }, { "username": "harperaa", "avatarUrl": "https://avatars.githubusercontent.com/u/1330944?v=4", "totalScore": 4, "prScore": 0, "issueScore": 4, "reviewScore": 0, "commentScore": 0, "summary": null }, { "username": "madjin", "avatarUrl": "https://avatars.githubusercontent.com/u/32600939?u=cdcf89f44c7a50906c7a80d889efa85023af2049&v=4", "totalScore": 2, "prScore": 0, "issueScore": 2, "reviewScore": 0, "commentScore": 0, "summary": null }, { "username": "0xRabbidfly", "avatarUrl": "https://avatars.githubusercontent.com/u/93952856?v=4", "totalScore": 2, "prScore": 0, "issueScore": 2, "reviewScore": 0, "commentScore": 0, "summary": null }, { "username": "yungalgo", "avatarUrl": "https://avatars.githubusercontent.com/u/113615973?u=92e0f29f7e2fbb8ce46ed13c51f692ca803de02d&v=4", "totalScore": 0.2, "prScore": 0, "issueScore": 0, "reviewScore": 0, "commentScore": 0.2, "summary": null } ], "newPRs": 17, "mergedPRs": 15, "newIssues": 4, "closedIssues": 14, "activeContributors": 15 }