## User Feedback Analysis — 2026-02-23 (based on 2026-02-20 to 2026-02-22 signals) ### Data snapshot (what we analyzed) - Sources represented: Discord (public + core-devs + partners), GitHub issues/PR discussions, monthly repo rollup. - Total distinct feedback signals captured in the provided data: **~18** (questions, bug reports, and recurring confusions). --- ## 1) Pain Point Categorization (Top recurring 5–7) ### 1) UX/UI — ElizaCloud dashboard interaction bugs (high severity) **Frequency:** ~2/18 signals (11%), but **high impact** because it blocks basic usage. **Examples** - Broken mousepad scrolling on API Explorer; scrollbar “hard-to-see” (Discord, 2026-02-20). - “Send request” returns **“api key is required”** even when a key appears present (Discord, 2026-02-20). **Who it affects most** - New and evaluative users trying to validate ElizaCloud quickly (first-session blockers). --- ### 2) Documentation — “v2 is recommended but under-documented” + unclear feature semantics (high frequency) **Frequency:** ~6/18 signals (33%). **Examples** - v2.0.0 is “recommended” yet has “less documentation” (Discord, 2026-02-21). - “Use a different key” is ambiguous; users ask if it supports **BYOK** like OpenRouter (Discord, 2026-02-20). - Requests for **per-model pricing breakdown** in the dashboard (Discord, 2026-02-20). - Confusion about **Spartan infra vs Milady wallet** relationship; request to clarify (Discord, 2026-02-22). - Confusion about which token gets “upgrade” benefits (ElizaOS vs $milady.ai) (Discord, 2026-02-22). **Who it affects most** - Builders onboarding to v2 (need “truthy” docs + upgrade/migration guidance). - Community members tracking ecosystem benefits (token/NFT), where ambiguity creates distrust. --- ### 3) Technical Functionality — Database migration/refactor path ambiguity (high severity for upgraders) **Frequency:** ~3/18 signals (17%). **Examples** - Removal of auto-migration **1.4.x → 1.6.x** (PR #6521 discussion): assumption users aren’t on 1.4.x; suggestion to **throw an error telling users to migrate to 1.6.x first** (core-devs, 2026-02-22). - Users asking “How many users still need to migrate?” (Discord, 2026-02-20). - plugin-sql supports **pglite vs Neon** chosen by env vars, but **no automatic data migration** between them (Discord, 2026-02-21). **Who it affects most** - Anyone upgrading legacy deployments or switching DB backends (self-hosters, teams). --- ### 4) Performance — Context/prompt bloat and token-limit blowups with many plugins (high severity for power users) **Frequency:** ~1–2/18 signals (6–11%), but repeatedly emphasized by core contributors. **Examples** - Odilitime “regularly encountering **200k token limits** due to numerous plugins,” with bootstrap providers and evaluations as main contributors (core-devs, 2026-02-22). **Who it affects most** - Power users running large plugin suites; teams building “agent platforms” vs single agents. --- ### 5) Integration — Provider flexibility gaps (OpenAI-compatible endpoints) (medium-high severity) **Frequency:** ~1/18 signals (6%). **Examples** - GitHub #6490: request to support **custom OpenAI endpoint URL** to use OpenAI-compatible providers (e.g., SiliconFlow). **Who it affects most** - Cost-sensitive users and enterprise/self-host users who rely on alternative inference vendors. --- ### 6) Technical Functionality — Duplicate or incorrect LLM calls / content handling bugs (high cost + poor UX) **Frequency:** ~2/18 signals (11%). **Examples** - GitHub #6486: Messages containing a URL trigger **duplicate LLM calls** (processed as both text and attachment), causing duplicated output + **2× token cost**. - Prior cloud issue (now closed) showed **image content stripped** from requests (#6494), indicating message conversion pipeline fragility. **Who it affects most** - Webapp users, cloud chat users, anyone sharing links/media frequently. --- ### 7) Community / Communication — Ecosystem benefits & token/NFT/wallet confusion (high frequency, trust impact) **Frequency:** ~4/18 signals (22%). **Examples** - Confusion whether holder “upgrade” applies to **ElizaOS vs $milady.ai** (Discord, 2026-02-22). - milady.ai token not visible in Phantom because **not launched yet** (Discord partners, 2026-02-22). - Repeated questions on where leadership posts updates (Twitter vs Farcaster split) and how to contact Shaw (Discord, 2026-02-20). - Chain validity confusion for Milady (BNB vs Solana) remained unanswered in provided slice (Discord, 2026-02-20). **Who it affects most** - Newcomers and token holders; partner communities monitoring deliverables. --- ## 2) Usage Pattern Analysis (actual vs intended) ### Observed “actual usage” 1. **Agents as productized vertical tools**, not just demos: - DarmaStef: reservation agent, travel agent, “database analysis agent for non-technical users” (Discord, 2026-02-22). 2. **Crypto-native autonomous operations**: - SolCex BD agent plugin: exchange listing automation, scoring, multi-source intelligence (Discord coders, 2026-02-22). 3. **Heavy plugin stacking**: - Power users loading many plugins hit token/context ceilings (200k token limit report). 4. **Multi-chain + identity + on-chain primitives**: - SAID Protocol identity work; ERC standards discussions; NFT drop benefits (daily summary + Discord). ### Gaps vs intended usage (implied by friction) - Platform is being used like a **“bring-your-own-provider / OpenRouter-like router”**, but BYOK semantics and endpoint configurability are not consistently exposed or documented. - Users treat ElizaCloud as a quick “API playground,” but dashboard bugs (“send request” + scrolling) undermine that onboarding loop. ### Emerging requests aligned with real usage - **Cost transparency + control** (per-model pricing; cost evaluators in scenarios; token-limit concerns). - **Provider flexibility** (custom OpenAI endpoint). - **Operational tooling** for large plugin environments (relevance filtering, bootstrap optimization, evaluation slimming). --- ## 3) Implementation Opportunities (2–3 concrete solutions per major pain point) ### A) ElizaCloud UI blockers (scroll + API Explorer “api key required”) **High impact / Low–Medium difficulty** 1. **Add an in-dashboard “API key sanity check” panel** - Show active key source (workspace key vs overridden key), last-4 display, and whether the explorer is using it. - Similar pattern: Postman / Supabase API explorers show explicit “Using key: …” context. 2. **Fix scrolling via CSS containment + consistent overflow strategy** - Ensure scroll container captures trackpad events; improve scrollbar visibility. - Similar pattern: GitHub’s code viewer and Linear use visible, consistent scroll containers. 3. **Add a “Copy curl” fallback** - If “Send request” fails, still let users copy a fully formed curl with headers. Reduces support load. --- ### B) v2 documentation + unclear semantics (BYOK, pricing, HTTP wrapper examples) **High impact / Medium difficulty** 1. **Create a single “v2 Quickstart (known gaps)” page** - Explicitly list: recommended branch, missing docs areas, migration caveats, and canonical examples (e.g., Milady’s HTTP wrapper examples mentioned by Stan). - Similar pattern: Next.js “App Router” rollout docs included “migration notes + known differences.” 2. **Document “Keys & Billing” semantics with a truth table** - What does “use a different key” do? Does it override per-request/per-workspace/per-agent? Which providers supported? - Similar pattern: OpenRouter docs clearly separate “platform key” vs “BYOK.” 3. **Expose per-model pricing in UI + API** - Provide “effective rate” for current model and token counts; link to pricing source-of-truth. --- ### C) DB migration path + backend switching confusion (1.4→1.6→2.0; pglite vs Neon) **High impact / Medium difficulty** 1. **Implement a hard preflight check with actionable errors** - If a 1.4.x DB is detected, stop and print: “Upgrade to 1.6.x first” with commands + doc link (as suggested in core-devs). - Similar pattern: Prisma and Rails migrations fail fast with “You must run X before Y.” 2. **Publish a “supported upgrade paths” matrix** - Include: 1.4→1.6→2.0, and state what is unsupported (direct jump). 3. **Provide a simple export/import tool for pglite↔Neon** - Even a “best-effort” CLI (`elizaos db dump` / `elizaos db restore`) would reduce confusion that “it supports both” implies portability. --- ### D) Token/context bloat & 200k token limit issues with many plugins **High impact / Medium–High difficulty** 1. **Make action/provider filtering default in large-plugin setups** - Leverage/extend the ActionFilterService approach (vector + BM25) to reduce prompt payload automatically when action count is high. - Similar pattern: LangChain tool selection and “tool retrievers” limit tool lists. 2. **Introduce “evaluation budgets”** - Allow caps: max tokens for evaluations, max providers to include, or “evaluation sampling” in dev mode. 3. **Add a “prompt payload inspector”** - Show breakdown: bootstrap providers tokens, evaluations tokens, message history tokens—so users can trim intentionally. --- ### E) Provider integration flexibility (custom OpenAI endpoint URL) **High impact / Low difficulty** 1. **Add `baseURL` (endpoint) to OpenAI provider config** - Support OpenAI-compatible services without adding bespoke providers. - Similar pattern: OpenAI SDKs and many OSS clients support `baseURL`. 2. **UI/CLI validation + presets** - Provide presets for common OpenAI-compatible vendors; validate path (`/v1`), show test call. --- ### F) Duplicate LLM calls when URL present (webapp) **High impact / Medium difficulty** 1. **Deduplicate message processing early** - Decide: URL stays as text OR becomes attachment metadata, not both (per issue #6486). 2. **Add regression tests around SSE stream assembly** - Verify only one completion per user message; ensure `done` terminates exactly once. - Similar pattern: Vercel AI SDK tests for streaming assembly and partial chunks. --- ## 4) Communication Gaps (expectations vs reality) ### Recurring expectation mismatches - **“v2 recommended” vs “v2 documented”:** users are told to use v2.0.0, then immediately hit doc gaps (Discord, 2026-02-21). - **“Supports pglite and Neon”** interpreted as “supports moving between them,” but there is **no auto migration** (Discord, 2026-02-21). - **“Upgrade for holders”** created confusion about which asset/token gets benefits (Discord, 2026-02-22). ### Recurring questions indicating onboarding gaps - “Where does Shaw post updates?” + needing Twitter and Farcaster (Discord, 2026-02-20). - “What does ‘use a different key’ mean?” (Discord, 2026-02-20). - “Which version is stable?” (Discord, 2026-02-21). - “How do migrations work for v2?” (core-devs, 2026-02-22). ### Specific improvements to align expectations - Add a **single canonical “Official Updates” page** that aggregates Twitter + Farcaster + Discord announcements (even if just embeds/links). - Add **one pinned Discord FAQ**: version choice, migrations, keys/BYOK, Spartan vs Milady relationship, token/NFT benefit scope. --- ## 5) Community Engagement Insights ### Power users / high-signal contributors (and what they need) - **Odilitime (core)**: needs performance headroom (token limits), clearer migration telemetry (“how many users still need to migrate?”), and reduced baseline prompt overhead. - **Stan ⚡ (core)**: pushing safer DB isolation patterns + explicit migration error handling; needs coordinated merge/rebase workflows for large refactors. - **DarmaStef (builder)**: building practical agents for non-technical users; would benefit from a “Solutions Gallery” and packaged templates. - **Ogie / buzzbysolcex (ecosystem builder)**: shipping real plugins + registry PRs; needs clearer registry contribution pipeline and discovery. - **King Nebuluz (builder)**: exploring Erlang “unbreakable agents”; needs integration guidance / extension points. ### Newcomer friction signals - Confusion about tokens (milady.ai not launched; which token benefits). - Confusion about where to ask certain questions (price discussion redirected to trading channel). - Unanswered “help me open a ticket” request indicates support workflow is not obvious. ### Converting passive users into contributors - Create **“Good First Issue: Docs for v2 gaps”** labels and route frequent Discord questions into trackable issues. - Run a monthly **plugin showcase + office hours**: builders demo agents (like DarmaStef) and maintainers map needs to PRs. --- ## 6) Feedback Collection Improvements ### Current channel effectiveness - **Discord** is high-volume but low-structure: many questions remain unanswered or are answered ad-hoc (e.g., Milady chain validity, live codebase Q&A agent, Solana seeker deployment). - **GitHub** captures technical issues well (e.g., duplicate URL calls, custom endpoint request) but lacks linkage back to Discord reports and ElizaCloud UX bugs. ### Improvements for more structured, actionable feedback 1. **Add an in-app “Report a bug” button in ElizaCloud** - Auto-attach page URL, browser info, workspace ID (if allowed), and console logs snippet. 2. **Create Discord-to-GitHub bridge workflow** - A simple form that posts to GitHub with tags: `cloud-ui`, `docs-v2`, `migration`, `provider`. 3. **Standardize issue templates for “Cloud UX” vs “Core runtime” vs “Provider”** - Ensures reports include reproduction steps, expected/actual, and environment. ### Underrepresented segments (missing feedback) - **Non-crypto users** building business/productivity agents (only hinted by reservation/travel/database agent examples). - **Enterprise/self-host teams** (concerns like SSO, audit logs, compliance are implied but not directly represented in this slice). - **Windows/Linux environment specifics** (only one Linux signal: Ollama embeddings issue). --- ## Prioritized High-Impact Actions (next 2–4 weeks) 1. **Fix ElizaCloud API Explorer blockers** (scroll + “api key is required”) and add “Copy curl” fallback. 2. **Publish a v2 “Quickstart + Known Gaps + Migration Paths” doc**, including explicit 1.4→1.6→2.0 guardrails and Milady HTTP wrapper example links. 3. **Implement fail-fast migration preflight checks** with actionable error messages (detect legacy DB versions; block unsupported paths). 4. **Reduce baseline context bloat**: make relevance-based action/provider filtering the default when plugin count is high; add prompt payload inspector. 5. **Add OpenAI provider `baseURL` support** (custom endpoint URL) to unlock OpenAI-compatible vendors and reduce integration churn.