# User Feedback Analysis — 2026-01-30 (elizaOS) ## 1) Pain Point Categorization (Top recurring friction areas) ### A. **Documentation (Highest frequency + highest reputational risk)** **Recurring problems** 1) **Token utility and distribution is unclear/contested** - Discord 💬-discussion had **7/9 (≈78%)** of the surfaced “FAQ” questions centered on token value/utility, allocation, or migration mechanics (e.g., “Why should anyone buy elizaos token?”, “Why did team get 40%…?”, “wallets holding 40%… vesting schedule?”). - Multiple users explicitly stated the only understandable utility is **Jeju gas fees**, and asked for “actual use case beyond Jeju.” 2) **Migration instructions are fragmented and support-heavy** - Users are repeatedly redirected to “the migration channel” without a canonical step-by-step runbook (e.g., ETH-chain zero liquidity / 98% loss report; Phantom wallet not detecting tokens; Tangem snapshot mismatch in GitHub issue #6369). - The “migration portal didn’t detect holdings” issue appears across Discord (Phantom) and GitHub (#6369), suggesting systemic confusion, not one-off user error. 3) **Embedding/knowledge configuration is easy to misconfigure** - Users hit `EMBEDDING_PROVIDER` enum validation errors when setting it to `openrouter` (Discord 2026-01-27). - Users also report “only OpenAI works reliably” for embeddings (Discord 2026-01-28), implying docs don’t match observed reliability. **Most affected users**: newcomers attempting setup/migration; builders adopting knowledge plugin early. --- ### B. **Technical Functionality (High severity: breaks core behaviors)** **Recurring problems** 1) **Eliza 1.7.2 action callback ordering bug (critical)** - Reported by Victor Creed: callbacks execute **in reverse order**, and the **structured return message is omitted**. - Impacts custom plugins using `plugin-sql`, `plugin-openai`, `plugin-bootstrap`. This is both a correctness bug and a “trust in framework” issue. 2) **CLI/project generation breakage (recently fixed, but shows fragility)** - GitHub issue #6388: `elizaos create` failed with `ERR_PACKAGE_PATH_NOT_EXPORTED` after following docs; this directly blocks onboarding. - Weekly summary indicates this “front door” was restored, but it’s a warning sign that packaging/doc alignment can regress. **Most affected users**: plugin developers; anyone building custom actions; new developers starting projects. --- ### C. **Integration (High frequency in builder channels)** **Recurring problems** 1) **Social / OAuth integration uncertainty and duplicated effort** - Core team discussing building a “connection page” redirect flow, then discovering Composio already solves “in-chat auth” and has 100+ integrations. - Risk: building bespoke auth flows while also planning n8n/Composio leads to parallel, incompatible approaches. 2) **Workflow automation direction is shifting (native plugins vs n8n)** - Strong internal signal: “95% want AI workflow builder” and preference to adopt n8n plugin architecture (Discord 2026-01-27). - Users will need clearer guidance on “the blessed path” for integrations (native plugin vs workflow nodes vs Composio). **Most affected users**: teams building productized agents (email/calendar/SMS); developers deciding what to build against. --- ### D. **Performance / Cost (High severity for production usage)** **Recurring problems** 1) **Knowledge plugin cost surprises** - Reports of **$0.03–0.04 per query** and “tens of thousands of tokens” for small docs when `CTX_KNOWLEDGE_ENABLED=true`. - Users needed expert intervention (0xbbjoker) to discover minimal configs and cost controls. 2) **Embedding provider reliability mismatch** - “Only OpenAI has worked so far” sentiment in community (Discord 2026-01-28). Even if OpenRouter is “supported,” perceived reliability is a performance/reliability issue in practice. **Most affected users**: anyone deploying RAG/knowledge features at scale; cost-sensitive indie builders. --- ### E. **UX/UI (Moderate frequency; high onboarding impact)** **Recurring problems** 1) **Error messages lack “next step” guidance** - Examples: `Cannot find module '@elizaos/plugin-web-search'`, `MIME_TYPE_MISMATCH` for SSE, `Invalid enum value` for embeddings—users depend on Discord to interpret errors. 2) **Streaming transport confusion (SSE vs socket.io)** - SSE setup triggers MIME mismatches; users are advised to switch transports (socket.io) rather than guided through correct SSE deployment. **Most affected users**: newcomers integrating chat clients; builders deploying servers behind proxies. --- ### F. **Community / Governance (High severity: trust + participation)** **Recurring problems** 1) **Unanswered governance questions amplify distrust** - “Are the wallets holding 40% known and vesting schedule?” and “why did team get 40%…?” were **explicitly unanswered** in Discord logs (2026-01-29). - This pushes discussion toward speculation, increases moderation load, and distracts from building. **Most affected users**: token holders, prospective contributors evaluating legitimacy. --- ## 2) Usage Pattern Analysis (Actual vs intended usage) ### What users are actually doing 1) **Using elizaOS as an automation/workflow platform, not just “agent chat”** - The dominant builder narrative is: connect Gmail/Calendar/Notion/Linear + if-logic + scheduled tasks (n8n). - “95% want AI workflow builder” is a strong directional statement: users want outcomes and integrations more than agent “auto-coding.” 2) **RAG/knowledge plugin is a core adoption path—but users treat it as plug-and-play** - Users enable contextual embeddings without anticipating cost, suggesting they assume “smart by default,” while the system is “powerful but sharp.” 3) **Multi-transport chat deployments are common (SSE, socket.io)** - Users deploy behind various backends/proxies, encountering MIME issues—indicating elizaOS is being used in real hosting scenarios, not just local demos. 4) **Token/migration support is functioning like a product support queue** - Many “how do I…” questions are operational/accounting issues rather than dev issues, but they occur in main discussion channels. ### Emerging / unexpected use cases - **Model routing as a first-class feature request (“ElizaRouter”)** - Community is thinking about distributed routing + QoS decay, not just selecting “small/large” models. This aligns with production optimization needs. - **AI persona/social presence as a product feature** - Calls to make @elizaos more “soulful,” implying brand/UX is part of perceived product quality. ### Feature requests aligned with actual usage - n8n-based workflow generation + execution (GitHub issue #6429; Discord 2026-01-27) - OAuth gateway + “connection page” redirect flow (Discord core-devs) - Model router (RouteLLM-like) rewritten to TS/Rust (Discord coders) - Cost controls and clearer modes for Knowledge plugin (Discord 2026-01-27/28) --- ## 3) Implementation Opportunities (Concrete solutions per major pain point) ### 1) Token utility/distribution confusion (Documentation + Community/Governance) **Impact**: Very high (trust + retention). **Difficulty**: Medium (mostly comms + some dashboards). **Solutions** 1) **Publish a single canonical “Token Utility & Flows” page + diagram** (High impact, low dev effort) - Include: where token is used (products, Jeju gas), buyback mechanics, and explicit non-goals. - Add a “So why separate tokens for Hyperscape/Babylon?” section (already asked and answered, but not discoverable). 2) **Transparency package for allocation + vesting** (High impact, medium effort) - Provide: wallet addresses, vesting schedule, and on-chain proof links. - Similar projects: ENS + Uniswap publish allocation breakdowns and vesting/wallet visibility; many ecosystems maintain public “treasury dashboard” pages (often via Dune). 3) **Move token debates to a structured format** (Medium impact, low effort) - A pinned Discord post that links to the above docs; require new token questions to include a checklist (“Have you read X?”). - Create a GitHub “tokenomics” discussion category to centralize and reduce repeat threads. --- ### 2) Migration failures & liquidity confusion (Documentation + Integration) **Impact**: High (users reporting catastrophic losses). **Difficulty**: Medium–High. **Solutions** 1) **“Migration Runbook” with decision tree and warnings** (High impact, medium effort) - Separate concepts clearly: *swap vs bridge vs migration claim*. - Include a red-box warning: “Do not attempt to swap on ETH if liquidity is zero—this is not migration.” - Add wallet-specific steps (Phantom, Tangem) since these were explicitly problematic. 2) **Migration portal UX improvements: “diagnostics mode”** (High impact, higher effort) - When holdings aren’t detected, show: snapshot block/time, detected addresses, and common causes (wrong chain, wrong address, token account mismatch). - Offer “submit evidence” flow (like GitHub issue #6369: address + tx hashes) instead of pushing users into chat. 3) **Status page + liquidity/migration notices** (Medium impact, low–medium effort) - Similar approaches: bridges often maintain a status page + pinned incident updates. - Reduces panic and repeated Discord questions. --- ### 3) Action callback ordering bug in 1.7.2 (Technical Functionality) **Impact**: Very high for plugin developers. **Difficulty**: Medium. **Solutions** 1) **Add a regression test that asserts message sequence** (High impact, low effort) - Test expected order: initial feedback → structured return → detailed callback. - Ensure coverage across affected plugins (`plugin-sql`, `plugin-openai`, `plugin-bootstrap`) or at least the message service layer. 2) **Introduce a “callback message contract” API** (High impact, medium effort) - Make callback phases explicit types/events (e.g., `CallbackPhase.INITIAL`, `STRUCTURED`, `DETAIL`) and enforce ordering in the message service. 3) **Document version pin + workaround until fixed** (Medium impact, low effort) - Provide a mitigation path for users shipping now (e.g., disable callback streaming or restructure messages). *Comparable patterns*: LangChain and Temporal-style workflow engines enforce deterministic event ordering via explicit event logs rather than ad-hoc message sends. --- ### 4) Knowledge plugin cost + embedding confusion (Performance + Documentation) **Impact**: High (unexpected spend). **Difficulty**: Low–Medium. **Solutions** 1) **Safe defaults + “cost mode” presets** (High impact, medium effort) - Default `CTX_KNOWLEDGE_ENABLED=false` with an explicit “High quality (higher cost)” toggle. - Add a startup warning when contextual embeddings are enabled (“estimated cost multiplier”). 2) **Cost instrumentation + logging** (High impact, medium effort) - Print chunk counts, embedding model, and estimated tokens per query. - Similar projects: LlamaIndex exposes chunking/token stats; many SDKs show estimated token usage pre-call. 3) **Fix docs and validation messaging around providers** (Medium impact, low effort) - If OpenRouter embeddings are “supported but flaky,” say so and provide known-good configurations (e.g., OpenAI embedding model via OpenRouter routing, as users did). --- ### 5) Plugin install / module resolution errors (Integration + UX) **Impact**: Medium (blocks features). **Difficulty**: Medium. **Solutions** 1) **Standardize plugin packaging + export maps** (High impact, medium effort) - The `@elizaos/plugin-web-search` “Cannot find module” suggests inconsistent module resolution/exports. 2) **Add `elizaos doctor` diagnostics** (Medium impact, medium effort) - Check node/bun versions, package manager, module resolution, and known misconfigs. 3) **Docs: “Known-good install paths (npm/bun/pnpm)”** (Medium impact, low effort) - Include exact commands and lockfile expectations. --- ### 6) Transport confusion (SSE MIME mismatch) (UX + Integration) **Impact**: Medium. **Difficulty**: Medium. **Solutions** 1) **Provide “Deployment recipes” for SSE behind common proxies** (High impact, medium effort) - Nginx/Cloudflare/Render/Fly configs; correct headers; avoid buffering. 2) **Explicitly recommend socket.io when appropriate** (Medium impact, low effort) - If socket.io is the “happy path,” say so and explain tradeoffs. 3) **Improve error messages** (Medium impact, low effort) - When MIME mismatch occurs, print expected Content-Type and likely misconfiguration. --- ## 4) Communication Gaps (Expectations vs reality) 1) **“Supported” vs “works reliably” (embeddings/providers)** - Official stance: Ollama/OpenAI/OpenRouter supported. User experience: “only OpenAI worked.” - Needed: compatibility matrix (“tested combinations”) + “known issues” section. 2) **Knowledge plugin is perceived as plug-and-play, but it’s an expert system** - Users enable contextual embeddings without understanding cost. - Needed: onboarding warnings + a “first 30 minutes” guide for knowledge/RAG with safe defaults. 3) **Migration guidance relies on Discord tribal knowledge** - Redirecting users to a channel is not a substitute for a maintained runbook. - Needed: a single canonical doc + incident/status updates during critical periods. 4) **Workflow strategy is evolving but not explicitly communicated** - Users/builders need to know: “Should I build a native plugin? Or an n8n workflow node? Or use Composio?” - Needed: a short “Integration architecture” RFC + roadmap. Recurring questions indicating onboarding gaps: - “Invalid enum value for EMBEDDING_PROVIDER…” - “Cannot find module @elizaos/plugin-web-search…” - “MIME_TYPE_MISMATCH with SSE…” - “Migration portal doesn’t detect my wallet…” - “Why separate tokens / what’s token utility?” --- ## 5) Community Engagement Insights ### Power users (high leverage) and needs - **DorianD**: driving model routing architecture + ecosystem risk analysis; needs a formal RFC lane and maintainer feedback loop. - **Stan ⚡**: already built **plugin-composio** and is drafting an RFC; needs a clear path to upstreaming and alignment on “official integrations strategy.” - **jin**: MCP support + dynamic repo/channel tracking + onboarding automation; needs prioritization and product support to turn these into “default DX.” - **0xbbjoker**: de facto expert on knowledge plugin costs/config; needs a way to convert guidance into official docs quickly. - **Victor Creed**: found a critical callback bug; needs maintainer acknowledgment + triage + a target fix version. ### Common newcomer questions (onboarding friction) - CLI/project creation and package manager issues (GitHub #6388). - Basic Git workflow (“commit vs PR”)—suggests many first-time OSS contributors. - Config errors with embeddings and knowledge plugin costs. - Migration/eligibility detection issues (Phantom/Tangem). ### Converting passive users into contributors 1) **“Docs-as-code” fast lane (already emerging)** - Reinforce sayonara’s PR pathway to `elizaOS/docs` with templates and “good first doc issue” labels. 2) **RFC program with clear ownership** - Lightweight RFC repo/process for: routing (“ElizaRouter”), integrations (n8n vs Composio), auth connection page spec. 3) **Bug bounty / “repro reward” for high-severity issues** - Especially for callback ordering + embedding provider reliability; pay in recognition or small bounties to motivate deep debugging. --- ## 6) Feedback Collection Improvements ### Current channel effectiveness - **Discord** is effective for rapid troubleshooting but creates: - repeated questions, - lost answers, - “redirect to channel” loops. - **GitHub issues** capture reproducible bugs well (e.g., #6388, #6369), but many user problems (migration/liquidity, config confusion) never become actionable issues with logs. ### Improvements for more structured, actionable feedback 1) **Introduce “Support Bundles”** (High impact) - A CLI command to generate a sanitized bundle: versions, config summary, provider selection, recent logs. Users attach to GitHub issues. 2) **Triage taxonomy + required fields** (Medium impact) - Issue templates for: Migration portal, Plugin install, Knowledge costs, Streaming transports. - Require: environment, steps, expected vs actual, logs, wallet address (if migration). 3) **Monthly “Top 10 friction” survey pinned in Discord** (Medium impact) - Multiple-choice to quantify pain (cost surprises, migration detection, install errors, etc.) and capture underrepresented voices. ### Underrepresented segments (missing feedback) - **Non-crypto builders** who only want automation/agents and may be discouraged by token-heavy discussions. - **Production deployers** (DevOps/SRE) encountering SSE/proxy issues—need more structured deployment feedback. - **Python-first users** (v2 Python runtime is active) who may not participate in Discord but hit runtime/packaging issues. --- ## Prioritized High-Impact Actions (Next 2–4 weeks) 1) **Ship a canonical Migration Runbook + portal diagnostics UX** - Goal: reduce “redirect to migration channel” dependence; prevent catastrophic swap mistakes; handle wallet detection failures with evidence-based escalation. 2) **Publish Token Transparency & Utility page (allocation + vesting + flows)** - Resolve unanswered questions (“40% wallets + vesting schedule”) with verifiable links; reduce reputational drag in main channels. 3) **Fix (and regression-test) the Eliza 1.7.2 callback ordering/omission bug** - Add deterministic ordering tests in message service; publish workaround/version guidance until patched. 4) **Make Knowledge plugin “safe by default” with cost presets + warnings** - Default CTX embeddings off; add visible cost instrumentation; provide known-good configs for embeddings. 5) **Clarify the official integration strategy (n8n vs Composio vs native plugins) via an RFC** - Decide the “blessed path,” align connection page work with Composio/n8n, and prevent duplicated auth/integration implementations.