## User Feedback Analysis — 2026-01-28 (covering feedback surfaced in Discord 2026-01-25 to 2026-01-27 + recent GitHub issues) ### Data note (how counts/percentages were estimated) This analysis is based on ~23 distinct user questions/issues visible in the provided Discord summaries (partners/coders/core-devs/discussion) plus recent GitHub issues referenced in the dataset. Percentages below are approximate shares of these observed items (not statistically representative of the full user base). --- ## 1) Pain Point Categorization (Top recurring 5–7) ### 1. Documentation — **Knowledge plugin setup + cost behavior is unclear** (High frequency, high severity) **Observed frequency:** ~6/23 items (~26%) cluster around Knowledge plugin confusion/cost. **Examples from feedback** - Validation error: `EMBEDDING_PROVIDER` received `openrouter` but enum only allows `openai|google` (YogaFlame; fix suggested by DigitalDiva). - Unexpected high costs: $0.03–$0.04 per query and “tens of thousands of tokens” for small markdown docs due to `CTX_KNOWLEDGE_ENABLED=true` (Tyrone; diagnosed by 0xbbjoker). **Who it affects most** - New adopters trying Knowledge for the first time. - Cost-sensitive builders evaluating viability for production. --- ### 2. Technical Functionality — **Provider/config validation is brittle and error messages aren’t self-correcting** (Medium frequency, high severity) **Observed frequency:** overlaps with above; also seen in CLI/create and plugin behavior confusion. **Examples** - Knowledge plugin: enum mismatch yields a hard failure without “did you mean…” guidance. - CLI create failure with `ERR_PACKAGE_PATH_NOT_EXPORTED` after `bun i -g elizaos` and `elizaos create` (GitHub issue #6388, closed quickly but indicates onboarding fragility). **Who it affects most** - New developers following docs “as-is” (most likely to churn on first run). --- ### 3. Documentation + Communication — **Airdrop/token ecosystem questions dominate discussion and remain unanswered** (High frequency, high severity for community trust) **Observed frequency:** ~9/23 items (~39%) are token/airdrop/migration/liquidity/“what is Babylon” questions. **Examples** - “Will Babylon be 100% airdrop for holders?” → “That’s the idea but details aren’t finalized” (Odilitime). - “Move tokens off Kraken for airdrops?” → “No details announced” (Hexx). - “Does Solana DEX trading count?” / “What’s Babylon?” / “Are Gold and Eliza Town official tokens?” → unanswered in-thread. - ETH chain token liquidity/swap confusion (CRYPTONIAN). **Who it affects most** - Non-dev community members and investors. - Newcomers trying to understand legitimacy/scope. --- ### 4. Integration — **Workflow/integration direction is shifting; users need clarity on “plugins vs n8n vs MCP”** (Medium frequency, high strategic severity) **Observed frequency:** ~4/23 items (~17%) explicitly about integration architecture; implicitly impacts many more. **Examples** - Core pivot: integrate n8n with Eliza v2 (`@elizaos/core@next`) for natural-language workflow generation. - Users asking about OpenAI-compatible chat completions endpoint supporting OpenAI JSON API/OpenRouter/Kobold API (DigitalDiva; unanswered). - External MCP servers via Eliza Cloud API (`POST /api/v1/mcps`) (SATA; answered by SOLOMON VANDY), but docs/checklist requested. **Who it affects most** - Builders choosing architecture for production deployments. - Partners exploring platform integrations (e.g., Seeker phone Dapp store). --- ### 5. Technical Functionality — **Plugin action callbacks and multi-step planner behavior are confusing** (Lower frequency, moderate severity) **Observed frequency:** ~1–2/23 items (~4–9%), but impacts plugin developers. **Example** - In v1.7.2, user thought callbacks only sent first response; confirmed multiple callbacks supported; troubleshooting centered on planner config (onestep vs multistep) (Victor Creed + Odilitime). **Who it affects most** - Plugin developers building interactive actions that need progressive feedback. --- ### 6. Community + Security — **Scam/support-bot impersonation risk** (Lower frequency, high severity) **Observed frequency:** ~1/23 items (~4%), but severe impact. **Example** - “Create A Ticket” bot requesting wallet addresses; suspicious links after asking questions; confirmed scam (SATA + Odilitime). **Who it affects most** - Newcomers, token holders, anyone requesting support. --- ### 7. Performance/Cost — **Hidden cost multipliers and unclear “safe defaults”** (Medium frequency, high severity) This is distinct from docs: it’s about default configuration choices that can create surprise costs. **Example** - Contextual embeddings dramatically increase token usage; minimal config using `openai/text-embedding-3-small` via OpenRouter worked well for a 3.1MB PDF / 517 chunks (0xbbjoker). --- ## 2) Usage Pattern Analysis (Actual vs intended usage) ### What users are actually doing 1. **Using ElizaOS as an automation/workflow product, not a “coding agent”** - Core-devs explicitly estimate **“95% want AI workflow builder”** (s), aligning with user interest in Gmail/Calendar/Notion/Linear + if/then logic. 2. **Self-hosting on always-on low-power hardware** - Mini PCs used to avoid keeping a primary workstation running 24/7; interest in Oracle Cloud free tier deployments (LarpsAI). 3. **Extending agents via external tool servers (MCP) for specialized tasks** - AI red teaming with human-in-the-loop via external MCP servers registered in Eliza Cloud (SATA). 4. **Trading/crypto intelligence use cases** - Interest in PumpFun token tracking, wallet ROI monitoring, and “mapping human decision patterns” into agent logic (MATTIOBOY, web3buidl). 5. **Multi-channel chat deployment** - Demand for iMessage/SMS and “any chat app interface” (Sam + Agent Joshua). ### Emerging / unexpected use cases - **Partner distribution via alternative app ecosystems** (Seeker phone Dapp store): desire to ship a standout assistant where existing apps are low quality. - **Workflow generation + debugging loop with users**: “experimental workflow” labeling, then backend review/fix → add to library (s). This is closer to a “marketplace of validated automations” than a classic agent framework. ### Feature requests that align with actual usage - Conversational “add skills/plugins” flow through chat (Agent Joshua) maps directly to workflow-builder behavior. - OAuth gateway + credential reuse from plugins into n8n (Stan, Odilitime) matches multi-tenant automation needs. - Prebuilt workflow templates (check email, send email, etc.) plus safe experimentation path (s). --- ## 3) Implementation Opportunities (solutions per major pain point) ### A) Knowledge plugin confusion + cost spikes (Docs + Performance) **High impact / Low–Medium difficulty** 1. **Add a “Cost-safe defaults” preset** - Default `CTX_KNOWLEDGE_ENABLED=false` unless explicitly turned on. - Ship a “starter config” that pins a low-cost embedding model (e.g., `text-embedding-3-small`) and explains tradeoffs. - Similar pattern: **Supabase** and **Vercel** templates provide “production-safe defaults” with explicit opt-ins for expensive features. 2. **Add runtime cost/behavior warnings** - On startup, detect `CTX_KNOWLEDGE_ENABLED=true` and print an estimated multiplier warning (“contextual embeddings can increase token usage by X–Y; expected $/1k docs”). - Similar pattern: **Terraform** plan output shows “what will change” before apply; do the same for “what will cost.” 3. **Improve validation errors with actionable suggestions** - If `EMBEDDING_PROVIDER=openrouter`, error should suggest: “Use `openai` for embeddings; set OpenRouter in TEXT_PROVIDER instead (link).” - Similar pattern: **Next.js** config validation often includes “Did you mean …” plus docs links. --- ### B) Onboarding failures (CLI create + first-run reliability) **High impact / Medium difficulty** 1. **Add an automated “doctor” command** - `elizaos doctor` checks Node/Bun compatibility, confirms exports paths, verifies required env vars, and validates plugin config. - Similar pattern: **Flutter doctor**, **brew doctor** drastically reduce support load. 2. **Pin/declare supported runtime versions prominently** - The GitHub issue shows Node v22 + Bun global install edge cases; docs should specify tested combinations and “known good” versions. 3. **Provide a Docker-first quickstart for new users** - Given community self-hosting patterns, make “docker compose up” the primary path to avoid local env issues. - Similar pattern: **Airbyte** and **n8n** heavily rely on Docker as the fastest first success path. --- ### C) Integration architecture uncertainty (n8n vs native plugins vs MCP vs OpenAI-compatible endpoints) **High impact / Medium difficulty** 1. **Publish an “Integration Decision Guide”** - A single doc that answers: when to use **native Eliza plugin**, when to use **n8n node**, when to use **MCP tool server**, and how credentials flow (OAuth gateway). - Similar pattern: **LangChain** distinguishes Tools vs Retrievers vs Agents with clear selection criteria. 2. **Ship a reference “workflow builder MVP”** - 5–10 prebuilt workflows (Gmail triage, calendar scheduling, Notion task creation, Linear issue creation). - Mark AI-generated variants as “Experimental” with a structured debug report. - Similar pattern: **Zapier** starts with curated “Zaps” and then opens customization. 3. **Answer the OpenAI-compatible endpoint request** - Provide an official compatibility layer or adapter guide: OpenAI Chat Completions JSON, OpenRouter, Kobold. - Even a minimal “supported subset” spec would reduce repeated questions. --- ### D) Token/airdrop/migration information gaps (Communication + Community trust) **High impact / Low–Medium difficulty** 1. **Create a canonical “Airdrops & Eligibility” page** - Include: what’s confirmed vs unconfirmed, snapshot dates, exchange custody guidance (e.g., Kraken), DEX trading considerations, long-term holder criteria (if planned). 2. **Create a canonical “Token Ecosystem Map” page** - Clarify relationship of ElizaOS token vs other tokens mentioned (Babylon, Hyperscape, “Gold”, “Eliza Town”), with “official / unofficial / community” labels. - Similar pattern: **Optimism** and **Arbitrum** ecosystems maintain “official vs community” asset registries to reduce confusion. 3. **Publish a weekly “State of the Project” update with a fixed cadence** - Even short: what shipped, what’s next, what’s not decided. - Reduces speculation loops that dominate discussion. --- ### E) Plugin callbacks / multi-step planner confusion (Technical Functionality + Docs) **Medium impact / Low difficulty** 1. **Document callback timing semantics with examples** - “Callback as progress update” vs “callback at action completion” with a small code snippet. 2. **Add a test harness example plugin** - A “multi-callback demo action” that logs when callbacks fire under onestep vs multistep planners. - Similar pattern: **Discord.js** and **Slack Bolt** provide canonical examples for interactive flows. 3. **Expose planner config visibly in logs** - On action start, log `planner=multistep` and callback mode, so developers know which behavior they’re in. --- ### F) Scam/support-bot risk (Community + Security) **High impact / Low–Medium difficulty** 1. **Lock down support flows** - Official ticketing should be a single verified bot; all other bots removed/blocked where possible. 2. **Add automated scam detection messaging** - Auto-moderator posts when “wallet address” + “ticket” patterns appear, warning users. - Similar pattern: many large OSS Discords (e.g., crypto wallets, dev tooling) use AutoMod + pinned “Never share seed/private keys” banners. 3. **Create an “Official Links & Support” pinned message** - Include: migrate portal, docs, GitHub, ticket channel, and explicit “we will never ask for wallet address/seed.” --- ## 4) Communication Gaps (expectations vs reality) ### Recurring expectation mismatches - **“Eliza is ready as a polished assistant product” vs reality: active pivot** - Users see Clawdbot inspiration and expect immediate parity; core team is still assembling n8n/OAuth/workflow layers. - **“Holding tokens implies automatic airdrop eligibility” vs reality: not finalized** - Multiple questions show users assume exchange custody rules and DEX trading eligibility are already defined. - **“Knowledge plugin should be cheap for small docs” vs reality: contextual embeddings can explode tokens** - This is a major surprise cost factor. ### Recurring questions indicating onboarding/documentation gaps - Knowledge plugin: embedding provider enums, contextual embeddings costs, minimal config. - Integration: OpenAI-compatible endpoint support; MCP registration docs. - Project setup: `elizaos create` reliability; Git/Cursor workflow questions; “How do I open a ticket?” ### Specific documentation improvements to align expectations - Add “Known gotchas” section to Knowledge docs: enums + CTX cost. - Add “Integration paths” doc: plugin vs n8n vs MCP. - Add “Support & Safety” doc/pin: verified ticketing + scam warnings. - Add “Project status & cadence” page: weekly updates + roadmap snapshot. --- ## 5) Community Engagement Insights ### Power users observed (and their needs) - **0xbbjoker**: deep Knowledge plugin expertise; needs a path to upstream best-practice configs (could maintain a “knowledge cost baseline” doc). - **DigitalDiva**: integration-focused (OpenAI-compatible endpoints); needs official stance/adapter guidance. - **SOLOMON VANDY**: MCP/Cloud API helper; needs MCP docs + examples to reduce repeated support. - **LarpsAI**: self-hosting guidance; needs an official “recommended deployments” page. - **Core devs (s, Stan ⚡, Sam, Odilitime)**: pushing workflow pivot; need structured user testing channel for n8n workflow generation. ### Common newcomer questions indicating friction - “How do I open a ticket?” - “How do I generate a project / `elizaos create` errors?” - “How to move code from Cursor to git?” - “What is Babylon?” and “Do I need to move tokens off exchange?” ### Converting passive users into contributors - Create “Good first issue” packs around documentation gaps (Knowledge config, MCP docs, self-hosting guides). - Run a weekly “Workflow Template Jam”: users submit top requested Gmail/Notion/Calendar flows; contributors turn them into validated n8n templates. - Give visible credit (release notes shout-outs) to helpers who answer FAQs repeatedly (0xbbjoker, DigitalDiva, SOLOMON VANDY). --- ## 6) Feedback Collection Improvements ### Effectiveness of current channels - **Discord** is effective for rapid troubleshooting but produces **repeated, unstructured questions** (especially token/airdrop). - **GitHub issues** capture reproducible technical problems well (e.g., CLI create) but are underused for UX/documentation feedback. - Partner discussions are high-level but lack a structured intake for integration requirements. ### Improvements for more structured, actionable feedback 1. **Add a “Workflow Requests” intake form** - Required fields: target apps, trigger, conditions, desired outputs, auth requirements. - Auto-labels into a public backlog. 2. **Standardize “Cost bug report” template** - For Knowledge: model, chunk count, doc size, CTX enabled, tokens/query, $/query. 3. **Add opt-in telemetry for self-hosted (privacy-preserving)** - Capture anonymized: plugin usage, error classes, startup failures. This would validate whether Discord anecdotes reflect real frequency. ### Underrepresented user segments (missing feedback) - **Non-crypto developers** (noise from token discussions may be crowding out framework feedback). - **Enterprise/security users** (except the MCP red-teaming thread; needs more structured capture). - **Operators** running production agents (need uptime/perf/observability feedback loops). --- ## Prioritized High-Impact Actions (next 2–4 weeks) 1. **Ship “Knowledge Plugin: Safe Defaults + Cost Guide”** (disable CTX by default, add warnings, publish minimal configs + benchmarks). 2. **Publish canonical “Airdrops / Eligibility / Token Ecosystem” docs + pinned Discord message** (reduce ~39% of repeated community questions). 3. **Release an “Integration Paths” guide + MVP n8n workflow template library (5–10 workflows)** (aligns with the “95% workflow builder” usage reality). 4. **Add `elizaos doctor` + Docker-first quickstart path** (reduce first-run churn from CLI/runtime mismatches). 5. **Harden support/security in Discord (verified ticketing + anti-scam automod + official links hub)** (low effort, high risk reduction).