## User Feedback Analysis — 2026-01-04 (based on data through 2026-01-03) ### Data volume & confidence notes - Sources included: Discord (discussion/coders/core-devs), GitHub issues/PRs for `elizaos/eliza`. - GitHub UX issues are dominated by a single reporter (borisudovicic). Treat as *high-signal directional feedback*, but not necessarily representative of the whole user base. - Discord shows a split between newcomers asking “what is this?” and builders trying to ship integrations (web, Discord, multi-model, cloud containers). --- ## 1) Pain Point Categorization (top recurring 5–7) ### 1) UX/UI — Public agent experience & navigation friction (high frequency, high severity) **Evidence** - GitHub issues: - #6318 “Scroll should work on whole page” - #6317 “Connect wallet should ideally go straight to wallet options” - (Also in monthly top issues list) #6313 “Separate public agent states”, #6311 “Chat summaries don’t really make much sense”, #6312 “Limit messages for non-signed up user to ~2-3”, #6314 “Public agent cards should have chat number” - Pattern: the same user opened multiple UX issues; nevertheless, they cluster into a coherent funnel problem: *visitor → try agent → sign in / connect wallet → understand agent value*. **Problems affecting most users (by likely impact)** - Visitors landing on a public agent link see controls that don’t match their intent (edit/toggles/fork expectations). - Wallet connection flow adds extra steps and cognitive load. - Page scrolling inconsistently works, creating “broken site” perception. - Chat summaries are confusing, reducing perceived usefulness of history and “memory”. --- ### 2) Documentation — “How do I…?” gaps for core workflows (high frequency, medium-high severity) **Evidence (Discord coders FAQ)** - Repeated “how” questions, several unanswered in-channel: - Website integration via API key (unanswered) - Calculation/forecasting approach (unanswered) - Add “knowledge & lore” sections like previous versions (unanswered) - Twitter poll posting (unanswered) - Direct request in action items: “Create examples of multi-model agent implementations”, “Document deploying custom plugins in cloud containers”, “Update documentation for new logging standards”. **Problems affecting most users** - Users can’t find canonical examples for multi-model routing, cloud deployment with custom plugins, and web embed/API patterns. - Version-to-version config changes (e.g., “knowledge & lore”) are not explained with migration guidance. --- ### 3) Technical Functionality — Integration errors & config regressions (medium frequency, high severity when hit) **Evidence** - “Model not found” error when integrating agent with a website (BAOVERSE🌟). - Agent “inability to recall info from bio” / missing “knowledge & lore” sections (YogaFlame). - SQL plugin bug fixed via PR #6316 (raw SQL for `SET LOCAL` to support data isolation); indicates sharp edges in production settings. **Problems affecting most users** - Integration failures are hard to diagnose without a clear error taxonomy and fixes. - Memory/knowledge configuration regressions feel like “the agent forgot who it is,” which is severe for agent quality. --- ### 4) Integration — Plugin versioning & deployment constraints (medium frequency, medium-high severity) **Evidence** - Discord plugin versioning confusion: v1.3.3 vs v1.3.5; “failed publishing of v1.3.4 update” mentioned (YogaFlame). - Questions about deploying agents with custom plugins; guidance exists (“cloud containers”), but lacks step-by-step docs. **Problems affecting most users** - Plugin distribution/versioning creates trust issues (“which version works with current ElizaOS?”). - Users are unsure whether cloud supports their plugin stack, and how to package it. --- ### 5) Performance — MultiStep/provider execution & perceived latency (low-medium frequency, high impact for power users) **Evidence** - PR #6263 merged: parallel provider handling using `Promise.allSettled` in MultiStep to improve speed. - Logging improvements and linting indicate ongoing operational maturity work. **Problems affecting most users** - Power users building multi-provider agents feel latency quickly; improvements are happening, but users need clarity on best practices and expected performance. --- ### 6) Community / Positioning — “What is ElizaOS really?” skepticism + token/migration confusion (medium frequency, medium severity) **Evidence** - Discord: repeated framing questions (“Eliza is an operating system?”, “Like linux?”). - Skepticism: whether it’s “just wrapping existing AI models.” - Token migration questions (AI16Z → ElizaOS) largely unanswered on Jan 1; still a visible gap. **Problems affecting most users** - Unclear mental model: framework vs hosted product vs “OS” metaphor. - Web3-adjacent users have expectations around migration/support that don’t map to current support bandwidth and docs. --- ## 2) Usage Pattern Analysis (actual vs intended) ### How users are actually using ElizaOS 1) **Multi-model orchestration in one agent** - Users want Anthropic for calculations/forecasting and OpenAI for reasoning, with task-based routing. - They are reaching for OpenRouter to unify provider config via env. 2) **“Agent as an integration surface” more than “agent as a chat toy”** - Website integration via API keys. - Social posting automation requests (Twitter polls; Farcaster scheduling mentioned earlier). - Discord plugin compatibility remains a gateway use case. 3) **Game worlds + autonomous NPCs** - Ongoing interest in “Zelda with agents” / RuneScape fork: agents populating towns, quest generation, NPC setup. 4) **Advanced autonomy research** - roseOS using ElizaOS as substrate for constraint-aware reasoning, explicit agency boundaries, accountability layers (traceability and policy constraints). ### Emerging / unexpected applications - **Lead-generation education funnel**: “agentic AI crash course” concept tied to ElizaOS cloud services. - **Crypto-earning/metaverse hooks** (Hyperscape earning functionality), which may pull product expectations toward wallets, gating, and UX polish. ### Feature requests aligned with observed usage - First-class **multi-model routing patterns** (beyond “set model in env”). - **Web embed / API quickstart** (agent-to-website). - **Social actions** (Twitter poll creation) and stronger plugin capability matrices. - **Public agent funnel redesign** (visitor vs signed-in vs owner states). --- ## 3) Implementation Opportunities (solutions per major pain point) ### A) Public agent UX funnel confusion (issues #6313/#6317/#6318 + related) **Proposed solutions (prioritized by impact vs difficulty)** 1) **High impact / Medium difficulty:** Implement *three explicit public-agent UI states* (Visitor, Authenticated Non-Owner, Owner) as proposed in #6313. - Visitor: clean chat, agent bio, limited messages, soft gate to sign-up. - Auth non-owner: Chat + Fork, creator attribution, no edit/toggles. - Owner: full management UI. - *Similar approaches:* ChatGPT share links (limited capabilities when not signed in), GitHub repo pages (read-only vs contributor permissions). 2) **High impact / Low difficulty:** Fix “scroll should work on whole page” (#6318) as a priority “paper cut” that affects first impressions. - Add an automated Playwright/UI test for scroll containers to prevent regression. 3) **Medium impact / Medium difficulty:** Streamline wallet connection (#6317) to go directly to wallet options. - *Similar approaches:* WalletConnect modal patterns that surface wallet choices immediately and remember last-used wallet. --- ### B) Documentation gaps for multi-model + cloud containers + web integration **Proposed solutions** 1) **High impact / Low-medium difficulty:** Publish 3 canonical recipes: - “Multi-model agent with task routing” (OpenRouter + per-tool/provider selection) - “Deploy custom plugins in cloud containers” (build/push/run + environment templates) - “Website integration with API key” (minimal JS snippet + backend proxy example) - Include “expected errors & fixes” section (“Model not found”, auth failures). 2) **Medium impact / Low difficulty:** Add a “Config migrations” doc page: - Explicitly address “knowledge & lore” / “bio recall” changes and recommended replacements. - Provide a before/after JSON example and version markers. 3) **Medium impact / Medium difficulty:** Add a docs-driven “capability matrix” for plugins (Discord/Twitter/OpenRouter/etc.): - Supported ElizaOS versions, required env vars, known limitations. *Comparable projects:* LangChain’s “cookbooks”, Supabase “Quickstarts”, Vercel integration guides with copy-paste deploy buttons. --- ### C) Integration errors: “Model not found” + memory/knowledge regressions **Proposed solutions** 1) **High impact / Medium difficulty:** Introduce a standardized error code system and troubleshooting links for common integration failures. - Example: `E_MODEL_NOT_FOUND` with actionable hints (invalid model slug, provider not enabled, OpenRouter mapping missing, env not loaded in container). 2) **High impact / Medium difficulty:** Add a validation step to agent startup: - Validate model/provider availability and required env vars before serving requests. - Fail fast with clear UI + logs. 3) **Medium impact / Medium difficulty:** Restore or replace “knowledge & lore” ergonomics: - Either re-introduce structured fields (with migration) or provide an official “knowledge injection” mechanism (seed memory store from bio/lore at creation time). - *Comparable projects:* Rasa’s NLU training data validation; OpenAI Assistants “instructions + files” separation. --- ### D) Plugin versioning confusion (Discord plugin, browser stores) **Proposed solutions** 1) **High impact / Low difficulty:** Maintain a single “Supported plugin versions” page (or registry metadata) listing: - Latest stable, compatibility ranges, and deprecation notices. 2) **Medium impact / Medium difficulty:** Add runtime plugin compatibility checks: - Warn users when plugin version mismatches core APIs (especially messaging plugins like Discord/Telegram). 3) **Medium impact / Medium difficulty:** Improve release automation: - Ensure failed publishes (e.g., missing v1.3.4) are surfaced as CI alerts and mirrored in changelogs. *Comparable projects:* Homebrew formula version pinning, VS Code extension compatibility constraints. --- ### E) Performance & observability (MultiStep + logging linter adoption) **Proposed solutions** 1) **Medium-high impact / Low difficulty:** Document “How to debug provider latency” using the new logging format + linter, with example traces for MultiStep. 2) **Medium impact / Medium difficulty:** Add built-in timing spans around provider calls and surface in a developer dashboard. 3) **Medium impact / Medium difficulty:** Provide “recommended provider concurrency limits” to avoid rate-limit storms when running `Promise.allSettled`. *Comparable projects:* OpenTelemetry spans, Next.js route timings, Temporal workflow visibility. --- ## 4) Communication Gaps (expectations vs reality) ### Where expectations don’t match reality - **“ElizaOS is an operating system” metaphor** helps marketing, but confuses newcomers: they still ask “How does eliza work?” and whether it’s “just wrapping models.” - **Cloud vs self-host boundaries**: users ask whether they must deploy outside cloud to use custom plugins; answers exist but aren’t discoverable. - **Multi-model support**: users expect first-class task routing UI; current answer is “use OpenRouter + env configuration,” which feels indirect. ### Recurring questions indicating onboarding gaps (Discord) - “How does eliza work?” / “How is it composable?” - “How do I integrate my agent into a website with an API key?” (unanswered) - “Can I add knowledge & lore sections like previous version?” (unanswered) - “Is there a Discord plugin that works with current version?” (seen earlier) ### Specific improvements to align expectations - Replace/qualify the “OS” tagline in onboarding with a concrete diagram: - Core runtime (memory/tools/providers) + plugin ecosystem + cloud hosting option. - Add a “What you can build in 30 minutes” onboarding path with 3 tracks: - Chat agent, Website agent, Social/Discord agent. - Publish a transparent “Supported / Experimental / Not supported” list (e.g., Twitter poll posting likely “not yet”). --- ## 5) Community Engagement Insights ### Power users & their needs (observed) - **Stan ⚡ (core dev)**: focuses on performance/logging quality; needs adoption of logging linter across plugins and consistent instrumentation. - **borisudovicic (high-signal UX reporter)**: wants refined public-agent funnel, clearer states, better summaries and metrics (chat number). - **Omid Sa / YogaFlame (builders)**: need reliable plugin/versioning, multi-model recipes, configuration migration clarity. - **Kenk / Shaw (product/community drivers)**: push new use cases (crash course funnel, game/NPC quest generation); need clearer platform narratives and examples. ### Newcomer questions indicating friction - Basic positioning: “What is ElizaOS?” “Like linux?” - Where to find docs / examples (Borko had to link docs directly). - Integration basics (API keys, model selection) not answered consistently in chat. ### Converting passive users into contributors - Create a “good first issue” stream specifically for: - Docs recipes (multi-model, web integration) - UI papercuts (scroll/container fixes) - Plugin compatibility metadata - Add a contributor pathway in Discord: - Weekly “triage hour” where core team labels issues and pairs newcomers with maintainers. - Recognize non-code contributions (docs PRs, reproduction repos, minimal test cases). --- ## 6) Feedback Collection Improvements ### Effectiveness of current channels - **Discord**: fast, but answers get lost; multiple unanswered technical questions indicate throughput limits. - **GitHub issues**: actionable and detailed (especially UX), but currently skewed toward one reporter and lacks structured reproduction/impact fields for integration errors. ### Improvements for more structured, actionable feedback 1) Add **issue templates** for: - “Integration error” (provider, model slug, env, logs snippet, deployment mode, reproduction steps) - “Public agent UX” (state, signed-in/out, expected vs actual, screenshots) 2) Add a **Discord bot command** (`/report`) that: - Captures question + context + links to docs tried - Opens a prefilled GitHub Discussion/Issue when unresolved after X hours 3) Run a **monthly in-product micro-survey** (1–2 questions) targeting: - First-time public agent visitors (“What stopped you from continuing?”) - Builders deploying plugins (“Where did deployment fail?”) ### Underrepresented segments whose feedback is missing - Teams running **production workloads** (SLOs, rate limits, security/compliance needs). - **Non-Web3** users (wallet flow dominates UX discussion; may not represent SaaS/enterprise expectations). - **Non-developer creators** (prompt/agent designers who need UI tools rather than env-file configuration). --- ## Prioritized High-Impact Actions (next 2–4 weeks) 1) **Implement distinct public-agent UI states (Visitor / Auth Non-Owner / Owner)** + streamline wallet connect entry (issues #6313 + #6317). 2) **Ship the “Web integration + API key” and “Multi-model routing” canonical recipes** (with copy-paste examples and common errors like “Model not found”). 3) **Fix top UI papercuts immediately** (scroll-on-whole-page #6318) and add regression tests for layout/scroll containers. 4) **Establish plugin compatibility/version transparency** (supported versions page + registry metadata; address Discord plugin version confusion). 5) **Add startup validation + standardized error codes** for model/provider availability and env configuration to reduce integration dead-ends (“Model not found”, missing knowledge/lore expectations).