## User Feedback Analysis — 2026-01-29 (covering community activity through 2026-01-28)

### 1) Pain Point Categorization (Top recurring friction areas)

> Basis: Discord threads from 2026-01-26 to 2026-01-28 (coders/discussion/partners/core-devs) + recent GitHub issues referenced in the weekly/monthly rollups. Percentages are approximate shares of *distinct help threads/questions* observed in this dataset.

#### A. **Technical Functionality** (highest frequency + highest severity)
1) **Embeddings provider confusion + breakage (OpenRouter/enum mismatch)**
   - Symptoms:
     - Knowledge plugin errors when using OpenRouter embeddings; reports that “only OpenAI worked reliably.”
     - Validation error: `EMBEDDING_PROVIDER` rejecting `openrouter` (only `openai | google` valid) in certain versions/configs.
   - Evidence examples:
     - “OpenRouter embeddings … kept giving errors; only OpenAI has worked so far.” (YogaFlame)
     - “Invalid enum value… receiving ‘openrouter’” (Jan 27 Q/A)
   - Estimated frequency: **~20–25%** of observed technical support questions.

2) **Migration portal not detecting ai16z in Phantom / snapshot mismatches**
   - Symptoms:
     - Migration site fails to detect tokens in Phantom.
     - Eligibility mismatch tied to snapshot wallets (e.g., hardware wallet + hot wallet split).
   - Evidence examples:
     - “Migration site isn’t detecting ai16z in phantom wallet” (ai16zbags, NBA).
     - GitHub: “[Migration] Eligibility Mismatch & Snapshot Bug” (Tangem hardware wallet) (#6369).
   - Estimated frequency: **~15–20%** (but very high severity because it blocks users near deadlines).

3) **Plugin compatibility drift across branches (develop vs released plugins)**
   - Symptoms:
     - `plugin-anthropic 1.x` not working with develop branch; type fixes needed.
   - Evidence:
     - Core-devs action item: “Fix plugin-anthropic 1.x compatibility with develop branch” (Odilitime).
   - Estimated frequency: **~5–10%**, but high impact for contributors testing “next/develop”.

4) **Action handler callback behavior unclear / perceived broken**
   - Symptoms:
     - Users expected multiple callbacks “as documented” but only first callback appears; callbacks arrive as completion responses rather than immediate feedback.
   - Evidence:
     - Jan 26 plugin callback troubleshooting (Victor Creed + Odilitime).
   - Estimated frequency: **~5–10%**, moderate-high severity for real-time UX.

#### B. **Performance**
5) **Knowledge plugin cost explosions due to contextual embeddings**
   - Symptoms:
     - “$0.03–0.04 per query” / “tens of thousands of tokens” on small docs traced to `CTX_KNOWLEDGE_ENABLED=true`.
   - Evidence:
     - Jan 27 Q/A and configuration workaround (0xbbjoker).
   - Estimated frequency: **~5–10%**, but severe when it hits (billing shock).

#### C. **Integration**
6) **SSE streaming setup fragile (MIME_TYPE_MISMATCH); fallback to socket.io**
   - Symptoms:
     - SSE streaming fails with `MIME_TYPE_MISMATCH`; users switching to socket.io to get working behavior.
   - Evidence:
     - Jan 28 coders: Chucknorris recommends socket.io; root traced to backend deployment config.
   - Estimated frequency: **~5–10%**, moderate severity but blocks streaming UX.

#### D. **Documentation / Onboarding**
7) **Onboarding gaps: Git workflow basics + “what does SEARCH_KNOWLEDGE return?”**
   - Symptoms:
     - New contributors unsure about commit vs PR, staging, forks.
     - Misunderstanding that `SEARCH_KNOWLEDGE` summarizes (it does not; returns top fragments).
   - Evidence:
     - Jan 28 Git guidance (Odilitime); SEARCH_KNOWLEDGE clarification (0xbbjoker).
   - Estimated frequency: **~10–15%**, low-medium severity but contributes to slow Time-to-Value.

---

### 2) Usage Pattern Analysis (actual vs intended usage)

#### How users are actually using elizaOS
- **RAG / Knowledge plugin as a core capability**, not “optional”: repeated focus on embedding providers, chunking, cost, and query behavior indicates users treat knowledge retrieval as foundational for agents.
- **Workflow automation is emerging as the “default app” expectation**: strong pull toward **n8n-based automation** (Gmail/Calendar/Notion/Linear + conditional logic), aligning with the statement that “95% of users want AI workflow builders.”
- **Real-time conversational UX expectations**: questions about SSE streaming, callbacks, and summarization show users expect “chat-like immediacy” and “assistant-style” responses rather than raw tool outputs.
- **Social agents and community-facing personas**: repeated feedback that the @elizaos account should feel like an agent persona, not corporate; interest in a Twitter agent.

#### Emerging / unexpected use cases
- **Onchain agent identity & reputation (ERC-8004)**: users are tracking agent legitimacy (“larps”) and reputation verification as part of ecosystem trust.
- **Autonomous “avatar networks” (fetch.ai-like matching + meetings + reports)**: requests for similarity-matching agents that proactively connect members and summarize outcomes.

#### Feature requests aligned with observed usage
- **Intelligent model routing** (small local router model) to optimize cost/load/complexity.
- **Custom SEARCH_KNOWLEDGE action** that performs query planning + summarization (bridging raw fragments to usable answers).
- **Bootstrapping skill** that auto-connects agents to docs/GitHub/Discord knowledge sources on first run.

---

### 3) Implementation Opportunities (2–3 concrete solutions per major pain point)

#### Pain Point: Embeddings provider confusion + OpenRouter reliability
**High impact / Medium difficulty**
1) **Make embedding provider selection self-validating and auto-suggesting**
   - On startup, detect invalid enum (`openrouter`) and print a “did you mean” fix with the correct setting for that version.
   - Add a runtime preflight: “try embedding 1 sentence” and fail fast with actionable logs.
   - Similar pattern: *Supabase CLI* and *Terraform* both emphasize config validation with targeted remediation text.

2) **Ship a compatibility matrix + known-good presets**
   - Provide versioned presets for Knowledge plugin:
     - “Local (Ollama)”
     - “OpenAI low-cost”
     - “OpenRouter (experimental)”
   - Include expected env vars + example `.env`.
   - Similar pattern: *LangChain* “integrations” docs + *Dify* “provider presets”.

3) **Move embeddings fully behind plugin boundaries (accelerate v2.x direction)**
   - If v2.x is already planned, publish a clear “migration path” doc and deprecate ambiguous runtime flags.

#### Pain Point: Migration portal detection failures (Phantom / snapshot mismatches)
**Very high impact / Medium difficulty**
1) **Add “manual verify” flow with proof-of-ownership**
   - Let users submit: address, snapshot proof (tx hashes), and sign a message with the wallet.
   - Auto-triage into a queue; provide ticket ID + SLA.
   - Similar pattern: many token migrations (e.g., Arbitrum airdrop disputes) use signed-message dispute flows.

2) **Improve wallet compatibility diagnostics**
   - When Phantom connects, show detected token accounts + snapshot status; if none, display “what we checked” and common reasons (wrong network/account, token account not associated, etc.).

3) **Expose a “snapshot lookup” public endpoint**
   - Users can paste an address and see snapshot balance + eligibility without connecting a wallet (reduces confusion and support load).

#### Pain Point: SSE streaming / MIME mismatches
**Medium impact / Low-medium difficulty**
1) **Document “supported streaming transports” as a decision tree**
   - If deploying behind X reverse proxy, recommend socket.io/WebSocket first; list exact headers for SSE.
   - Provide known working templates (Docker + nginx + cloud providers).

2) **Add server-side content-type assertions + clearer errors**
   - If route returns wrong MIME, surface the exact route and expected `Content-Type: text/event-stream`.
   - Similar pattern: *FastAPI* and *Next.js* often include example middleware for SSE headers.

3) **Provide a one-command “streaming sanity check”**
   - A CLI command that hits the streaming endpoint and verifies incremental events.

#### Pain Point: Knowledge plugin cost spikes (CTX embeddings)
**High impact / Low difficulty**
1) **Rename and hard-guard “contextual embeddings”**
   - Make `CTX_KNOWLEDGE_ENABLED` default off; if enabled, require explicit “I understand costs” flag or print cost warnings.
   - Add a cost estimator: show expected token usage per ingest/query.

2) **Add “cheap mode” preset**
   - Auto-select `text-embedding-3-small` (or local embeddings), smaller chunk sizes, no context expansion.
   - Similar pattern: *OpenAI cookbook* recommends small embeddings for most retrieval tasks; *LlamaIndex* provides “cost-aware settings”.

3) **Telemetry hook (opt-in) for cost per query**
   - Log approximate tokens and estimated $ for embeddings + LLM calls.

#### Pain Point: Callback behavior / perceived brokenness
**Medium impact / Medium difficulty**
1) **Clarify the contract: progress events vs completion**
   - Rename “callbacks” to “progress events” in docs if that’s the actual model; show timing guarantees.

2) **Provide a reference action demonstrating multi-callback**
   - A built-in sample plugin action that emits 3 progress updates and a final result.

3) **Expose planner configuration in templates**
   - If multistep is required, ensure starter projects include a commented section explaining `onestep vs multistep`.

---

### 4) Communication Gaps (expectations vs reality)

1) **“Supported” embeddings vs “works reliably”**
   - Users hear “Ollama/OpenAI/OpenRouter supported,” but multiple reports indicate OpenRouter fails in practice.
   - Fix: label OpenRouter embeddings as **experimental** until verified across environments; publish known working models and configs.

2) **SSE advertised/assumed vs recommended approach**
   - Community guidance trends toward “just use socket.io,” implying SSE is brittle in real deployments.
   - Fix: explicitly document SSE limitations and recommended defaults per deployment type.

3) **SEARCH_KNOWLEDGE expectations**
   - Users assume summarization; reality is raw fragment retrieval.
   - Fix: rename or document “SEARCH_KNOWLEDGE returns top-k fragments; summarization is a separate action.”

4) **Token/migration/airdrop uncertainty**
   - Repeated questions about migration eligibility, token value proposition, and airdrop requirements show an expectations vacuum.
   - Fix: a single canonical “Token & Migration” page with snapshot rules, deadlines, supported wallets, and dispute flow.

---

### 5) Community Engagement Insights

#### Power users (and what they need)
- **Odilitime**: heavy support + architecture decisions (n8n, embeddings, tooling). Needs: tighter “official answers” surfaced in docs to reduce repeated support.
- **jin**: onboarding + knowledge infrastructure (MCP, repo/channel tracking). Needs: structured feedback loops from new devs on Time-to-Value blockers.
- **0xbbjoker**: deep Knowledge plugin expertise. Needs: ability to publish “official” cost/config guidance prominently.
- **Stan ⚡**: n8n workflow plugin + OAuth spec coordination. Needs: clear contributor roadmap + integration test harness.
- **DorianD**: ecosystem risk analysis + product persona feedback. Needs: governance guidance / disclaimers around legally risky dapps.
- **Hexx 🌐**: migration support. Needs: better tooling + clearer user-facing diagnostics to reduce manual explanation.

#### Newcomer friction signals
- Git basics (commit vs PR) and “where do I open a ticket?” recur—suggesting onboarding should include:
  - “How to contribute (Git 101)”
  - “Where to get help (tickets vs Discord channels)”
  - “First successful run” checklist.

#### Converting passive users into contributors
- Add “good first issue” funnels tied to the exact pain points:
  - docs fixes (embedding matrix, SSE guide)
  - starter templates (streaming sanity check)
  - migration portal UX copy improvements
- Run monthly “support-to-PR” sprints: take the top 3 Discord FAQs and turn them into merged doc PRs.

---

### 6) Feedback Collection Improvements

#### Current channel effectiveness
- **Discord** is effective for rapid debugging but produces repetitive answers (embedding config, streaming, migration).
- **GitHub issues** capture high-severity blockers (CLI create failure, migration bugs) but may miss “soft” UX friction (confusing defaults, unclear expectations).

#### Improvements for more structured, actionable feedback
1) **Standardize Discord → Issue intake**
   - Add a bot command (e.g., `/issue`) that creates a prefilled GitHub issue template with logs/env/version and links the Discord thread.

2) **Introduce a “Known Issues / Known Good Configs” living page**
   - Updated weekly; linked by helpers when questions repeat.

3) **Add lightweight onboarding survey**
   - After `elizaos create` and first run: ask 3 questions (what blocked you, what confused you, what you’re building).

#### Underrepresented segments
- **Non-technical builders** (who want workflows) are discussed as 95% of target, but their direct feedback is sparse in the dataset.
- **Production deployers** (cloud/reverse proxies) appear mainly through streaming MIME issues; need targeted feedback from teams deploying behind common infra (nginx, Cloudflare, Vercel-like setups).

---

## Prioritized High-Impact Actions (next 2–4 weeks)

1) **Publish a “Knowledge Plugin: Known-Good Configs + Cost Guardrails” page**
   - Include CTX embeddings cost warning, minimal configs, and provider reliability labels.

2) **Fix/clarify embeddings provider handling**
   - Add config validation + improved error messages; mark OpenRouter embeddings experimental until verified; ship presets.

3) **Migration portal: add diagnostics + dispute workflow**
   - Snapshot lookup tool + signed-message manual verification path + clearer Phantom/Tangem guidance.

4) **Streaming transport guidance + templates**
   - Officially recommend socket.io/WebSocket for most deployments; add SSE decision tree and a streaming sanity-check CLI.

5) **Turn top Discord FAQs into docs via a weekly “FAQ PR” cadence**
   - Commit/PR basics, SEARCH_KNOWLEDGE behavior, multistep vs onestep, and “where to open tickets.”