# Hyperfy Discord - 2025-02-09

## Overall Discussion Highlights

### Build Mode & Controls
- **New Pointer-Lock Building Interface**: Ashxn has implemented a significant upgrade to the building system, replacing the previous right-click system with a more intuitive pointer-lock based interface featuring snap-to-grid, rotation controls, and duplication functionality.
- **Build Controls**: Users can press Tab to toggle build mode, then use left click to move objects, right click to duplicate, X to delete, and mouse wheel to rotate.
- **UI Improvements**: New dropdown components have been added, along with improved build mode controls for grab, duplicate, destroy, and rotate functions.
- **Community Feedback**: Users are requesting additional features like undo/redo functionality, grouping capabilities, and more precise rotation controls.

### Agent Development
- **Agent Behaviors & Needs Systems**: MetaMike is developing sophisticated agent behaviors with needs systems that could support MMO/RTS-style gameplay.
- **Multilingual Agents**: Users are implementing AI agents like Eliza and Victor that can navigate the virtual world, recognize locations, and interact with users in multiple languages.
- **Agent Configuration Dashboard**: Szjanko previewed an agent configuration dashboard for Hyperfy that will allow users to customize agent settings through a form interface with preset options.
- **Location Awareness**: Developers are training agents to recognize locations and provide directions within virtual environments.

### 3D Asset Optimization
- **VRM Avatar Creation**: Significant discussion around proper rigging, weight painting, and mesh optimization for VRM avatars, with the insight that V2 requires properly skinned meshes with weight painting rather than just parenting objects to bones.
- **Asset Quality Rating**: TheMattEmpire proposed a quality rating system for assets based on optimization and appearance, similar to rarity levels in games.
- **Optimization Techniques**: Community members shared workflows for converting FBX files to GLB format using Blender and further optimizing with gltf.report.

### Infrastructure & World Portability
- **World Export/Import**: Ash suggested using zip files of world folders as the simplest solution for world exports/imports between different Hyperland hosting platforms.
- **Implementation Challenges**: Discussion around standardizing the zip format, handling folder structure, and ensuring version compatibility for portable worlds.
- **Security Considerations**: Maximus raised concerns about versioning information and security checks for imported worlds.

### Integration & Interoperability
- **IoT Integration**: HPrivakos demonstrated integration between Hyperfy and real-world IoT devices, controlling Philips Hue lights from within the platform.
- **Blockchain Integration**: Development of a Solana branch was confirmed, with multiple people working on integrating other chains as well.
- **VRM in NFT Metadata**: Roustan shared a project that successfully incorporated VRM files in NFT metadata, allowing 3D avatar models to be associated with NFTs.

### Documentation & Resources
- **Consolidated Documentation**: Omka created a new "Awesome Hyperfy" site (hyperfy.how) to consolidate scattered resources in a searchable format.
- **Metaverse Design Resources**: Community members shared links to metaverse design toolkits and articles about open standards for XR interoperability.

## Key Questions & Answers

**Q: What is the new build mode being implemented?**  
A: A pointer-lock based building interface with features like snap-to-grid, rotation, duplication, and destroy functionality. (Ashxn)

**Q: How do I use the new build controls?**  
A: Press Tab to toggle build mode, then use left click to move objects, right click to duplicate, X to delete, and mouse wheel to rotate. (Ashxn)

**Q: How did you train the agent to know where things are? How easy is it to re-train when things move?**  
A: Repeatedly asking/telling it to scan for locations and points of interest, then asking for directions from different spots until it gets accurate. (drdoge.eth)

**Q: How do you usually convert fbx to glb optimized for hyperfy?**  
A: Import fbx to Blender, export glb. Before dropping into a world use the gltf.report site to compress it down further if needed. (Saori and maximus)

**Q: What causes VRM models to fail in Hyperfy V2?**  
A: V2 requires properly skinned meshes with weight painting rather than just parenting objects to bones. (Ashxn)

**Q: How do you parent with automatic weights in Blender?**  
A: Select all meshes first, then the rig, right-click and choose "parent > Automatic weights". (TheMattEmpire)

**Q: What would be the simplest approach for world portability?**  
A: A zip of the world folder, with import/export buttons in hosting dashboards. (ash)

**Q: Do we have a way to have secrets in scripts yet?**  
A: Not yet but could be implemented with a command like `/secret agentApiKey value` that would be accessible only on the server. (Ashxn)

**Q: Is Claude Sonnet 3.5 going to be available in the agent configuration dashboard?**  
A: Not sure, as it's "x30 more expensive than other models on OpenRouter". (HPrivakos)

## Community Help & Collaboration

### VRM Avatar Creation
- **Valiant helped devilsadvocate.sol** create a functional VRM avatar with a cube-based character by replacing the original armature with default VRM armature, merging vertices, resizing bones, and parenting with automatic weights.
- **TheMattEmpire assisted devilsadvocate.sol** with understanding how to parent meshes to bones with proper weighting, explaining the process of selecting meshes then rig before using "parent > Automatic weights" option.

### 3D Asset Optimization
- **Maximus, Saori, and MetaMike helped treed** with converting 3D models to GLB format by explaining the process of importing FBX to Blender and exporting as GLB, with additional compression using gltf.report.
- **𝕽𝖔𝖚𝖘𝖙𝖆𝖓 shared Clone Tools** with devilsadvocate.sol via WeTransfer when the tool was no longer available for download.

### World Creation & Building
- **b0gie guided treed** through basic steps for world creation, providing step-by-step instructions including installing WSL, downloading git and nvm, checking node version, cloning repo, installing dependencies, and running localhost.
- **Ashxn helped TheMattEmpire** with snap functionality in positioning, explaining that ctrl currently disables snapping but controls are being updated.

### Agent Development
- **MetaMike shared resources with peezy** for animation code for walking agents.
- **Maximus shared resources with MetaMike** for agent development in virtual environments, providing links to articles about embedding intelligence into virtual environments.

### Documentation & Resources
- **Omka created hyperfy.how** to help the community find scattered Hyperfy documentation, consolidating resources in a searchable format.
- **HPrivakos shared code for Discord bot functionality** with the general community, providing a GitHub repo for a bot that posts messages with star reactions to different channels.

## Action Items

### Technical
- **Build Mode Improvements**
  - Implement undo/redo functionality for build mode (devilsadvocate.sol, maximus)
  - Add grouping functionality for objects (Omka, devilsadvocate.sol)
  - Add ability to type in specific rotation angles (Omka)
  - Add gizmos for easier object rotation and placement (ToxSam)
  - Fix UI scaling for smaller screens (Omka, maximus)

- **Agent Development**
  - Implement context injection for agents to understand their environment and needs (MetaMike)
  - Create a manifest of objects and coordinates for agents (maximus)
  - Finalize list of AI models to be offered in the agent configuration dashboard (szjanko)
  - Implement navmesh for agent pathfinding (MetaMike, peezy)

- **Asset Optimization**
  - Implement a quality rating system for assets based on optimization and appearance (TheMattEmpire)
  - Fix logarithmic depth buffers in next release to address clipping issues (Ashxn)
  - Explore batch processing for assets using the same material to reduce size (TheMattEmpire)
  - Implement material color changing in app scripts (Shiffty)

- **World Infrastructure**
  - Implement export/import functionality for world folders as zip files (ash)
  - Develop world folder switching functionality (HPrivakos)
  - Consider version compatibility tracking for imported worlds (maximus)
  - Implement security checks for imported world files (maximus)
  - Implement world settings tab for admins (Ashxn)
  - Implement fog effects in core engine for better depth perception (Ashxn)

- **Integration & Interoperability**
  - Add secrets management for scripts (HPrivakos, Ashxn)
  - Continue development of Solana integration branch (maximus)
  - Integration of other blockchain chains (maximus)
  - Convert 3D scan into VRM format (𝕽𝖔𝖚𝖘𝖙𝖆𝖓)
  - Implement VRM in NFT metadata (𝕽𝖔𝖚𝖘𝖙𝖆𝖓)

- **User Experience**
  - Add controller/gamepad support (Omka, devilsadvocate.sol)
  - Fix controller issues for VR implementation (voxvienne)
  - Implement audio-reactive visual effects for live events (Ashxn)
  - Add animation playback capabilities to enable cutscenes (᲼)
  - Make stats panel interactive to control asset properties (ToxSam)
  - Combine and rig Clone VRM (devilsadvocate.sol)

### Features
- Complete and release agent configuration dashboard for Hyperfy (szjanko)
- Create a Discord bot or tag system to easily find and catalog useful information (Omka)
- Develop a ground/grass app that allows easy swapping of terrain textures (voxvienne)
- Implement avatar equipping system (Saori)
- Create an app that can generate PDFs and share them to email addresses (bitpixi)
- Add ability to delete objects that glitch through the ground plane (vox)
- Update Hyperfy Tools to allow clicking on vertices in edit mode to automatically create snap points (~/HowiEDuhzit)

### Documentation
- Create documentation for smooth interpolation pattern (Saori)
- Create a VRM rigging tutorial/demonstration for the community (devilsadvocate.sol)
- Create documentation for hosting platforms on standardized world export/import implementation (ash)
- Consolidate documentation from HackMD to the new hyperfy.how site (maximus)
- Add a form to hyperfy.how for community contributions (Omka)
- Metaverse design toolkit resources (maximus)
- Open standards for XR interoperability (maximus)