_{Wizard Otter by arseniic}

Hairy Otter

Experience a magical world and cast spells in VR.

Table of Contents

1. About The Project
2. Getting Started
3. Current Features
4. Roadmap
5. Contributions
6. Acknowledgments

About the Project

We're a team of UC Berkeley students (computer scientists, game developers, artists, and engineers) joined together to create an immersive and inspiring experience: spellcasting in virtual reality.

Demo

05/06/2024

Built With

• 
• 
• 

(back to top)

Getting Started

Help improve the world of hairy otters! Follow these steps:

Installation

1. Fork and clone the project

   git clone https://github.com/your_username_/hairy-otter.git

2. Install Unity packages

• XR
  • XR Interaction Toolkit
  • Oculus XR Plugin
  • XR Plugin Management
  • Meta Voice SDK
• General
  • TextMeshPro
  • Unity UI
  • Universal RP
  • AI Navigation

Contributing

1. Fork the Project
2. Create your Feature Branch (git checkout -b feature/AmazingFeature)
3. Commit your Changes (git commit -m 'Add some AmazingFeature')
4. Push to the Branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

(back to top)

Current Features

Refer to this section to learn the technical details of current features.

Voice Recognition and Spell Casting

We utilized the Voice SDK powered by Wit.ai to recognize voice commands. The user can activate and deactivate the voice service using quest controllers.

Current commands

• lumos - illuminates the wand
• nox - deilluminates the wand
• attack - launch a fireball
• shield - create a shield
• book - spawn a book

Relevant scripts

Assets/Scripts/
│   Spells.cs
│
└───VoiceSDK/
│   │   VoiceController.cs
│   │   CastSpellConduit.cs
│   │   ...
...

Development Progress

We initially implemented voice recognition using TensorFlow and sent the recognized voice command to the Unity application, which allowed us to build and deploy custom machine learning models.

To keep our solution contained within Unity, we later switched to using Voice SDK provided by the Meta Quest.

References

Learn how to use Voice SDK:

• Incoroporate Voice SDK into Unity - Medium
• Voice-enabled VR Unity App - GitHub

(back to top)

Roadmap

• Speech Recognition
• Demo - 05/06/2024 (Meta Quest 2, Mac, Windows build, with basic spellcasting)
• Graphics
  • 3D modeling and animation
  • Post-processing (fog, lighting)
  • Shaders
  • Water
• AI & Machine Learning
  • Gesture Recognition
  • Speech Recognition
• Spells
  • Wand grabbing skill (w/o wand)
  • AR interaction (MR Utility Kit, passthrough layer)
  • Spellbook
• Minigames & Features
  • Broom riding (Quidditch)
  • Spell making
  • Potion making
  • Wand making
  • Shop (diagon alley)
  • Mythical animal taming
  • Sorting Hat
  • Chess (life-sized, sorcerer's stone)
  • Platform 9 3/4
  • Unique enemy battles (basilisk, giant spider, dementor)
  • Dueling (npcs, lord voldemort)
  • Triwizard Tournament
  • Storymode
  • Multiplayer mode
• UI / UX
  • Inventory
  • Game menu
  • User flow

(back to top)

Contributions

• Jonas So Gutierrez - Implemented VR hands, VR movement (continuous and teleport), VR grab, and flying
• Eugene Kang - Implemented spell casting, and player movement; Created game environment and target dummies
• Yena Kang - Created spellbook, animal summoning (expecto patronum); Integrated voice recognition models (TensorFlow)
• Jaeha Yi - Implemented spell casting, VR movement, and enemies; Created game environment, UI, and demo; Integrated voice recognition models (TensorFlow, Wit.ai)
• John Yoon - Implemented projectile firing, wand movement; Integrated voice recognition models (TensorFlow)

(back to top)

Acknowledgments

Thank you to XR@Berkeley for supporting our project!

Resources we found helpful and utilized throughout our journey:

XR

• XR setup
• XR movement

Voice SDK

• Incoroporate Voice SDK into Unity - Medium
• Voice-enabled VR Unity App - GitHub

Features

• Animal Summoning
• Magic Effects
• Endless Book
• Wand

Game Environment

• Dungeon

(back to top)