Prototype of a mini game and a sensor-based glove to test the interactions and desirability of a glove-shaped game controller.
01/2017 - 03/2017
UX Design, Game Design, Graphics Design, 3D Modeling, Electronics Assembly, Software Development
Rhinoceros, Photoshop, Arduino UNO/Arduino IDE, p5.js
HOW TO USE
Make a fist to get ready.
Spread fingers to fire!
Cover the glove to charge.
Feel the vibration feedback of charging.
Release to cast Super Fireball!
1. FLEX SENSOR
Senses curled finger when you make fist gesture.
2. LIGHT SENSOR
Senses ambient light change when you cover the glove to charge.
3. VIBRATING MOTOR
Provide haptic feedback when charging.
4. ARDUINO-BASED SYSTEM
Core system is created with an Arduino UNO single-board computer.
5. 3D-PRINTED CHASSIS
Contains all electronic components and comes with a solid wristband. 3D model created with Rhinoceros.
It all started from a game...
My bro Mike and I made a mini game with p5.js, in which the player cast fireball to battle with the Evil Cat. (Sorry, we are dog people.)
We originally used the accelerometer on a smartphone to control the game on laptops (player swings the phone to fire), but the inaccuracy and latency urged us to find another way of game control.
Secondary Research: Sensors & Controllers
We read several academic papers about sensors on mobile devices to gain basic understanding of available technologies that we could take advantage of.
In addition to academic materials, we also did a competitive analysis on game controlling mechanisms, including button-based control like traditional Sony PS4 game controller, in-air gesture detection like Microsoft Kinect, and gyroscope/accelerometer like Nintendo Wii.
We then used three criteria, including Accuracy, Intuitiveness, and Fun to evaluate all the options.
We eventually came to the idea of building our own controller with suitable sensors to detect in-air gestures. Most of our participants agreed that simple gestures like the "Ironman" gesture were much more intuitive and fun than available input options, but it needed to be accurate.
Exploring controlling mechanisms
We experimented with different input devices to control the game, including keyboard, mouse, smartphone, camera and microphone. Several storyboards and low-fidelity prototypes were used to explore the user scenario.
In addition to the glove, we 3D-printed a chassis with wristband to contain the whole system. The chassis consists of five pieces of detachable parts to make printing more manageable and allow for player customization through different color combinations.
Circuit and Sensor Assembly
An Arduino board is used as the core of the controller. We tried various sensors including accelerometer, gyroscope, and proximity sensor, but eventually found the flex and light sensors to give us the highest returns with the lowest implementation difficulty.
Connecting Arduino to P5.js codes
Our game is written in P5.js. We use serial port for Arduino to communicate with P5.js codes. The most challenging parts were determining the threshold for the sensors and transferring the data. For example, how bended should the flex sensor be to be registered as an event?
4. USABILITY TEST AND ITERATION
We went through three stages of usability test. In the first stage we used smartphone as controller to test the game mechanics. In the second prototype we combined accelerometer and proximity sensor, but found compatibility issues because sensors shared SCL and SDA ports. In the third iteration we used flex sensor and light sensor as alternatives, and they work very well. We also added haptic feedback in our latest build to enhance the immersive experience.
1st Iteration: Smartphone
Goal: Test basic game mechanics and control
We used the accelerometer in a smartphone as the controller. (Connection by network socket.) However, the latency of network connection hinders gaming experience.
(By the way, in this first iteration, the Little Guy is the enemy, and you can change the attributes of the fireball by voice!)
2nd Iteration: Arduino and Sensors
Goal: Find suitable sensors and control mechanism
We tried to build our own controller with Arduino and sensors. In the beginning we were still trying to emulate the control scheme of smartphones (accelerometer), but we also experimented with a variety of sensors including gyroscope, proximity sensor, light sensor, and flex sensor. The goal was to use these sensors to detect the so-called "Iron Man" gesture. (Spreading and pointing fingers forward).
3rd Iteration: Flex Sensor and Light Sensor
Goal: Test usability of the final hardware design
The 2nd iteration failed to meet our goal because of compatibility issues between multiple sensors, but it also helped us realize that we could use cheaper and less complicated sensors to have the same Iron Man effect.
In the end we used Flex Sensor to detect curling/spreading fingers, and a light sensor for the cool "Iron Man charging" gesture.
5. FINAL PRODUCT SHOWCASE
The feedback we got was generally positive thanks to the intuitive gaming experience and playful graphics, but most people also felt it was exhausting to use the controller longer because of the weight. The haptic feedback that came when players charge was also highly praised for providing immersive experience.
We will make our next iteration a slimmer form factor with only necessary sensors at the glove side, while leave all the computing units including the Arduino mainboard in a set-up box. By doing this we can dramatically reduce the weight of the glove and improve user experience.
Our ultimate goal is to make the glove wireless, but considering current compatibility issues between sensor modules on Arduino, for the near future we will still stick to a wired design.
Wei-Hung Hsieh (Left): I worked on sketching/storyboarding/graphic design/3D modeling, and also took care of 10% of the software development including game balance, background objects, and game menu.
Michael Frampton (Right): My best friend Mike was in charge of most of the software development and game mechanics. He's currently an interaction designer and professional Pokémon master based in Utah. I miss you! Come back to Seattle!
Winter 2016 MHCI+D Prototyping Studio Project
Justin Hamacher, Lecturer, MHCI+D, University of Washington
Justin Lund, Lecturer, MHCI+D, University of Washington
Mini Game with the Evil Cat
Winter 2016 MHCI+D User Interface Software and Technology Project
Jeff Heer, Associate Professor, Computer Science & Engineering, University of Washington