Animation Studio/Wii Remote Data Visualization


Summary
This program allows the creation of forward kinematic animations on a hierarchy of objects. Animations can be recorded on screen using the mouse or sliders or using a Nintendo Wii Remote. Raw Wii Remote data is also visualized.

Bio
My name is Bryan Blackford. I am a junior undergraduate in the computer game design program at University of California Santa Cruz.


Background

Forward Kinematics allows a set of objects which are connected hierarchially (parent-child relationship) to be animated by performing transformations on parent objects, and the transformations are propogated into child objects.

In 2006, Nintendo's Wii video game console introduced an innovative form for player interaction. A wireless device, called the Wii Remote (or wiimote), about the size of a television remote controller, held by the player in one hand, which could detect the motion of the player's hand. Prior to this, games were played by pressing buttons on a controller. This enabled possibilities for new kinds of games and new ways of playing traditional games. For example, the Wii console shipped with a sports game where a player could play tennis by swinging the remote like a tennis racket and see the characters on screen move the racket with nearly 1:1 motion.

The Wii remote communicates with the Wii console (wii) over a bluetooth connection. For this program, I am using a usb bluetooth adapter, the BlueSoleil bluetooth stack, and the WiiUse library.

The wiimote contains an accelerometer, which detects acceleration along 3 axes, and an infrared camera, which is used to determine the orientation (rotation) of the wiimote with the help of a bar with a number of infrared lights, which is placed near the display that the wiimote is being pointed at. The wiimote also has several buttons, a small speaker, vibration feedback (rumble), and an expansion port.

Recently, a device called MotionPlus was released by Nintendo which plugs into the wiimote expansion port and uses a tuning fork gyroscope to provide angular rate sensing, giving motion sensing which is more accurate. (This program does not yet support MotionPlus.)

Project Details

Steps to create a forward kinematic animation:
select a model or shape from the scene (or by using < and > buttons)
select a shape from the dropdown by the 'child' button
click child to create a child object
transforms to parent will apply to child

Transforms that are recorded include:
Translation (position)
Rotation
Scale
Color (mostly useful for lights, the 4 spheres in the scene)
Time Delay ('Delay' button)

Animations can be created by:
Select 'Animation' from the dropdown.
Click New.
Click the new animation (bottom of list).
Select an object from the scene.
Click 'attach' to bind the animation to the object.
Click 'Record' and perform transformations.
Click 'Record' to end recording.
Click play and loop for looping animation.
Select and play another animation for layered animations.

The Erase button will erase the transforms in the selected animation, allowing for a new animation on the same object. The Delete button will delete the selected animation, as long as it isn't the last one. The Copy button will duplicate the animation, which could be useful for attaching to a different object.

Primitive shapes can be created by selecting Sphere, Cylinder, or Cone. Then click Create to create an independent object, or Child to create a child to the currently selected object.

The Child button does nothing when Model or Animation is selected.

Click "Eye" to select the viewing camera, animations can be attached to the camera to record camera movements.

Use the "set RePos" button to set the reset position. Then press "reset pos" during recording to return the object to that position during animation playback.

The wiimote can be used as a tool to transform and animate an object (such as an enterprise) on the screen.
Features in this function:

First press "+" to enable motion capture.
Rotation on the roll and pitch axis.
Press A for continuous relative rotation, or B for a single absolute rotation
Press home to reset rotation (this is not recorded in animation).
Left/right arrows rotate on the yaw axis.

The absolute roll data from the wiimote is visualized using a line graph. The horizontal axis is time and the vertical axis is the value (-180 to 180, though data gets jumpy near edges). The graph autoscales on both axis.

Wiimote sensor data is obtained using the wiiuse library.[see reference]

Videos
Videos are posted to my youtube channel:
http://www.youtube.com/mintygreencow
Or download local videos:
bblackfo-161final-1.3gp
bblackfo-161final-2.3gp

Source
Download the source here:
bblackfo-161final-src-data.zip

Binaries
Download Windows (XP tested) binaries (includes 3d models) here:
bblackfo-161final-win32-bin-data.zip

Hardware

- Wireless sensor bar purchased from eBay. (4$)
- USB bluetooth device purchased from eBay. (3$)
- Original Nintendo Wii Remote purchased from eBay. (17$)

Software

- Wiiuse library
- OpenGL via MesaGL
- FLTK 1.3
- Bluesoleil bluetooth stack
- Emacs text editing/debugging IDE
- MSYS unix-like terminal for windows
- make/g++/gdb MSYS binaries for compile/debug
- Windows XP (tested) Linux/UNIX should compile with minimal change

Timeline

3/4 - Fall from tree, dislocating elbow.
3/5 - X-rays reveal no elbow fractures.
3/9 - More X-rays reveal possible wrist fracture.
3/10 - GlovePIE testing yields positive results. Unfortunately, GlovePIE only works through a virtual joystick driver.
3/13 - Discovery of other wiimote libraries look promising, unfortunately these fail to compile. Focus of project moves to animation.
3/14 - Most animation work is done. Wiiuse libraries finally compile.
3/15 - Wiimote can rotate objects. Graph added.
3/16 - Further X-rays find piece of bone near elbow has been knicked off. Wrist is not fractured. Object hierarchy from 160 final works perfectly for forward kinematics.
3/17 - Happy birthday to me! Finals all day! Writeup and presentation.

Future Work

Wii MotionPlus adapter for more accurate motion sensing, yaw rotation, force translation.
Use data from IR camera for wiimote position, for object translation.

References

http://en.wikipedia.org/wiki/Wii_Remote
http://people.cs.vt.edu/~bowman/3dui.org/course_notes/chi2009/wiimote.pdf [paper on wiimote visualization]
http://carl.kenner.googlepages.com/glovepie
WiiUse Library
Bluesoleil bluetooth stack