Researchers with Microsoft and several U.S. universities are envisioning an immersive game experience that could turn walls, floors, ceilings and objects in a room into a life-size virtual play area. The prototype system they've developed, called RoomAlive, uses six combination projector-depth cameras to cover a room's walls and furniture with input/output pixels.
The concept could turn a living room into a 360-degree Whack-A-Mole game, or have users chasing hostile robots across the floor, chairs and coffee table. The team working on the project describe the system in a research paper for the ACM User Interface Software and Technology Symposium (UIST) currently taking place in Honolulu.
"Imagine playing a video game in your living room without a television. Instead, the game happens in the room, all around you. When the game starts, the room magically transforms into an ancient castle, the walls turn to stone, and flaming torches emerge from the walls casting flickering shadows onto the furniture. Out of the corner of your eye, you see a glowing idol appear on your couch," the research team said.
"You walk towards the idol when suddenly, a trap opens on the wall next to you exposing blow darts ready to fire. You leap out of the way, only to land on the floor face-to-face with a giant cockroach," the team explained. "You quickly get up and jump on the roach. You reach the idol successfully, and a scoreboard drops down showing that you have just scored the best time for the adventure course. This is just one of many interaction scenarios that are made possible by RoomAlive."
'Possibilities are Endless'
The RoomAlive concept builds on the IllumiRoom research project Microsoft described during the 2013 Consumer Electronics Show (CES). With IllumiRoom, researchers used Kinect for Windows to map the features of a room in real-time and turn projected images into fully interactive game elements.
While the IllumiRoom demo presented at CES focused on educational and entertainment uses, "the possibilities are endless," Microsoft noted at the time. "It's rare for a company to pull back the curtain and share research in such raw form at the world's largest technology tradeshow. However, we think it's vitally important to get the next generation of students excited about computer science -- and what better way than to show off research that makes gaming more fun."
RoomAlive extends the capabilities Microsoft envisioned with IllumiRoom, although there are still numerous issues that need to be resolved before the concept becomes commercially viable, the research team noted. Calibration errors cause some visual content to appear as ghosted images in overlapping projection regions, for example, and latency issues remain a concern.
Of course, there are also the potential physical impacts of playing virtual games in a real-world setting -- something the researchers acknowledge could be an issue, especially with very young gamers.
"(C)ertain parts of a living room environment may be less suitable for touch-based experiences," the team said. "For example, an expensive vase or painting may be inappropriate for direct contact. Particularly for children’s games, designers may want to limit touch interaction to the floor and walls (large vertical planes)."
In theory, parents might also be able to tag fragile items in a room so they are taken out of the area of play, the team noted.
"Alternatively, the experiences could rely solely on whole body movement, pointing and shooting, or traditional controller input," they said. "Pointing and controller input are ideal for controlling objects at a distance, such as a virtual driving across a wall. We generally found that experiences were most enjoyable when all forms of input were available to users."