FCS News

Originally shared in Fall 2012 CS Magazine.

Stretching the Boundaries: Ubiquitous Computing @ Dal

Digital technology pervades our everyday lives. We can find nearby Thai restaurants, compare reviews, menus and prices, and view relative distances and locations on interactive maps or superimposed onto our field of view. We can follow a route to a chosen spot and be reminded that we need to buy a birthday card as we pass the local stationary store. At the restaurant, we can capture and immediately share images and videos, annotated with time, place and the people present. Before leaving, we can even add our own review.

The situation at home is no less dizzying. We are building complex networks of entertainment and communication devices—even security systems, energy systems and appliances, not to mention laptops, tablets and an increasing array of peripherals—all of which we expect to work together seamlessly. Even in sport, GPS-enabled watches and heart rate monitors allow athletes to track, visualize and analyze their performance across multiple dimensions.

These scenarios, and many of the technologies underlying them, were first explored in an area of computer science called ubiquitous computing (ubicomp). The mission of ubicomp is to support people in their everyday lives. Ubicomp is the interdisciplinary study of computing away from the desktop or laptop. It draws from many areas of computer science, including: human-computer interaction (HCI), computer vision, digital signal processing, knowledge-based systems, distributed systems and networking. The research also draws on the theories and tools of psychology, sociology and anthropology, and derives inspiration from art and design disciplines when exploring new horizons.

At Dalhousie, Derek Reilly and his students are exploring applied ubicomp scenarios within healthcare and heavy manufacturing. They are also developing and evaluating new technologies to support healthy activities such as going for a run, maintaining a good posture at the office, or whole body interactive gaming.

“I believe that interdisciplinary collaboration is critical in order to build compelling ubicomp experiences,” says Dr. Reilly “and to convert these experiences into research outcomes.”

An example of this type of collaboration is Tweetris, an interactive art exhibit involving faculty and students from Dalhousie, University of Toronto and OCAD University (OCAD U). Visitors play a friendly game of full body “Tetris brick” (ortetromino) making. As tetrominos are created, a video snapshot of the person making the shape is tweeted to @TweetrisTO. The tweeted tetromino snapshots are, then, used in a game of Tetris which other visitors can play, in real-time, by using their mobile phones or a wall-projected kiosk next to the main exhibit.

Tweetris has been used to contribute to our understanding of how settingimpacts whole body interaction, and how low fidelity cues (such as tetrominos) can elicit whole body interaction styles from users without prematurely constraining them: a very useful approach for designers of these types of systems. Tweetris has been exhibited at art events and both gaming and HCI conferences.

Limber is another great example of interdisciplinary research collaboration, with Kate Hartman and Emma Westecott of OCAD U. Limber explores how gaming elements and ambient feedback can promote healthy behaviours within office environments. Jokingly described as "the gamification of sitting down", Limber tracks a person’s seated posture over the span of a day. It periodically rewards the player with "Limber points", or penalizes them with "Un-Limber points". Players also accumulate points for stretching and getting up off of their seats. 

This summer, Dr. Reilly is working with a team of students from Dalhousie Faculty of Computer Science (FCS) and OCAD U to develop an ambient device that gives unobtrusive feedback about these behaviours and visualizations. This enables office workers to track their performance over time, to compare their scores with those of their colleagues and to view the cumulative score of their workgroup in relation to that of another group. The team will be studying how these different motivators influence behaviour by deploying Limber in several targeted office environments.

A recurring theme in all of Dr. Reilly's research is the blending of physical and digital. In another project involving Boeing and other FCS faculty, Dr. Reilly's students are exploring ways that mobile devices can be used to query and explore relationships between paper documents and other surrounding objects, inspired by techniques from desktop information visualization.

He has developed a toolkit for blending these physical and virtual worlds called TwinSpace. “This allows people to meet and collaborate across distances without resorting to talking over a videoconference window,” he says. “It allows collaborators to move fluidly between different workspaces—a whiteboard and a conference table, for example—without losing contact with those connected remotely.”

TwinSpace has shown promise in controlled studies and demos, but has yet to be deployed "in the wild". Over the next two years, this technology will be adapted to a very distinct applied context: a Brazilian care facility, serving people with neurological disorders. Researchers will explore how it can be used to support impromptu staff meetings, and when transitioning patients into independent living.

As ubiquitous computing enters the mainstream, the work of researchers like Dr. Reilly becomes more immediately relevant. By exploring new opportunities for ubiquitous interaction, Dr. Reilly and his students hope to find ways to augment—rather than overwhelm–our everyday lives.