Rapid Prototyping for Gesture-based Interaction,via Commercial-off-the-Shelf Input Devices

The goal of the workshop is to gather and consolidate expertise on rapid prototyping of gesture-based user interaction, relying on commercial-off-the-shelf available input devices.

In the past decade, games that rely on body movement have gained momentum, not in the least by commercial successes such as the Wii, developed by Nintendo or the EyeToy brought on the market by Sony PlayStation. What these commercial gesture-based games have in common is that they rely upon relatively cheap, mass produced input devices such as the WiiMote, the EyeToy camera or the Wii balance board. While many research labs have developed comparable motion sensing systems themselves, or have developed specific rapid prototyping tools, for other game or UX researchers, it can be preferable to make use of and/or combine commercial-off-the-shelf (COTS) available input devices. Rather than having to prototype these sensor and sensor related devices themselves, UX research can use COTS input devices which are cheap and easy to acquire and do not require extensive expertise about hardware design. Especially when the focus is on user interaction and not on hardware development itself, COTS input devices are appealing, as this leaves more resources toward the study of the actual user interaction.

Due to reverse engineering, already a lot is known about these COTS components . E.g. with respect to Wii related input devices several software libraries exist that allow researchers and developers to connect and work with these input devices. Active forums surround these libraries that offer support and lend a hand. However, researchers that already experimented with COTS input devices soon discover that a lot of knowledge cannot be found on these forums. Furthermore, it is hard to make an informed choice regarding which libraries, which API’s, which algorithms, etc. are more at hand in specific situations. Finally, the whole tool chain used restricts the possible choices since we don’t want to spend our time in generating converters, translators…

Therefore we organize this workshop to gather, consolidate and share this expertise. While we do not pretend that one workshop will be able to accumulate all knowledge surrounding this topic, it can be the start of a prolonged exchange of knowledge on rapid prototyping with COTS input devices.

Participants will submit a video tutorial, where the rapid prototyping technique they want to show, is explained. This can be both of a COTS based prototype they have built or an illustration of how they use COTS devices or applications in prototypes or how they would like to use them but have not succeeded so far due to certain problems or limitations.

The tutorials can be about one or more of the following subjects:

• Hardware specifications of COTS


• Dedicated software libraries for COTS


• The right signal processing algorithms when using COTS


• How to put all previous components together in the most efficient way

The video tutorial will be submitted with the Vimeo online video sharing service. Make sure, if you post your video, to make it private. Use the option “Private->People you choose” and select “Maarten Van Overveldt”. Then we can easily see your video tutorial without making it public. Confirm your submission by mail (Maarten.Van.Overveldt@groept.be). Add a link to the video and other necessary links which refer to the research project(s) in which these video tutorials are made.

Six or more video tutorials will be selected and their producers will get the chance to participate in the workshop. Participants will be selected on the basis of the relevance of their work regarding the topic of the workshop.

The selected tutorials will be posted on this workshop blog so that the future participants get the chance to select a tutorial they find the most appropriate to work with during the workshop. The selection is done by a voting system on the blog and will lead to 2 or 3 tutorials that are going to be used in the workshop. Besides a general discussion about the pros en cons regarding rapid prototyping with COTS there will be a thorough discussion about these tutorials. The outcome of these discussion, like a proposal for possible tutorial adjustments, will be posted on this blog.

9:30 – 11:00 Presentation of participants

The workshop will start with a presentation of all participants and their work related to the topic or rapid prototyping with COTS input devices.

11:30 – 13:00 Inventory of problems/answers

In the next block, a discussion will be held among participants with respect to the most salient and urgent problems? Which problems did participants encounter? Which solutions were found? What are known limitations of certain sensor types? Which topics have not yet been addressed? Possibly other participants have answers to these questions. The discussion and inventory of problems will also follow the four sub domains presented above, namely 1) hardware specifications, 2) dedicated software libraries, 3) signal processing and 4) putting everything together. At the end of the inventory, a decision will be made on which groups of participants will work with which of the selected tutorials.

14:00 -15:30 The tutorials

To apply the knowledge shared during the morning, we will ask participants to work with one of the selected tutorials in the afternoon and complete them as good as possible. A descent preparation beforehand and support during the workshop by the creators of these tutorials is needed to guarantee a successful completion of this part. The necessary tools and COTS have to be at hand. We will ask participants to also focus on documenting their process of how they proceed step by step in following the tutorial and document the pros and cons about the way of working that is presented in the tutorials.

16:00-17:30 - Revising the inventory

The practical experience in the previous part can be used to revise the inventory and open the discussion about the most important issues regarding the subject for the last time. A deeper discussion about the tutorials will lead to a proposal for possible adjustments that can be made in order to enhance the content of the tutorial or the way it is set up. This so that people coming from a varied background are able to understand and ‘re-create’ the things presented in the tutorials.