User Clustering Visualization and Its Impact on Motion-Based Interaction Design

Movement-based interaction design relies on sensor data analysis and higher-level feature extraction to represent human movement. However, challenges to effectively using movement data include building computational tools that allow exploring feature extraction technology as design material, and the need for visual representations that help designers better understand the contents of movement. This paper presents an approach for visualizing user clustering descriptors to enhance the practitioners’ ability to use human motion in interaction design. Following a user-centered strategy, we first identified perceptions of, and barriers to, using motion-based features in a group of interaction designers. Then, a multiple-view multiple-people tracking system was implemented as a detection strategy that leverages current models for 3d pose estimation. Finally, we developed a computational prototype that performs instantaneous and short-term clustering of users in space and presents simple descriptors of the algorithm’s output visually. Our approach was validated through a qualitative study with interaction designers. Semi-structured interviews were used to evaluate design strategies with and without the assistance of the computational prototype and to investigate the impact of user clustering visualization on the design of interactive experiences. From practitioners’ opinions, we conclude that feature visualization allowed designers to identify detection capabilities that enriched the ideation process and relate multiple dimensions of group behavior that lead to novel interaction ideas.