Pianolens: An Augmented Reality Interface For Piano Instruction

Evan Strasnick, Austin Chambers, Liu Jiang, Xiaonan Tong

Stanford University, 2016.

Conventional methods for teaching, rehearsing, and performing on the piano fail to assist the player in learning new songs, navigating and rehearsing particular sections of music, and adding emotional/musical subtleties into their performance. Advancements in mixed reality systems have enabled complex visualizations that can enhance the piano experience. We present Pianolens, an augmented reality system which assists in learning and performance through an interface which 1) provides a dynamic and interactive sheet music display, 2) visualizes musical elements through intuitive modifications to the sheet music, 3) teaches new songs by demonstrating proper timing and fingering of notes, and 4) allows for easy navigation between and within songs for demonstration of rehearsal of arbitrary sections of music. We hypothesize that these aids will not only facilitate the learning of new music, but also make the rehearsal process more engaging for learners.