A beginner at a piano has to connect the notes on a score with the keys in front of them. Before playing a piece, they must work out which key to press and when. That translation can stand between the learner and the experience of making music.
INTRO used illuminated keys and falling notes to turn piano practice into a rhythm-game interface. It began in 2019 as a student project in Dongyang Mirae University’s MIT project club. I led the team, designed the hardware and software, and took the working device into public exhibitions.
Bringing the guidance closer to the keys
Showing the next note on a screen still leaves the learner to locate it on the instrument. INTRO connected the approaching note with an illuminated physical key. The screen showed the progression of the music; the keyboard showed where to act.
The input was MIDI performance data. Notes and timing in a MIDI file became the source for the visual notes and keyboard guidance. The work translated structured musical information into a form the learner could follow with their eyes and hands.
The user still performed the physical action of playing a piano. Visual guidance helped connect the written music to that action, while the falling-note format made the timing visible.
Making software and hardware point to the same note
The device combined two single-board computers, LEDs, light-diffusing acrylic and a Unity interface. UART linked the hardware and software, with MIDI information feeding the visualisation and control.
This required designing across different materials. Unity provided the moving notes and virtual keyboard. The physical device had to make the same guidance visible at the real key. A note approaching on screen and the place a person reached for needed to form one instruction. As team lead, I designed the hardware and software around that connection.
Contemporary coverage also records colour cues on the virtual keyboard and automatic page turning for the on-screen score. The visual guidance extended across the playing interface. A 2020 interview with the team preserves the demonstration and implementation details.
From an exhibition device towards an instrument
We exhibited INTRO at the Korea Electronics Show and the Future of Education Conference. It won the grand prize at the Dongyang Mirae Expo held within KES, and appeared in Kazakhstan national broadcaster 24.kz’s coverage of education technology.
At an exhibition, a visitor could sit at the keys and follow the guidance directly. The product was experienced through the screen, the device and the visitor’s hands together.
A demonstrable device and an instrument someone wants to keep practising on still have different requirements. The contemporary interview identified the keyboard-cover material and the feel of playing as further development work. Visibility was only one part of the design; the material under the player’s fingers mattered too.
What happened after the prototype
The graduation project led to a startup and licensing discussions with an instrument manufacturer, but those discussions did not result in a deal. This outcome is recorded in a 2021 interview.
INTRO produced a working exhibition prototype, an award, broadcast coverage and a route into commercialisation discussions. It combined a game interface and an embedded device to bring the information a user needed closer to the place their hands had to act.
