How I got this idea
I wanted to make a fun instrument that translates hand movements in the air into actual guitar sounds in real-time, using a motion sensor and code instead of just playing pre-recorded audio files.
Problem
- Digital MIDI instruments lack tactile feedback and physical expressiveness during performance.
- High latencies (above 20ms) between gestural movement and acoustic output disrupt the user's musical timing.
- Synthesizing high-fidelity, realistic guitar acoustics typically requires heavy software libraries.
Solution
- Engineering Approach: Developed a spatial controller glove that transmits acceleration vectors to a local, high-speed mathematical sound compiler.
- Architecture: Deployed an Arduino microcontroller reading MPU6050 data over I2C, streaming bytes via 115200 baud serial connection to a local Python synthesis thread.
- What I Personally Built: Programmed the Arduino I2C reading scripts, wrote the Python Karplus-Strong acoustic loop, and wired the spatial glove prototype.
- Current Status & Result: Functional hardware and software prototype demonstrating real-time gestural sound compilation.
PythonArduinoMPU6050NumPyKarplus-Strong SynthesisSerial Communication
Hardest Technical Challenge
Executing string synthesis equations in real-time under a strict 10ms execution deadline to prevent audio cracking. Solved by writing the Karplus-Strong algorithm using optimized NumPy array operations, eliminating standard loop lookup overhead and reducing latency to sub-8ms.
Project Demo / Interface
What I learned
- Digital Signal Processing (DSP) and wave synthesis modeling.
- High-baud serial data transmission and latency isolation.
- Sensor calibration and motion threshold filtering.