Data collection doodles

I am not sure how this will interface with the whichever electronic survey system we adopt. (UIC has one I think we can use for free.) But I think step one is to come up with some way for pianists to tell us their preferred fingerings for predefined repertoire. I am thinking we can do something in Java or JavaScript and HTML5 that can render ABC or LilyPond on the fly. We will also leverage this system as an essential component of the "Semi-Automated" system below, to allow automatic correction of detected fingerings. In my mind, this should all be very straightforward. I am quite sure there is a JavaScript library for ABC that should make this pretty easy to implement. (I was planning to use this library for BowTIE.)

Sketch of Manual Data Collector (MDC)

The display is needed for the Manual Data Collector (MDC) to allow pianists to correct system errors. We could also track these corrections to help us analyze automation errors.

Sketch of Semi-Automated Data Collector (SADC)

We may want to dispense with the pre-defined content and let the pianists just play. Once the automation is sufficiently robust, it really should not matter what they play (only how they play it). Why do I suspect these words will come back to haunt me? Anyway, this will be the most painless way to collect data. Even if we always want to have the screen and people playing from a score we provide and know a priori, eventually we will want to stop asking them to correct things.

Sketch of Fully Automated Data Collector (FADC)

Deployment

1. MDC as part of needs analysis survey.
2. Prototype SADC in CS lab environment for testing by Didactyl team. Ivana, I am looking at you. Please don't avert your eyes. Maybe Paul Landes from the NLP Lab could help too.
3. SADC in private studio.
4. SADC at home?
5. FADC in piano lab (classroom) environment.
6. FADC at home.

Deployment speculation