2015-03-20 survey

Done

Administrivia

• Received exemption from IRB. I wasn't expecting that.
• Exchanged email with Richard Parncutt regarding committee membership. He is open to the idea, but seemed to have a hard time with the concept of a committee member being both examiner and consultant on the research. He thinks he should be consulting with faculty on the matter, not me. 

Data Collection

• Defined more survey questions about hand size (reach) in Qualtrics.
• Improved MDC JavaScript tool to support slurs, ties, and annotations in ABC.
• Deployed new MDC to Qualtrics.
• Met with IB to refine survey. It did not appear to be too long at all, even with the fingering exercises, some of which she could do almost at a glance. We started talking about content for the in-person data collection, but this is still an open question. The piano lab now has one piano outfitted with an overhead camera and a secondary camera on a tripod. We may be able to leverage some of this equipment for data collection. 

Doing

1. JavaScripting the Manual Data Collector (MDC).
2. Requesting letter of support from head of the Department of Music.
3. Trying to quantify variability in the Parncutt sight-reading corpus with a mind toward defining statistical approaches, as requested in protocol template.
4. Trying to run finger6 over input file.
5. Playing with Synthesia AFP (one .exe, one .dll, and that's it).
6. Collecting all the Synthesia data there is from their forum.
7. Implementing basic color tracking synchronized with MIDI events.

Struggling

• A lot of work at work.

2015-03-13 status

Done

Administrivia

• Submitted IRB application for expedited review.
• Submitted application for Provost's Award.

Data Collection

• Refined survey questions in Qualtrics.
• Proved concept of embedding MDC JavaScript tool in Qualtrics.
• Stored required JavaScript libraries on my UIC Google Drive.
• Explored VexTab as alternative to abcjs for MDC. The +1+ fingering annotations are not well rendered in abcjs, but text annotations work well. So I decided to stick with abcjs.

Doing

1. JavaScripting the Manual Data Collector (MDC).
2. Meeting with IB to refine survey questions.
3. Trying to quantify variability in the Parncutt sight-reading corpus with a mind toward defining statistical approaches, as requested in protocol template.
4. Trying to run finger6 over input file.
5. Playing with Synthesia AFP (one .exe, one .dll, and that's it).
6. Collecting all the Synthesia data there is from their forum.
7. Implementing basic color tracking synchronized with MIDI events.

Struggling

• We need to define the set of content we want to focus on. The last survey question will ask subjects to identify/select pieces that raise fingering difficulties. We need to narrow down the list of usual suspects in a way that will make the whole process tractable and scalable. I don't want to have to typeset of bunch of music.

Second sight

Could sight reading performance be improved with full (automatically generated and customized) fingerings? Or is this too much to process? Would full annotation affect look-ahead span a la Sloboda? How much annotation is ideal in a sight reading situation? Does this vary by individual? There is a literature on the subject of sight reading. Maybe I should (sight) read it.

What is the optimal amount of fingering annotation? How do editors currently decide what notes to annotate? Instead of aligning to the full annotations of performers, we could try to predict the partial annotations of editors. Ultimately we want to generate the most useful annotations. This implies determining both the correct fingerings and the ideal set of notes to be annotated. We could have a verbosity knob on the application, or the application could learn the player's preferred level of verbosity. Annotations could disappear from the displayed score, as the program determines they are note needed. They could reappear if the player falters. Would this drive the player nuts? Would they demand a static representation? Would they do this because that is what they are used to (because that is how paper works) or because this has inherent merit. Does this inherent merit, if it exists, apply only to performance situations, but not to a practice situations?

The idealized performance is memorized. As progress is made toward this ideal, should not fingering annotations gradually vanish? What about other annotations? What should disappear last--the notes, dynamic contours? Could practice for (memorized) performance benefit from this sort of reverse scaffolding?

Questions about the adequacy of editorial annotations should be included in our survey. What do the people think? Are the annotations useful even if they are ignored or crossed out? Pianists say they are at least some of the time (interpretation uber alles), but can this utility be measured?

Content manifesto

KernScores' ABC links are all broken.

KernScores seems to be the best source of symbolic music, not IMSLP, which really just gives us PDF versions of public-domain music. But it is not clear if KernScores is still being curated.

We need ABC or a way to translate data to ABC if our MDC solution based on ABC is to have legs. Guido and MuseData (md2) formats are present, but their renderings are rather unsightly. These formats do not have the JAvaScript support that make ABC and Vex attractive. Need to try hum2abc, but I am dubious. Another thing to try, since MusicXML format is available, is xml2abc. 

Will we live to regret the design decision to standardize on ABC?! I think it is sufficiently expressive, but is there something newer, cheaper, faster, stronger that we are overlooking?

ABC gives us the following awesome things:

• Readability
• Rich toolset for high quality rendering
• A JavaScript library (abcjs) for easy implementation of MDC
• Interoperability with music21.
• MIDI note numbers
• In the Dactyler soup already

All we need, as I see things now, is a clear (read: automated?) path from KernScores to ABC.

The last survey question will ask subjects to pick piece that raise fingering difficulties. I really hope this is a question they can answer, or we can do something to tease this information out.

To do  this, we need to narrow down the list of usual suspects in a way that will make the whole process tractable and scalable. I don't want to have to typeset of bunch of music. 

A modest proposal: We limit the size of our repertoire to the 108,703 scores available through KernScores (and the few other scraps we are able to create or scrounge from Synthesia). We start with some subset of Bach. Then move through Mozart and Beethoven, or stay in the baroque period. Perhaps the latter approach is preferred, as I think we want to avoid polyphony as much as possible. Two hands playing intertwining monophonic (rapid or finger legato) lines are the focus. One-voice, then two-voice, then three-voice inventions? Maybe we just stick with Bach?

2015-03-06 status

Done

Administrivia

• Drafted consent request, protocol, and recruitment email for expedited IRB review.
• Extracted draft questionnaire from Qualtrics for inclusion in IRB submission.

Data Collection

• Met with Dr. Zefran. He thinks CV alone should get the job done. Other sensors have noisy signals and are a hassle. He thought this would be a good project for CV class.
• Contacted Dr. Jezekiel Ben-Arie for advice on CV. He is considering the problem in consultation with his wife, who is a pianist.
• Drafted more questions for needs analysis survey and entered them into Qualtrics. 

Doing

1. Finalizing application for Provost's Award.
2. Drafting questions with accompanying rationales for survey.
3. Trying to quantify variability in the Parncutt sight-reading corpus with a mind toward defining statistical approaches, as requested in protocol template.
4. Prototyping Manual Data Collector (MDC).
5. Trying to run finger6 over input file.
6. Playing with Synthesia AFP (one .exe, one .dll, and that's it).
7. Collecting all the Synthesia data there is from their forum.
8. Implementing basic color tracking synchronized with MIDI events.

Struggling

• Do I need a complete survey for IRB review? I think I do.