Thursday, November 19, 2015

2015-11-19 status

Done

    Data Collection

    • Paid lottery winner for Survey I.
    • Invited two people who asked for a second chance at Survey I (fingering exercises).
    • Followed up again on Survey II, providing a 21 November deadline. Received 14 responses. So we have 198 full responses to date.
    • Found apparent problem with fingering collection script, so we are missing fingering data from 10 subjects. I suspect a browser incompatibility with the JavaScript code or something wonky in Qualtrics. 
    • Wrote and tested script to load data from Survey I to SQLite.

    BowTIE

    • Met with Jackson. He is going to work on "native" Java code for the performance component, as his programming experience is limited to Java.
    • Played around with more tutorials.

    Doing

    1. Analyzing survey data.
    2. Integrating MDC4 into draft Qualtrics survey for WTC, Book 1, Prelude 1.
    3. Adding file save and print features to MDC4.
    4. Adding support for non-exhaustive input in MDC4.
    5. Assembling prerequisites (JavaScript libraries, data layer) for BowTIE.
    6. Standardizing file format for fingering.

    Struggling

    • What to do with partial results from people who completed Part I, said they had time for Part II, but never finished? Protocol says we delete data from people who walk away.
    • Should we contact these people to try to rectify missing fingering data in Survey I? Ivana is one of them.
    • Should we offer small direct payment for Survey II in addition to lottery?
    • Should we offer larger payment to fewer people?
    • Should we give subjects in Phase II the option of photocopying their annotated scores and mailing them to us?

    Wednesday, November 11, 2015

    2015-11-11 status

    Done

      Data Collection

      • Closed out Survey I (fingering exercises) after collecting 203 responses. Response per email rate is 3.8%. Completion rate for all people who started the survey is 203/352 = 57.7%.
      • Followed up with 20 subjects who did not complete profile part of Survey I, asking them to complete Survey II. Received 9 responses. So we have 193 full responses to date.
      • Exchanged more emails with people who had problems with the survey tool or questions about the study.
      • C-MIDI source code provided is incomplete and cannot be built. We will inspect the code and port any bits that look promising.
      • Defined SQLite schema for (anonymized) survey data.

      Doing

      1. Writing script to load SQLite.
      2. Integrating MDC4 into draft Qualtrics survey for WTC, Book 1, Prelude 1.
      3. Rethinking incentive scheme going forward.
      4. Adding file save and print features to MDC4.
      5. Adding support for non-exhaustive input in MDC4.
      6. Assembling prerequisites (JavaScript libraries, data layer) for BowTIE.
      7. Standardizing file format for fingering.

      Struggling

      • What to do with partial results from people who completed Part I, said they had time for Part II, but never finished?
      • Should we give subjects in Phase II the option of photocopying their annotated scores and mailing them to us?

      Wednesday, November 4, 2015

      2015-11-04 status

      Done

      Data Collection

      • Distributed initial survey recruitment to new email lists of 4,424 piano teachers, assembled from music teacher association sites in Ohio, Pennsylvania, California, Massachusetts, Florida, New Jersey, and Georgia.
      • Received 154 responses so far (with 139 completing both parts).
      • Exchanged more emails with people who had problems with the survey tool or questions about the study.
      • Obtained C-MIDI source code from Dr. Dimitry Gorodnichy of the University of Ottawa. This may give us a head start on our contemplated automatic data collector (ADC).
      • Created new repo for ADC at https://github.com/dvdrndlph/dactylize. It is supposed to be a pun, as in dactyl (finger) + ize (eyes). But I don't think anyone is going to get it.
      • Created virtual Windows host and opened the C-MIDI code in Visual Studio. Not too surprisingly, the projects won't build.
      • Gnashed teeth when survey participant pointed out mistake in Exercise 3.

      BowTIE

      • Reached out to Ashkan Rezaei, a first year CS PhD student at UIC and jazz aficionado about maybe helping us evaluate BowTIE. He has an interest in automatic transcription, which would be very useful for our evaluation. He is receptive to the idea.

      Doing

      1. Defining SQLite schemata for (anonymized) survey data.
      2. Writing script to load said schemata.
      3. Adding file save and print features to MDC4.
      4. Adding support for non-exhaustive input in MDC4.
      5. Integrating MDC4 into draft Qualtrics survey for WTC, Book 1, Prelude 1.
      6. Assembling prerequisites (JavaScript libraries, data layer) for BowTIE.
      7. Standardizing file format for fingering.

      Struggling

      • Response rate stinks. Currently, its upper bound is 154/5292 = 2.9%.
      • Do I need to worry about selection bias? (Are we only going to get outlier subjects with unusually diverse notions of fingering?)

      Tuesday, November 3, 2015

      Visual Studio, ho!

      Here are the steps I took to get Visual Studio working on my Mac (a puny MacBook Air to be specific):
      1. Download the ISO image for Visual Studio license through UIC's WebStore.
      2. Copy the Visual Studio license in a safe place.
      3. Download the ISO for Windows Server 2012 R2.
      4. Save its keys to a safe place.
      5. Download and install VirtualBox for OS X hosts.
      6. Launch Virtual Box.
      7. Click the New button to create a new virtual host.
      8. Enter a name and select "Windows 2012" as the version for the host. Configure at least 2GB of RAM.
      9. Click Create and select your Windows Server ISO file as the File location.
      10. Click Create.
      11. Run the new virtual host. You will be asked to install Windows.
      12. Click Devices --> Insert Guest Additions CD image.
      13. Run VBoxWindowsAdditions.exe from the CD image.
      14. Click Devices --> Shared Folders --> Shared Folder Settings and share the folders you stored the Visual Studio ISO and our Git repo.
      15. Reboot your virtual host.
      16. On the virtual host, install MagicISO.
      17. Use MagicISO to open the VisualStudio ISO and install VisualStudio from it.
      18. Open the .dsp files from our repo in VisualStudio.
      19. Figure out why none of them will build.

      C-MIDI, ho!


      In a rather exciting development, I just received the source code from Dimitry Gorodnichy for the C-MIDI project. This might give us a head start on our automatic data collection system.

      Here is the conference paper that describes the marker-less C-MIDI system:
      D. Gorodnichy and A. Yogeswaran, “Detection and tracking of pianist hands and fingers,” in Proceedings of the Third Canadian Conference on Computer and Robot Vision, 2006.
      The paper is long on technical details, but short on performance data. This is all they have to say on the subject of system performance:
      The professional pianists, as they play and see at the same time on a computer screen the visual annotation of their playing, are content acknowledging the correct finger annotation in about a half of cases. Of the other half, the fingers are either left unmarked or can provide a set of possibilities to choose from.​
      ​Still, it is worth a long hard look.

      I have checked the C-MIDI source code into our new GitHub repo. (I think it is good to keep this project separate from the other Didactyl code in case this one gets a little bloated, which seems likely.)

      Unfortunately, the code provided is a Microsoft Visual Studio (Windows) project, which I have little experience with. But we do have a free Visual Studio license through UIC, so we should be able to compile the code easily on a Windows box.

      Pablo and I are looking into running Windows on our Macs using VirtualBox. I have had luck doing this in the past, and we have free Windows Server 2012 R2 licenses through UIC, so this can be done at no cost.

      Another option, and a better one if we actually want to test this system, is to run Windows under Boot Camp.

      Wednesday, October 28, 2015

      Fill in the blanks

      Given an ABC score and a non-exhaustive set of fingerings, we should be able to determine if the fully annotated score can be inferred, based on the fingerings provided and some set of formalized editorial conventions.

      The only conventions that seem apparent to me at present are the "next note, next finger" (NNNF) convention and the "only one way" (OOW) convention. There may also be a simile convention that repeats fingerings for repeated sections of (similarly beamed?) notes in cases when NNNF does not strictly apply.

      NNNF seems to have a straightforward linear-time algorithm (for melodies at least), where we move left-to-right through a list of notes n_1, n_2, n_3, .... The first notes will infer their fingerings from the first downstream annotation encountered. Subsequent notes will infer fingerings from themselves (trivially) and from the most recent prior annotation encountered. We keep track of n_i.finger (the controlling annotation) and n_i.offset (the signed difference--as a count of lines and spaces--between n_i and the note with the controlling annotation.) And we simply assert that 1 ≤ n_i.finger + n_i.offset ≤ 5. There may be some subtlety when accidentals are involved.

      Dealing with polyphony is another kettle of fish. OOW seems to be in play for chords in The Little Pianist. There may also be standard fingerings (SF) that are assumed for chords.

      Chunking into phrases seems very material here. We should look at Radicioni's work in this area. I know he had some things to say about phrases, and this is its own research topic, I believe.

      If we can implement something reasonable in this area, we should also be able to auto-fill fingerings in MDC4 (MDC5, anyone?). We could pad with xes in the input field and (optionally) render the complete annotations on the score. Or maybe we need separate input fields for each note or a pop-up for each note as it is clicked for more surgical entry in this mode of input. And do we want to revisit the utility of voice fingering (as opposed to the left-to-right, low-to-high approach currently employed in MDC4) in this new context?

      Another application would be to identify "unnecessary" annotations in a score and to learn (infer rules for) when such unnecessary annotations are appropriate (i.e., appreciated by performers). We could try to come up with some sort of model of good editorial practice.

      2015-10-28 status

      Done

      Administrivia

      • Accepted invitation to talk to the Pardo Lab at NU about piano fingering on December 8.

      Data Collection

      • Sent reminder email to the 203-strong Midwest piano faculty list.
      • Distributed initial survey recruitment email to newly assembled list of 168 music departments across the U.S. and Canada.
      • Distributed initial survey recruitment email to 51 additional piano faculty, mostly in Canada.
      • Distributed initial survey recruitment email to list of 280 piano teachers, assembled from music teacher association sites in Milwaukee, Madison, Kansas City, and Minnesota.
      • Received 62 responses so far (with 53 completing both parts).
      • Exchanged several emails with people who had problems with the survey tool or questions about the study.
      • Outlined simple algorithm for classifying scores as fully (unambiguously) specified, identifying unnecessary annotations, and generating fully annotated scores from fully specified scores. I currently only contemplate the "next note next finger" convention, but even if this is as sophisticated as it gets, this should be a nice little sub-project that could yield a quick workshop paper.

      BowTIE Revival

      • Explored Yeoman/Angular/Ionic approach to PhoneGap development, completing this tutorial.
      • Dr. Forbes and his advisee Alex Pieczynski think Alex should be able to get CS 398 credit for BowTIE work next semester. Sounds good to me.

      Doing

      1. Defining SQLite schemata for (anonymized) survey data.
      2. Writing script to load said schemata.
      3. Adding file save and print features to MDC4.
      4. Adding support for non-exhaustive input in MDC4.
      5. Integrating MDC4 into draft Qualtrics survey for WTC, Book 1, Prelude 1.
      6. Assembling prerequisites (JavaScript libraries, data layer) for BowTIE.
      7. Standardizing file format for fingering.

      Struggling

      • Response rate stinks (but with a big enough denominator, all things are possible). Currently, its upper bound is 62/868 = 7.14%.
      • Do I need to worry about selection bias? (Are we only going to get outlier subjects with unusually diverse notions of fingering?)