Vision pops back into view

Okay, we are now turning back to the ADC rigamarole. Here is what I just sent to my new mentee Pablo on the subject:

Dear Pablo: 

Here are some links:

• Open Frameworks

• Processing

• OpenCV

• C-MIDI

• GitHub

• Mendeley

​And I attach a few other relevant papers I have collected on your new favorite topic. I recommend that you read these papers and try to do some sort of "Hello CV" program in Processing, just to get your feet wet. 

The last link is to Mendeley, which I use to manage my references and make bibliographies easy. We can also set up a common folder to share references for this project. I think it is handy.

Also, if you haven't already done this, you should sign up for a GitHub account. We will use it as a central repository for our code, and rumor has it that many developers in Silicon Valley don't have resumes. They just have GitHub profiles. Anyway, send me your user ID when you have one. I am dvdrndlph. (This will probably live as a directory under https://github.com/dvdrndlph/didactyl.) 

I think that is enough mentoring for now. Have fun.

Cheers,
Dave

2015-10-21 staus

Done

Administrivia

• Hosted (with a lot of help from colleagues at UIC and NU) Chris Raphael's distinguished lecture, "The Informatics Philharmonic." It was a real career highlight for me.
• Interviewed two undergraduate mentee (yes, that is a word) candidates for WiCS DREAM program. They were both interested in the BowTIE project. One (trumpeter Jackson Price) will start this semester. One (guitarist Alex Pieczynski) will join next semester.
• Reached out to CR for help identifying a more research oriented to piano pedagogue to collaborate with.
• Sent a status update to Dr. Forbes per his request at the Raphael dinner. It would be great to engage him more.
• Received a nice email from Eugene Graub, a piano professor at Grinnell, asking to be kept apprised of the Didactyl project.

Data Collection

• Distributed initial survey recruitment email to list of 203 piano faculty in Midwest universities. Received 20 (with 15 completing both parts) responses so far.
• Sent reminder email to CAMTA mailing list.
• Started compiling music department contact list, as third recruitment wave will apparently be needed to get to 50 respondents, which is still my minimal goal.
• Met with SJ to demo MDC4 and discuss my planned data collection approach.
• Exchanged email with Dr. Cruz, whose brother is a classical pianist. As tradition mandates, I tried to rope him in as a subject or collaborator. It was nice of her to think of my work.

BowTIE Revival

• Researched cross-platform mobile development.
• Created new GitHub project for this development work.
• Created "Hello World" PhoneGap application. PhoneGap seems a promising way to get a research system up fast.
• Jackson has trumpet and horn performance undergraduate friends (at Northern Illinois and UIUC, as I recall) who (along with me and him) might form be a subject pool to evaluate this system. I am very excited to have Jackson on board.

Doing

1. Defining SQLite schemata for (anonymized) survey data.
2. Writing script to load said schemata.
3. Adding file save and print features to MDC4.
4. Integrating MDC4 into draft Qualtrics survey for WTC, Book 1, Prelude 1.
5. Assembling prerequisites (JavaScript libraries, data layer) for BowTIE.
6. Standardizing file format for fingering.

Struggling

• Response rate still stinks.
• Do I need to worry about selection bias? (Are we only going to get outlier subjects with unusually diverse notions of fingering?)

Annotation conventions and ambiguity

I received a nice email from Eugene Gaub, a music professor at Grinnell. In my response to him, I outlined my current thinking on the minimal annotation in edited scores. Here it is so I don't lose track of it. . . .

Actually, we just started seriously looking at how completely "determined" or "specified" a given passage is, given the apparently sparse annotations included in the score. I wonder if there are standard conventions for how to decide when to include an annotation and how much variability there is among editors and composers. Would you be able to point me to any good pedagogical sources that might talk about this? 

I am actually only a beginning pianist (and to say even this is an insult to beginning pianists everywhere), and I have been struck by how few numbers appear on an exercise, yet the fingerings are in fact unambiguously specified--given an implied "next-note-next-finger" convention. But as you describe, I have often experienced the desire to have more numbers on the page, to be sure.

We plan to explore the notion of optimal verbosity of annotations, perhaps for varying skill levels or familiarity with a given piece. Given a "fully determined" set of fingerings, I envision a sort of verbosity knob where we display just the right number of numbers. Measuring intrinsic "optimality" in this context might pose a special challenge. But we could do some usability testing with actual pianists if push came to shove. The more I think about this, the more I like it as a related sub-project. 

Status and near-term plans

Here is some information I sent to AF to summarize where I am at and where I think I am going. . . .

The happy news is that we do have survey data starting to trickle in now. If you want to see the survey, it lives here.

This survey includes a little JavaScript program to facilitate fingering data entry. This code has been largely rewritten recently to support the more complex music (from Book 1 of Bach's Well Tempered Clavier) planned for a second phase of "manual" data collection. So the dominoes are mostly in place now for data collection through the end of the year (during which we will happily pay $700 of my $2000 Provost's Award to pianists).

However, by November 15, I should have sufficient data to say something new. (Indeed, at Dr. Jordan's suggestion, I am starting to analyze data now. I have 19 responses so far.) So I plan to be shopping my first conference paper around by the first of the year. I really want to get into ISMIR 2016, as it is in New York this year in August. But something smaller and sooner will also be targeted.

In parallel, work will resume in earnest soon on our computer vision system to collect fingering data from actual performances. (There was a lot of rumination on this topic in the early going.) This part should be a lot of fun, but it is unexplored terrain for me. I am therefore hoping to rope in an undergraduate student (under the auspices of the WiCS DREAM mentoring program) to help. We have $500 budgeted to spend on equipment for this and then our last $800 to compensate pianists to evaluate and leverage it.

Conferences

The ISMIR conference will be in New York in 8-12 August 2016! So I am setting my sights squarely on that. I will probably go even if I can't get a paper accepted there. The conference was in October last year. Not sure why they moved it up (and I wish they hadn't).

MCM 2016 does not appear to be scheduled yet. While it was in London in June this year, it has been in the US, Canada, and Mexico in the recent past.

Sound and Music Computing also does not seem to have their 2016 act together yet. It was in Ireland in 2015 at the end of July, but it seems always to be in Europe.

The International Computer Music Conference was in Texas this year at the end of September. It will be in the Netherlands 2016 on September 12-16.

Another, lower key, event to keep in mind is the one that Bryan Pardo is trying to get established: The Midwest Music Information Retrieval Gathering. He and CR were actually talking about revitalizing this event at CR's distinguished lecture at UIC. It would be good if UIC had a presence there.

It sucks that all these things are right on top of each other on the calendar.

2015-10-14 status

Done

Administrivia

• Obtained $200 to pay lottery winners for first two surveys.

Data Collection

• Invited several more acquaintances to participate in survey and did some ineffectual nagging.
• Distributed initial survey recruitment email to list of 165 Chicago area piano teachers. Received 6 responses so far.
• Added character buffer to make MDC4 fingering entry more fluid. Performance seems good now.
• Added ability to specify hand in use for fingerings to allow either hand to be used on either staff.
• Implemented output and validation functions in MDC4 to enable interoperation with Qaultrics.
• Fixed problem with whitespace entry in MDC4.
• Changed MDC4 rendering of left and right hand fingerings to be in different fonts, so they can be easily differentiated.
• Implemented method in MDC4 for annotating synchronous pitches from multiple voices.
• Supported "redundant" identical synchronous pitches from multiple voices in MDC4.
• Took a stab at defining BNF grammar for fingering entry strings, using ANTLR. Regular expressions currently in use are buggy and restrict expressiveness. But they are probably sufficient for the current task.

Doing

1. Auditing Music 170 (Keyboard Skills I), TuTh 3:30-4:45.
2. Adding file save and print features to MDC4.
3. Integrating MDC4 into draft Qualtrics survey for WTC, Book 1, Prelude 1.
4. Standardizing file format for fingering.

Struggling

• Response rate stinks.

2015-10-02 status

Done

Administrivia

• Resubmitted program advance for initial surveys per Sherice.
• Obtained IRB approval for data collection activities.
• Worked out some scheduling details for CR's Distinguished Lecture at UIC.
• Reached out to the local academic friends of the project about the lecture.
• Reached out to Peter Snyder about grad student panel to talk to CR when he is here. 
• Made pitch for lecture and panel at Friendly Friday.
• Posted two flyers for the lecture in the Music Department (with their office's approval).
• Got positive response from Bryan Pardo at NU. It seems likely there will be a contingent from his lab.

Data Collection

• Fixed two typos in the survey, which I caught during another "final" review.
• Sent out initial survey to my short list.
• Improved latency between fingering entry and rendered annotation by more than 50%. But it still feels sluggish.
• Pinged Jef Moine (abc2svg developer and general mensch) and learned how to make the fingering symbols bigger.
• Tested MDC4 with 5-voice piece (P&F 1 from WTC). Fixed several issues.

Doing

1. Auditing Music 170 (Keyboard Skills I), TuTh 3:30-4:45.
2. Adding character buffering to MDC4 to make input smoother.
3. Implementing output hooks in MDC4 to play nice with Qualtrics.
4. Standardizing file format for fingering.

Struggling

• Pinged M. Moine to complain about bad associations of fingerings to notes that start at the same time in different voices. This really seems like a bug. Waiting for response. If no fix is forthcoming, I will need to code around this.