Friday, June 19, 2015

Surveying the survey

From: David Randolph
Date: Fri, Jun 19, 2015 at 3:48 PM
Subject: Re: Help needed for piano fingering study
To: Ivana Bukvich


​I like the term *holistic*. You sound like my kind of teacher.

On trumpet, Hanon goes by the name of Arban. I hate that guy--or at least the first half of his "conservatory method." But I have always assumed that my playing stalled because I couldn't bring myself to master all of his exercises.​ Every time I pick up the horn again, there he is.

Cheers,
Dave

On Fri, Jun 19, 2015 at 2:44 PM, Ivana Bukvich wrote:


Between 8-11. Teachers use them with all age groups though. My personal opinion is that they are boring and most of the students agree. However, they can very useful. In terms of sight reading they reinforce interval if reading so that is why I like them. Also, for the same reason they are easy to transpose and useful that way. You are correct about the repetitive nature that allows one "not to think" so the motor memory is primarily used. I think that can be useful as well with an average student , but I like more "holistic" approach to practicing .


On Jun 19, 2015, at 2:18 PM, David Randolph wrote:


Dear Ivana:

I am going to leave the Bach out. We will have activities down the road for larger scale manual annotation. These activities should have their own incentives, I think, and deserve an interface with no warts on it. Moreover, we don't want to scare people off with too much work up front.

Can you tell me how old you were when you played Hanon, or, more generally, at what ages it tends to be used? I may add a question specifically about if, when, and for how long Hanon was used by the subject.

And when you say you "use" Hanon, do you mean play it yourself or use it with students? I assume the latter. I am surprised by your using it as "sight reading exercises," as my assumption is that these exercises would be the least surprising (and most internalized) pieces in the canon. They would pose technical challenges more than cognitive challenges, no? I would like to discuss these matters further. I wonder what we could do in the way of simulating a practice regimen or capturing the essence of a player's total experience with their repertoire. If we actually get to the point where we can record people's fingerings as fast as they can play, a lot of options will open up. We are long way from there, but it is fun to think about.

Are there any alternatives to Hanon? Or does it have a corner on the market of such drudgery?

Thanks,
Dave


On Fri, Jun 19, 2015 at 12:58 PM, Ivana Bukvich wrote:


Well, since I only worked on fragments it took about 15 min, it was clearly an optional activity. Hanon is probably useful as data, but , really doesn't tell you anything about fingering choices as there is only one way to play them. On the other hand,it tells something about how to finger patterns. I find them terribly boring and only use them as remedial sight reading exercises, of course , had to practice them as a child.. I agree on things you can leave out.


On Jun 19, 2015, at 11:49 AM, David Randolph wrote:


​Dear Ivana:

Thanks for the great feedback.

How long did the "Invention 1" annotation take? I expect this to add something like 30-40% to the ​time it takes to complete the survey. Was it clear that this was an optional activity? If we are keeping this in, I will add language to make it clear that this may be time consuming and is purely optional. Or do you think we should just leave it out?

We could also remove one or more of the other fingering exercises. Any thoughts on this?

Also, do you have anything similar to Hanon that you use with your students or use/used in your own studies? The thing is, we have fingering data (in machine readable form) for Hanon that we can just use. I have (vague) plans to incorporate these data as foundational fingering patterns in our models. Are you familiar with Hanon at all? Are you "anti-Hanon" for some reason? What else is out there that is similar? I am just trying to find out if there are any canonical rudiments that might influence fingering choices significantly.

Here are the changes I plan to make. . . .

I will implement the left-to-right, low-to-high fingering input you suggested. This is a good change.

I will investigate using anonymous survey links.

Having to select each of the 24 inventions had to be tedious. I will look for a way to select all or revise how we get at this information.

I will remove the questions about problematic reach between finger pairs. They show little promise of providing us anything useful.

I will add questions about any Hanon alternatives that you think are worthwhile.

Many many thanks,
Dave


On Fri, Jun 19, 2015 at 9:07 AM, Ivana Bukvich wrote:

I see, it looked like short fragments, but perhaps it was like that because of the fact that I did it on the phone, I have i-phone 5c. The whole survey took about 45 min to an one hour, it is a bit long.


On Jun 19, 2015, at 8:22 AM, David Randolph wrote:


Dear Ivana:

You should have seen the whole "Invention" score with separate input fields for each of the lines. The only problem should have been with missing continuation ties across lines. I will take another look to see if anything is missing. If you say this looks like fragments, then we may just drop this question. I have a plan to update the interface to make this look better, but I don't want to hold up the survey waiting for it.

Did you really do this whole thing on an iPhone? That is really good news. What kind of iPhone is it? I couldn't do it on my dinky little Android phone.

How long did it take you to do this in total? Do you think it is too long?

Thanks,
Dave



On Thu, Jun 18, 2015 at 10:59 PM, Ivana Bukvich wrote:


I just completed it, Bach Invention w/o score . Did you mean to have only fragments , that's what I got on I phone. I


On Thu, Jun 18, 2015 at 4:47 PM, Ivana wrote:
Hi Dave, I've just completed most of it ,and had to stop as I was on I phone and it ran out of batteries.I will continue tonight afterI get home. It looks good, small suggestion:for the left hand, when having double notes(intervals) it would be better to enter bottom finger first, then the top as in reading chords one always looks at the root of the chord first then the rest of the notes,same with fingering. I
Also, I will forward the survey to my UIC  students. Grant looks good but I have no idea how to do it really.
Sent from my iPad


Thursday, June 18, 2015

Grant me a grant


Monday, June 15, 2015

A fool's errand in four phases

After the survey, I am leaning toward an initial follow-up study that involves manual annotation (similar to what is requested for "Invention No. 1" in the survey, but a bit more complex) of one or more selections from Bach's Well Tempered Clavier (WTC).

This is because I have discovered high-quality machine-readable (MuseScore) editions of WTC and the Goldberg Variations are available. Also, the pianists can do this work anywhere and therefore need not be local.

Oh, and building the input interface for this is a much lower risk proposition than building the sensor/computer-vision system, and the manual system is seen as a necessary component of the "semi-automated" system we also described in the Provost's Award proposal.

So the plan is 1) survey, 2) MDC4, 3) SADC, 4) FADC.

The fully automated system is still planned, but actual work on it probably won't start in earnest until school ramps up in the fall. The next development milestone will be a robust manual data collector that copes with up to four voices. (The current system only deals with two voices and has at least one problem with pieces that contain multiple lines. It is only just barely able to support the music in the survey.)

The survey has been updated to randomize the order of presenting exercises B, C, D, F, and G. Then, based on whether the subject's age is odd or even, it either presents Czerny's fingered context or a completely unfingered context for exercise A. Then it does the same for E.

I chose A and E for the experiment because they had the highest variability reported by Parncutt et al. However, I probably should have normalized by the length of fingered fragment. A and E are the longest fragments at 8 notes each.
  • A: 10 fingerings / 8 notes = 1.25 fingering/note 
  • B: 5 fingerings / 4 notes = 1.25 
  • C: 9 / 5 = 1.8 
  • D: 8 / 7 = 1.143 
  • E: 18 / 8 = 2.25 
  • F: 5 / 6 = 0.833 
  • G: 9 / 7 = 1.286 
But this doesn't really measure the consensus I see, as A and E clearly have the widest disagreement. I need to get my head around this.

Agreement, expressed as mode count over total fingering count.
  • A: 8/28 = 0.2857 
  • B: 17/28  = 0.6071
  • C: 10/28 = 0.3571
  • D: 15/28  = 0.5375
  • E: 4/28 = 0.1429 
  • F: 23/28 = 0.8214
  • G: 14/28  = 0.5000
This seems like one avenue to a Kappa score. Or maybe a weighted Kappa score, as the fingers do indeed have a natural order. Or should we think of every note as being "categorized" with a particular (weighted) finger? This probably amounts to the same thing.

I think the average edit distance from the mode (most popular fingering) gets us a pretty good measure of overall similarity for a bunch of fingerings. The greater the consensus on a single fingering, the more edit distances of zero we have. We can normalize the edit distance by dividing by the note count. This should give us a number between 0 (perfect agreement) and 1 (no agreement).

But what about the case, like piece A, where there are two popular--and quite dissimilar--fingerings? Is this showing agreement or disagreement? Using our normalized edit distance approach, this is going to register as disagreement. Is this fair? Should we be calculating all of the distances between all of the fingerings instead? Yes, I think so.

This probably just boils down to Fleiss's Kappa. But can this be weighted? It seems as though this should be possible.

I have also added a mechanism in the survey to measure how much time the user spends on each fingering exercise, though it is still unclear what happens to this data the user backtracks and visits the exercise more than once. We apparently need to keep the BACK button active to cope with the inconsistent way that different browsers cope with our error message for incomplete fingerings. (Safari takes you to the next screen. Chrome leaves you on the current screen after sometimes printing a confusing message about the BACK button.

Thursday, June 11, 2015

Zombie survey

The survey just won't die.

I am thinking that we need to do something more experimental with the fingering exercises. In including the full exercises, it occurs to me that we could include a few exercises with Czerny's fingerings included. This plus a measure of variability would give us a way to measure the influence and see how far it reaches across hand sizes. I really think this is worth doing, but how do we pick the exercises that tip their hands? The pieces with the highest and lowest variability? Randomly?

Having timing information would also be useful--if we randomly show fingerings or not (and make it so the surveys are not uniform). I know Barbara will worry about this, as we will need more subjects to say anything with statistical significance. Can we mitigate this somehow?

We have 7 exercises. In half the surveys (the experimental group), we show 3 fingered exercises and 4 unfingered.  In the other half (the control group), all pieces are unfingered. The pieces are displayed in random order.

Throw out the piece with three voices?

I also want to ask about specific pieces that we actually have transcriptions for (or are willing to commit to transcribe). The idea is to take the intersection of a university or publisher "standard baroque/classical repertoire" and the contents available at MuseScore. We could even provide links to the current transcriptions, since I am not sure how memorable the titles are going to be, even to people who know the pieces well.

Is there a "standard" Czerny method that competes with Hanon? Are there any other comparable methods with significant mindshare?


Tuesday, June 2, 2015

2015-06-02 status

Done

Administrivia

  • Received notification that CS Department had provided feedback on annual review. Asked Santhi how to access.

Data Collection

  • Integrated new Manual Data Collector (MDC) into Qualtrics survey.
  • Fixed Safari MDC rendering problems.
  • Added cursor management on failed input validation.
  • Improved note highlighting.
  • Fixed problem annotating notes with accidentals.
  • Revised survey recruitment email and integrated it into Qualtrics.
  • Validated end-to-end survey flow in Qualtrics.
  • Discovered rendering problem with multiline tunes.
  • Drafted alternative "MDC2" interface for longer tunes with scrolling windows for both input and rendered output.
  • Identified performance problem with abcjs library for longer tunes.
  • Opened several issues against abcjs.
  • Discovered and evaluated abcjs alternative abc2svg and found it much more performant.
  • Removed several questions from survey and debugged several problems.
  • Decided to remove "bonus" annotation question from survey and to plan for a manual data collection follow-up survey with a separate $100 gift card lottery.
  • Decided to use email to send gift cards to winners.

Doing

  1. Updating IRB documents to reflect changing plans.
  2. Piloting abbreviated survey with my friend Alex.
  3. Defining survey distribution lists.
  4. Migrating MDC2 to abc2svg.

Struggling

  • The UI for MDC2 needs to keep everything visible and rendering quickly.
  • I think we should have three consent forms for the three planned data collection stages. Subjects don't need to learn about things they aren't going to do.