Onion of assumption

Assumptionion?

There are a lot of assumptions we are making in our planned data collection and how it relates to the the fingerings that are actually used in performance.

1. (MDC) Fingerings for an imaginary (unmotivated, unvalidated) performance are the same as those generated for an actual performance.
2. (MDC4) Fingerings made for a performance in the past remain relevant for a new performance.
3. Entering fingerings into a web application produces the same results as those collected on paper. (Using an electronic device might disrupt usual procedures for determining fingerings.)
4. Full annotations, and not the presumably more typical minimal or targeted annotations, are accurate.
5. Annotations reflect the fingerings used in actual performance.
6. Actual performances are consistent over time.
7. Read performances are the same as memorized performances.

This all again makes me want to study how people (composers, editors, and performers) decide which notes to annotate and which to ignore. The notion of fingering ambiguity should be explored. When does it exist? How can it be measured? Are annotations primarily to flag difficult passages--warning signs in a sea of automation? Or markers of orthodoxy in a sea of freedom/chaos? Or do they, with the help of some conventions, unambiguously define the entire performance? Where are these conventions documented? Ivana and Anne should be able to help with these questions.

Rage about order

Barbara advises I think a bit more about the randomization of the fingering exercises in the first survey. Here goes. . . .

There may be a natural order for the exercises (A-G)--assuming Czerny was presenting these in an order that makes pedagogical sense. The motivation of randomizing is to disperse any influence that fingering one piece would exert on the fingering of another. But why is that important to do? Why not just let this influence exert itself. If such an effect is significant, is it not better to have it be applied in a uniform way? And if it is subtle, wouldn't testing it repeatedly help us detect it. Also, we are not randomizing the order in which A and E are presented. I think showing them in a fixed order might be the way to go. And we just stick A and E (with their coin toss on recommended fingerings) back into the original order.

The other important matter is minimizing the differences between what we did and the stated procedure of Parncutt et al.:

Each pianist received the complete score of the seven pieces. The scores included tempo and dynamic markings, but not fingerings, which had been carefully erased (by whiting out and recopying). The 12 professional pianists and 3 students received the scores after partici- pating in a companion study that involved sight-reading each piece twice. They took the scores home before writing down their preferred fingerings. The other participants received the scores by mail. None of the participants were directly paid for their services; however, the pianists who had participated in the sight-reading study were reimbursed (£25 for pro- fessionals and £5 per hour for students). 

Pianists were informed that the pieces were by Czerny, but were not told the title or opus number of the collection. They were asked to write on the score their “preferred fingerings” for the right hand of the first two measures of each piece. In order to avoid ambiguity, they were asked to write a finger number against each and every note. A preferred fingering was defined as “the fingering that you would probably use in performance.” They were asked to disregard any fingerings that they thought or knew that the composer intended for the pieces; retrospectively, we found no reason to believe that any of them were influenced by such knowledge or belief. They were also asked to write brief explanations for their choice of fingerings directly on the scores; the aim of asking for this commentary was primarily to ensure that the pianists had given the matter some thought and were sure of their fingering solutions.

Note they say nothing about the order in which the pieces were presented, and note that there were different procedures followed followed for two groups of participants. I think it safe to assume that the players in the sight-reading experiment were exposed to the pieces in a fixed order, and that all of the participants were delivered a stack of papers in a particular order. At this point, they could solve the problems in any order they chose. But it seems likely that expert pianists, unless they dropped the papers on the floor, would simply work through the pieces in order.

Our electronic survey interface allows subjects to move between the pieces as well by way of NEXT and BACK buttons, but I doubt much use of this will be made. Also, we are not asking for piece-by-piece written rationale, as I am not sure how doing so "ensures" anything about how diligent the subjects were. They say nothing about how these "commentaries" were evaluated to vet subjects--not too surprising for researchers starved for data.

Also, we are not going to tell the students who wrote the pieces, so we don't later have to tell them to ignore this fact. Also, we are simply asking them to "Enter the fingerings you would use to perform this music." We do not say anything about probability. To align with Parncutt (and likely with the variability of such things in reality), we should change this to "Enter the fingerings you would probably use to perform this music." This change can be made easily to the MDC JavaScript.

Data plumbing

Okay, the prime directive is this: No PII gets out of Qualtrics. The following should work:

1. Deliver "Survey I" (which leverages MDC) via anonymous links.
2. Use the survey settings to suppress the IP and geolocation data and leverage the Question selection when we download the CSV to skip My contact information. . . .
3. Load the data from the CSV to a MySQL (or SQLite) database, using the "ResponseID" as the primary key (our "arbitrary label") for user records.
4. Assemble a Qualtrics panel for the "Survey II"/MDC4 activities, adding "ResponseID" as a field. (The automatically generated "Survey I Complete" and "Survey II Complete" panels will include this setting as "TriggerResponseID.")
5. Deliver "Survey II" (which will leverage MDC4) via individualized links.
6. Store the original "ResponseID" (pulled from the panel record field "TriggerResponseID") as embedded data in "Survey II." You will need to use the name "TriggerResponseID," as there is no apparent way to rename this field.
7. Use the survey settings to suppress the IP and geolocation data when downloading the CSV.
8. Load to MySQL using the anonymized "TriggerResponseID" as primary key.
9. Use the same original "ResponseID" to identify subjects in the in-person SADC and FADC data collection activities.

2015-07-07 status

Done

Administrivia

• Assembled amendment materials for IRB.

Data Collection

• Integrated new Manual Data Collector (MDC) into Qualtrics survey, with low-to-high input ordering for chords (per Ivana's feedback).
• Completed pilot with Ivana.
• Drafted recruitment emails for faculty and for department staff.
• Established project web site at http://drando2.gsites.uic.edu (https://sites.google.com/a/uic.edu/didactyl).
• Compiled mailing list (Qualtrics panel) of nearly 200 piano faculty in Illinois, Iowa, Wisconsin, Indiana, Missouri, Michigan, and Ohio. The hope is that a significant subset of these people and their students will a) respond and b) hail from Chicago.
• Settled on "Open Well-Tempered Clavier" (OpenWTC) and "Open Goldberg Variations" (OpenGoldberg) for our initial corpus, as these were well-publicized (Kickstarted) efforts by MuseScore to produce high-quality editions.
• Translated OpenWTC to abc format.
• Added a little experiment to the survey for two of the fingering exercises. Half of the subjects will see the suggested Czerney fingerings.
• Finalized survey.

Doing

1. Leveraging abc2svg for MDC4.
2. Standardizing file format for fingering.

Struggling

• Contemplating a major rewrite for MDC4, where I get out of the business of parsing abc myself. This should simplify the code greatly, but the way is not clear.
• Prelude and Fugue 18 appears to be corrupted by my hacking efforts to migrate to abc.

Surveying the survey

From: David Randolph
Date: Fri, Jun 19, 2015 at 3:48 PM
Subject: Re: Help needed for piano fingering study
To: Ivana Bukvich


​I like the term *holistic*. You sound like my kind of teacher.

On trumpet, Hanon goes by the name of Arban. I hate that guy--or at least the first half of his "conservatory method." But I have always assumed that my playing stalled because I couldn't bring myself to master all of his exercises.​ Every time I pick up the horn again, there he is.

Cheers,
Dave

On Fri, Jun 19, 2015 at 2:44 PM, Ivana Bukvich wrote:


Between 8-11. Teachers use them with all age groups though. My personal opinion is that they are boring and most of the students agree. However, they can very useful. In terms of sight reading they reinforce interval if reading so that is why I like them. Also, for the same reason they are easy to transpose and useful that way. You are correct about the repetitive nature that allows one "not to think" so the motor memory is primarily used. I think that can be useful as well with an average student , but I like more "holistic" approach to practicing .


On Jun 19, 2015, at 2:18 PM, David Randolph wrote:


Dear Ivana:

I am going to leave the Bach out. We will have activities down the road for larger scale manual annotation. These activities should have their own incentives, I think, and deserve an interface with no warts on it. Moreover, we don't want to scare people off with too much work up front.

Can you tell me how old you were when you played Hanon, or, more generally, at what ages it tends to be used? I may add a question specifically about if, when, and for how long Hanon was used by the subject.

And when you say you "use" Hanon, do you mean play it yourself or use it with students? I assume the latter. I am surprised by your using it as "sight reading exercises," as my assumption is that these exercises would be the least surprising (and most internalized) pieces in the canon. They would pose technical challenges more than cognitive challenges, no? I would like to discuss these matters further. I wonder what we could do in the way of simulating a practice regimen or capturing the essence of a player's total experience with their repertoire. If we actually get to the point where we can record people's fingerings as fast as they can play, a lot of options will open up. We are long way from there, but it is fun to think about.

Are there any alternatives to Hanon? Or does it have a corner on the market of such drudgery?

Thanks,
Dave


On Fri, Jun 19, 2015 at 12:58 PM, Ivana Bukvich wrote:


Well, since I only worked on fragments it took about 15 min, it was clearly an optional activity. Hanon is probably useful as data, but , really doesn't tell you anything about fingering choices as there is only one way to play them. On the other hand,it tells something about how to finger patterns. I find them terribly boring and only use them as remedial sight reading exercises, of course , had to practice them as a child.. I agree on things you can leave out.


On Jun 19, 2015, at 11:49 AM, David Randolph wrote:


​Dear Ivana:

Thanks for the great feedback.

How long did the "Invention 1" annotation take? I expect this to add something like 30-40% to the ​time it takes to complete the survey. Was it clear that this was an optional activity? If we are keeping this in, I will add language to make it clear that this may be time consuming and is purely optional. Or do you think we should just leave it out?

We could also remove one or more of the other fingering exercises. Any thoughts on this?

Also, do you have anything similar to Hanon that you use with your students or use/used in your own studies? The thing is, we have fingering data (in machine readable form) for Hanon that we can just use. I have (vague) plans to incorporate these data as foundational fingering patterns in our models. Are you familiar with Hanon at all? Are you "anti-Hanon" for some reason? What else is out there that is similar? I am just trying to find out if there are any canonical rudiments that might influence fingering choices significantly.

Here are the changes I plan to make. . . .

I will implement the left-to-right, low-to-high fingering input you suggested. This is a good change.

I will investigate using anonymous survey links.

Having to select each of the 24 inventions had to be tedious. I will look for a way to select all or revise how we get at this information.

I will remove the questions about problematic reach between finger pairs. They show little promise of providing us anything useful.

I will add questions about any Hanon alternatives that you think are worthwhile.

Many many thanks,
Dave


On Fri, Jun 19, 2015 at 9:07 AM, Ivana Bukvich wrote:

I see, it looked like short fragments, but perhaps it was like that because of the fact that I did it on the phone, I have i-phone 5c. The whole survey took about 45 min to an one hour, it is a bit long.


On Jun 19, 2015, at 8:22 AM, David Randolph wrote:


Dear Ivana:

You should have seen the whole "Invention" score with separate input fields for each of the lines. The only problem should have been with missing continuation ties across lines. I will take another look to see if anything is missing. If you say this looks like fragments, then we may just drop this question. I have a plan to update the interface to make this look better, but I don't want to hold up the survey waiting for it.

Did you really do this whole thing on an iPhone? That is really good news. What kind of iPhone is it? I couldn't do it on my dinky little Android phone.

How long did it take you to do this in total? Do you think it is too long?

Thanks,
Dave



On Thu, Jun 18, 2015 at 10:59 PM, Ivana Bukvich wrote:


I just completed it, Bach Invention w/o score . Did you mean to have only fragments , that's what I got on I phone. I


On Thu, Jun 18, 2015 at 4:47 PM, Ivana wrote:
Hi Dave, I've just completed most of it ,and had to stop as I was on I phone and it ran out of batteries.I will continue tonight afterI get home. It looks good, small suggestion:for the left hand, when having double notes(intervals) it would be better to enter bottom finger first, then the top as in reading chords one always looks at the root of the chord first then the rest of the notes,same with fingering. I
Also, I will forward the survey to my UIC  students. Grant looks good but I have no idea how to do it really.
Sent from my iPad

Grant me a grant

• http://www.grammy.org/grammy-foundation/grants (Deadline: Oct 1?)
• Chancellor's Graduate Research Award (née Fellowship) (Submission from department September 23, 4:00 p.m. CDT)
• http://www.qualtrics.com/innovation-exchange/research-grants/ (Rolling deadline)
• http://www.ams-net.org/fellowships/ams50.php (Deadline: Dec 15)

A fool's errand in four phases

After the survey, I am leaning toward an initial follow-up study that involves manual annotation (similar to what is requested for "Invention No. 1" in the survey, but a bit more complex) of one or more selections from Bach's Well Tempered Clavier (WTC).

This is because I have discovered high-quality machine-readable (MuseScore) editions of WTC and the Goldberg Variations are available. Also, the pianists can do this work anywhere and therefore need not be local.

Oh, and building the input interface for this is a much lower risk proposition than building the sensor/computer-vision system, and the manual system is seen as a necessary component of the "semi-automated" system we also described in the Provost's Award proposal.

So the plan is 1) survey, 2) MDC4, 3) SADC, 4) FADC.

The fully automated system is still planned, but actual work on it probably won't start in earnest until school ramps up in the fall. The next development milestone will be a robust manual data collector that copes with up to four voices. (The current system only deals with two voices and has at least one problem with pieces that contain multiple lines. It is only just barely able to support the music in the survey.)

The survey has been updated to randomize the order of presenting exercises B, C, D, F, and G. Then, based on whether the subject's age is odd or even, it either presents Czerny's fingered context or a completely unfingered context for exercise A. Then it does the same for E.

I chose A and E for the experiment because they had the highest variability reported by Parncutt et al. However, I probably should have normalized by the length of fingered fragment. A and E are the longest fragments at 8 notes each.

• A: 10 fingerings / 8 notes = 1.25 fingering/note 
• B: 5 fingerings / 4 notes = 1.25 
• C: 9 / 5 = 1.8 
• D: 8 / 7 = 1.143 
• E: 18 / 8 = 2.25 
• F: 5 / 6 = 0.833 
• G: 9 / 7 = 1.286 

But this doesn't really measure the consensus I see, as A and E clearly have the widest disagreement. I need to get my head around this.

Agreement, expressed as mode count over total fingering count.

• A: 8/28 = 0.2857 
• B: 17/28  = 0.6071
• C: 10/28 = 0.3571
• D: 15/28  = 0.5375
• E: 4/28 = 0.1429 
• F: 23/28 = 0.8214
• G: 14/28  = 0.5000

This seems like one avenue to a Kappa score. Or maybe a weighted Kappa score, as the fingers do indeed have a natural order. Or should we think of every note as being "categorized" with a particular (weighted) finger? This probably amounts to the same thing.

I think the average edit distance from the mode (most popular fingering) gets us a pretty good measure of overall similarity for a bunch of fingerings. The greater the consensus on a single fingering, the more edit distances of zero we have. We can normalize the edit distance by dividing by the note count. This should give us a number between 0 (perfect agreement) and 1 (no agreement).

But what about the case, like piece A, where there are two popular--and quite dissimilar--fingerings? Is this showing agreement or disagreement? Using our normalized edit distance approach, this is going to register as disagreement. Is this fair? Should we be calculating all of the distances between all of the fingerings instead? Yes, I think so.

This probably just boils down to Fleiss's Kappa. But can this be weighted? It seems as though this should be possible.

I have also added a mechanism in the survey to measure how much time the user spends on each fingering exercise, though it is still unclear what happens to this data the user backtracks and visits the exercise more than once. We apparently need to keep the BACK button active to cope with the inconsistent way that different browsers cope with our error message for incomplete fingerings. (Safari takes you to the next screen. Chrome leaves you on the current screen after sometimes printing a confusing message about the BACK button.