Dactylize proof of concept

Initial breadboard layout supporting 16 keys and 10 fingers. 

Foil tape key overlays on four keys on mini keyboard, wired with
alligator clips for proof of concept.

Dactylize drawings

Finger and key connectors

Enclosure concept

Connector details

Architecture and thoughts on output

Circuit details

Dactylize comes alive

We have finally come up with a design for automatically collecting fingering data from actual performances that I think has a good chance of working (details to follow).

There are two catches. First, we will be attaching very light wired "sensors" (actually just connection points) to the pianist's unsuspecting finger tips. Minimizing the effects of these sensors on performance is a design challenge that we will need to address.

Second, and more important for this email, we will be making some modifications to the keyboard itself. These modifications should not damage the keyboard, but they are not things I would want to see done to a digital piano I cared about.

Therefore, I am considering two options. First, I have a very old and cheap 61-key keyboard (a Roland EP5) that I would happily donate to the cause. It is velocity sensitive, but does not have a weighted action. It is very nearly beneath *my* contempt. Indeed, I may use it for prototyping just for spite.

Option two is to purchase a $500 Casio Privia PX-160. This keyboard has a remarkably good graded hammer action that seems to be the best at this price point. In my almost utterly worthless opinion, it is reasonably comparable to the actions in the piano lab at UIC, but of course opinions will vary on this point. Anyway, it is the best keyboard I can afford with the grant money I have left.

My question to you: Is it worth it to have a graded hammer action? Part of the data we want to collect is a reasonably nuanced MIDI performance. I think it will help in this regard, and it should be more enjoyable for subjects to play. But I could also spend this money to compensate subjects. They would enjoy that too.

2016-03-09 status

Done

Data Collection

• Fixed problem with Survey III in Safari. I reinitialized the editor to be empty when NEXT is clicked, and everything seems to work normally. This should help our response rate a lot.
• Validated data entered in abcDE.
• Entered Czerny's recommended fingerings for Prelude 9 and 17 and Fugue 17. Fugue 9 posed too many problems for me to finger from Czerny. Decided to leave it in survey to see if people working from other editions fare better.
• Drafted recruitment email for Survey III.
• Finalized Survey III and asked IB and AFL to review.
• Transcribed 7 of 60 studies from Czery's Opus 821. 
• Fixed several abcDE bugs and warts.
• Dropped computer vision plans for automatic data collector (ADC) in favor of more straightforward electronic approach inspired by suggestion from Alex Demos. Dr. Demos suggested that using the MIDI data and closing a circuit tied to a particular finger should be enough: attach wires to the fingers and foil to the keys and just use finger-to-key contacts as switches. But I think the problem is a little more complicated than that, as having multiple fingers resting on the keys when a note is struck and the timing granularity of MIDI would likely leave us with considerable ambiguity. I think this needs to be handled as a patch-bay circuit, where a specific finger is tied to a specific key at the point of contact just as two parties were connected on an old switchboard.
• Found incredibly informative book explaining a patch-bay circuit using the Arduino and Pure Data (Pd), a graphical programming language I have actually used before. So I have a pretty clear plan in mind for prototyping ADC. It should take more soldering than thinking.
• Ordered about $300 of components and tools needed to build the system.
• Reached out to a EE friend to sanity check my plans. Waiting to hear back. Will reach out to Dr. Zefran if push comes to shove.
• Met with Dr. Demos and discussed my research plans and musical phrasing at some length. He thought WTC was a good corpus choice, and the missing annotation (fill-in-the-blank) problem seemed to resonate with him (as it did with CR). We concluded that we did not need to address phrasing in the WTC data collection, as we can make assumptions about phrasing because of the style of the music. Anyway, we will be leaving this for future work. By the way, he teaches advanced statistics in the Psychology Department. Hail fellow, well met.

Data Analysis

• Analyzed a number of descriptive charts and clarified some methodological matters.

 Doing

1. Discussing each descriptive chart in detail--its purpose, insight (if any). Top priority now because I want to share with subjects, as promised.
2. Transcribing Czerny Opus 821 to use in missing annotation study. 10% done.
3. Determining predictive value of one fingering sequence choice on another--initial Chi Square foray.
4. Calculating mean edit distance as measure of IAA and "influence."
5. Calculating adoption rate as measure of "influence."
6. Performing Chi Square analysis of exploratory dataset to correlate abbreviated Parncutt fingerings with gender, reach, age, Hanon usage, technical practice, preparation actions, injury, etc.
7. Finishing Parncutt algorithm implementation.

Struggling

• What, me worry?

2016-02-24 status

Done

Administrivia

• Completed required "Preventing Discrimination and Sexual Violence Training."
• Determined current plans are consistent with IRB protocol and terms of Provost's Award, which stipulate "funds must be expended substantially in accord with the budget submitted." (So we have some wiggle room.)

Data Collection

• Drafted "monolithic" Qualtrics survey to collect WTC fingering data for 9 P&Fs and to spend $1000 (or less?) in the process.
• Added undo/redo and revert functionality to abcDE.
• Added "tags" button to open modal dialog for metadata entry in abcDE.
• Added support for "copy" and "paste" of complete fingering sequences.
• Downloaded four public domain editions of WTC Book 1 with different editors (Czerny, Busoni, Czerny and Roitzch, and Mugellini) from IMSLP.
• Split preludes from fugues to be more digestible by subjects.
• Entered Czerny's recommended fingerings for seven WTC P&Fs (1, 2, 3, 5, 6, 15 and 21) into abcDE, identifying several bugs and warts in the process. The interface does make this a relatively painless process, so I can generate a fair amount of data myself.
• Fixed several bugs and warts. Several remain.
• Decided to implement a MIDI/image capture system independent from future computer vision system. Will prototype in Python and SimpleCV.
• Reached out to a friend and former film major for advice on lighting and cameras, with the idea of assembling a shopping list.

Data Analysis

• Soon, soon. Started typing R code today.

 Doing

1. Determining predictive value of one fingering sequence choice on another--initial Chi Square foray.
2. Drafting recruitment email for Survey III.
3. Calculating mean edit distance as measure of IAA and "influence."
4. Calculating adoption rate as measure of "influence."
5. Testing monolithic Qualtrics survey.
6. Discussing each descriptive chart in detail--its purpose, insight (if any).
7. Performing Chi Square analysis of exploratory dataset to correlate abbreviated Parncutt fingerings with gender, reach, age, Hanon usage, technical practice, preparation actions, injury, etc.
8. Entering Czerny annotations for P&F 9 and 17. The plan is to annotate--and in the process validate--the 10 (minus 1, since the content of Czerny's P&F 8 diverges significantly from OpenWTC) most well-known selections as reported by our survey respondents.

Struggling

• The editor is not working for multiple exercises in Qualtrics using Safari desktop. Fingerings from prior exercises appear in subsequent ones, and prior scores appear "under" current ones. Very strange and very bad for our response rate.
• We have complete Czerny and Hart fingerings, but only partial data for Parncutt. I think we need functioning Parncutt code before we can move forward with last week's proposed study to apply our evaluation method.
• How do we combine mean agreement, mean edit distance, and adoption as a measure of overall "influence" or quality of a model?
• Musing about ambiguity (completeness) in editorial fingering annotations. What questions can we ask our subjects to clarify? What exercises can we pose?

2016-02-10 status

Done

Administrivia

• Pinged Sherice about reimbursement matters.
• Obtained LaTeX template from Rachel and Nick to use for prelim report.
• Filled in some blanks and checked it all into GitHub.

BowTIE

• Created new "pure" Cordova app. 
• Got notation to display.
• Created button graphics and icons.
• Took stab at multitouch implementation.
• Opened issues in GitHub to define next steps for team.

Data Collection

• Outlined follow-up survey for extrinsic "influence" evaluation of Parncutt fragments.

Data Analysis

• Created descriptive statistics report with complete data (191 subjects).

 Doing

1. Calculating mean edit distance as measure of IAA and "influence."
2. Calculating adoption rate as measure of "influence."
3. Discussing each descriptive chart in detail--its purpose, insight (if any).
4. Determine predictive value of one fingering sequence choice on another.
5. Performing Chi Square analysis of exploratory dataset to correlate abbreviated Parncutt fingerings with gender, reach, age, Hanon usage, technical practice, preparation actions, injury, etc.

Struggling

• Must spend money.
• How do we combine mean agreement, mean edit distance, and adoption as a measure of overall "influence" or quality of a model?
• Musing about ambiguity (completeness) in editorial fingering annotations. What questions can we ask our subjects to clarify? What exercises can we pose?

2016-02-03 status

Done

Data Collection

• Moved abcDE editor to new release home on github.io.
• Proved concept (mostly) of integrating new editor into Qualtrics survey.
• Declared abcDE feature complete.
• Wrote end-user and administrator documentation.

Data Analysis

• Created new GitHub repo.
• Played around with Bibtex.
• Drafted some nullable hypotheses.
• Spun wheels.

 Doing

1. Discussing each descriptive chart in detail--its purpose, insight (if any).
2. Determine predictive value of one fingering sequence choice on another.
3. Performing Chi Square analysis of exploratory dataset to correlate abbreviated Parncutt fingerings with gender, reach, age, Hanon usage, technical practice, preparation actions, injury, etc.

Struggling

• New (?) Qualtrics and new abcDE editor are not getting along.