Survey split

We want to split the survey but give people the option of continuing directly to Part II at the completion of Part I. I don't see a way to do this with separate surveys in Qualtrics, but we can just add the Part II questions in the Part I survey after a question about whether the person would like to continue:

Congratulations, you have completed the fingering exercise survey. Thank you for sharing your expertise.

The next step would be for you to answer some questions about your background and attitudes about fingering. Completing this "player profile" will earn you a chance to win another $100 Amazon gift card and will make the fingering data you have just provided even more valuable to our study. Would you like to do this now?

• Yes, I have time.

• No, maybe later.

If the subject says yes or no, we add her name, email, and response ID to the "Survey I Complete" panel (via a panel trigger) and send her a thank you email that includes the consent agreement.

If the subject says yes, we press on. When she complete the survey, we add the subject's name, email, and response ID to the "Survey II Complete" panel. We also send a thank you and copy of the consent agreement via an email trigger.

(Another important change: Add a panel trigger to add the user to the "Lottery I" panel if she says yes to participating in the Part I lottery. Do the same for the Part II lottery. This will make it easier to process the survey awards. Unfortunately, or perhaps fortunately, this mechanism cannot be used to capture contact information from people who do not complete the survey within two weeks and effectively withdraw from the study. This is because the panel updates are only triggered on survey completion. Moreover, I can't delete all information about a person while still maintaining her contact information. So I am changing the consent form to promise compensation only for in-person sessions that are terminated early. Whew.)

If the subject says no, we add her information to the "Survey II Separate" panel, so we can follow up with her later. The profile part will be advertised to members of this panel as a separate, abbreviated survey.

Lottery II contact information for this follow-up survey will be written to the "Lottery II" panel and derived from the "recipient information," which should be available because we will be using unique survey links (from the "Separate" panel) for all surveys but the first one.

Should we include a second question about participating in the lottery for the integrated Part II? What about the standalone Part II? I say yes to both.

The progress bar is really discouraging, since it includes a bunch of future questions that a lot of people won't be answering and also because it does not have any notion of weighting the effort required for a given question. The progress bar creates a distinct impression that very little progress is being made. So I am not going to show it in Survey I and am instead including a heading (e.g. "Exercise 1 of 7") at the top of each fingering exercise. This should give the subject a better sense of progress. I just wish I could start the progress bar for the embedded Part II, but this does not seem possible. So they will have to do without.

Per BDE's request, I am also changing required to initial when talking about the survey activities in the consent agreement.

These changes will be made to version 5 of the survey (now surveys).

I must add our single v4 respondent to the "Complete" and "Lottery" panels manually.

Also, the follow up survey must write the original "${e://Field/ResponseID}​" value (extracted from the panel data field "TriggerResponseID") to an embedded data field called "TriggerResponseID." Then we should be all set. This should add only minor complexity to the previously planned data plumbing. We just have another opportunity to use "TriggerResponseID," and whatever process loads the data to MySQL will have to be a teeny bit smarter. While I still need validate all this end to end, I am quite certain it will work.

2015-08-07 status

Done

Administrivia

• Started reworking application materials for Chancellor's Graduate Research Award (née Fellowship). Statement of intent no longer required. DGS must submit by 23 September (4 p.m. CDT).

Data Collection

• Added email trigger in Qualtrics to send thank you message and copy of consent document on survey completion, as promised in survey text.
• Announced "final" survey to three people I know using generic recruitment email.
• Received one response from someone I don't know.
• Acting on advice from BDE, who was rightfully worried about length of monolithic server, split survey into two parts (fingering and player profile), to be sent out separately with separate $100 lottery incentives. AFL thought this was a good idea.
• Decided fingering part should go out first.
• After a budgetary epiphany, decided that all online activities will be incentivized with separate $100 lotteries.
• Updated Qualtrics and affected IRB documents to reflect these changes.
• Redrafted recruitment emails to reflect these changes.
• Refinalized surveys, leaving question content unchanged.

Doing

1. Leveraging abc2svg for MDC4.
2. Standardizing file format for fingering.

Struggling

• Contemplating a major rewrite for MDC4, where I get out of the business of parsing abc myself. This should simplify the code greatly, but the way is not clear.
• Prelude and Fugue 18 appears to be corrupted by my hacking efforts to migrate to abc.

The curse of the monolithic consent agreement

On Thu, Aug 6, 2015 at 12:04 PM, David Randolph wrote:

Dear Barbara:

I think 45 minutes is a pretty fair estimate [for the full-blown v4 survey]. Yes, they can do it in more than one session if they use the same browser. (I should highlight this to the recruitment email.) Ivana did it with a big extra fingering exercise in about an hour total, across two sessions. She would probably do what remains in the current survey in 40 minutes.

If this is a major concern, we could split this into two parts--the player profile (10-15 minutes) and the fingering exercises (25-30 minutes) and give away a gift card for each part. But this adds complexity to keeping the pieces together, and I really need both parts.

Will we lose more people to attrition between part 1 and part 2 than we scare off with the 45 minute estimate? That is the big question. There is some value to fingering data without the player profile, I suppose.

And would this imply another run through the IRB gauntlet?

The more I think about it now, the more I think it might be worth it.

What do you think?

Thanks,

Dave

On 8/6/15 1:25 PM, David Randolph wrote:

Dear Barbara:

Another problem has occurred to me. Doling out $50 for each set of two scores is going to deplete our funds pretty rapidly. Ideally (?), we would have each of say, ten pieces, fingered by five different pianists (preferably with a range of hand sizes). This would cost 5 x 5 x $50 = $1250. After the $200 we now plan to spend on the initial surveys, we are $50 over budget, and we have nothing to pay anyone for on-campus visits and whiz-bang computer-vision experiments, which I am still pretty keen to do.

So the other change I would make is for all online activities (not just the required ones) to use lotteries for incentives. If we simply treat each of the five (most well-known to our survey respondents) pieces as separate $100 lotteries, we are covered for $500 + $200 = $700. And we still have $700 to play with. This sounds like a better approach than what I have in the consent form right now.

I am going to lock the current survey. I only have one response so far (from someone I don't know actually--I think my sister-in-law forwarded the link). We should meet soon to strategize.

Thanks,

Dave

And finally. . . .

Dear Barbara:

Yes, please hold off until we get our story straight.

I have closed the survey I sent out. I can reopen it if we decide to press on. But I think changes are in order.

I am creating alternate surveys in Qualtrics now. The protocol changes I have in mind should be pretty minor, but should improve our response rate and bang for buck. I think it is worth the extra hassle.

The idea now is to start with the (25-30 minute) fingering exercises and then do the (10-15 minute) profile questions. It might help response rates if we did it in the other order (and ease them into our clutches). But you had recommended doing the data collection first to eliminate any prejudice the questions might exert on the fingerings. So there is a trade-off to consider. Separating the questioning from the annotation in time probably makes me lean more toward starting with the shorter questioning part.

I am still worried about attrition between the two surveys, to be sure. But this seems like our best guess at an optimal approach.

Thanks,

Dave