Respondent Mode Choice in a Smartphone Survey: 2012 [United States]

Principal Investigator(s): View help for Principal Investigator(s) Frederick G. Conrad, University of Michigan, Institute for Social Research, Survey Research Center; Michael F. Schober, The New School for Social Research, Department of Psychology

Version: View help for Version V1

Name	File Type	Size	Last Modified
RMCSSdata.csv	text/csv	1.1 MB	02/16/2017 11:01:AM
Respondent-Mode-Choice-in-a-Smartphone-Survey-Codebook.pdf	application/pdf	281.7 KB	02/16/2017 11:07:AM

Project Citation:

Conrad, Frederick G., and Schober, Michael F. Respondent Mode Choice in a Smartphone Survey: 2012 [United States]. Ann Arbor, MI: Inter-university Consortium for Political and Social Research [distributor], 2017-02-17. https://doi.org/10.3886/E100429V1

Project Description

Summary: Now that people on mobile devices can easily choose their mode of communication (e.g., voice, text, video) survey designers can allow respondents to answer questions in whatever mode they find momentarily convenient given their circumstances or that they chronically prefer. We conducted an experiment to explore how mode choice affects response quality, participation and satisfaction in smartphone interviews. Respondents were interviewed on their iPhone in one of four modes: Human Voice, Human Text, Automated Voice, and Automated Text. Respondents were either assigned the mode of their interview (Assigned Mode) in which case the contact and interviewing mode were the same or they were required to choose the mode of their interview (Mode Choice) after being contacted in one of the four modes. 634 respondents completed the interview and a post-interview online debriefing questionnaire in the Assigned Mode group and 626 respondents completed the interview and online debriefing in the Assigned Mode group. This dataset contains 2691 cases, the 1260 respondents who completed the interview and debriefing as well as 1431 cases that were invited to participate but ended their participation somewhere shy of the last debriefing question: either they did not choose a mode, did not answer the first question, started but did not finish the interview, finished the interview but did not complete the debriefing. All respondents (who completed the interview) answered 32 questions from US social surveys. 13 interviewers from the University of Michigan Survey Research Center administered voice and text interviews (Five administered interviews in both experimental conditions, three conducted only Assigned Mode interviews, and five conducted interviews in just the Mode Choice condition. Automated systems launched parallel text and voice interviews at the same time as the human interviews were launched.

Respondents who chose their interview modes provided more conscientious (fewer rounded and non-differentiated) answers and they reported greater satisfaction with the interview. Although fewer respondents started the interview when given a choice of mode, a higher percentage of Mode Choice respondents who started the interview completed it. For certain mode transitions (e.g., from automated interview modes) there was no reduction in participation. The results demonstrate clear benefits and relatively few drawbacks resulting from mode choice, at least among these modes and with this sample of iPhone users, suggesting that further exploration of mode choice and the logistics of its implementation is warranted.

Funding Sources: National Science Foundation. Directorate for Social, Behavioral and Economic Sciences (SES-1026225); National Science Foundation. Directorate for Social, Behavioral and Economic Sciences (SES-1025645)

Scope of Project

Subject Terms: survey mode choice; mode choice; mode choice and data quality; survey mode preference; mode preference; survey mode switching; mode switching; survey interviewing; paradox of choice; choice overload; choice and self-determination; data quality; automated interviewing; mode choice and participation; response rates; rounding; heaping; straightlining; nondifferentiation; survey satisficing; satisficing; iPhone; breakoff; text message interviewing; survey methodology; IVR; text message; speech IVR; nonresponse; interview satisfaction; mobile devices; smartphone; SMS

Geographic Coverage: United States

Time Period(s): 3/28/2012 – 9/12/2012 (March - September 2012)

Collection Date(s): 3/28/2012 – 5/3/2012 (March-May 2012); 7/25/2012 – 9/12/2012 (July-September 2012)

Universe: iPhone users 21 years of age and older

Data Type(s): survey data

Collection Notes: iPhone users were recruited from Craigslist, Facebook, Google Ads, and Amazon Mechanical Turk. They were asked to complete a screening questionnaire to determine if they were eligible to participate. To be eligible, one needed to be 21 or older, and own an iPhone with a US area code. Eligible participants who provided a telephone number in the screening questionnaire were sent a text message with a link to a web page. The web page captured the user-agent string to determine if the device was an iPhone. Once eligible, phone numbers were assigned a contact mode. Respondents in the Mode Choice group could choose to be interviewed in the contact mode or choose to be interviewed in one of the other three modes. Once the interview had been completed, respondents were sent a link via text message to a post-interview debriefing questionnaire concerning their experience. At the conclusion of the post-interview debriefing, respondents were sent a text message with a $20 iTunes gift code as a token of appreciation for their time.

Methodology

Response Rate: The calculations below use AAPOR RR2.

46.4% (654/1,409) - Mode Choice
50.5% (648/1,282) - Assigned Mode

Sampling: A convenience sample of iPhone users was recruited from Google Ads, Amazon Mechanical Turk, Facebook Ads, and Craigslist (see Antoun et al, 2016 for a discussion of sampling sources).

Participants recruited were not intended to represent the US population, iPhone users, or smartphone users. The sample was designed to test experimental manipulations through random assignments and conditions on a consistent platform.

Data Source: The data for this project were collected in an original survey experiment of iPhone users conducted in 2012. Participants were either assigned an interview mode or required to choose one. The interview modes were: Human Voice, Human Text, Automated Voice, and Automated Text.

In the "Collection Mode" field that follows this one, it was not possible to indicate that some interviews were conducted via Text, and therefore not possible to indicate that both human-administered and automated text interviews were conducted. Similarly CATI refers to Human Voice interviews and TACASI refers to automated voice (although the Automated Voice system required respondents to speak their answers and used speech recognition to classify the spoken answers while TACASI typically involves touch tone input from respondents). Web-based data collection was used for the screener (from which most demographic information is derived) and the post-interview debriefing (from which the satisfaction data are derived).

Collection Mode(s): computer-assisted telephone interview (CATI); mixed mode; other; telephone audio computer-assisted self interview (TACASI); telephone interview; web-based survey

Scales:

How often do you now smoke cigarettes?

Every day
Some days
Not at all

Have you smoked at least 100 cigarettes in your entire life?

Have you ever, even once, used marijuana or hashish?

During the past 30 days, on how many days did you drink one or more drinks of an alcoholic beverage?

[Open-ended numeric]

During the past 30 days, on how many days did you have 5 or more drinks on the same occasion?

[Open-ended numeric]

In a typical week, about how often do you exercise?

Less than 1 time per week
1 or 2 times per week
3 times per week,
4 or more times per week

How many sex partners have you had in the last 12 months?

[Open-ended numeric]

During the past 12 months, have your sex partners been ...

Exclusively male
Exclusively female
Both male and female
Had no partners

About how often did you have sex during the last 12 months?

Not at all
Once or Twice
About once a month
Two or three times a month
About once a week
Two or three times a week

Now thinking about the time since your eighteenth birthday (including the recent past that you've already told us about), how many female partners have you had sex with?

[Open-ended numeric]

Thinking about the time since your eighteenth birthday (including the recent past that you've already told us about), how many male partners have you had sex with?

[Open-ended numeric]

Now I'm going to read a list of terms that people sometimes use to describe themselves: "A" heterosexual or straight; "B" homosexual, gay or lesbian; and "C" bisexual. Which option best describes how you think of yourself?

Heterosexual or straight
Homosexual, gay or lesbian
Bisexual

Now I'd like to ask you some questions specific to your diet. [pause] For the next group of questions, I'll ask you how much you favor or oppose engaging in certain behaviors. After I say the behavior, please say one of the following: ‘strongly favor’, ‘somewhat favor’, ‘neither favor nor oppose’, ‘somewhat oppose’ or ‘strongly oppose’. The first behavior...avoiding fast food.

Strongly favor
Somewhat favor
Neither favor nor oppose
Somewhat oppose
Strongly oppose

The next behavior: Maintaining a healthy diet.

Strongly favor
Somewhat favor
Neither favor nor oppose
Somewhat oppose
Strongly oppose

The next behavior: Monitoring cholesterol levels closely.

Strongly favor
Somewhat favor
Neither favor nor oppose
Somewhat oppose
Strongly oppose

The next behavior: Emphasizing the taste of food rather than its nutritional value.

Strongly favor
Somewhat favor
Neither favor nor oppose
Somewhat oppose
Strongly oppose

The next behavior: Paying close attention to the nutritional information on food packaging.

Strongly favor
Somewhat favor
Neither favor nor oppose
Somewhat oppose
Strongly oppose

The next behavior: Limiting the amount of red meat in your diet.

Strongly favor
Somewhat favor
Neither favor nor oppose
Somewhat oppose
Strongly oppose

And finally, the last behavior: Balancing one's diet across the key food groups.

Strongly favor
Somewhat favor
Neither favor nor oppose
Somewhat oppose
Strongly oppose

During the last month, how many times did you eat spicy food?

[Open-ended numeric]

How often do you attend religious services?

At least once a week
Almost every week
About once a month
Seldom
Never

How often do you read the newspaper?

Every day
A few times a week
Once a week
Less than once a week
Never

On the average day, about how many hours do you personally watch television?

[Open-ended numeric]

During the last month, how many movies did you watch in any medium?

[Open-ended numeric]

During the past 12 months, how many movies have you seen in movie theaters?

[Open-ended numeric]

During the last month, how many times did you shop in a grocery store?

[Open-ended numeric]

During the last month, how many times did you eat in restaurants?

[Open-ended numeric]

How many songs do you currently have on your iPhone?

[Open-ended numeric]

How many apps do you currently have on your iPhone?

[Open-ended numeric]

How many text messages have you sent and received on your iPhone in your current billing cycle?

[Open-ended numeric]

Finally, we have two last questions on earlier topics in the interview. Let's return to smoking. This time be sure to include any puffs on any cigarettes, whether or not you inhaled AND whether or not you finished them. Keeping this in mind, have you smoked at least 100 cigarettes in your entire life?

And for the last question, let's return to newspapers. This time be sure to consider any newspaper content no matter whether it appears in print, online or on a mobile device. And be sure to count only newspapers from recognized journalistic outlets. Keeping this in mind, how often do you read the newspaper?

Every day
A few times a week
Once a week
Less than once a week
Never

Weights: Our primary goal was to test experimental effects, not to generalize to a population beyond our convenience sample. Thus no weights were calculated. In addition, there was no evidence that nonresponse affected the composition of the respondent sample so no weights were calculated to adjust for nonresponse. This is discussed in Conrad et al., 2017.

Unit(s) of Observation: iPhone users 21 years of age and older

Geographic Unit: United States

Related Publications

Download this project

Published Versions

V1 [2017-02-17]

Export Metadata

Dublin Core

DDI 2.5

Report a Problem

Found a serious problem with the data, such as disclosure risk or copyrighted content? Let us know.

This material is distributed exactly as it arrived from the data depositor. ICPSR has not checked or processed this material. Users should consult the investigator(s) if further information is desired.