Precision and Disclosure in Text and Voice Interviews on Smartphones: 2012 [United States]

Principal Investigator(s): View help for Principal Investigator(s) Frederick G. Conrad, University of Michigan. Institute for Social Research. Survey Research Center; Michael F. Schober, The New School for Social Research. Department of Psychology

Version: View help for Version V2

Name	File Type	Size	Last Modified
Precision and Disclosure in Text and Voice Interviews on Smartphones Codebook	application/pdf	862.4 KB	05/03/2015 11:53:AM
Precision_and_Disclosure_in_Text_and_Voice_Interviews_on_Smartphones_Dataset	text/csv	610.7 KB	05/03/2015 11:59:AM
Precision_and_Disclosure_in_Text_and_Voice_Interviews_on_Smartphones_Dataset	text/plain	610.7 KB	05/03/2015 11:59:AM

Project Citation:

Conrad, Frederick G., and Schober, Michael F. Precision and Disclosure in Text and Voice Interviews on Smartphones: 2012 [United States]. Ann Arbor, MI: Inter-university Consortium for Political and Social Research [distributor], 2017-02-17. https://doi.org/10.3886/E100113V2

Project Description

Summary: As people increasingly communicate via asynchronous non-spoken modes on mobile devices, particularly text messaging (e.g., SMS), longstanding assumptions and practices of social measurement via telephone survey interviewing are being challenged. This dataset contains 1,282 cases, 634 cases that completed an interview and 648 cases that were invited to participate, but did not start or complete an interview on their iPhone. Participants were randomly assigned to answer 32 questions from US social surveys via text messaging or speech, administered either by a human interviewer or by an automated interviewing system. 10 interviewers from the University of Michigan Survey Research Center administered voice and text interviews; automated systems launched parallel text and voice interviews at the same time as the human interviews were launched. The key question was how the interview mode affected the quality of the response data, in particular the precision of numerical answers (how many were not rounded), variation in answers to multiple questions with the same response scale (differentiation), and disclosure of socially undesirable information. Texting led to higher quality data—fewer rounded numerical answers, more differentiated answers to a battery of questions, and more disclosure of sensitive information—than voice interviews, both with human and automated interviewers. Text respondents also reported a strong preference for future interviews by text. The findings suggest that people interviewed on mobile devices at a time and place that is convenient for them, even when they are multitasking, can give more trustworthy and accurate answers than those in more traditional spoken interviews. The findings also suggest that answers from text interviews, when aggregated across a sample, can tell a different story about a population than answers from voice interviews, potentially altering the policy implications from a survey.

Funding Sources: National Science Foundation. Directorate for Social, Behavioral and Economic Sciences (SES-1026225); National Science Foundation. Directorate for Social, Behavioral and Economic Sciences (SES-1025645)

Scope of Project

Subject Terms: survey interviewing; rounding; automated interviewing; response rates; straightlining; completion; nondifferentiation; statisficing; iPhone; breakoff; sensitive questions; data quality; text message interviewing; survey methodology; IVR; precision; text message; speech IVR; heaping; nonresponse; interview satisfaction; multitasking; mobile devices; disclosure; smartphone; SMS; mode comparison

Geographic Coverage: United States

Time Period(s): 3/28/2012 – 5/3/2012 (March - May 2012)

Collection Date(s): 3/28/2012 – 5/3/2012 (March - May 2012)

Universe: iPhone users 21 years of age and older.

Data Type(s): survey data

Collection Notes: iPhone users were recruited from Craigslist, Facebook, Google Ads, and Amazon Mechanical Turk. They were asked to complete a screening questionnaire to determine if they were eligible to participate. To be eligible, one needed to be 21 or older, and own an iPhone with a US area code. Eligible participants who provided a telephone number in the screener were sent a text message with a link to a web page. The web page captured the user-agent string to determine if the device was an iPhone. Once eligible, phone numbers were assigned an interview mode. Once the interview had been completed, respondents were sent a link via text message to a post-interview debriefing questionnaire concerning their experience. At the conclusion of the post-interview debriefing, respondents were sent a text message with a $20 iTunes gift code as a token of appreciation for their time.

Methodology

Response Rate: The response rate below was calculated using AAPOR RR2.

46.4% (654/1,409) - Mode Choice

Sampling: A convenience sample of iPhone users was recruited from Google Ads, Amazon Mechanical Turk, Facebook Ads, and Craigslist (see Antoun et al, 2016 for a discussion of sampling sources).

Participants recruited were not intended to represent the US population, iPhone users, or smartphone users. The sample was designed to test experimental manipulations through random assignments and conditions on a consistent platform.

Data Source: The data for this project were collected in an original survey experiment of iPhone users conducted in 2012. Participants were either assigned an interview mode or required to choose one. The interview modes were: Human Voice, Human Text, Automated Voice, and Automated Text.

In the "Collection Mode" field that follows this one, it was not possible to indicate that some interviews were conducted via Text, and therefore not possible to indicate that both human-administered and automated text interviews were conducted. Similarly CATI refers to Human Voice interviews and TACASI refers to automated voice (although the Automated Voice system required respondents to speak their answers and used speech recognition to classify the spoken answers while TACASI typically involves touch tone input from respondents). Web-based data collection was used for the screener (from which most demographic information is derived) and the post-interview debriefing (from which the satisfaction data are derived).

Collection Mode(s): computer-assisted telephone interview (CATI); mixed mode; other; telephone audio computer-assisted self interview (TACASI); telephone interview; web-based survey

Scales: The questionnaire contained 32 questions after answering “yes” to a “safe-to-talk question”. Twenty five questions came from ongoing social surveys (Behavioral Risk Factor Surveillance System [BRFSS], the National Survey on Drug Use and Health [NSDUH], the National Health Interview Survey [NHIS], the General Social Survey [GSS], and published methodological studies [Conrad, Brown, and Cashman 1998; Tourangeau, Couper, and Conrad 2007] administered in the United States. Five questions were developed for this study. Two questions were asked a second time with a definition presented.
The 32 questions contained a variety of response options (detailed below).

How often do you now smoke cigarettes?

Every day
Some days
Not at all

Have you smoked at least 100 cigarettes in your entire life?

Have you ever, even once, used marijuana or hashish?

During the past 30 days, on how many days did you drink one or more drinks of an alcoholic beverage?

[Open-ended numeric]

During the past 30 days, on how many days did you have 5 or more drinks on the same occasion?

[Open-ended numeric]

In a typical week, about how often do you exercise?

Less than 1 time per week
1 or 2 times per week
3 times per week,
4 or more times per week

How many sex partners have you had in the last 12 months?

[Open-ended numeric]

During the past 12 months, have your sex partners been ...

Exclusively male
Exclusively female
Both male and female
Had no partners

About how often did you have sex during the last 12 months?

Not at all
Once or Twice
About once a month
Two or three times a month
About once a week
Two or three times a week

Now thinking about the time since your eighteenth birthday (including the recent past that you've already told us about), how many female partners have you had sex with?

[Open-ended numeric]

Thinking about the time since your eighteenth birthday (including the recent past that you've already told us about), how many male partners have you had sex with?

[Open-ended numeric]

Now I'm going to read a list of terms that people sometimes use to describe themselves: "A" heterosexual or straight; "B" homosexual, gay or lesbian; and "C" bisexual. Which option best describes how you think of yourself?

Heterosexual or straight
Homosexual, gay or lesbian
Bisexual

Now I'd like to ask you some questions specific to your diet. [pause] For the next group of questions, I'll ask you how much you favor or oppose engaging in certain behaviors. After I say the behavior, please say one of the following: ‘strongly favor’, ‘somewhat favor’, ‘neither favor nor oppose’, ‘somewhat oppose’ or ‘strongly oppose’. The first behavior...avoiding fast food.

Strongly favor
Somewhat favor
Neither favor nor oppose
Somewhat oppose
Strongly oppose

The next behavior: Maintaining a healthy diet.

Strongly favor
Somewhat favor
Neither favor nor oppose
Somewhat oppose
Strongly oppose

The next behavior: Monitoring cholesterol levels closely.

Strongly favor
Somewhat favor
Neither favor nor oppose
Somewhat oppose
Strongly oppose

The next behavior: Emphasizing the taste of food rather than its nutritional value.

Strongly favor
Somewhat favor
Neither favor nor oppose
Somewhat oppose
Strongly oppose

The next behavior: Paying close attention to the nutritional information on food packaging.

Strongly favor
Somewhat favor
Neither favor nor oppose
Somewhat oppose
Strongly oppose

The next behavior: Limiting the amount of red meat in your diet.

Strongly favor
Somewhat favor
Neither favor nor oppose
Somewhat oppose
Strongly oppose

And finally, the last behavior: Balancing one's diet across the key food groups.

Strongly favor
Somewhat favor
Neither favor nor oppose
Somewhat oppose
Strongly oppose

During the last month, how many times did you eat spicy food?

[Open-ended numeric]

How often do you attend religious services?

At least once a week
Almost every week
About once a month
Seldom
Never

How often do you read the newspaper?

Every day
A few times a week
Once a week
Less than once a week
Never

On the average day, about how many hours do you personally watch television?

[Open-ended numeric]

During the last month, how many movies did you watch in any medium?

[Open-ended numeric]

During the past 12 months, how many movies have you seen in movie theaters?

[Open-ended numeric]

During the last month, how many times did you shop in a grocery store?

[Open-ended numeric]

During the last month, how many times did you eat in restaurants?

[Open-ended numeric]

How many songs do you currently have on your iPhone?

[Open-ended numeric]

How many apps do you currently have on your iPhone?

[Open-ended numeric]

How many text messages have you sent and received on your iPhone in your current billing cycle?

[Open-ended numeric]

Finally, we have two last questions on earlier topics in the interview. Let's return to smoking. This time be sure to include any puffs on any cigarettes, whether or not you inhaled AND whether or not you finished them. Keeping this in mind, have you smoked at least 100 cigarettes in your entire life?

And for the last question, let's return to newspapers. This time be sure to consider any newspaper content no matter whether it appears in print, online or on a mobile device. And be sure to count only newspapers from recognized journalistic outlets. Keeping this in mind, how often do you read the newspaper?

Every day
A few times a week
Once a week
Less than once a week
Never

Weights: Our primary goal was to test experimental effects, not to generalize to a population beyond our convenience sample. Thus no weights were calculated. In addition, there was no evidence that nonresponse affected the composition of the respondent sample so no weights were calculated to adjust for nonresponse. This is discussed in Conrad et al., 2017.

Unit(s) of Observation: iPhone users 21 years of age and older

Geographic Unit: United States

Related Publications

Download this project

Published Versions

V2 [2017-02-17]

V1 [2015-05-03]

Export Metadata

Dublin Core

DDI 2.5

Report a Problem

Found a serious problem with the data, such as disclosure risk or copyrighted content? Let us know.

This material is distributed exactly as it arrived from the data depositor. ICPSR has not checked or processed this material. Users should consult the investigator(s) if further information is desired.