This course presents research conducted to increase our understanding of how data collection decisions affect survey errors. This is not a “how–to-do-it” course on data collection, but instead reviews the literature on survey design decisions and data quality in order to sensitize learners to how alternative survey designs might impact the data obtained from those surveys. The course reviews a range of survey data collection methods that are both interview-based (face-to-face and telephone) and self-administered (paper questionnaires that are mailed and those that are implemented online, i.e. as web surveys). Mixed mode designs are also covered as well as several hybrid modes for collecting sensitive information e.g., self-administering the sensitive questions in what is otherwise a face-to-face interview. The course also covers newer methods such as mobile web and SMS (text message) interviews, and examines alternative data sources such as social media. It concentrates on the impact these techniques have on the quality of survey data, including error from measurement, nonresponse, and coverage, and assesses the tradeoffs between these error sources when researchers choose a mode or survey design.
4.1.3 Text Message Surveys
Loading...
來自 University of Michigan 的課程
Data Collection: Online, Telephone and Face-to-face
134 個評分
從本節課中
Module 4: Emerging modes, new data sources
In this lesson, we focus on some new data collection modes such as mobile web surveys and SMS text interviews, as well as alternative data sources such as sensor data, administrative data, and social media. By the end of this lesson, you will have a sense of the issues to which survey methodologists and survey researchers are devoting much of their attention these days. You will be able to weigh the pros and cons of these new methods and data sources.
與講師見面
Frederick Conrad, Ph.D.Research Professor, Survey Methodology
Institute for Social Research
Welcome back.
We're continuing with our discussion of mobile data collection.
We've talked about mobile web surveys ,and we'll
now shift to a discussion of text message surveys, using texting or
SMS to collect data, not just to invite or contact participants.
So the first thing to really think about is that texting is ubiquitous.
It's the primary method of daily communication in many parts of the world.
Even the most basic mobile phones support texting.
Billions of text messages are sent each month in the US and
billions more globally, so it's at least worth exploring as a survey mode.
As we've discussed for recruiting or contacting participants into
data collection and other modes like web, including mobile web and telephone, but
our focus now is on texting as a data collection mode, as an interview mode.
So what do we mean by texting,
because it's possible to interpret the term differently?
Sending and receiving character-based messages limited to 160
characters in SMS, or short messaging service,
or in Internet-based app the number of characters is unlimited.
Both of those are within scope for the current purposes.
These text messages are created either with an alphanumeric keyboard or
keypad which generally requires pressing the same
key multiple times to produce different letters.
Versus virtual alphabetic keypads, which are more common on smartphones,
so they're on-screen keypads with all alphabetic characters available.
The messages are either displayed individually or
they're threaded in the case of smartphones.
So when they're threaded, the entire kind of history, the conversational history,
is visible.
When just individual messages are available
in older cell phones, it's harder to reconstruct the history.
And the arrival of a message generally alerts the user via a tone or
a vibration or some kind of visual notification that the message has arrived.
It kind of grabs their attention.
This is one of the potential advantages of texting for survey data collection,
is that even when one isn't particularly thinking about being a survey participant,
the arrival of a message may well grab the respondent's attention.
And then finally, texting may require the sender and
recipient to subscribe to a service, such as WhatsApp.
But because texting is native to virtually every mobile phone,
much text communication can occur without any subscription services involved.
This table kind of lays out the attributes of
texting in comparison to the attributes of voice communication.
So if we think about survey data collection,
this would be telephone surveys, telephone interviews, versus text-based interviews.
But really, these features apply to these media in general.
And I won't go through the entire table,
I'll just mention a few of the features in the table.
So the first is synchrony.
So the idea is that voice is fully synchronous.
That is, when a speaker asks, say,
a question or produces some speech, it's really expected that
the listener will produce some speech almost immediately, often within a second.
With texting, it's much less synchronous.
While there can be that kind of rapid back and forth communication in texting,
quite common for the response to be sent when the user has a moment,
has time, when it's convenient.
There just isn't the same expectation for
an immediate response in text that there is in voice.
Another feature that differs between voice and text is the conversational structure.
So spoken communication occurs turn by turn.
That is, that one person speaks, and
then the person who was the listener becomes the speaker.
[COUGH] And it works quite well, but there are occasionally errors in
which the participants get kind of tripped up and they speak at the same time.
But it's pretty rare and
there are remarkably good mechanisms in place to overcome simultaneous speech.
In texting, well, it is also turn by turn.
Each message is a turn.
There really is no analogue to simultaneous speech in texting,
although occasionally text message may arrive out of order, out of sequence,
but that's pretty rare.
So while they're both turn by turn,
the idea of stepping on the other person's toes is pretty unlikely in texting.
Another feature that really distinguishes texting and
voice communication is persistence.
So texting is persistent.
That is, one sends a text message and because it's visual it
doesn't go away unless the user deletes it.
It remains on the user's phone.
It also remains in the cloud, but that's a separate issue.
In voice communication the turns are ephemeral.
They go away once they're uttered.
Unless they're audio recorded, there's no record of what has been said.
So that's a big difference.
You can imagine that that might affect people's willingness to provide sensitive
information.
They might be reluctant to provide sensitive information if there's
a permanent record.
And another distinction between the two ways of communicating has to do with
nonverbal cues, such as what are sometimes called paralinguistic features,
like ums and uhs.
With voice, ums and uhs just come with the territory.
It's rich with respect to these paralinguistic cues.
With texting there's virtually none of this unless somebody
intentionally includes it.
So you can type into a text message, to communicate perhaps,
uncertainty about the rest of the message.
But short of that,
there's virtually no analogue to paralinguistic forms of speech.
And then a final attribute that's quite different between voice and
texting is the way multitasking occurs.
So there are types of multitasking with voice and text, but they're different.
So with voice it tends to be simultaneous, so one might be speaking on the phone
while cooking, kind of doing the two things in parallel.
At some deep internal level shifting between the tasks, but
really engaging in the two tasks at the same time.
But with texting it's pretty unlikely that one is both entering text
into a text message and doing another task.
But once a text message is sent, it's completely normal to
switch to another task between the time the text message is sent and
the time the user reads the reply to that text message.
So all kinds of, the rest of life takes place between text messages.
So it's really a task switching approach versus a simultaneous
multitasking with voice.
So these are a number of differences, and there are others in this table,
that could lead to differences between the data,
and particularly the data quality, collected with text and voice.
So we'll turn shortly to evidence about measurement quality in text versus voice.
But before doing that, let's talk about coverage and
coverage error in text interviews because if the people
who text are different from the people who don't text, then we have the same
kind of problem we've discussed with web access in general.
So the idea is that, of course,
people who are not mobile phone owners are not likely to be texters.
But it could well be that among mobile phone owners,
those who do not text are different on some of the attributes we might want to
measure in a survey than those who do text.
But we just don't know.
So a first investigation of coverage error is reported in
an in-press chapter by Conrad and
colleagues in which the authors looked at results from a telephone survey
conducted by the Pew Organization in which they asked, among other things, whether
they were mobile phone owners, and if they were, they asked them if they texted.
And the survey also produced demographic data for
these respondents, ao we can see if those mobile phone owners who text
differ demographically from those mobile phone owners who do not text.
And in the table here you can see that the demographic variables produced by this
survey are age, sex, education, race, ethnicity, and income.
And if you look under the column A for mobile phone owners and
compare that to the values in column B for mobile phone owners who text,
you can see that there are a few differences, but not many.
There are differences in age, so
fewer mobile phone owners who text are older than 34 years old.
In other words, texters are younger than non-texters and
there are slightly more college graduates, significantly but slightly,
more college graduates among mobile phone owners who text than those who don't text.
But other than that, there are no differences on the other attributes.
And so the question is whether these two differences are likely to affect
the answers that are collected in a survey that's conducted via texting.
Another thing to consider is that the proportion of texters among
mobile phone owners is growing, and that as it grows, any bias,
in terms, for example, of age and education, is likely to be reduced.
Also from Pew, texting rates among US mobile phone owners have been on the rise
and are continuing to rise.
So if we were to project where texting by mobile phone owners is going,
we can see that we're approaching full coverage and that
there may be no difference between mobile phone owners who text and those who don't.
So we'll turn next to a discussion of measurement in text message surveys,
focusing on a laboratory experiment that compares text and voice data collection.
I'll see you in just a minute.
