Guidelines For Interpreting Abortion Opinion Polls
Raymond J. Adamek, Ph.D.

A.I.R.V.S.C.
Association for Interdisciplinary Research in Values and Social Change
Vol. 7 No. 2, Sept/Oct 1994
Reproduced with Permission

The author, Raymond J. Adamek Ph.D. is a professor of sociology at Kent State University, Kent. Ohio. He has extensively studied and written on abortion and public opinion polls.

When people think of opinion polls, they often remark, “You can prove anything with a poll.” While there may be an element of truth in the statement, there are some less cynical guidelines to evaluate poll results. These guidelines will be illustrated with abortion polls, although they generally can be applied to polls on any issue.

Guidelines for Interpreting Abortion Polls

1. Polls Can Accurately Reflect Opinions.
2. Sampling Error Is A Fact Of Life.
3. One Poll Doesn’t Prove Anything. Look For Consistent Results.
4. Words Are Important.
5. Context Is Important.
6. Knowing Who Sponsored, Conducted, and Reported A Poll Is Important.
7. Look At The Data Before You Read The Narrative.
8. Do Not Assume Respondents Are Knowledgeable.
9. Several Specific Items Are Better Than One General Item.
10. Some Items Generally Elicit Seemingly Pro-Choice Answers.
11. What Is A Question’s Premise? Does It Reflect Reality?
12. Can Responses Be Interpreted Unambiguously?
13. Life is Paradoxical. Respondents Often Are Too.
14. Does The Poll Exhibit Balance?
15. What Questions Were Not Asked?

Polls Can Accurately Reflect Opinions

Modern polling methods do permit the accurate measurement of the opinions of all adult Americans. They usually involve probability samples of 700 to 1500+ respondents. Probability samples give each person in the population an equal or known chance of being selected, and thereby generally yield a mirror image of the population. Poll results based on non-probability samples (e.g. subscribers of a particular magazine) cannot be generalized to the whole population, however.

Sampling Error Is a Fact of Life

While probability sample results are generally accurate, they seldom match population values exactly. Sampling error (fluctuation) is the difference between a sample value (e.g. percent who favor a given item) and the true population value. In most national probability studies, sampling error varies from ±2% to ±5% and pertains to the total sample.

Where subgroup responses are considered (e.g. males vs. females), sampling error will be slightly higher.

Consideration of sampling error may be particularly important when attitude trends are studied. Thus, assume two polls with a sampling error of ±3% ask the same question. The fact that 50% of the sample agree with a given position in 1993 and 56% agree in 1994 may reflect only fluctuation about the true rate of agreement of 53% of the population in both years, rather than an increase of 6% agreement.

One Poll Doesn’t Prove Anything - Look for Consistent Results

Because of sampling error, the possibility of selecting a non-representative sample, and the effects of question wording, one poll doesn’t prove anything. Consistent results over several polls increase the credibility of the findings. (A poor question may consistently yield misleading results, however.)

The consistency guideline may be illustrated by the polls in Table 1. A majority of Americans rejected the notion of paying for abortions under a national health care plan in five of these six polls. Only the second poll found more approval than disapproval. Given the short time frame, this inconsistency is probably due to question wording rather than opinion change. The most consistent result indicates the public is against health care abortion funding.

Words Are Important

The way a question is asked influences the answers obtained. This is clearly demonstrated by the items in Table 2, which were asked of the same respondents at different points in the same poll. Referring to an amendment as “protecting” rather than “prohibiting” something, and mentioning the unborn child causes 21% to 28% of the respondents to change their positions.

Since wording can influence response patterns, trends in attitudes are best measured by considering polls, which ask exactly the same questions over time.

Context is Important

The context in which questions are asked also influences responses. We may expect a different response pattern to abortion questions preceded by women’s rights questions than to the same items preceded by children’s rights questions. Even changing the order in which abortion times are asked has an effect on responses. Consider the following two items:

Do you think it should be possible for a pregnant woman to obtain a legal abortion if she is married and does not want anymore children?

Do you think it should be possible for pregnant woman to obtain a legal abortion if there is strong chance of serious defect in the baby?

Researchers Schuman, Presser and Ludwig (1981) found agreement with the first, social item was 17% higher when it preceded the second, physical item. While context cannot always be determined from media reports, it should be considered when more complete reports are available.

Knowing Who Sponsored, Conducted, and Reported a Poll is Important

Knowing who sponsored, conducted, and reported a poll is important since their frames of reference, assumptions, and terminology will be reflected in the questions asked, the responses obtained, and the way findings are reported. While not all polls sponsored by advocacy groups are necessary biased, objectivity may be difficult to attain. Consider the following item asked in a poll conducted for the National Abortion Rights Action League and reported by Hickman-Maslin Research and American Viewpoint (1988).

…which of the following three positions is closest to the way you feel: abortion should be available to any woman who wants one; abortion should only be available under certain extreme circumstances; or abortion should not be allowed.

The study reported that 39% of the respondents chose “available to any woman,” 49% chose “available under certain extreme circumstances,” and 10% chose “abortions should not be allowed.” However, the report (pp. 5-8): 1) virtually combines the first two groups to form a “pro-choice” category comprised of 88% of the respondents; 2) adds the phrase “under any circumstances” when describing the position of the 39% who said abortion should be available to any woman; 3) omits the word “extreme” from the “extreme circumstances” phrase in discussing the second group; 4) includes a question to probe the “extreme circumstances” and “not be allowed” responses to see if more pro-choice attitudes could be detected among those respondents; 5) doesn’t include a question to probe the “available to any woman” response to see if more pro-life attitudes could be detected there.

Look at the Data Before You Read the Narrative

If tabular data are presented, study them to see what they indicate before reading the accompanying narrative. Sometimes one finds that emphasis is given to minority patterns ("almost half of all Americans believe abortion should be available if the woman wants it for any reason") rather than to majority patterns. Occasionally one finds the data actually contradict the author’s interpretation.

Do Not Assume Respondents are Knowledgeable

People may respond to questions on topics they know little about for a variety of reasons: they mistakenly believe they are knowledgeable, they wish to cooperate and answer all questions, they do not wish to admit they lack knowledge. Particularly for complex and technical matters such as court decisions, it may be advisable to take responses with a grain of salt.

Consider the two questions in Table 3, which were asked within three months of one another. What confidence can we place in the finding that 56% favor the Court’s ruling when 1) the pollster describes the ruling incompletely, and 2) only 18% of the public realizes that “abortions are legal for the duration of the pregnancy regardless of a woman’s reason for wanting one"? Earlier studies also indicate that relatively few Americans know what the Supreme Court’s abortion policy entails (Blake, 1977; Research and Forecasts, 1986; Hickman-Maslin Research and American Viewpoint, 1988).

Several Specific Items are Better Than One General Item

Particularly on complex issues, it is better to ask a series of detailed questions to measure attitudes than to ask one general question. This guideline is dramatically illustrated by five national polls conducted shortly after the Supreme Court’s Webster decision in July, 1989. The polls were taken by Newsweek, Time, the Los Angeles Times, Gallup and Media General. Each first asked a general question similar to the one asked by Newsweek, (1989): “Do you agree with the recent Supreme Court decision permitting States to pass laws restricting abortion?” In each poll, only a minority (31% - 43%) said they agreed with the decision.

However, each pollster then asked from three to six questions about specific “restrictions” requiring 1) the woman to receive counseling regarding abortion alternative, fetal development, and abortion complications; 2) parental consent for minors’ abortion; 3) viability testing for fetuses beyond a certain age; 4) exemption of public hospitals, personnel, or funds from providing abortion. Nineteen specific items were asked across the five surveys. On 17 items, respondents reversed the position they had taken on the general question. A majority endorsed specific restrictions in 15 instances, and pluralities endorsed them in two other instances (see Adamek, 1989b, for details). Hence, had only the general question been asked, we would have an erroneous picture of people’s judgment of Webster.

Part of the public’s reaction to the general question may be due to the fact that Americans seem reluctant to restrict, ban, prohibit, or forbid almost anything, although as we noted earlier, they are more willing not to allow something. We might also note that referring to the regulations that Webster allows as “restrictions” reflects a pro-choice perspective. Pro-lifers would consider them “protections” for the pregnant woman, the parents of minors, the viable fetus, etc. More neutral language would simply refer to such provisions as regulations.

Cook, Jelen and Wilcox (1993) offer insightful analyses of other situations where several specific abortion items are to be preferred to one general item.

Some Items Generally Elicit Seemingly Pro-Choice Answers

Various polls, including the one presented in Table 4, indicate that while “woman’s decision and “woman and her doctor” items typically elicit large seemingly pro-choice majorities, they do not identify pro-choice persons. While 71% of the respondents in the Table 4 poll said they “favor leaving the decision to have an abortion to the woman and her physician”, 55% indicated abortions should not be legal, or should be legal only for relatively rare, physical indications.

Two Virginia Slims polls conducted in 1980 (N = 3,007) and (N = 3,000) found that while 76% and 72% of women agreed that “where abortions are legal, the decision about an abortion should be left up to the woman and her doctor,” 41% in each year agreed that “where abortions are legal, the father should have the right of veto” (The Roper Organization, 1980, 1985).

Similarly, 56% of voters in a 1987 poll said they generally favored “keeping it legal for women to be able to choose to have abortions when they decide to have one.” But when asked, “ …which of the following three positions is closest to the way you feel: abortions should be available to any woman who wants one, extreme circumstances only or should not be allowed,” 59% said abortions should be available only in extreme circumstances (49%) or not at all ( 10%). (Hickman-Maslin, 1988).

To characterize the public as pro-choice on the basis of responses to general “woman’s decision” or “woman and her doctor” items, therefore, simply demonstrates inadequate analysis (Adamek, 1989a).

What is a Question’s Premise? Does it Reflect Reality?

On fourteen occasions between 1973 and 1993, Harris polls asked some version of the following question:

In 1973, the U. S. (United States) Supreme Court decided that State laws which made it illegal for a woman to have an abortion up to three months pregnancy were unconstitutional, and that the decision on whether a woman should have an abortion up to three months of pregnancy should be left to the woman and her doctor to decide. In general, do you favor or oppose this part of the U. S. Supreme Court decision, making abortion up to three months of pregnancy legal? (Harris, 1993).

Because it is incomplete, this description of Roe v. Wade does not reflect reality, and is biased in a pro-Court direction. It is biased, first because it mentions that the Court found State’s restriction on abortion unconstitutional. This invokes the authority of the Court on one side of the issue, and respondents are likely to defer to this authority. A more neutral way to phrase this item is simply to state what the Court’s position was, without indicating it found other positions unconstitutional.

The question is biased, secondly, because it includes the phrase “should be left to the woman and her doctor to decide.” As we saw above, this wording typically inflates responses in a pro-choice direction. Mentioning the doctor also suggests physical indications for abortion, which most Americans endorse (Cook, Jelen and Wilcox, 1992). Further, although Harris could not have known this when the question was initially framed, the phrase, “left to the woman and her doctor to decide,” does not reflect reality. Six large studies with N’s varying from 195 to 1,505 indicate that less than 25% of American women consult a doctor when making their abortion decision (Faria, Barrett and Goodman, 1985; Friedlander, Kaul and Stimel, 1984; Henshaw and Kost, 1992; Jain, 1977; Rosen, 1977; Shaw, Funderburk and Franklin, 1979).

The most serious biasing element of the question, however, is the phrase, “up to three months of pregnancy, “ which is repeated three times. This phrase gives the uninformed respondent the impression that the Court did not legalize abortion beyond the first trimester, and instructs the informed respondent to confine his/her judgment to the first trimester. The “up to three months of pregnancy” language not only biases the responses, but invalidates the results as a measure of approval of Roe v. Wade. If we want to know the public’s opinion about Roe v. Wade, we have to ask about the entire ruling, which sets policy governing the second and third trimesters as well. Employing the first-trimester language is akin to asking persons who have never seen the Empire State Building to give an opinion of it while showing them a photo of only the first 34 stories.

Harris is not alone in employing this incomplete description of Roe v. Wade. On 42 occasions between 1986 and 1993, at least ten other pollsters used similar language in measuring opinion about the Court’s decision (Adamek, 1994).

Can Responses be Interpreted Unambiguously?

Sometimes questions are posed in such a way that we cannot be sure what the responses indicate. For example, respondents may be asked if they favor or oppose passing a law which regulates abortion in a number of specific ways. If responses are not clarified by subsequent questions, we cannot tell if those who say they oppose such a law do so because they believe the law is too restrictive or too permissive.

Another type of question, which yields ambiguous results, is the “double-barreled” question, illustrated by the following item:

When it comes to the policies and goals of the Republican party on … social issues, such as abortion and gay rights … do you strongly agree, mainly agree, have mixed feelings, mainly disagree, or do you strongly disagree with the party’s policies and goals? (Hart and Breglio Research Companies, 1992)

This item mentions two issues, and should be separated into two questions. Respondents agreeing with Republican policy on one issue but not the other will not know how to respond. Two persons who have the same mixed opinion may choose different responses, and one cannot tell their true opinion.

Life is Paradoxical, Respondents Often Are Too

Life is beset by paradoxes and contradictions. On complex issues like abortion, one can expect respondents to be ambivalent and inconsistent. This is illustrated by the data in Table 2 and in Table 5. Poll consumers should not be surprised that such apparent contradictions occur. More pointed questions, requiring the respondent to choose between the rights of the unborn and the rights of the mother, would help to resolve the paradox.

Does the Poll Exhibit Balance?

A single question, groups of questions, and entire polls can be evaluated to determine whether they exhibit balance. That is, are they objective, do they use parallel language to describe opposing viewpoints, and do they give “equal time” to those viewpoints? The following item is unbalanced in that it uses non-parallel terms to describe two positions, thereby reflecting a pro-choice perspective.

The two main groups in the abortion debate are the so-called pro-life group, which opposes abortion, and the so-called pro-choice group, which supports women’s right to have an abortion. Which of these two groups do you tend to support more? (Harris, 1992)

More neutral phrasing, which uses parallel language to describe the two positions, is illustrated by the following alternatives.

The two main groups in the abortion debate are the so-called pro-life group, which opposes abortion and the so-called pro-choice group, which supports abortion.

The two main groups in the abortion debate are the so-called pro-life group, which supports the unborn’s rights to life and the so-called pro-choice group, which supports women’s right to have an abortion.

The items found in Table 6 illustrate a lack of balance across a series of items. The poll employs seven questions asking which circumstances justify legal abortion. Five of the seven deal with relatively rare health indications for abortion, and only two deal with more frequently occurring social indications. Yet at least 93% of abortions in the United States are done for social reasons (Torres and Forrest, 1988). Pollsters should focus on the most prevalent conditions and not on the relatively rare physical indications.

What Questions Were Not Asked?

This author is currently conducting a study of all abortion polls reported to The Roper Center for Public Opinion Research for 1986 ñ 1993. The Roper Center receives the results of virtually every major poll of the American Public.

The study sought to determine the relative amount of attention the pollsters paid to the two main parties (the woman and the fetus) whose rights clash in the abortion controversy. The rights of government, parents of the minor girls seeking abortions, and fathers of the unborn have also been at issue in abortion litigation. A content analysis was made of all questions asking what Americans thought about the rights of these five parties. The analysis omitted polls done for advocacy groups, and those whose total or sub-population N’s were less than 500. This yielded 22 polling organizations which asked 297 “rights”-questions. (See Table 7)

Major pollsters asked many questions to ascertain what Americans thought about women’s rights, but never once asked what they thought about the rights of the unborn! One could argue that since the rights of women and the unborn clash directly on this issue, asking only about women’s rights also informs us of the public’s view of the rights of the unborn. However, the data in Table 2 and Table 5 indicate that this is not the case. Thus, if pollsters were equally interested in both major parties in the abortion controversy, a different picture of American public opinion would emerge.

Conclusion

While our review has been critical of some questions which are quite biased, we should recognize that devising insightful, valid and reliable questions on complex and controversial topics such as abortion is a difficult task. This is particularly true when the pollster is attempting to measure attitudes toward court decisions, laws, or policies that evolve over time, and change in ways that cannot be anticipated. In such circumstances, even initially valid questions may become less than adequate (Adamek, 1994). Hence, the value of employing these guidelines to interpret polls is underscored.

References

Adamek, Raymond J., 1989a, “The Myth of ‘Pro-Choice’ America” in Dave Andrusko (ed.), The Triumph of Hope, Washington D.C.: National Right to Life Committee, pp. 31-40

Adamek, Raymond J., 1989b, “Public Supports Protective Legislation.” National Right to Life News, 16, (November 16), 1,10

Adamek, Raymond J., 1994. “Public Opinion and Roe v. Wade: Measurement Difficulties.” Public Opinion Quarterly. 58:409-418

Americans United for Life, 1991, Abortion and Moral Beliefs, Washington, D.C.

Blake, Judith.1977. “The Supreme Court’s Abortion Decisions and Public Opinion in the United States,” Population and Development Review, 1 and 2: 45-62

Cook, Elizabeth A., Ted G. Jelen and Clyde Wilcox, 1992, Between Two Absolutes, Boulder, CO, Westview Press

Cook, Elizabeth A., Ted G. Jelen and Clyde Wilcox, 1993, “Measuring Public Attitudes on Abortion: Methodological and Substantive Considerations,” Family Planning Perspectives, 25: 118-121, 145

Faria, Geraldine, Elwin Barrett and Linnea M. Goodman, 1985, “Women and Abortion: Attitudes, Social Networks, Decision-Making,” Social Work in Health Care, 11: 85-99

Friedlander, Myrna L., Theodore J. Kaul and Carolyn A. Stimel, 1984, “Abortion: Predicting the Complexity of the Decision-Making Process,” Women & Health. 9:43-54

Harris, Louis and Associates, 1992a, August 12, 1992 poll. Storrs, CT: The Roper Center for Public Opinion Research

Harris, Louis and Associates, 1992b. June 10, 1992 poll. Storrs, CT: The Roper Center for Public Opinion Research

Harris, Louis and Associates, 1993, October 6, 1993 poll. Storrs, CT: The Roper Center for Public Opinion Research

Hart and Breglio Research Companies, 1992, December 15, 1992 poll. Storrs, CT: The Roper Center for Public Opinion Research

Henshaw, Stanley K. and Kathryn Kost, 1992, “Parental Involvement in Minor’s Abortion Decisions,” Family Planning Perspectives, 24: 196-207, 213

Hickman-Maslin Research and American Viewpoint, 1988, National Abortion Rights Action League Abortion Poll, December 17, 1987 poll

Jain, A. K., 1977, Single Service Organizations: A Comparative Study of Twelve Abortion Clinics in Ohio, Ph. D. Dissertation, Cleveland, OH: School of Applied Social Services, Case Western Reserve University

Newsweek, 1989, “The Future of Abortion,” July 17, 1989, 14-27 Research and Forecasts, 1986, “Knowledge of the U.S. Constitution,” Poll conducted for the Hearst Corporation, Storrs, CT: The Roper Center for Public Opinion Research

Roper Organization, 1980, 1985, The 1980 Virginia Slims American Women’s Poll, and The 1985 Virginia Slims American Women’s Poll, Storrs, CT

Rosen, Raye H., 1977, “The Patient’s View of the Role of the Primary Care Physician in Abortion,” American Journal of Public Health, 67:863-865

Schuman, Howard, Stanley Presser and Jacob Ludwig, 1981, “Context Effects on Survey Responses to Questions About Abortion,” Public Opinion Quarterly, 45:216-233

Shaw, Paul C., Charles Funderburk and Billy J. Franklin, 1979, “An Investigation of the Abortion Decision Process,” Psychology: A Quarterly Journal of Human Behavior, 16:2:11 - 19

Torres, Aida and Jacqueline D. Forrest, 1988. “Why Do Women Have Abortions?” Family Planning Perspectives, 20:169-176

Top