Tools for Listening: Quantitative and Qualitative

by Mark Snowden

Twenty years ago, I had cancer. Chemotherapy brought me to the brink of death. After receiving a transfusion, a single question burned within me: If I died right now, would my Christian media production work make a difference?

Twenty years ago, I had cancer. Chemotherapy brought me to the brink of death. After receiving a transfusion, a single question burned within me: If I died right now, would my Christian media production work make a difference?

As the Lord healed me and led me into other missions-related vocations, my perspective was transformed. I didn’t set out to learn communications research techniques; they simply became tools I needed to advance the gospel. You may need them, too.

Survey tabulations and focus group interpretations became as important to the production process as a video camera and an editing bay. Sometimes the research already existed. Sometimes on-site observation was so revealing, it saved me weeks of work. At other times, I needed the input of a wise leader to understand what I was receiving. And a funny thing happened along the way: as I began listening to others, I found that God was speaking.

A CHRISTIAN PERSPECTIVE ON COMMUNICATIONS RESEARCH
God speaks through people. Pages of questions can form an instrument that God can use to speak to researchers. This “instrument” can aid in discernment, which is a spiritual gift (1 Cor. 12:10). Survey research provides a basis for prioritizing kingdom work.

Communications research is about listening. Missions leaders should start research by listening to God and praying for wisdom and discernment. From Scripture we know that God expects his stewards to invest their talents wisely (Matt. 25:14-30). Good stewardship of time, personnel and money requires today’s field personnel to use a variety of research methods.

Are you listening? If so, how? Survey research is an important tool for cross-cultural workers. The first step is recognizing the need for information that will help you make quality decisions. Implementing effective research means seeking godly wisdom and avoiding “falsehood and lies” (Prov. 30:8).

LISTENING TO GROUPS
Surveys provide an efficient way to hear thousands of people. In a populous world, having a “voice” in matters is often frustrating, if not impossible, for both the message’s communicator and its recipient. An organization may use surveys to listen to many. Large numbers of people can reveal their opinions through representative “subgroups.” Researchers call this a sample. The surveying itself is called “sampling.”

Missionaries desire rapid multiplication of indigenous churches among all peoples. Effectively communicating the gospel is a bedrock requirement. Factual information—not guesswork or hunches—aids decision-making. “Gathering the facts of growth is among the more demanding and yet most important phases of the effort to discover the whys of growth” (Smith 1984, 80). Donald McGavran, a pioneer in church-growth understanding, equates using statistics to a physician using a thermometer to measure a fever (McGavran 1990, 130).

Diagnostic tools merely help someone understand what God is doing. Survey information helps determine where we’ve been and any future progress that might be made for the kingdom.

An effective communication strategy will, therefore, have as its starting point a clear description of the audience and their needs. We need to study their relationship to God and their position in the decision process. Such detailed information gathered prior to the start of a Christian media ministry will provide a basis for later evaluation of results. (Søgaard 1993, 79)

Quantitative research measures attitudes, experiences and other information that defines behaviors. For Christian researchers, this means looking at more than, say, evangelism. It also means considering the most pivotal step in a Christian’s life—the point of conversion. Each stage of the spiritual decision-making process requires analysis that aims to find comprehensive results. Research should be put within the larger framework of an overall missions strategy (Engel 1979).

SMART QUESTIONS
Focus on the Family founder James Dobson has said in a radio broadcast that when he was young his father wanted to know whether he had asked any “smart questions” at school that day. What smart questions does the Holy Spirit prompt you to ask?
When urged to collect answers to “smart questions,” look in several places:

Existing research: What has been done on this subject? Check libraries, social research journals, with missions colleagues and on the Internet. This saves valuable time.

One team of missionaries benefited from a report on selling cosmetics to Muslim women in Indonesia. The “positioning” of make-up also gave clues into communications preferences (Sack 2000).

Observation: “You can observe a lot by watching,” Yogi Berra said. Visiting an area can reveal obvious needs. This may sound simplistic, but it saves time and money in the planning process.

In 1999, a four-man team traveled across Bolivia into southern Peru noting possible locations for church plants. At eleven thousand feet, the thin air at Lake Titicaca created an atmospheric “skip” that extended the Christian broadcasters’ signals beaming from La Paz.

That meant radio ministry had great reach across this area. The team saw many churches in Bolivia. In contrast, in southern Peru, members saw mounds where local people made sacrifices. This revealed to the team not only a need for churches in Peru but also a need for addressing their spiritistic belief system. Based upon the team’s findings, members recommended that the radio station manager and local pastors broadcast listener-friendly messages for people with lower literacy skills.

Questionnaires: This is a common tool used by researchers to get people who represent their sample to fill out surveys using different types of questions. Unfortunately, this often is the only type of research used. Questionnaires involve written answers but not essays. The other four research categories help explain or can amplify what is tabulated from answers received from a myriad of questions such as:

• True or false: Christians can know the will of God.
• Forced choice: Do you prefer studying the Bible by reading it yourself or by listening to it on audiocassette?
• Open choices: Circle any of the following media that help you study the Bible: Devotion guided, Bible study curriculum, radio programs, audiocassettes or other (please list: ______________).
• Yes, No, Sometimes or No answer: I listen to the radio daily.
• Likert Scales: With 5 being greatest and 1 being least, “How much does a magazine’s design affect your decision when purchasing it from a newsstand?”
• Written answers: Please comment on how you feel about the daily radio program, “Experiencing God.”

Focus groups: Dialogue sessions with about a dozen people representative of those you are studying may clarify findings and serve as sources of information and correction.

In 1991, for Kazakhstan’s first media survey, we held a dialogue with a group of aspiring businesspersons. I will never forget participants’ stunned faces when we asked, “Do you have a radio?” In the former Soviet Union, virtually every home and office was pre-wired with “cable radio.” Kazakhs would not have asked this question; however, I’m glad we did, even if we looked silly. Our written surveys only asked about radio. The focus group helped us discern that broadcast radio would not penetrate homes and offices. We needed to work with cable radio operators. This vital decision-making information boosted our strategy’s success and helped guide international Christian broadcasting ministries in the early days of the Commonwealth of Independent States (CIS).

Interviews: Leaders and influential representatives from your interest group can provide guidance, perspective and wisdom. Interviews also tend to help leaders be more receptive during the implementation phases of your project or campaign.

The tally of questionnaires and cross-tabulation of pastors’ responses prompted an interviewer to ask Brazilian Baptist Convention leaders, “Why are periodicals credited by so many pastors as having started new churches?” Convention leaders credited a Women’s Missionary Union (WMU) in Brazil for publishing a magazine that instructs women to include their husbands, children and Bible study leaders when contacting a seminary about pastors who can help the group become a church. The Baptist leaders reported that more than half of Baptist churches planted in the past decade in Brazil were started through WMU publications.

GETTING STARTED
You will need help. Your research team should be comprised of competent believers. Team members may provide insights into the people group or population segment being researched. Other groups may be willing to partner with the project. Involve many capable people. The project leader will decide whether to use survey-takers, interviewers, translators or mathematicians to help with final tabulations.

Recruit team members: Find the right people to carry out the project. Each project will have its own limitations on money, time and human resources. Workers must understand and work within these limits. When the research team meets for the first time, set a timeline and establish checkpoints along the way. Present this document to cast a vision for the project and ask them for tips and insights that can help the project succeed. Each team member needs some individual attention from the project leader.

• The project coordinator manages the project, overseeing timelines and laying plans to carry out the research elements (reading existing research, observing, surveying, conducting focus groups and interviewing leaders). The coordinator drafts the final report.
• An assistant handles information as it is received. Name an assistant early so tallies can begin immediately. Transcribe and, if necessary, translate the recorded dialogue from focus groups as you conduct them.
• At least five surveyors may get paid—fifty cents per fully completed questionnaire and twenty cents for partial completion.
• The prayer leader meets regularly with the team to identify prayer concerns, keep the spiritual focus sharp and intercede through prayer channels.

Work with your team to determine:

• Schedule: Projects usually take eighteen months; six months is typically required for survey work.
• Ongoing contact: Commit to staying in touch with team members.
• Doing all five elements of the required research: Categories include reading existing research, observing, surveying, conducting focus groups and interviewing leaders.
• Suspending current work: The project director should delay other work until the research project is finished. This is a worthwhile sacrifice.
• Confidentiality: The research coordinator keeps participants’ information secure.
• Sharing data and results: Determine what the results reveal. Write a final report discussing the findings. Monitor progress toward the project’s objective, such as adjusting plans for radio ministry in Kazakhstan or deciding where to plant churches in the Andes. Accountability is vital to ensure money is spent properly and that information gleaned will actually make a difference. Share results with administrators and project leaders so they can strategize and keep you accountable.
• Prayer support enlistment: This is vital to each project.
• Future meetings: Set locations and dates.
  Anticipated budget: Set a budget by anticipating travel expenses, surveyors’ payments per questionnaire and other costs such as photocopy or telephone charges.
• Timeline: The project leader should back-time each deadline so that detailed schedules can be worked out for the team’s survey. When should copies be made? Who makes them? When is the best time for a focus group? Your planning needs flexibility. Satan will seek to thwart your efforts.

  Checkpoints in a research timeline: (approximately six months in duration)

  • Survey project introduction
  • Test run planning and project development
  • Test run using small samples/tallies
  • Project coordinators meet and finalize the project
  • Implement real research
  • Team meets to discuss findings
  • Project coordinator’s meet
  • Compile and analyze
  • Report relevant findings
  • Decide based on the information learned

Testing phase: Even before testing your first survey draft, allow at least two hours for several representative people to examine your focus group questions and questionnaire (especially the one for the unchurched). Have some trusted representatives take the survey, then honestly discuss what they liked and didn’t like. Consider a micro-test with another small group.

Implementation phase: The full implementation should be much smoother than the testing phase; however, it will touch more people. The testing phase should have worked out any “bugs” so the process will flow smoothly with fewer surprises. The survey will attract attention from those being surveyed and from church leaders interested in your work. The fact that you are researching the issue draws attention to it. Be ready to handle questions as they arise. If the situation changes drastically during implementation, do not be afraid to shut down the research and ask for help or wait until the environment is conducive to beginning again. In such cases, of course, deadlines must be renegotiated.

Preparing a report: Your team will have large amounts of reliable information. Use bar graphs or pie charts to quickly communicate numbers. An outline includes: (1) how the research was conducted, (2) findings, both quantitative and qualitative, (3) discussion of findings and (4) recommendations. The fourth point should be based on master strategy planning needs. “Endvision” should be addressed frequently throughout the report.

Reporting: Anticipate delivering a twenty-minute report and ten-minute discussion. An accompanying printed handout details your information. Consider making the data available on disk or CD.

DEVELOPING AND INTEGRATED MEDIA STRATEGY
Each people group has a preference for the way it likes to receive decision-making information. Certain types of media work better at conveying messages than others. Some content reinforces good things in the culture (a bridge) or addresses what in the worldview may cause people to resist the gospel (a barrier). Understanding bridges and barriers within a specific worldview makes for effective communication.

The communications process requires sensitivity and proper timing. For example, don’t do a survey in the middle of monsoon season. And you won’t get a good focus group at the peak of harvest.
The research results should not launch a flurry of media activity apart from the overall objectives for church planting work within a people group. Instead, whatever communications approaches are identified as most appropriate should be used in the overall approach involving every aspect of church planting.

Research isn’t an end in itself. Research findings should be integrated into the bigger picture of church planting, evangelism, discipleship and leadership training. Communications research shows that media flow together like a river. When people step into a river, they immediately know which direction the current will take them. The same should be true with communications. The lost enter at multiple points. Radio leads people to a film showing. At the film showing they move to the next stage, perhaps a Bible study where they receive cassettes or curriculum. The flow keeps serving as a catalyst to move people to the cross for a lifetime of maturity as a spiritually-reproducing believer.

It takes discernment to choose what information best contributes to your ultimate missions objective(s). As in the best movies where editing, lighting and acting become transparent, the same goes for media strategies. The best ones flow seamlessly as a part of the comprehensive whole under the authority of the Lord. If appropriate, missions leaders and local believers should receive copies of the plan and the research behind it.

Additional media strategy training is available at http://newWway.org/strategy. A cross-cultural communications research toolbox can be found at http://newWway.org/research. Case studies, models and papers are also available.

Among the Muslim Yao people of southern Africa, an evangelism strategy benefited from an anthropological survey conducted by a South African university. Using this data and their own observations, missionaries produced radio programs that helped listeners understand biblical principles and broke down barriers to the gospel. The same radio programs advertised events such as medical clinics or film showings. At the events, Christians distributed Scripture portions on cassette or in print and invited people to Bible study groups where believers introduced the lost to Christ. One strategic element led to another over time, using media to build relationships.

This integrated approach worked well for evangelism. Additional strategies were developed for discipleship and training leaders.
It’s important to re-think media saturation projects and precision harvesting. In our zeal to “make disciples of all peoples,” many missionaries unfortunately launch communications efforts without reflecting biblically on how they can steward their resources in a godly way. Veteran missions leader Jim Engel tells of Christians distributing tracts in an Asian village only to discover that no one in the village could read. At month’s end, some missionaries in South America with tract distribution quotas would drive to bus stops and throw out tracts. Mission planners in East Asia lament that pseudo-Christian cults have followed evangelical mass media blitzes, leading more than a few seekers astray.

There is nothing wrong with media saturation campaigns if culturally appropriate and relevant media are used to cluster seekers for follow-up and church-planting. Rather than attack these well-intentioned efforts, cross-cultural witnesses would be better stewards if they used research methods outlined in this article to make informed media selection decisions.

Precision harvesting is defined as “a strategic placement of church planters in contact with seekers or new believers who already have been identified and cultivated through their response to mass evangelism” (Garrison 1999, 60). Mass evangelism using communications media works best when response mechanisms bring people together. For example, clustering individuals or families is possible by mapping where they live or some other commonality between them.

In a Central Asian capital, church planters ran ads offering free Bibles in two major newspapers. In fourteen months respondents requested more than eighteen thousand Bibles. Church planters marked each request on a map. The clusters of responses helped church planters identify neighborhoods in which to begin ministry. A team of eight conducted observation research to discover that one highly responsive sector was extremely conservative Muslims. Within a year, several Bible studies were launched in homes and four new churches were planted.

THE LISTENING TASK
It took four rounds of chemotherapy to make me stop work long enough to start asking some important questions. It was reassuring to not flail around from one type of research to another, but to boil down the process into five specific aspects that really work. We can never know the full extent of our efforts, but we can plan wisely to make a larger impact and evaluate as we go to ensure the best results. Watching missionaries implement strategies based on real information has been a blessing for which I give God praise and glory.

The Lord wants us to listen. Sometimes listening to the needs or cries of those closest to us muffles or drowns out those from the larger group. As God tugs on your heartstrings for planting, watering or harvesting, may these tasks be done with ears only God can provide. Communications research is a wonderful toolbox that the Holy Spirit can use to help you hear the cries of the many over those of the few.

References
Engel, James F. 1979. Contemporary Christian Communications: Its Theory and Practice. .

Garrison, David. 1999. Church Planting Movements. Richmond, Va.: International Mission Board.

McGavran, Donald. 1990. Understanding Church Growth. Grand Rapids, Mich.: Eerdmans.

Sack, Michael. April 2000. “The Value of ‘Practiced Ignorance’ in Cultural Research.” .

Smith, Ebbie. 1984. Balanced Church Growth. Nashville: Sunday School Board.

Søgaard, Viggo. 1993. Media in Church and Mission. Pasadena, Calif.: William Carey Library.

—–

Mark Snowden is a missions writer based in Richmond, Va. He has been a communications researcher for more than twenty years, eighteen of them with the Southern Baptist Convention’s International Mission Board.

Copyright © 2005 Evangelism and Missions Information Service (EMIS). All rights reserved. Not to be reproduced or copied in any form without written permission from EMIS.

Comments are closed.