Tools for Rapid Market Assessments

by Larry Lev, Linda Brewer and Garry Stephenson
Oregon State University Extension Service
Revised March 2003

Introduction

Farmers Markets have few resources to collect information that can help them to change and improve. We have designed three low-cost methods to address this information gap. The first two methods (attendance counts and dot surveys) can be used independently by individual markets. They are discussed in the first two sections of this report.

The third section presents a more integrated study of what we call a Rapid Market Assessment (RMA), which adds a third element called Constructive Comments/Observations (CCO). In our experience an RMA works best as a collaborative learning process in which a team of market managers (and others) jointly visits and studies a host market. The learning is two-way with both the host market and the visiting RMA team members gaining new knowledge and experience.

The host market gets the outputs of the three data gathering methods. The RMA team members learn how to make use of the data gathering methods. They also have the opportunity to establish relationships with and learn from their peers. They begin to see beyond the hectic routines of market day and making constructive observations of other markets they can come to view their own markets with greater understanding. Throughout we try to make the process inexpensive in terms of time and money.

Attendance Counts

Knowing how many people shop at your market is a valuable but generally not collected piece of information. Vendors want to be able to estimate their potential sales, surrounding businesses are interested in spillover sales, and community leaders want to know how important the market is as a social center.

Most markets have numerous entry points. Therefore it requires both planning and personnel to get an accurate estimate of attendance. While greater accuracy would be achieved if everyone who enters the market were counted, that approach would require stationing someone at each market entrance for the entire market. It is our experience that counting everyone who enters the market for a ten-minute period each hour provides an acceptable estimate. This approach greatly reduces the time demands for people doing the counting. When the attendance count is part of a broader RMA, it frees the team members to do other RMA activities for the other 50 minutes per hour.

Collecting this data requires clear communication and cooperation among the individuals involved. Here are the key items to take into account.

• Determine lines of sight in the market between and among all entrances (this will determine how many counters you need).
• Some shoppers enter the market between booths, rather than through established entrances. It’s important that these shoppers be counted only once. Count takers at opposite ends of the same line of site must agree on a boundary beyond which they, individually, will not count these shoppers.
• Determine the exact line for each entrance that separates people who are in the market from those who are not.
• For markets that open on the hour (e.g., 9:00), the count should take place every hour from 00:25 to 00:35. Consistency and precision in timing the counting periods yields a more accurate estimate.
• Only adults are counted (no dogs or kids).
• We have found that counting is easier and more accurate if “clickers” are used. They can be purchased at stationary stores.
• Only shoppers entering the market during the counting period should be included in the count. If possible, counters will perceive which shoppers have placed purchases in their cars and are returning to the market. These re-entering shoppers would not be counted again.
• The ten-minute hourly totals are multiplied by 6 to give an estimate of the rate of shopper flow into the market per hour. These hourly estimates are summed to provide an estimate for the market.
• Pre-shoppers: a special case. All markets have rules, and their rules vary. Some markets do not allow shoppers into the market before opening time. Others do, but prohibit sales until the opening of the market. These markets can acquire a significant number of shoppers, waiting in line before specific vendors. These “pre-shoppers” may not be well estimated by the standard market counting technique. They can be counted just before the opening of the market, not multiplied by six, and added into the daily estimated total.

Dot Surveys

Introduction

Conducting consumer research in farmers’ markets presents significant challenges. Traditional techniques, such as interviews and written questionnaires, are not well suited to this venue. The accuracy of face-to-face interviews is constrained by small sample size and a potential bias toward those willing to be interviewed. Written questionnaires, whether to be completed in the market or returned by mail, have very poor response rates, once again biasing the sample. The Dot Survey approach greatly increases the number of survey participants and therefore should result in more accurate assessment of consumer preferences and actions. This is a “self-service” research approach addressing a limited number of questions displayed on easels in the market. We call the individual questions “Dot Posters” because consumers indicate their responses using colorful, round, self-stick labels. Because of the nature of this technique, respondents can see others’ answers to the questions. This is both strength and a weakness. It is strength because respondents view the process as much less extractive – they appreciate inclusion in the research process. It is a weakness to the extent that respondents may be influenced by what they observe on the posters. This is not an issue for the majority of questions. For example, people don’t change their answer to, “Where do you live?” based on others’ answers. For a few question this may be an issue and it is important to be sensitive to this point when crafting the survey questions.

This potential source of bias can be reduced by taking the following steps:

• “Seeding” posters with randomly placed dots that are later removed.
• Replacing the poster sheets with fresh sheets at intervals throughout the market day.

Six Steps to Successful Market Research Using Dot Posters

1. Identify information needs and carefully craft up to four closed-ended questions.
• Develop questions with the market manager and/or market board to address their most urgent concerns. Table 1 in the Appendix shows questions that have been successfully used with this technique.
   
• The questions should be clear and succinct. Define the possible answers – fewer is better. Overall you want each question and the answers to be easily understood by respondents.
   
• Through careful wording one poster may accommodate two closely related questions. Example: “Will you be dong any additional shopping or eating in the downtown district this morning? If yes, indicate how much you anticipate spending.”
   
• In general, consumers are quite willing to answer four questions; more questions may reduce the response rate. Each additional question also increases the space needed in the market to conduct the research.
   
2. Prepare the “questionnaires” for use at the market
• Write each question at the top of a large flipchart sheet. Legibility is critical so either print very clearly or paste typed questions on the flipchart sheet.
   
• Define the answers and place them in a horizontal line across the bottom of the sheet (we often also place the answers across the tops of each column as well.
   
• Divide the answer scale with vertical lines extending the length of the sheet. When dividing the space recognize that while it is preferable to divide the space evenly (so that the space allocated doesn’t influence the respondents), it is necessary to allocate more space to the more frequent responses. Example: When asking, “How much did you spend?” we know more respondents will answer $10, $15, and $20 than $40, $60, or $80.
   
• For quantitative questions the order of the answers is obvious. In certain cases the lines between answers can serve as additional responses. For example, the line between $20 and $30 represents $25 (however, note that within a column all the answers are counted the same). Ordering the answers for subjective questions is less clear-cut. There is no logical order for the answers to the question, “Why did you come to the market today?” As a result there is no way to allow respondents to split their votes between all two-category combinations.
   
3. Prepare sufficient color coding labels or dots
• The ¾-inch, self-stick dot is most easily handled; however, the ½-inch size dots take up much less space on the posters. In a crowded market using the ¾-inch dots will necessitate changing the posters at least once. Although inconvenient, the additional poster preparation has the advantages of reducing prior information given to respondents and allows the data to be analyzed by time period.
   
• Cut the dots into strips, providing exactly one dot per question. Participants are limited to one response per question. This high volume survey process permits as many as 3-4 respondents per minute; simplicity and consistency are vital.
   
• Use of a single dot color reduces participant confusion. Individuals receiving multiple colors frequently ask, “Does it make any difference which color I use for which question?” Individuals receiving one color and observing that others have used a different color may draw conclusions regarding the demographic significance of the colors.
   
• Changing dot colors periodically during the market day permits time analysis of market activity (are the responses from early in the market different than those from late in the market). Changing to a fresh poster sheet provides the same opportunity.
   
4. Position the dot posters in the market.
• Confer with the market manager in advance to understand the traffic pattern through the market. Select a site passed by most or all of the shoppers.
   
• Set up the flip charts in a row on market day.
   
• Select a site that is in the main flow of foot traffic, but that does not interfere with sales of surrounding vendors.
   
• Use your institution’s signage to identify yourself to the public. An awning will provide protection from rain and sun, and better blend into market culture.
   
• Be prepared, especially, for the effect of wind and rain on the easels. Large binder clips will eliminate paper blowing in the wind. In very windy sites, concrete blocks or other weights may be necessary to keep the easels from blowing over.
   
5. Actively recruit participants.
• Most people will not participate without a direct, personal invitation to do so. Approach potential participants, giving visual cues that you’re part of the market’s business for the day.
   
• Respondents are limited to one set of dots per “shopping group.” A group of adults shopping together, and managing their money in common, defines a shopping group. The shopping group idea is particularly important for questions about purchases, both inside the market and at surrounding businesses.
   
• From scripted comments, explain the market research, and ask for a minute or two of the participant’s time. Be brief. Examples of introductory lines we have used: “Do you have just a minute to help out the market?” or, “Have you had a chance to do the dots?” People tend to associate the word, “survey” with something much more time-consuming; avoid its use.
   
• Distribute strips of dots to willing participants. Our university prohibits distributing dots directly to minors, although parents can have children place their dots. Offer to place dots for customers whose hands are full. The loss of anonymity does not appear to effect participation.
   
• Instruct participants to place dots “where it most makes sense,” and to place only one dot per poster. Team members must pay attention to the posters. Gently stop participants who are placing more than one dot on a single poster. A percentage will miss a poster, and have an extra dot; they may need help determining which poster was missed.
   
• Most shoppers have never participated in this type of data collection before. It is important that each team member be familiar with the process and be able to answer questions about it.
   
• The question, “How much did you spend in the market today?” will be handled in various ways by participants. While it is best to poll shoppers as they leave the market, that isn’t always possible. Some will save a dot, and return after shopping. The dots are removable so shoppers can return and move a dot if their actual spending differs from the initial estimate.
   
• Encourage participants to indicate their actual expenditures on the day of the survey, rather than what they “usually” spend at the market. Markets vary week by week; consumer spending is a reflection of what is in season. The survey is a snapshot of a specific market day rather than an indicator of average shopping practices.
   
6. Data analysis
• Tabulate the data by counting the dots assigned to each answer.
   
• Ensure accuracy of the count by leaving a small mark on each dot as it is counted. While each question should have the same total number of responses, this won’t happen. In all surveys, some respondents will fail to answer a specific question. Conversely, some will have used two dots for one question. It’s not possible to correct for either of these problems during data analysis. The variation in total responses across questions will be quite small.
   
• Compute responses as percentages of total responses. Quantitative data can be further analyzed. For example, mean spending per shopping group or total spending per time segment may be of interest to a market manager.
   
• The sampling unit in this technique is the “shopping group,” not the individual, although many shopping groups will consist of individuals. The average size of a shopping group will vary by market and is best determined using the dot poster approach. We have found the average size of shopping group to be in the range of 1.5 to 1.8 people.
   
• When determining the total spending for the market, it is first necessary to divide the market population count by the average size of the shopping group to determine the estimated number of shopping groups. This number can then be multiplied by average purchase amount. An estimated market total based on the simple market population count would result in a significant overestimate of market sales. You can read more about market counts in this publication under “Attendance Counts.”
   
• Analysis of identical or similar questions across markets is often undertaken to compare and contrast locations.
   

Constructive Comments / Observations (CCO) as a part of an RMA

This is the third component of the RMA and requires the greatest amount of preparation and coordination. Whereas attendance counts and dot surveys can be stand-alone exercises, we have always conducted a CCO as a part of a complete RMA.

In an RMA a small group of outside market managers (or board members, etc.) travel together and study a specific host market. The host market profits from the insights provided by “fresh” eyes of outsiders. The RMA team members gain from establishing relationships with and learning from their peers. They begin to see beyond the hectic routines of market day and can begin to view their own markets with greater perception.

We have organized the RMA process so that there are virtually no demands on the participating team members beyond the day of the study itself. They travel to the market, do the various components of the study at the market, hand in their worksheets, and participate in a phone debriefing several weeks later

The RMA process should not be regarded as a comprehensive assessment or evaluation of the market. Instead, what the host market receives are the attendance counts, results of the dot survey and a report that provides constructive comments (what is going well and what could be changed or improved).

Here are the steps in the RMA process with a focus on what is needed to conduct the CCO part of the process.

• Form ad-hoc teams of 4-5 managers or board members from markets around the state. These teams will study a specific market on a specific date and collect qualitative and quantitative information.
• While it would be ideal to share information about the market in advance, we generally haven’t done that.
• For morning markets, we try to arrange a pre-market dinner to allow the host market’s manager and board to provide market history, current situation and goals for the next 5 years. The dinner also forms relationships between the RMA team members and host market representatives for the next day
• On market day, the team members are responsible for three things:
• Attendance counts
• Dot surveys
• Filling the out the three CCO sheets based on their own observations of the market.
• Sometimes a fourth element is added – interviews with key people in the community on their interactions with the market.
• The team leader sets a schedule to make sure that each team member has a chance to participate in all three activities and that everything that needs to be done gets done.
• The RMA participants consider three major themes for each market with each theme developed on a separate worksheet: atmosphere, physical characteristics of the market site, and vendors and products.
• Market atmosphere includes the “feel” of the market, type of shoppers, conversations, and educational activities.
• Physical characteristics of the market site include access, flow of people, liability issues, and organization.
• Vendor and products addresses product mix, product quality, signage, display, and customer service.
• On each worksheet the team members provide constructive comments that reflect both what is going well in the market and what could be changed or improved. Whenever possible they are encouraged to write down things that they observe that they will apply back to their own market.
• Before they finish with the worksheets at the end of the market the team members are encouraged to go through each and select out
• Their most important message to the host’s manager,
• The most important lesson/observation they will take back to their own markets, and
• Any observations that should be excluded from the public report.
• At the end of the market the team gets together briefly to debrief the experience. While it would ideal to have the host manager participate, he/she is generally too busy (and he/she is also generally too tired to be able to absorb much at the end of the market).
• Before they leave the market each team member hands over his/her sheets to the person who will prepare the report.
• The collated report pulls together all of the comments made by individual team members. There is a not an attempt to provide a single view of the market so individual comments may conflict. But there is also not a separate report for each participant. The guarantee of anonymity for each participant greatly eases any potential for tension.
• In general the report of the attendance counts and dot survey are distributed to the host market and back to the team members in 3 or 4 days of the RMA.
• The compiled CCO is distributed within two weeks of the RMA. At the end of the compiled CCO, 3 to 5 discussion questions are suggested.
• The 3 to 5 discussion questions form the focus of the telephone debriefing for the RMA. The debriefing only looks at these major issues rather than touching on everything that occurred during the RMA.