Tips for Citizen Scientists
Tips for Designing Bee Research Projects
Randy Oliver ScientificBeekeeping.com
Revised 2 Jan 2016 (in purple)
Beekeepers are known for being of curious and experimental minds. Since factors affecting beekeeping are continually changing, new unanswered questions are bound to arise; the beekeeper “citizen scientist” can often answer them himself by performing a well-designed experiment, and then share those results to the benefit of everyone. But for the results of any experiment to be meaningful, it is important that the experiment follow certain scientific principles. Following are some tips for running a successful experiment.
You can perform a “trial” to test the hypothesis that this “treatment” (e.g., some sort of management technique or application of a medication) will result in a certain effect (e.g., lower mite level, greater honey production). Or, you could perform a “study” to learn something about bees or beekeeping (e.g., whether feral colonies have a higher incidence of X than do managed colonies).
#1 point—what is the single specific question that you are trying to answer with this experiment? Write that question on the wall, and eliminate anything from your experimental design on that doesn’t have to do with specifically answering that question (and make sure that your experimental design will unambiguously answer the question). Fill in the following blanks before you go further (take some time to think them through):
Your question ______________________________________________________________________
Your hypothesis ____________________________________________________________________ (You will set up the experiment to determine whether actual data either “support” or “refute” your hypothesis).
The treatment to be tested _____________________________________________________________
The predicted effect _________________________________________________________________
Perform an internet (e.g. Google Scholar) literature search of previous research, and read any previous studies that are applicable to this treatment. Such studies should also give you a model for your write up. Speak to researchers familiar with this topic.
Your background research will introduce you to the format of scientific papers, which follow the typical format of:
Title, Author, and Date
Materials and Methods
Acknowledgements and Funding Source
It would be wise to plan and execute your experiment to fit the above format (see the example at the end of this doc for more details). This model works, and allows you to share what you learn with others. You may also wish to present the results to one or more groups of beekeepers. Knowing the above, may I strongly suggest that you:
Plan Ahead and Work Backwards
Here’s a tip: first run the experiment backwards in your head, and then on paper. Begin by imagining exactly how you plan to present the results to an audience in a manner so that they can fully understand the significance of your experimental findings. Are you going to use scatter, line, or bar graphs? Plan and draw on paper, the exact sort of graphs you will be using to present your findings, and how the results would look if they either support or disprove your hypothesis. Then work backwards to set up every detail of the experimental design to produce those graphs (e.g., what will be your x and y axes?). This approach will help to save you from kicking yourself later on for wishing that you’d done something differently! However, it is unlikely that any experiment will go exactly as planned. It is the rare experiment that I wouldn’t do somewhat differently the second time—read the Discussion section in published similar studies to learn from others’ errors and experience. See end note [] for further discussion.
Materials and Methods
This is how you actually run the experiment. You are going to try to determine whether your treatment(s) cause any measureable differences in the test group(s) compared to an identical control group (which should receive a sham treatment, e.g., plain syrup, or a simple opening and smoking). Every colony in the entire experiment must receive identical location, handling, feeding, etc. other than the specified “treatment.” You want the specific treatment to be the only “variable” between the groups.
Or if you are doing a study, how exactly did you sample and why; how did you avoid bias; how were the samples analyzed.
The M&M should be written with enough detail that another researcher could exactly duplicate your experiment (replication).
Relevance—to be relevant to “real-life” beekeeping, the size, condition, feeding, and environment of the test colonies should approximate that of normal beekeeping practices.
The best design is generally randomized block—if there are 3 treatments and a control, then group colonies into groups of 4, and randomly assign all 4 treatments in the group (use random.org; use different randomization for each group). This will minimize location, pathogen transmission, and orientation effects.
If you are trying to determine, for instance, the effect of a miticide, I like to take mite counts from all colonies on Day 0, rank the hives by mite count, and then assign treatments alternately down the ranking. If you have 3 treatments and control, then randomly assign the 4 treatments to the 4 most infested colonies, and continue likewise down the list.
Number of colonies (generally a minimum of 12 in each group). One of the toughest problems in bee research is the intrinsic variability between colonies. This “noise” often makes it difficult to tease out any effects due to treatment. The more colonies in the trial (the larger the “n”) the better. Practically, I wouldn’t waste my time with fewer than 12 colonies per group; 18-36 would be even better.
Incubator trials—consult with advisors for best practices.
Replication—typically a trial is replicated three times in order to confirm you get the same result each time (you may not). However, a single replicate is better than nothing.
Source of colonies (queens, etc) should be randomized per treatment group. In some trials it may help to found all colonies with sister queens so as to minimize variation. If queens from different sources are used, best to use each source in one randomized block.
Grouping of hives—the further apart the hives the better, in order to minimize the “drift” of (especially, infected) bees from hive to hive. The distance factor must be weighed against changing the variable of “location” if such spacing puts some of the hives, say, into shade or some other change in the environment.
Hives should not be placed in a line (in order to minimize the effect of drifting bees) and should all have the same solar, wind, etc exposure, or exposures rotated. Use landmarks (shrubs or yard trash) in the apiary to help the bees to orient back to their proper hive. See [] for cautions.
Generally, each pallet of hives receives one treatment, due to drift from hive to hive on that pallet. Pallets should be placed so that each treatment group has pallets facing each compass direction.
Randomization–Treatment or control should be determined by flip of a coin or random number generator (random.org). This decision must be entirely independent of the Investigator, so that there is no bias, and should be done only after all other colony preparations have been done. Note: “arbitrary” assignment of treatment is not random! One problem that I’ve had with the random assignment of treatments is that sometimes the coin flip or random number generator (http://random.org/) puts all one treatment in one location. This is where a randomized block design helps. If you are testing only two groups (Treatment vs Control) then call each pair of hives a block, and randomly assign treatment to one of each pair.
Systematic assignment of treatment—you may instead treat every other colony, or some other systematic assignment of treatment. If so, rotate or randomly assign the order for each group (e.g., rotate the solar exposure of hives receiving each treatment—don’t have all the test colonies facing the same direction). Another method is to first number and then grade all colonies for the main metric (say mite count). Then use a spreadsheet to rank all colonies by mite count from highest to lowest. Then assign treatment alternately down the list (I like this method, since it is applies the treatment equally across the range of starting points).
Number the hives—you don’t want wind, rain, or skunks to mess up your numbering (practical experience here). Firmly affix weatherproof numbers [] to each hive.
Bias and Blinding—It is virtually impossible to avoid bias. Go out of your way to “blind” yourself as to which colonies were receiving which treatment until all the data is collected and analyzed. Ideally, the Investigator, field inspectors, or graders will be completely “blinded” as to which colonies received which treatments.
Log book.–A chronological log book, in ink, must be kept of every detail and action taken in the yard. Many important discoveries are made after the fact by reviewing the log book. Never trust your memory! The printing up of a log entry sheet in advance will help you to clarify which data to collect, and the form in which it should be recorded. I print the attached data entry sheet on heavy cardstock, and fill it with a waterproof pen in the field. All raw data should be recorded on paper, in case of computer mishaps. As soon as possible, enter the data into an Excel spreadsheet.
Treatments–Specify in advance how treatments will be delivered, and how control colonies will be given sham treatments. If one smokes and opens the treatment hives, then one must do the same for the control hives; if one feeds a treatment in syrup, then the control hives must receive an equivalent amount of untreated identical syrup.
Effect–How will you measure any effect? Examples would be regular weighing of hives, cluster size estimate, mite counts, nosema spore counts or prevalence determination, etc.
Time points—Assign approximate time points for taking measurements in advance. Day 0 is the starting day (after equalization and set up)—the baseline to which future measurements will be compared. Collect data at at least three time points (four would be better) so as to be able to detect trends.
Grading–Specify what metrics will be measured (number of frames of bees, hive weight, etc) and how they will be measured, and at what time (Day 0, midpoints, End). The more time points at which measurements are taken, the better that a trend can be detected (generally plan at least three data time points).
You should be aware of the amount of tedious time involved in grading, and in processing samples. I have a lot of practice, and can tell you how long it takes us.
Be sure to practice each measurement several times at least before you even think about touching a hive in the experiment!!!!! You do not want to be learning or practicing on test hives or actual samples!!!!
Two of us—one tipping, one grading—can grade hives in the morning at a rate of around one hive per minute. Remember, you need a blinded grader. I prefer the relatively noninvasive “cluster grading,” in early morning, before the colony has broken cluster.
Gently smoke and tip the hive up off the bottom board, and count the number of frame interspaces filled with bees (I typically estimate to the nearest half or quarter frame). For colonies in a single box, the grade from below is enough (although you may count some very tiny colonies as zeroes). If there is more than one box, take a count for each box from both the top and bottom view. Then average the two counts for each box. Be sure to randomize the order in which you grade at each time point, so as to minimize the effect of time (clusters may start to break if it warms, thus affecting the grades).
Taking a ½ cup sample of bees (for varroa infestation rate and/or nosema count) takes another couple of minutes per hive. Allow a couple more minutes to check for brood. You should be able to grade and sample 24 hives before noon.
Now comes the hard part—processing the samples. Allow 2-3 minutes ea for varroa counts (faster if you build a shaker table). Then 5-10 minutes for each nosema count. For an experiment with 24 hives, if you allow 15 minutes total for each sample, that would add up to 6 hours of bench work for each time point.
There are three main methods of measuring nosema: field of view, mean spore count, or prevalence (all covered at this website). The latter two are the most accurate, and take about the same amount of time. For mean spore count I suggest using 25 bees by the “ziplock method and a hemacytometer. For prevalence, perform gut squashes of 10 individual bees. Either method gives roughly the same results, but prevalence is more biologically relevant.
Label EVERYTHING! Label and date every data sheet and sample–do not trust your memory!
Don’t cherry pick your data—there may be annoying outliers or results that don’t make sense to you. Exclude or censor data with the greatest caution! Decide, in writing, in advance, which criteria you will use to remove colonies from the experiment—queenlessness, disease, starvation, etc.
Removal of colonies from the experiment—allowing colonies to collapse may flood adjacent hives with parasites. You should physically remove any colony from the test yard(s) when it has “no further chance to survive by itself with regard to the local conditions.”
Statistics—you should run your design by a statistician prior to beginning the experiment! The simplest results would be date entered into an Excel spreadsheet, means and medians calculated, and standard errors of each mean indicated on a column (bar) or line graph []. Scattergrams, histograms, and line charts can also be used. Perform the Student’s T test to determine statistical significance (http://studentsttest.com/) for data that follows a “normal” (bell curve) distribution, or the Mann-Whitney test if it does not (http://www.socscistatistics.com/tests/mannwhitney/Default2.aspx). Use the Henderson-Tilton’s formula to determine treatment efficacy (http://www.ehabsoft.com/ldpline/onlinecontrol.htm#HendersonTilton).
Even better is to run an Anova and comparison of means, but that’s a bit more complicated than I wish to go here. In any case, don’t let the statistics scare you—collect the data, and then have someone help you with the stats.
However, of even more import than statistical significance is the size of the effect—was there enough difference due to the treatment to be of biological or practical relevance? Don’t rely upon statistics alone—look carefully at the bees during the course of the experiment, and look for patterns and trends in the data. The human eye/brain is wired to spot trends and patterns, which is why graphing the data is so important; statistical tests for probability check our inclination to “see” patterns where none actually exist.
In order that your experimental results can be compared to those of others, I strongly suggest that you download and read the appropriate sections in The COLOSS BEEBOOK Volume 1, Standard methods for Apis mellifera research: http://www.coloss.org/beebook/I/introduction
You do not necessarily need to follow their suggested guidelines (e.g. I prefer the quick and non intrusive cluster strength grading method to the suggested Coloss methods), but you had better have good reasons for not doing so.
In order to truly make sure that your results weren’t a fluke due to random chance, or some variable that you weren’t aware of, you should replicate the experiment. Each experimental run at any location is called a replicate. If you ran the trial with test groups at two different locations, or repeated the trial at a different time, then you’d have run two replicates. Statistical analysis of a single test group is considered by some as pseudoreplication, and thus lacking strong validity. In better studies, the experiment is typically replicated three times in order to see whether you get the same result each time.
Before you start, write out the exact protocol that you plan to follow, in minute detail. You can later “deviate” from the protocol if necessary, with an explanation of why. Include exactly how you will set up the experiment, apply treatments, care for the bees during the course of the experiment, and the time points at which you will collect data, and exactly which data you are going to collect, and exactly how you are going to measure it (e.g., how many bees in a sample, taken from where in the hive, at what time of day, and on what dates). Writing a detailed protocol in advance can save you a lot of decision-making in the field, and grief in the end when you say, “I wish that I would have thought that out before I started!”
Additional Pages and Resources
Data Sheet Template 24-hive data sheet
Parts of a Scientific Paper
THE PARTS OF A SCIENTIFIC PAPER.
Title in Bold (Latin names in italic)
Key words: for computer search
by Your Name
A very brief summary of what you did, your results, and your key conclusions.
A brief summary of the state of our knowledge on the specific subject of this experiment, including appropriate citations (Author Date) of any background research, your hypothesis, why you feel that the experiment is worth doing, and its potential practical applications. Be sure to clearly state the question that this experiment is trying to answer (every action and word of this experiment and write up should apply to the answering of this question).
MATERIALS AND METHODS
Exact and detailed instructions of what you did, so that other researchers could exactly duplicate your experiment. Use appropriate brand names and measurements. Refer to figures or tables as needed for clarity; refer to them in the text by citing them (e.g., “The feeder design included a screen to guide the bees ( Fig. 1).”). Each figure or table should contain enough information to be self explanatory without reading the text of the paper; include a brief descriptive title at the top, and an explanatory legend below. Describe the number of replications, the variables, and controls.
This is where you present your results. Explain them in writing, but also include data tables or graphs to illustrate your findings visually (cite them). Calculate standard errors of the mean and levels of significance and show them in the illustrations. Do not discuss your interpretation of the results yet!
This is where you discuss what you learned. Interpret the data. Does your collected data support or refute your hypothesis? Is the difference statistically significant? List practical applications, problems with the experiment, and suggest future experiments.
Who helped you, who reviewed the paper, and who funded your research (was there any conflict of interest?).
Use the following style:
Genersch, E and M Aubert (2010) Emerging and re-emerging viruses of the honey bee (Apis mellifera L.). Vet Res. 41(6): 54.
 The following is from an email exchange that I had with someone who wanted to do a study to “prove” a point:
Who do you expect to have an interest in the results of your study? The first thing to do is to imagine that you are standing in front of that group, making a presentation of your findings and discussion.
What I’m asking you to do is to make up, prior to starting the experiment, dummy powerpoint slides, as though the study had gone exactly as you had hoped, in order to show yourself how you would make your presentation points. This is similar to storybooking a movie prior to starting to film it.
I do not recommend presenting Excel spreadsheets to groups—charts (graphs) are much better. I understand that you may not yet be proficient in Excel. Go ahead and draw up your presentation in pencil, how you hope the graphs will look. Pretend that you are actually making the presentation to the groups that you mention above. Allow yourself to make up data for now, as though the study gave you exactly the results that you hoped for. Although I do not recommend putting more words on a Powerpoint slide than you would put on a T-shirt, for now type out the presentation points that you would wish to make—why the experiment was relevant, what you learned from it, and how others can apply your findings.
Then see if the presentation would be robustly convincing for those groups. I mean really imagine that you are standing up in front of them!
Only then can we really see whether the study is worth doing, and exactly how to design it. I hope that you understand what I’m trying to get you to do. Too many studies are started without really thinking them through to the end, and you just waste your time and effort in collecting a bunch of data that doesn’t amount to anything worthwhile!
 A really tough problem to surmount in field experiments with hives of bees is that if one hive gets sick, the bees tend to drift to the immediately adjacent hives and make that colony sick too! So think out your hive placement, and how treatments are assigned. If all hives on one pallet get the same treatment, and one gets sick, all will likely get sick, and then your results would suggest that the treatment caused them to get sick! On the other hand, if you have paired hives, one treated and one control, then if the treated hive gets sick, its adjacent control hive will likely get sick too. Take some time and think this through before you start!
 Error bars in Excel are a bit tricky. Put your calculated SEMs [=STDEV(data range)/SQRT(COUNT(same data range)] below the means in your data table. Highlight the data series in the chart, then in the Layout page, go to Error Bars, more error bar options, custom, specify value. Then highlight the calculated SEMs to use.