Introduction to Pilot Studies

Why do a pilot study?

Pilot studies are useful to develop a preliminary understanding of a specific subject (e.g. will magnitude and frequency of freeze-thaw cycles on Maine’s bridge decks change in the future?). They are typically conducted to investigate a question with the intent of possibly of conducting a full research project in the future.

Conducting a pilot study can answer many questions about the research topic without investing too much time and effort:

  • Is it feasible? Decide if this research can be conducted or whether or not there is a practical way to continue.
  • Does it matter? Determine if preliminary results justify performing a more extensive study.
  • Who needs to know, or who is the audience? Pilot studies can sometimes provide adequate preliminary results to communicate via conference presentations, targeted publications, or proposals.
  • What investment is required for additional results? Develop budgets and timelines as well as identify research collaborators.

Details and recommendations for conducting a pilot study can be found here.

What are the limitations of a pilot study?

Pilot studies are often limited because they are typically not extensive enough to provide conclusive results or because they do not examine an adequate range of sites or systems needed to develop policies or guidelines. If a pilot study shows there is an impact that needs to be examined in a given system, a more complete study or full-fledged research project can be conducted. Information on conducting a larger study can be found here, and services by the ICNet for partnering on a project or having a research study conducted for you are located here.

Additionally, pilot studies are often not publishable as full papers, though some journals have an option to publish preliminary findings as a “technical note”. Another alternative is to present preliminary findings at conferences where the results become part of the conference proceedings. A full research project still may follow.

The basics

Pilot studies that use climate models will generally only use one to three global climate models rather than a full ensemble, have limited locations, may use simplified infrastructure models, and perform rapid data analyses. The idea is that a pilot study should be limited in scope so one or a few people can conduct it in a relatively short time period (6 to 18 months is typical). Well-designed pilot studies help create networks, lay the groundwork for a proposal of a larger study, and provide a learning opportunity for the research process.

Note: Pilot studies recently funded by the FHWA on climate and infrastructure, described here, tend to be much larger scale than a typical research pilot study.

How much effort should this take?

The effort should not be overwhelming. Exactly how long your study takes, how much effort it will require, and what process you should use are heavily dependent on the specifics of your study, but you can get a good idea by searching for and reading other pilot studies that are similar to yours.

Resist the urge to find some climate model output and start running numbers. Instead, do preliminary research about the topic you are studying early, and leverage existing studies and data collection from other ICNet members where possible. It is likely that you will find a study (or several) that is similar enough to guide your research.

Climate model output, sample figures and tables, downscaled model output, and methods that are adequate for most pilot studies already exist and are publicly accessible. Therefore, unless you have very specific climate model output needs, we recommend that you take advantage of the available model output sources. At this stage, it is generally easier to simplify your engineering analysis to match available climate model output than to try to generate climate output that perfectly matches your engineering model or tool.

Caution: There are a tremendous number of global climate models (GCMs), emission scenarios, and sources for model output as well as analysis methods. Do not let your pilot study get away from you and become a larger study. You can transition what you have learned from this pilot project to a more comprehensive research experiment with a different goal and plans.