for Engineers and Researchers
- Starting a Study
- Background Research
- Project Scope
- Climate Model Output
- Baseline Analysis
- Future Analysis
- Communication of Findings
This section describes the preliminary administrative steps required to begin a research study.
Pick a Question
The scientific question that you want to answer or the hypothesis that you want to test must be specific enough to narrow down variables, a timeframe, and a location(s) in order to determine a conclusive answer.
Example: How will climate change alter the freezing and thawing of roads in Madison, Maine, in the mid century and end of century as compared to present day? When will the winter weight premium thresholds no longer be exceeded in a typical winter for roads in Madison, Maine?
Acquire a Team
Assemble a team of interdisciplinary specialists and/or students to help you work on the study. This team may involve those in academia, practice, or some combination.Designate team member roles.
File Storage System
Choose a file storage system for your team to compile all of the documents and datasets necessary for and created during the study.
Ideally, the system should:
- Allow everyone to be able to view and work on files
- Meet immediate and future storage space requirements
- Be economically feasible
- Be easily accessible and useable
Some examples of file storage systems include:
- Cloud Storage: DropBox, Box, and GoogleDrive
- Agency, Company or University storage systems
This section describes the background research needed before starting the planning and analysis stages of a study. Background research on both the subject matter and on the specific study or project approach is necessary.
Research the subjects and concepts you need to understand prior to data collection. Expect to find an overwhelming number of climate change studies, but relatively few studies specific to climate change and transportation infrastructure. Rely on refereed papers that have been peer-reviewed and agency reports on the topics. Refereed papers can be found via Web of Science, JSTOR, Google Scholar, and the TRB database.
Some of the topics you might want to investigate could include:
- Global Climate Change
- Why do we need climate projections?
- How is climate change likely to affect my study region?
- Why are climate projections uncertain?
- What is downscaling and when do I need it?
- Infrastructure and Environmental Forcings
- What environmental factors affect my system?
- What models are available to link system performance and environmental variables?
- Infrastructure and Climate Change
- What are the anticipated impacts?
- How vulnerable is the asset?
- What are the adaptation approaches?
Research similar studies to focus your research questions and to ensure that you are building on previous research. To do this, conduct a literature review of peer-reviewed papers. In addition to searching the transportation sector, consider looking at methods applied by other sectors with a longer historical of climate change analyses including water resources, agriculture, and energy.When reviewing the articles, focus on the:
- Environmental variables
- Climate models, emissions scenarios, and downscaling
- Data collection and analysis methods
- Processes that could be modified for your study
Compile your findings and use this research to decide on the following components of your study:
- Source of data output
- Climate and Engineering models you will use
- Emissions scenario(s)
- Timeframe of data/output
- Variables of interest
- Site(s) of interest
Develop a reference database at this early stage to track your references. Consider using a reference database tool to help track citations such as EndNote, Mendeley, and Zotero.
This section reviews the key steps in creating and finalizing your project scope. It is important to complete a consultation with specialists who have the knowledge and experience with the climate research process and are able to provide input on your plan.
Scope of Work
Create a detailed project scope that outlines the tasks required to accomplish your goal. Include the critical tasks and milestones. Break each main task into subtasks. Create a timeline for the project that includes all of the critical tasks. Divide the tasks among members of the team. Assign a lead for each task, even if it is a team task.
If there are gaps in your team, consider reaching out to the ICNet climate scientists and infrastructure engineers for feedback. Ask for their perspectives on your:
- Research scope
- Planned source of model output
- Method for synthesizing results
In addition to climate and sea level rise experts, ICNet members have experience analyzing pavements, bridges, ports, tunnels, and hydrology/hydraulics for design as well as operations and maintenance. Members also have experience and expertise in top-down and bottom-up analysis methods, uncertainty/risk analysis, adaptation approaches, and communicating climate change results. The overall goal of the consultation step is to refine your research process and target goals.
Make revisions to your research plan or project scope based on the input you receive.
- Scope of work example
- Contact list of ICNet Climate Scientists and Engineers
- Summary of findings from ICNet Climate Scientist discussion
This section covers the collection of output from your desired source. If your team chooses to use the Bureau of Reclamation site, as recommended in the Federal Highway Agency (FHWA) guides, refer to the detailed instructions on how to gather those data provided on the ICNet and the FHWA websites.
Desired Variables, Resolution, Geographic Area, and Timeframe
Decide on the following:
- The variables you want to investigate (e.g., temperature, precipitation, etc.)
- The time and space resolution for your climate model output if using gridded output, or whether station information is more desirable for the study.
- The geographic area or the individual station sites of the study (coordinates)
- The timeframes for the analysis including
- Baseline: historic period for which you have both climate model output (later referred to as baseline output) and documented observations (later referred to as observed data) of your environmental variables (e.g., temperature data from 1950 – Present)
- Future: the period you are interested in studying (e.g., temperature data from Present – 2100)
When considering possible climate projections for the future, one source of uncertainty is the future rate of human emissions. Social and climate scientists have developed different categories of potential future emissions rates called emissions scenarios. These scenarios are based on combinations of varying levels of economic growth, population growth, energy usage, and technological development.
When choosing emissions scenarios for research, it is important to show or incorporate the variability of your projections. For this reason, most researchers choose at least two emissions scenarios, one from each extreme. However, fewer researchers are including the lowest category of greenhouse gas concentrations (RCP 2.6), as current rates are already higher than that from RCP 2.6 projected trajectory.
Source of Output
Deciding where to obtain climate model output is based on the emissions scenarios, variables, resolution, geographic area, and timeframe you choose, and whether or not you want downscaled data. See the “Downscale is Needed” step to learn about downscaling and why you might want to include it in your research. Not all models provide output for every emissions scenario, so this step can be based on the emissions scenario decision. This choice is based on whether model output source or emissions scenario is more important to your study.
The ICNet has a spreadsheet tool that summarizes some of the leading available climate model output sources. See the Summary Sheet in the resources tab below.
Climate Models and Model Runs
Once the decision on which climate model output source has been made, your team must decide which models to use from that source. Climate models are computer-based models that use physics, chemistry, and hydrology to simulate the climate system. Most websites that host climate model outputs have more than one climate model.When conducting climate change research for the infrastructure sector, the rule of thumb is to include as many different climate models as possible. Different models have better accuracy in different components or regions and worse accuracy in others, resulting in variability among the models. Using as many models as possible includes that variability and leads to more robust results. Eliminating models takes away some of the true variability of your study’s results. For this reason, the ideal choice is to use all of the models available through your output source.
Time and budget constraints often prevent using all models available. To help guide the choice of which models to use in these cases ICNet members have developed a new tool. This tool classifies the models provided by every source of climate model output based on their reliability and usefulness. For more information, see the “Summary Sheet” in the resources tab below.
Another aspect to consider is model “runs”. One simulation from a model is considered one “run” of the model. Due to time constraints, many climate scientists feel one model run is sufficient. However models contain their own internal variability. Using more than one run of the model(s) can help compensate for this variability. The most accurate results would be produced by selecting as many model runs as possible for all of the available models. Consider obtaining multiple model runs as appropriate for your available time and budget.
Translation to Engineering Variables
The information that comes out of a climate model is referred to as “output” rather than data because, though the projections are based on scientific processes, they are not concrete, real-life observations. Although the term “output” is used, this is not necessarily the final product researchers need. The output from the climate models is a variable that affects a study, but the “output” can then be taken and used as an input into other models or mathematical formulas that produce the final output of a study. This new output would be the main variable that is being measured in the study. Review the climate model temperature output in the freeze-thaw example study.
Downscale if Needed
The climate models described in the spreadsheet are Global Climate Models (GCMs), which means the entire globe is modeled. Due to the limitation of computational resources when running a model for the entire globe for a set time horizon and time resolution (e.g. 3-hour intervals for 200 years), the spatial resolution is often not adequate for researchers who work at a smaller scale (e.g., the local or regional level). Downscaling is a way to translate the global model information to higher-resolution information relevant to local studies. There are two types of downscaling: statistical and dynamical. Statistical downscaling uses statistics to relate global variables to local/regional variables. Dynamical downscaling runs a regional model with a finer resolution using GCM output as boundary conditions. Some of the sources provide output that is already downscaled, but it is also possible to downscale or correct biases from the already-downscaled output. This is not a trivial task – your team will need expertise in this area. For more information on downscaling, see the ICNet webinars on statistical and dynamical downscaling.
A baseline analysis (section 5) can help to determine if further downscaling is required.
Remember, models are not designed to match the day-to-day variability of the observations, but the overall statistics, such as mean, standard deviation, trend, etc. should be similar over a twenty to thirty year period.
Set aside time to review all of your climate model output decisions. Changes may need to be made to get the best quality output for your research. For example, not all emissions scenarios are used by every model. If a particular emissions scenario is important to the study, then it may be necessary to change your source selection. Consider your time and resources. Estimate the actual amount of data that your studies will yield and compare it with your scope of work and timeline to ensure that the study is logistically feasible.
Follow the instructions provided by your chosen source to input your decisions and collect output. If you are using CMIP3 or CMIP5 data from the Bureau of Reclamation site, review the ICNet’s instruction guide for data collection and data processing without using the interface.
This section covers how to run and collect output from the “baseline” or “historical” section of your timeframe. It also reviews how to conduct a baseline analysis to assure the selected models are replicating historical conditions and will produce credible information for future analyses.
If you are running the climate model output through an engineering model, the baseline analysis should run both the observed data and the baseline climate model output through that model to produce the desired variables. If you are not running the output through your own model, use the observed and baseline climate variables (e.g., precipitation or temperature).
The goal of the baseline evaluation is to determine the goodness of fit of the climate model output as compared to historical observations. A baseline analysis should be a relatively quick and simple statistical goodness of fit test. We recommend looking at the output using visual tools such as box plots and histograms/probability density functions (in PDF format) that compare the observed data to the model output for the baseline period. When analyzing the baseline output, use a 20-30 year window.
The models will most likely show similar results to the observed data. In this case, use all of the models in order to incorporate the overall variability of the models.
If any of the models produces results that are drastically different than the observed, consider removing those models from your analysis.If many of the models differ, then your output likely needs to be downscaled further to better reflect the climate in that area. This typically occurs if you are using GCM model output or using extreme values (e.g., annual maximums). Before proceeding to your future analysis, you should return to Step 4.
This section outlines how to conduct the “future” portion of your research. This methodology follows a “top-down” approach to climate and infrastructure research, but other methods including the “bottom-up” approach are also available. The top-down approach is applicable when considering how a variable changes over time. The bottom-up approach is applicable when attempting to characterize the vulnerability of an asset. It typically examines where or when a parameter of interest exceeds a particular threshold in a larger region.
The future analysis should use the same climate models that were identified as reasonable in the baseline analysis. Run the time series or summary parameters of the climate model output through the engineering model. Each combination of climate model, emission scenario, and model run should be analyzed individually initially. If you are not running the output through another model, just compile the future output based on your final model selection.
Compare the model output for the future period to the observed data from the baseline period. Comparisons should include graphical methods (e.g., time series plots, box plots) for the entire future period or for 20-30 year periods from mid-Century to end-Century. Note how the variables change over time and check to see if the model produces results that appear to be reasonable. Be warned: if the initial results are surprising, verify that the analysis was performed correctly, but it does not necessarily mean there is something wrong with the model. If your engineering model is fairly complicated, it might be useful to produce comparisons of intermediate parameters (e.g., temperature and precipitation) to understand how these values are likely to change over time.
During the evaluation process, questions often arise about when it is and is not permissible to combine results from multiple models or runs during analysis. It is never permissible to average output from the climate model before running your engineering model because averaging will remove the extremes from the climate parameters. However, it is permissible to average the final variables after the output from the climate model has been run through your own engineering model, and the final results have been reached.
Create an Ensemble of Model Results
After completing all calculations/analysis for each run individually, each separate GCM with multiple runs may be averaged. This will give you one result for each GCM. The output from all the GCM’s may then be combined but keep different emission scenarios separate. This is very important; if models are averaged before doing the analysis, all the extremes will be averaged out since they might not (and probably will not) happen on the same calendar day.
Most climate change studies report changes using the mean and variability across models known as the multi-model ensemble. The IPCC states, “The reason to focus on the multi-model mean is that averages across structurally different models empirically show better large-scale agreement with observations, because individual model biases tend to cancel.” In practice, this means that variation among the models is to be expected and that studies should report findings based on ensembles of models rather than a single model.
It is advisable to include figures when reporting your climate change research results that show how variables have changed from the past to a future time period of interest. However, it is not accurate to represent this trend with the output from a single model.
An ensemble of the output from each model compared to the historic observations can be depicted on a single graph. Typically the graphs include some measure of the central tendency (e.g., median or mean) as well as a measure of the variability or range across models. Some examples of the same information that is presented in multiple formats are provided in the Freeze-Thaw case studies. The layout of the ensemble is fairly subjective as long as it depicts the overall trend while considering information from as many models as possible.
- Freeze Thaw future output comparison to observed (appendix)
- IPCC on climate models
- Does climate adaptation policy need probabilities?
This section outlines recommendations for communicating your research results. Regardless of the findings, the results from your research as well as information about your research process will add to the relatively limited information available on conducting climate change and infrastructure research or understanding infrastructure vulnerability.
Depict your desired final parameters visually. These visuals may include line graphs, bar charts, and box plots. Choose the best graph to display each category of the parameters. For example, magnitude is usually depicted as line graphs and bar graphs whereas dates are usually depicted with line graphs and box plots. For site specific plots from ICNet pilot studies, computer codes and instructions are available to help you create some great visuals with your own data sets. There are other online tools to help you create and experiment with different visuals.
The technical communication should document your research questions, methods, and findings. For the methods, create a table listing the climate models, emission scenarios, and model runs used in your study as well as the source of the climate model output. Document the process used for the model output (e.g., models were removed from the study, was additional downscaling applied). Briefly depict your baseline analysis results. For your future results, choose up to four or five of the visuals that best depict your overall findings.
The ICNet Research Page
ICNet maintains a database of pilot studies and offers a means to communicate your findings and to help others find your work. Even if you do not anticipate writing a paper based on your findings or the results of your study are inconclusive, a summary of findings may still be useful. Please contact us for more information at [email protected]
- On how to make graphs readable: Visual Analysis Guidebook
- On prose and visualizations: Visual Communication—Document Design, Figures, and Tables
- On visualization: Graphical Display for Effective Communication of Research Results
- The ICNet Guide to Climate Communication
- Guide to creating visuals in Plot.ly
- Submit your research to [email protected]
- Case Study Research Report (coming soon)