High Throughput Computing: A Solution for Scientific Analysis

• High Throughput Computing
• Home
• HTC Computing Times

• HTC Research Applications
• Maximum Entropy
• 2D/3D Hydrodynamic Numerical Modeling
• Species Trend Analysis and the Effects of their Surrounding Environment
• Bioclimatic Variables and Continuous Surface Drought Indices
• Landscape-scale Ecological Modeling
• Geostatistical Simulations and Statistical Analysis

• For More Information Contact
• Catherine Jarnevich
• Jeff Morisette

Maximum Entropy (MaxEnt)

Background

The Fort Collins Science Center (FORT) has developed the Resource for Advanced Modeling (RAM) to provide a collaborative working environment for researchers and resource managers to address pressing natural resource problems, primarily related to invasive species. The RAM is used to produce habitat suitability models of current and forecasted species distributions to inform management decisions. FORT researchers in the RAM use a variety of techniques, including a machine-learning technique called MaxEnt (Phillips et al. 2006) that has consistently performed well in different comparison studies of habitat suitability models. 

Problem

In 2010 alone, seven different groups of scientists have visited the RAM for intensive modeling sessions using MaxEnt. Each group comprises experts on a given species for a focus location, such as invasive weeds in Alaska. During these sessions the goal is to use an iterative modeling process, where we run models, examine results, and re-run models with modified parameters as determined by the FORT modeling team and visiting species experts.  However, in some cases computations for a single model can require more than 12 hours to run. This lengthy run time makes iterative modeling difficult and diminishes the utility of the visiting experts’ intensive but time-constrained modeling session.

Solution

When we use MaxEnt in the RAM, we employ a cross-validation approach that requires multiple iterations of the same model. These iterations can take advantage of HTCondor¹, where different workstations can run iterations simultaneously. We developed a graphical user interface (GUI) between HTCondor and MaxEnt to provide all the parameters that are allowed by MaxEnt for user inputs. This application has the same look and feel as MaxEnt but allows jobs to be submitted to HTCondor without requiring user knowledge of HTCondor. It then returns all HTML graphic results and probability results generated by MaxEnt (Figure 1). Once all the iterations are completed, a single core can create the final summary. By developing a way to run MaxEnt using HTC, the computing time for the many needed iterations is reduced considerably. For example, when we examined Africanized honey bees in the United States, MaxEnt took 6.27 hours to run without HTCondor and 1.5 hours to run with HTCondor (see HTC Computing Times). This decrease in processing time allows model runs to be completed in a short enough period to be processed and acted upon within the short timeframe of a RAM modeling session.

Figure 1. Screen captures representing the HTCondor MaxEnt GUI and MaxEnt output generated by Steven Phillips’ software (MaxEnt).

References

• Phillips, S.J., R.P. Anderson, and R.E. Schapire. 2006. Maximum entropy modeling of species geographic distributions. Ecological Modelling 190:231-259.

¹The use of any trade, product, or firm names is for descriptive purposes only and does not imply endorsement by the U.S. Government.