Flocks of a feather dock together: Using Docker and HTCondor to link high-throughput computing across the USGS

USGS scientists often face computationally intensive tasks that require high-throughput computing capabilities. Several USGS facilities use HTCondor to run their computational pools but are not necessarily connected to the larger USGS pool. This project demonstrated how to connect HTCondor pools by flocking, or coordinating, within the USGS. In addition to flocking the Upper Midwest Environmental Science Center and the Wisconsin Water Science Center, we have flocked with the USGS Advanced Research Computing Yeti supercomputing cluster and other water science centers. We also developed tutorials on how to sandbox code using Docker within the USGS environment for use with high-throughput computing. A main accomplishment...

Categories: Project; Tags: 2017, All Funded Projects, Completed, Computational Tools and Services, Docker, All tags... HTCondor, High-Throughput Computing (HTC), Software, Source Code, Fewer tags