Skip to main content
Advanced Search

Filters: Tags: Source Code (X)

35 results (109ms)   

Filters
Date Range
Extensions
Types
Contacts
Categories
Tag Types
Tag Schemes
View Results as: JSON ATOM CSV
thumbnail
Recent open data policies of the Office of Science and Technology Policy (OSTP) and Office of Management and Budget (OMB), which were fully enforceable on October 1, 2016, require that federally funded information products (publications, etc.) be made freely available to the public, and that the underlying data on which the conclusions are based must be released. A key and relevant aspect of these policies is that data collected by USGS programs must be shared with the public, and that these data are subject to the review requirements of Fundamental Science Practices (FSP). These new policies add a substantial burden to USGS scientists and science centers; however, the upside of working towards compliance with...
thumbnail
Over the last few years, the ISO 19115 family of metadata standards has become the predominantly accepted worldwide standard for sharing information about the availability and usability of scientific datasets among researchers. The U.S. interests in the ISO standard have also been growing as global-scale science demands participation with the broader international community; however, adoption has been slow because of the complexity and rigor of the ISO metadata standards. In addition, support for the standard in current implementations has been minimal. Principal Investigator : Stan Smith, Joshua Bradley Cooperator/Partner : Chis Turner In 2009, the Alaska Data Integration Working Group members (ADIwg) mobilized...
thumbnail
Inventories of landslides and liquefaction triggered by major earthquakes are key research tools that can be used to develop and test hazard models. To eliminate redundant effort, we created a centralized and interactive repository of ground failure inventories that currently hosts 32 inventories generated by USGS and non-USGS authors and designed a pipeline for adding more as they become available. The repository consists of (1) a ScienceBase community page where the data are available for download and (2) an accompanying web application that allows users to browse and visualize the available datasets. We anticipate that easier access to these key datasets will accelerate progress in earthquake-triggered ground...
thumbnail
Deep learning is a computer analysis technique inspired by the human brain’s ability to learn. It involves several layers of artificial neural networks to learn and subsequently recognize patterns in data, forming the basis of many state-of-the-art applications from self-driving cars to drug discovery and cancer detection. Deep neural networks are capable of learning many levels of abstraction, and thus outperform many other types of automated classification algorithms. This project developed software tools, resources, and two training workshops that will allow USGS scientists to apply deep learning to remotely sensed imagery and to better understand natural hazards and habitats across the Nation. The tools and...
thumbnail
Insect pests cost billions of dollars per year globally, negatively impacting food crops and infrastructure and contributing to the spread of disease. Timely information regarding developmental stages of pests can facilitate early detection and control, increasing efficiency and effectiveness. To address this need, the USA National Phenology Network (USA-NPN) created a suite of “Pheno Forecast” map products relevant to science and management. Pheno Forecasts indicate, for a specified day, the status of the insect’s target life cycle stage in real time across the contiguous United States. These risk maps enhance decision-making and short-term planning by both natural resource managers and members of the public. ...
2012 Updates (from the FY12 Annual Review) The NWIS Web Services Snapshot represents the next generation of data retrieval and management. The newest Snapshot tool allows instant access to NWIS data from four different web services through ArcGIS, software available to all USGS scientists in all mission areas. Increased data retrieval efficiency reduces the steps required to retrieve and compile water data from multiple sites from what can be more than 30 steps to just a few clicks. As an end-user education tool, it promotes use of NWIS data from both web services and the NWIS database, which increases the production of scientific research and analysis that uses NWIS data. The Snapshot database design enables efficient...
Detailed information about past fire history is critical for understanding fire impacts and risk, as well as prioritizing conservation and fire management actions. Yet, fire history information is neither consistently nor routinely tracked by many agencies and states, especially on private lands in the Southeast. Remote sensing data products offer opportunities to do so but require additional processing to condense and facilitate their use by land managers. Here, we propose to generate fire history metrics from the Landsat Burned Area Products for the southeastern US. We will develop code for a processing pipeline that utilizes USGS high-performance computing resources, evaluate Amazon cloud computing services,...
thumbnail
Geotagged photographs have become a useful medium for recording, analyzing, and communicating Earth science phenomena. Despite their utility, many field photographs are not published or preserved in a spatial or accessible format—oftentimes because of confusion about photograph metadata, a lack of stability, or user customization in free photo sharing platforms. After receiving a request to release about 1,210 geotagged geological field photographs of the Grand Canyon region, we set out to publish and preserve the collection in the most robust (and expedient) manner possible (fig. 6). We leveraged and reworked existing metadata, JavaScript, and Python tools and developed a toolkit and proposed workflow to display...
thumbnail
Executive Summary Traditionally in the USGS, data is processed and analyzed on local researcher computers, then moved to centralized, remote computers for preservation and publishing (ScienceBase, Pubs Warehouse). This approach requires each researcher to have the necessary hardware and software for processing and analysis, and also to bring all external data required for the workflow over the internet to their local computer. To explore a more efficient and effective scientific workflow, we explored an alternate model: storing scientific data remotely, and performing data analysis and visualization close to the data, using only a local web browser as an interface. Although this environment was not a good fit...
thumbnail
Land-use researchers need the ability to rapidly compare multiple land-use scenarios over a range of spatial and temporal scales, and to visualize spatial and nonspatial data; however, land-use datasets are often distributed in the form of large tabular files and spatial files. These formats are not ideal for the way land-use researchers interact with and share these datasets. The size of these land-use datasets can quickly balloon in size. For example, land-use simulations for the Pacific Northwest, at 1-kilometer resolution, across 20 Monte Carlo realizations, can produce over 17,000 tabular and spatial outputs. A more robust management strategy is to store scenario-based, land-use datasets within a generalized...
thumbnail
USGS scientists often face computationally intensive tasks that require high-throughput computing capabilities. Several USGS facilities use HTCondor to run their computational pools but are not necessarily connected to the larger USGS pool. This project demonstrated how to connect HTCondor pools by flocking, or coordinating, within the USGS. In addition to flocking the Upper Midwest Environmental Science Center and the Wisconsin Water Science Center, we have flocked with the USGS Advanced Research Computing Yeti supercomputing cluster and other water science centers. We also developed tutorials on how to sandbox code using Docker within the USGS environment for use with high-throughput computing. A main accomplishment...
thumbnail
Large amounts of data are being generated that require hours, days, or even weeks to analyze using traditional computing resources. Innovative solutions must be implemented to analyze the data in a reasonable timeframe. The program HTCondor (https://research.cs.wisc.edu/htcondor/) takes advantage of the processing capacity of individual desktop computers and dedicated computing resources as a single, unified pool. This unified pool of computing resources allows HTCondor to quickly process large amounts of data by breaking the data into smaller tasks distributed across many computers. This project team implemented HTCondor at the USGS Upper Midwest Environmental Sciences Center (UMESC) to leverage existing computing...
thumbnail
Science is an increasingly collaborative endeavor. In an era of Web-enabled research, new tools reduce barriers to collaboration across traditional geographic and disciplinary divides and improve the quality and efficiency of science. Collaborative online code management has moved project collaboration from a manual process of email and thumb drives into a traceable, streamlined system where code can move directly from the command-line onto the Web for discussion, sharing, and open contributions. Within the USGS, however, data have no such analogous system. To bring data collaboration and sharing within the USGS to the next level, we are missing crucial components. The sbtools project team built sbtools, an R interface...
thumbnail
Access to up-to-date geospatial data is critical when responding to natural hazards-related crises, such as volcanic eruptions. To address the need to reliably provide access to near real-time USGS datasets, we developed a process to allow data managers within the USGS Volcano Hazard Program to programmatically publish geospatial webservices to a cloud-based instance of GeoServer hosted on Amazon Web Services (AWS), using ScienceBase. To accomplish this, we developed a new process in the ScienceBase application, added new functionality to the ScienceBase Python library (sciencebasepy), and assembled a functioning Python workflow demonstrating how users can gather data from a web API and publish these data as a cloud-based...
thumbnail
Web portals are one of the principal ways geospatial information can be communicated to the public. A few prominent USGS examples are the Geo Data Portal (http://cida.usgs.gov/gdp/ [URL is accessible with Google Chrome]), EarthExplorer (http://earthexplorer.usgs.gov/), the former Derived Downscaled Climate Projection Portal, the Alaska Portal Map (http://alaska.usgs.gov/portal/), the Coastal Change Hazards Portal (http://marine.usgs.gov/coastalchangehazardsportal/), and The National Map (http://nationalmap.gov/). Currently, web portals are developed at relatively high effort and cost, with web developers working with highly skilled data specialists on custom solutions that meet user needs. To address this issue,...
thumbnail
U.S. Geological Survey (USGS) scientists are at the forefront of research that is critical for decision-making, particularly through the development of models (Bayesian networks, or BNs) that forecast coastal change. The utility of these tools outside the scientific community has been limited because they rely on expensive, technical software and a moderate understanding of statistical analyses. We proposed to convert one of our models from proprietary to freely available open-source software, resulting in a portable interactive web-interface. The resulting product will serve as a prototype to demonstrate how interdisciplinary USGS science and models can be transformed into an approachable format for decision-makers....
thumbnail
A unique opportunity for USGS to collaborate with IRIS-PASSCAL (the national seismic instrument facility) has presented itself to develop a geophysical data archive format that follows FAIR principles. IRIS-PASSCAL is extending facility to include magnetotelluric (MT) instruments prescribing the need for them to archive collected MT data by extending their existing protocol. Concurrently, Congress has mandated the USGS to collect nationwide MT data (5000 stations) which will all need to be archived under FAIR principles. In collaboration with IRIS-PASSCAL, we propose to develop a generalized HDF5 format for archiving MT data which can easily be extended to other geophysical data in the future. This project will...
Wildfires are increasing across the western U.S., causing damage to ecosystems and communities. Addressing the fire problem requires understanding the trends and drivers of fire, yet most fire data is limited only to recent decades. Tree-ring fire scars provide fire records spanning 300-500 years, yet these data are largely inaccessible to potential users. Our project will deliver the newly compiled North American Fire Scar Network — 2,592 sites, 35,602 trees, and > 300,000 fire records — to fire scientists, managers and the public through an online application that will provide tools to explore, visualize, and analyze fire history data. The app will provide raw and derived data products, graphics, statistical summaries,...
thumbnail
Computational models are important tools that aid process understanding, hypothesis testing, and data interpretation. The ability to easily couple models from various domains such as, surface-water and groundwater, to form integrated models will aid studies in water resources. This project investigates the use of the Community Surface Dynamics Modeling System (CSDMS) Modeling Framework (CMF) to couple existing USGS hydrologic models into integrated models. The CMF provides a Basic Model Interface (BMI), in a range of common computer languages, that enables model coupling. In addition, the CMF also provides a Python wrapper for any model that adopts the BMI. In this project the Precipitation-Runoff Modeling...
thumbnail
This project team developed a Web-hosted application (that can also be used on mobile platforms) for automatic analysis of images of sediment for grain-size distribution, using the “Digital Grain Size” (DGS) algorithm of Buscombe (2013) (“DGS-Online,” 2015). This is a free, browser-based application for accurately estimating the grain-size distribution of sediment in digital images without any manual intervention or even calibration. It uses the statistical algorithm of Buscombe (2013) that estimates particle size directly from the spatial distribution of light intensity within the image. The application is designed to batch-process tens to thousands of images, utilizing cloud computing storage and processing technologies....