In modern-day “Big Data” science, the diversity of data (unstructured, uncurated, and of different formats) and software provides major challenges for scientific research, especially with the reproducibility of results. The NSF DIBBs Brown Dog Project[1] aims to build cyberinfrastructure to aid autocuration, indexing, and search of unstructured and uncurated digital data. It is focusing on an initial set of science use cases (green infrastructure, critical zone studies, ecology, social science) to guide the overarching design, with user-accessible extensibility as an important driving requirement for the project development. The Brown Dog is composed of two highly extensible services, Data Access Proxy (DAP) and Data Tilling Service (DTS). These services aim to leverage/reuse any existing pieces of code, libraries, services, or standalone software (past or present), accessible through an easy-to-use and programmable interface. DAP focuses on file format conversions; DTS does content based analysis/extraction on/from a file. These services wrap relevant conversion and extraction operations within arbitrary software with reusability purpose, manage their deployment in an elastic manner, and manage job execution from behind a deliberately compact REST API. Underpinning these core services are the foundational tools, which do the actual work of conversion or extraction. These tools are integrated into the Brown Dog services via a Brown Dog Tools Catalogue.
This tutorial aims to give the attendee the knowledge to be able to add a tool to the Brown Dog Tools Catalogue, and to be able to integrate Brown Dog capabilities into an application via the API.
Specifically, the two components to the tutorial will cover:
* Adding a conversion or extraction tool to Tools Catalogue - the user creates a Brown Dog wrapper script around a 3rd party tool exposing some data transformation functionality within that tool. We will walk attendees through the process of creating and adding the tool to the Tools Catalogue.
* Utilizing Brown Dog transformation services, through the brown dog API, through a new or pre-existing application.
We will provide attendees with stub code for integration of a tool into the Tools Catalogue, as well as a stub client application that will be used as an example of how to program against Brown Dog.
