Collaborative Research: Elements: Towards A Scalable Infrastructure for Archival and Reproducible Scientific Visualizations

Today’s science revolves around leading edge datasets – data that scientists need to carefully analyze so that they can draw reliable scientific conclusions. The rate at which these leading-edge datasets are becoming larger and more complex is accelerating every day. In many ways, having access to a dataset does not equal to, or even come close to, having access to the insights in the dataset.

This nuanced but crucial difference in accessibility creates a deep barrier to making scientific results reproducible. To this end, “Accessible Reproducible Research”, published by Science in 2010, presented a system for reproducible research. A decade later, unfortunately, accessible reproducible research is still in its infancy.

It turns out that this barrier is much more fundamental than previously believed, even though on the surface it seems solvable by investing resources and setting guidelines and policies. The real challenge is that the computing toolsets, the working environments, and the work processes of the original team of scientists are very difficult for a different team of scientists to recreate with precision.

Such difficulty stems from the rapid speed at which computing technology is advancing; so that freezing a computing environment in a practical manner is nearly impossible. In addition, scientific intuition is difficult to codify, simply documenting a new idea is not enough to communicate what a scientist saw before pursuing that idea. From that respect, making accessible reproducible research a reality requires better methods and tools.

In this project, the investigators will focus on the visualization step of data analysis, which is a central component of scientific discovery. This project’s aim is to develop an Archiving Infrastructure for Reproducible Interactive Visualization (AIRIV). Through this infrastructure, the investigators will demonstrate how visual explorations of large and complex data can be reliably captured, efficiently stored, easily shared, and freely reused by any user.

This project will improve accessibility of reproducible research and promote the progress of science. For areas such as medicine and pharmaceutical research, this project will provide an unprecedented channel to accelerate translational research and advance the national health.

This project will build upon research funded by a prior NSF CISE Research Infrastructure award. In that previous project, the investigators found a method to capture interactive user experience of visualization tools, and to share the captured experience without the need to share the original software or the original data. Furthermore, during the reuse of a captured experience, the user has freedom to explore beyond the exact sequence of how the previous user has used the tool with a method called Loom.

In this new project to create AIRIV, the investigators will focus on web-based visualization dashboards, which represent the standard way for scientists around the world to interact with their data and derive insights. This project will first build a general AIRIV Javascript library that can be imported by any web browser-based application. Using the AIRIV library, developers of web-based visual dashboards can easily implement automatic generation of Loom objects into their dashboards.

Developers will be able to instrument their applications to store new provenance information with Loom objects as well. The investigators will then conduct performance and scaling tests to understand the tradeoffs between hosting choices under settings of local, institutional clusters, and community shared data infrastructures. Operators of scientific facilities can use the findings to help science communities make informed choices as to where and how to host scientific visualization archives for better share-ability and cost efficiency.

The investigators will also develop machine learning methods that can compare Loom objects and externalize commonalities and patterns in an entire archive of Loom objects. Such new methods will lead to creating a search by example functionality for AIRIV archives. For requirements collection, continuous improvement, and deployment testing, the investigators will engage the Mayo Clinic & Illinois Alliance, which serves as a framework for several technologies in healthcare, many of which center around the research and development of dashboard/analytical tools.

We target two such analytics efforts, OmiX and KnowEnG, both of which are developed at National Center for Supercomputing Applications (NCSA).

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.