Elements: CRISPS: Cell-Centric Recursive Image Similarity Projection Searching

Materials scientists use microscopy to determine the structure, order, and periodicity that affect properties. The core challenge is that only a fraction of microscopy data is published. Thus, most of the information from these costly experiments is lost to an abyss.

CRISPS - Cell-Centric Recursive Image Similarity Projection Searching – is a computing infrastructure to make high-value materials microscopy Findable, Accessible, Interoperable, and Reproducible (FAIR). CRISPS is a multitouch, interactive research "buddy" that uses artificial intelligence to form associations like the human mind to identify similarities between images.

It has permanent, immutable recollection to search and discover collections of scientific images based on labels and associations. CRISPS will be made openly available and will be promoted at conferences, in user facilities, and online to foster a community of developers and users. Public scientific literacy will be enforced through interactive museum exhibits that use CRISPS to explore and discover materials microscopy.

The program also supports a first summer research experience for six under-represented persons in STEM.

CRISPS is a full-stack software solution for materials microscopy that seamlessly integrates three novel software concepts. 1. DataFed: a federated scientific database for collecting, collating, and searching scientific data and metadata. 2. Schema-Free Search: a tool for cell-centric indexing and searching metadata without schemas. 3.

Recursive Image Similarity Projections: a tool to interactively explore image similarity. Each of these efforts is intellectually innovative. DataFed provides an automated, secure, scalable generalized scientific data repository that supports metadata schemas, searches, and provenance graphs.

DataFed removes barriers to collaborative science through trusted authentication and secure managed file transfers using GridFTP. Schema-Free Search: an innovative index and ML-tokenization methodology to search unstructured metadata efficiently using ElasticSearch. Scientists will discover schemas and ontologies through an interactive graphical user interface (GUI).

Recursive Image Similarity Projections: a collection of deep learning models to conduct automatic symmetry-aware microscopy featurization. When coupled with manifold learning and a GUI, this software tool will enable filtering, similarity exploration, and rapid labeling of materials microscopy. Combining these tools will facilitate creative inquiry into unpublished microscopy, accelerating the discovery of new materials with novel functionalities.

CRISPS will be documented and released under a non-restrictive license which allows its reuse, modification, and commercialization. The PIs work with stakeholders in academia and industry to implement CRISPS for typical experiments in optical, electron, and scanning probe microscopies. It provides public access to a 0.5 PB allocation on a DataFed server with indexing and image similarity searching functionality through CRISPS.

Interdisciplinary concepts, including scientific data management, search ontologies, and machine learning, are being integrated into courses in materials and computer science; and will be broadly shared through the Lehigh Microscopy School and conference tutorials.

This proposal receives funds through the Office of Advanced Cyberinfrastructure in the Computer and Information Science and Engineering Directorate and the Division of Materials Research in the Mathematical and Physical Sciences Directorate.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.