NSF POSE: Phase II: Establishing an Observatory-Driven Sustainability Path for PypeIt, an Open-Source Ecosystem (OSE) for Astronomical Spectroscopy

Spectroscopy is a fundamental tool astronomers use to understand the physical Universe, allowing scientists to measure, for example, the physical state and motions of intergalactic gas and the demographics of stellar populations in distant galaxies. State-of-the-art instrumentation continually improves the ability to collect these data; however, equally important are sophisticated software packages that perform the careful processing of raw data into calibrated spectra ready for scientific analysis.

The PypeIt (pronounced "pipe it") software package has emerged as a comprehensive data-processing product that leverages modern, industry-driven support infrastructure for open-source code development. This NSF award enables PypeIt to expand into a full-fledged open-source ecosystem (OSE) that will support nearly all medium- and large-class telescopes owned and operated by institutions in the United States.

The wide reach of this effort reduces barriers to processing data from both new and existing spectrographs, improving the scientific productivity of both experienced astronomers and those relatively new to processing spectroscopic data. The OSE also introduces PypeIt support for commercially available spectrographs used by amateur astronomers, enabling them to quickly produce calibrated, science-ready spectra from their observations.

Although initially developed by a small team at the University of California Observatories for instruments at the Lick and W. M. Keck Observatories, PypeIt now supports processing data from more than 50 spectrographs at 18 observatories worldwide.

Its success can be attributed to two of its guiding principles: (1) focus low-level algorithms on the commonalities of spectroscopic data process such that they can be cleanly separated from the higher-level specifications of each spectrograph; and (2) employ state-of-the-art algorithms that ensure systematic errors are negligible compared to the fundamental noise limit set by Poisson counting statistics. Building from these core principles, this NSF award enables PypeIt to reach its full potential as a community-wide OSE, meeting a critical need outlined by the 2020 Decadal Survey of Astronomy and Astrophysics.

This is achieved by four primary activities: (1) strategically broadening the current PypeIt developer base to include key staff at seven US observatories (Lick, WMKO, Palomar, MMT, LBT, Lowell, and APO) by directly funding them to enable PypeIt to support processing data from their instruments; (2) building partnerships with additional observatories both, in the U.S. and abroad, by supporting open-source development by the worldwide astronomical community and by hosting bi-annual workshops that introduce new users to PypeIt and provide real-time collaboration between users and developers; (3) implementing a new community-driven governance structure for the project that improves prioritization of the development activities and is more broadly inclusive of all stakeholders (developers, observatory staff, and users); and (4) extending existing, but observatory-specific, tools that facilitate real-time decision making for observers at the telescope (key to improving observing efficiency) and that streamline long-term data archiving (a key priority of the astronomy community) to new observatories.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.