Collaborative Research: EarthCube Capabilities: ICESpark: An Open-Source Big Data Platform for Science Discoveries in the New Arctic and Beyond

The Arctic climate system is undergoing rapid change with rising air and sea surface temperatures, accompanied by declines in Arctic glaciers, sea ice and snow cover on land. Increases in global air temperatures and ice-sheet mass loss are driving sea level rise around the globe. Meanwhile, as the Arctic melts, maritime and commercial activities in the region are expanding, presenting new opportunities, as well as societal and cultural challenges.

As Arctic regions are largely inaccessible to traditional observation techniques, satellite remote sensing systems play a key role in monitoring their essential climate variables. However, the unprecedented volume and variety of geospatial big data collected by new satellites have reached far beyond the capacity of computing platforms accessible to most geoscientists.

This gap between data growth and data discovery capacity significantly undermines the value of emerging big datasets. Moreover, most existing software for geospatial big data do not offer advanced analytical capabilities to facilitate geoscience discoveries. The project aims to remove these barriers by developing a low-cost and large-scale system, namely ICESpark, to seamlessly support the lifecycle of big data enabled geoscience research in the New Arctic and beyond.

The results may improve the well-being of citizens by addressing key climate change questions, including extreme events, natural disasters, sea-level rise, drought, and wildfires. It will also improve science and engineering education via development of new course materials, cross-training of students from computing and geosciences fields, as well as an ICESpark webinar series.

ICESpark is a distributed platform that can combine local commodity computers into a powerful environment that is ready for geospatial big data (GeoBD). Built on Apache Sedona, ICESpark first develops data integration and cleaning tools to harness a wide variety of GeoBD across geoscience domains including oceanography, cryospheric science and ecology.

Moreover, ICESpark provides a scalable data discovery layer to efficiently identify all coincidental data across streams from heterogeneous sensing platforms (e.g., ICESat-2, Jason-3, Sentinel-3, GEDI) under various conditions. Third, ICESpark offers advanced data analytics capabilities, including a geo-feature identification system and a geo-pattern mining package, to equip geoscientists with geophysical or statistical tools to examine complex relationships and patterns embedded in GeoBD.

To enhance research infrastructure, ICESpark will provide a variety of pre-packed front-ends including Jupyter notebooks as well as interoperation with EarthCube’s QGreenland, improving the accessibility to the system across broad disciplinary communities. The system will also be open-sourced and follow the EarthCube GeoCODES Dataset schema for long-term sustainability.

The multidisciplinary team will work together on the design and development of ICESpark and optimize it to harness GeoBD and tackle challenging geoscience problems.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.