CAREER: A Time-Sensitive Large Model Training Platform for Dynamic Data Analytics

Large-scale deep learning models are widely used and are critical across many fields. However, the need to adapt and refine these models to changing conditions is becoming increasingly important. Achieving real-time adaptation poses a significant challenge due to its resource-intensive nature.

This research focuses on developing methods to dynamically refine and adapt existing models, enabling real-time updates and reducing the reliance on time-consuming retraining. Such advancements are critical for scenarios where static pre-trained models often underperform, including cases with incomplete data, dynamic environments, or evolving research objectives.

These methods support a wide range of applications requiring real-time adaptability in large-scale models, such as climate modeling, real-time traffic management, digital twins of fusion energy in plasma physics, and virtual infrastructure twins of supercomputer networks. By addressing this challenge, the project not only promotes innovation but also boosts scientific research and practical applications.

This project contributes to the U.S. national goal of broadening participation in science and engineering, developing the research workforce for advanced cyberinfrastructure, promoting the innovation economy, and maintaining a leading position in international technology competitions. The developed software as a foundational tool is shared with researchers and engineers in academia, national laboratories, and industry for broader societal impacts.

Also, this project promotes K-12, undergraduate, female, and underrepresented minority populations in the science, technology, engineering, and mathematics (STEM) fields and strengthens collaboration between academia and national laboratories/industries.

The primary objective of this project is to pioneer the effort of developing a time-sensitive large model training platform for dynamic data analytics in practice by taking advantage of the world-class accelerator computing infrastructure to fine-tune trillion-parameter large models cost-efficiently. Towards this end, this project has three research thrusts and one educational thrust.

First, this research automatically generates a parallelization plan at an affordable cost to minimize training iteration latency. Second, this project progressively grows models from pre-trained small models during fine-tuning to reduce the number of training iterations. Third, this project validates the practicality of the developed platform using realistic applications in weather forecasting, fusion energy experiment control, resilient streaming event prediction in virtual infrastructure twins, and scooter-sharing demand prediction.

Fourth, this project designs integrated research and education activities, including broadening the adoption of the time-sensitive training platform, undergraduate student research advising, curriculum development, and outreach for professional development and K-12 education.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.