III:Medium:Physics-guided Machine Learning for Predicting Cell Trajectories, Shapes, and Interactions in Complex Dynamic Environments

As advances in deep learning continue to revolutionize the field of computer vision, it is now possible for machine learning methods to predict the future trajectory and behavior of moving objects in benchmark problems such as pedestrian and vehicle tracking. Despite these developments, current standards in deep learning for predicting future trajectories of objects mostly assume the background to be static and the shapes of the objects to be invariant to motion.

However, in many real-world applications, we routinely encounter problems where the background environment is constantly changing its structure, which in-turn directly affects changes in shape, appearance, and future trajectory of the moving objects. For example, in the area of mechanobiology—the field of study of movements of living cells—cells undergo massive transformations in their shape, size, and trajectory as they move across fibrous environments in the human body, continuously tugging or pushing on the background fibers and remodeling the background environment in the process.

This project aims to develop novel machine learning methods to study the interplay between changes in cell shapes and background environments using microscopy imaging data and scientific knowledge of the physics of forces exerted by the cells on the background environments. Our ultimate objective is to discover the rules of cell behavior under varying background configurations and use these rules to predict future movements of cells in a number of scientific and societally relevant applications such as the study of embryo development, wound closure, immune response, and cancer metastasis.

One of the long-standing goals of artificial intelligence has been to teach machines how to predict or forecast the future. With advances in deep learning, it is now possible for machine learning (ML) frameworks to make predictions in several computer vision applications. We ask the question: can deep learning methods extract the rules of motion of dynamic “shape-shifting” objects—that are constantly adapting their appearance in relation to their environment—and use these rules to predict their future behavior?

We investigate this question in the context of a motivation application in mechanobiology to predict and explain how cells move, interact with each other, remodel their environment, and adapt their appearance with changing physiological environments inside our body. Despite the success of deep learning in predicting human motion and vehicle trajectories, fundamental gaps remain in the ability of these methods to predict the dynamics of cell motion in complex realistic environments.

This is primarily due to the highly dynamic nature of cell shapes that undergo limitless transformations as they sense and react to their environment during motion. In addition, the dynamics of cell motion is constrained by the physics of forces exerted by the cells on the background environment, as well as the complex nature of cell-cell interactions.

The vision of this project is to develop a novel physics-guided machine learning (PGML) framework to predict the motion of shape-shifting objects in dynamic physical environments. Our framework fully leverages the principles of “convergence research” by integrating data, knowledge, and methodologies from three different disciplines: machine learning, experimental cell imaging, and computational modeling.

The ultimate goal of our project is to catalyze the discovery of new “rules of cell behavior” by analyzing explainable theories produced by our PGML framework in the context of mechanobiology.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.