Reading Minds and Machines

Can we decode the training data of a Deep Neural Network (DNN) directly from its parameters?Training data of DNNs are assumed safe. Recent findings by us and by others indicate that this is not the case, with severe implications on Data-Privacy.

Yet, such findings shed light on why DNNs perform so well.On a different front: Can we decode what a person saw/heard/thinks directly from their brain activity?This may have huge benefits: communicate with “locked-in” patients, explore dreams, man-machine interfaces, enhance our understanding of the human brain.

No risk of violating human privacy here, as thoughts do not “float” in the air and person’s collaboration is required.Each of those 2 questions is intriguing on its own, with far-reaching implications. Despite the inherent differences between Human Brains & DNNs, they also have much in common.

Exploring the two in-tandem can lead to significant breakthroughs in both fields.

Recent advancements in both areas, with recent incorporation of Deep-Learning (DL) tools to analyze brain activity, opens the door to explore the two jointly.

Our expertise in both domains will enable explicit Encoding/Decoding between Brain activity & DNN activations, allowing to directly learn/infer from one about the other. Initial explorations indicate that our proposed goals, although ambitious, are within reach.

Our intermediate goals in each domain are worthwhile on their own, forming a strong safety net.Expected outcomes include:•Deep Data-Privacy•Insights on DNNs, their Generalization & Vulnerabilities•Insights on of “what is encoded where” in the brain•New scientific tools for brain-scientists to explore the brain •Allow “locked-in” (ALS) patients to communicate their thoughts/needs•Use Brain scanning to improve DNNs & DNNs to improve Brain scanningOur project requires no human subjects nor BrainScience expertise; only publicly available datasets.

All methods lie in Computer-Vision & DL, with impact on both DL & BrainScience.