Resource Center

Each of us has 43 muscles in our face, and we use those muscles to create 10,000 unique combinations of expressions — all of which subtly convey important information to other people during conversations, explains computer scientist Ehsan Hoque. Our physical gestures, language, and tone of voice add further layers of complexity.

“Anthropologist Edward Sapir said that when we speak to each other, there is a secret code that we all understand and comply with, but we don’t know how it works,” says Hoque, one of the 2020-2021 National Academy of Medicine Emerging Leaders in Health and Medicine.

In his research at the University of Rochester, he uses advanced computer technologies to analyze this unspoken language. “I want to be able to understand that code, using computation as a lens, while staying mindful of their ethical and appropriate uses.”

Advances in machine learning and artificial intelligence now allow the quick analysis of an unprecedented amount of human interaction data, enabling Hoque and his colleagues to gradually unravel the hidden patterns in communication, breaking new ground in human behavior research.

Hoque’s work aims not just to understand nonverbal communication but also to develop tools that channel those insights to improve human health and well-being. “As I started to do this, I realized there is a huge application on health … from autism, to Parkinson’s, to helping doctors improve quality of care and honor individual preferences of end-of-life patients, to helping elderly who might have difficulty with social communication due to social isolation, to PTSD, while improving equity and access.”

About the organization: