Conversation is never just about the words you say - everything from the tone and volume, to body language and gaze are just as important. 

These non-verbal communication signals can be easily misinterpreted, even more so for people with anxiety or Asperger’s. This prompted a team at Massachusetts Institute of Technology to build a smartwatch app that detects the mood behind a conversation.

How does it work? Well, we start with an algorithm created by researchers Tuka Alhanai and Mohammad Mahdi Ghassemi - which is built into a Samsung Simband fitness tracker for this research. 

It can analyse speech and tone (all of which was recorded with an iPhone 5S), crunch that data and detect what the person is feeling for every five seconds of conversation, essentially telling the user whether that piece of dialogue was happy or sad.

This info was paired with what else could be taken via the various sensors on the fitness band - heart rate, skin temperature, fidgeting movements, etc.

To test this, subjects wore the band and were asked to tell a story, inflecting different emotions throughout. Overall, this algorithm was able to determine the correct emotion with 83% accuracy.

While this AI is rough around the edges - making basic conclusions such as long-drawn-out monotonous conversation meaning sad - it’s clear the team are really onto something here.

“Imagine if, at the end of a conversation, you could rewind it and see the moments when the people around you felt the most anxious,” says graduate student Tuka Alhanai. “Our work is a step in this direction, suggesting that we may not be that far away from a world where people can have an AI social coach right in their pocket.”

Findings will be presented at next week’s Association for the Advancement of Artificial Intelligence (AAAI) conference in San Francisco - but you can get a look at this tech in action just below.