Programmer builds a better lie detector with computer vision and facial analysis
A remote lie detector based on biometric signals has been developed by Fletcher Heisler, host of the YouTube channel “Everything is Hacked”. Truthsayer software uses standard video feeds to analyze changes in subjects’ expressions to find indications that they are not telling the truth.
Heisler caught the attention of the computer vision community earlier this year when he created coding software with facial expressions.
Heisler notes the limitations of polygraph testing. Their effectiveness is “controversial at best”, they cannot be performed remotely, and the subject always knows they are taking one.
He used OpenCV, a pre-trained mood recognition library, and Face mesh from Google MediaPipe to create Truthsayer in the Python programming language.
Detecting the subject’s face, especially with images that include multiple people, was a temporary challenge in Truthsayer’s development, Heisler says.
Heart rate is measured from blood flow to and from the face, captured by a standard webcam. Changes are measured, along with differences in blink rate, mean direction of gaze, lip compression, and facial occlusions with the subject’s hand. Clusters of signals can indicate dishonesty.
“There’s a lot of pseudoscience here,” Heisler acknowledges, “but there’s a lot of very well-documented science.”
The result is a scrollbar with truth on the right, considered the baseline, and “deception” on the right.
Mark Zuckerberg’s U.S. Congressional testimony showed no signs of deception, but Truthsayer identified times in his European Parliament testimony when questions about shadow profiles prompted signs of deception.
Potential uses for the software suggested by Heisler include social engineering, but also increasing confidence in online negotiations and interviews or dating, assisting with therapy or guided meditation, and as an aid to people with autism to understand the expressions of others.
biometrics | computer vision | Def Con | expression recognition | facial biometrics | facial analysis | lie detection | research and development | video analysis