Michigan State University researchers received a $12 million, four-year federal grant from the Intelligence Advanced Research Projects Activity, or IARPA, under its Biometric Recognition and Identification at Altitude and Range, or BRIAR, program.
The IARPA BRIAR program is a 48-month effort to deliver end-to-end software systems capable of detecting individuals at severe imaging conditions, extracting biometric signatures from the whole-body (such as an individual’s gait and/or body shape) and face, and fusing biometric information for robust multi-modal matching.
The MSU project entitled “Physics-driven Modeling and Learning for Person Recognition at a Distance and Altitude” is led by Xiaoming Liu, an MSU Research Foundation Professor; Arun Ross, Martin J. Vanderploeg Endowed Professor; and Anil Jain, a University Distinguished Professor — all in the College of Engineering.
Biometric recognition refers to technology for real-time and automated recognition of individuals based on their body attributes such as face, fingerprints and iris. We use biometrics when we unlock our mobile phones using face or fingerprints, board international flights and go through immigration after returning from an overseas trip. In these applications, the camera or sensor needs to be in close proximity to the individual being recognized. But what about situations when law enforcement agencies want to find a missing person in the woods or locate a group of fugitives?
The MSU BRIAR team incorporated the use of drone technology for recognizing individuals at a distance. While the elevated imaging platforms increased the altitude and range, this vantage point often only showed the tops of people’s heads, making it difficult to identify any distinguishing facial features from one person to another.
The proposed FarSight system incorporates six components: detection and tracking; image restoration; face, gait and body shape feature extraction; and multimodal biometric fusion. Credit: Xiaoming Liu, Arun Ross and Anil Jain
This, according to Liu, necessitated the use of whole-body biometric recognition. “When we combined the benefits of each of the three biometric modalities (face, body shape and gait), we were able to achieve state-of-the-art accuracy as reported in the independent evaluation conducted by IARPA on sequestered data,” he said.
MSU’s system, called FarSight, accepts videos from drones as input and outputs a candidate list of identities from a database. To build this prototype system, the MSU team had to address a number of challenges, including low-quality imagery, atmospheric turbulence, large variations in viewing angle and distance, and limited amount of operational data to train the recognition system.
FarSight is a person recognition system that implements and fuses face, gait and body shape recognition modules in challenging conditions presented by full-motion videos. Credit: Xiaoming Liu, Arun Ross and Anil Jain.
“We have developed a successful prototype system that can look at one person from about 300 to 400 meters away,” said Liu. “During the next phase of the project, we’ll be looking at multiple people together at a range of 600 to 700 meters.
As the team improves the range and accuracy of the system, it is also learning the relative strengths of various biometric measurements in images captured at different viewpoints.
“We were surprised to learn that a person’s height, build and gait often contain more useful biometric information than their face when capturing images from a long distance,” said Liu.
MSU’s team, which serves as the lead institution for this project, also includes Christopher Perry in the Department of Computer Science and Erin Bunting and Robert Goodwin in MSU’s Remote Sensing and Geographic Information Systems Center. Other researchers and personnel for the project include Stanley Chan at Purdue University, Atlas Wang at the University of Texas at Austin and Humphrey Shi at the Georgia Institute of Technology.
