Although biometric technology is characterised by providing strong user authentication, there is very little informative discussion of accuracy in the biometric industry.
There is also very little understanding of what accuracy means in a real-world environment as opposed to a laboratory environment. Most vendor statements on accuracy bear little relevance to real-world performance of biometric systems. That is, most commercial and government organisations generally employee biometric systems without any knowledge and understanding of the real-world accuracy of the systems.
The lack of discussion and understanding of real-world performance on biometric technology is because biometric companies primarily concern themselves with performance in highly controlled environments. That is, biometric companies often assess their technology’s accuracy by using static or artificially generated templates, images and data, not by processing live data.
When biometric companies perform tests using actual users, the test environments do not generally replicate operation with untrained or poorly motivated subjects, as are often found in real-world deployments. The sample population used during the testing phase tends to be very general and has very little similarities to the target population and environment.
The basic algorithm testing is a necessary step in the development of any technology and most biometric technologies have proven their theoretical capabilities in a closed controlled environment.
However, it is necessary to evaluate and assess the capabilities of biometrics as a solution to problems in individual and institutional authentication, not as just a developmental technology. In order for companies to be comfortable in deploying biometrics technology, statements on biometric accuracy must reflect operation in real-world environments.
The key performance metrics in biometrics such as false match rate, false non-match rate, and failure-to-enrol rate must be well understood and explained. No single metric indicates how well a biometric system or device performs, the analysis of all three metrics is necessary to assess the performance of a specific technology.
In addition, biometric system performance is affected by a variety of external factors. These performance metrics must be assessed in the context of their usage and deployment environment. Assessing accuracy without understanding the reasons behind a company’s biometric deployment or the rationale for an individual’s biometric usage is largely fruitless.
Researchers at Biometric Research Laboratory (BRL) within Namibia Biometric Systems (NBS) made astonishing discoveries that the majority of institutions employing biometric technologies in third world countries have limited knowledge or do not know how accurate the biometric technology their institution is deploying.
Most technology companies employee highly skilled individuals to market their products. However, most commercial and government organisations have limited skills and knowledge to accurately make an independent assessment of the technology in relation to their needs.
*Dr Risco Mutelo studied Biometric Engineering at the New Castle University, United Kingdom.
He can be contacted at risco.mutelo@namibiabiometricsystem.com or beatrice.nardi@namibiabiometricsystems.com.