How can I calculate the Failed Acceptance Rate and False Recognition Rate?
How can I calculate the FAR and FRR for an authentication device? (Which I assume is using Biometrics). Otherwise is their any statistics on the subject that is easily accessible?
Answer
A biometric system can work in two modes, which must be distinguished during evaluation: verification and identification.
In verification mode, a user presents his identity and the biometric device verifies that the identity matches (imagine that you go to customs and you present your ID with face image, the officer verifies your real face and the image on ID).
In identification mode, no assumption of identity is made in the beginning and comparison to all templates has to be made (for example, a killer left his fingerprint at crime scene and then police identifies him using his fingerprints -- no assumption is made, they just use the acquired fingerprint and they compare it to all their fingerprints in their database).
It is therefore necessary to distinguish between these two situations, because identification is generally more demanding.
Lets assume, that you are evaluating a verification mode of biometric system. Imagine a biometric system, which assigns all authentication attempts a score from interval [0, 1]. 0 means no match and 1 means full match. Obviously if you set your threshold to 0, all genuine users are admitted, but all impostors are admitted also. On the other hand, if you set your threshold to 1, no one is admitted. So for real usage, you usually set the threshold somewhere between. This might cause, that not all genuine users are admitted and some impostors are admitted. As you can see, there are two error rates: FAR (False Accept Rate) and FRR (False Reject Rate).
FAR is calculated as a fraction of impostor scores exceeding your threshold.
FRR is calculated as a fraction of genuine scores falling bellow your threshold.
Example: We have a fingerprint system. In order to evaluate a performance of any biometric system, we need to gather a database. Assume, that we have done that and the database consists of 10 legitimate users (USER_1-USER_10) and each user provided his finger 10 times (10x10 = 100 images in total). Lets assume, that single image is sufficient for template creation. You select a user (e. g. USER_1) and one his fingerprint image and create the template. The rest of his images you use to verify the fingerprint and you receive 9 genuine scores. All images of other users you use as a impostors and you receive 90 impostor scores. You repeat the template generation for all images and all users and in total you receive 900 genuine scores and 9000 impostor scores. These scores are usually used to generate so called ROC curves to choose the best threshold suting your problem. If you have chosen a threshold, you can calculate the FAR and FRR using the definition I stated above.
Lets assume, that we have chosen as threshold 0.7 and 100 impostor scores exceed the threshold and 50 genuine scores fall below the threshold.
So FAR = 1.1% and FRR = 5.6%.