Citizenship and Immigration Canada
www.cic.gc.ca
Common menu bar links
Biometrics Field Trial Evaluation Report
Section 4. Evaluation of the Field Trial
This section describes the evaluation methodology and provides a brief overview of some key results obtained. These results are described in greater detail in the remainder of this report.
4.1 Evaluation methodology
The evaluation of the biometrics field trial was guided by performance indicators which were developed prior to the launch of the field trial. Detailed indicators were developed under four key objectives as listed below in Table 4-A.
Table 4-A: Field trial evaluation framework
Objective 1. Identity Management and Program Integrity
Goal: To measure biometric technology as an identity management and program integrity tool
Outcomes measured
- I1: Identity management outcomes
- I2: Facial recognition system performance
- I3: Fingerprint recognition system performance
- I4: Fingerprint and facial recognition fusion performance
- I5: Overall biometric system performance
Objective 2. Client Service
Goal: To understand the impact of the introduction of biometrics on client service for Canada’s visa and entry programs.
Outcomes measured
- C1: Client facilitation
- C2: Client relations
- C3: Public Opinion
Objective 3. Operational Impact
Goal: To explore the organizational and procedural impacts of biometric implementation.
Outcomes measured
- O1: Visa office impacts
- O2: Port of Entry Impacts
- O3: Usability of Refugee Biometric Data
- O4: Centralised Matching Impacts
- O5: Ergonomics
- O6: Human Resources Impacts
Objective 4. Cost Factors
Goal: To understand biometric program costs so that they can be measured against benefits.
Outcomes measured
- C1: Implementation costs
Performance indicators were developed for each objective (See Appendix C – Performance Indicators). The field trial yielded significant data through a broad range of measurement tools, including:
- Biometric system reports for measures relating to system performance, timing and suggested matches
- CIC system reports (CAIPS) and available local statistics for a portion of the operational impact measures for visa offices
- Forensic specialist review to judge photo and fingerprint quality, and to review system suggested matches
- Site visits where CIC Project Team members observed the field trial and interviewed employees, including supervisors
- Client surveys conducted at the Hong Kong and Seattle visa offices during the last month of the field trial; and
- Problem reports, status updates and periodic conference calls with supervisors at the field trial sites.
4.2 Field trial findings
The biometrics field trial yielded enough data to allow for a thorough evaluation. The following tables provide some notable statistical findings from an analysis of the field trial data:
Table 4-B: Biometrics field trial volumes at a glance
| ITEM | VOLUME |
|---|---|
| Total client enrolments at all field trial sites [note 1] | 18,264 |
| Enrolments at both visa offices | 6,979 [note 2] (photo only) 7,875 (photo and 10 fingerprints) |
| Hong Kong | 8,516 |
| Seattle | 6,338 |
| Number of enrolments at ports of entry | 338 [note 3] |
| Enrolments at Refugee Intake Centre | 3,410 (photo and 10 fingerprints) |
| Field trial arrivals detected at participating ports of entry | 548 (Douglas/Pacific Highway) 934 (Vancouver International Airport) 1,482 TOTAL |
| Number of times field trial clients presented one finger for verification at ports of entry | 1,020 |
It is interesting to note that even during the limited field trial period there were 364 repeat clients. Most of these clients applied twice (usually after being refused the first time) but a few applied three or four times during the six month period. This shows the potential to facilitate these repeat clients by enrolling their biometrics initially, and then only verifying them at subsequent encounters.
Table 4-C: Interesting field trial facts
| ITEM | VOLUME |
|---|---|
| Field trial clients claiming refugee status in Canada | 12 [note 4] |
| Multiple enrolments (biometric matches) | 394 182: only photos available for the repeat clients (mail-in applications, not seen in Vancouver) 195: both photos and fingerprints available for the repeat clients 17: only fingerprints available [note 5] |
| Number of clients who applied multiple times | 364 |
| Clients correctly matched using only facial recognition with system recommended threshold | 98.4% |
| Clients correctly matched using only fingerprints with system recommended threshold | 97.9% |
| Clients correctly matched using both facial and fingerprint recognition | 100% |
Footnotes:
1. Includes temporary resident visa applicants in Hong Kong and Seattle, refugee claimants at the Refugee Intake Centre, and multiple enrolments. Clients who applied more than once during the field trial period, either in Hong Kong or Seattle, had their photos and/or fingerprints enrolled every time.
2. Includes minors (under 14 years old) and mailed-in applications.
3. 10-fingerprint enrolments only – all photos were enrolled at visa offices.
4. Includes three irregularities found: one client committed identity fraud by assuming a different name; and two clients were refused visas and then travelled to Canada with improperly obtained or fraudulent documents using their original names. See Section 5.
5. See Section 6–Biometric System Performance under CICconditions–for more information.
