Evaluation of the Immigration Loan Program

2. Methodology

2.1 Evaluation Approach

The evaluation scope and approach were determined during a planning phase, in consultation with CIC branches involved in the delivery of the Immigration Loan Program. The logic model and evaluation framework were developed in discussion with the program areas. Terms of Reference for the evaluation were approved by the Departmental Evaluation Committee in October 2013. The evaluation was conducted in-house.

It was determined that a medium-sized evaluation would be conducted with the level of effort based on several factors including:

• the Immigration Loan Program had never been the subject of an evaluation;
• the Evaluation of the Government Assisted Refugees and Resettlement Assistance Program,^{Footnote 28} conducted in 2010-11 included a recommendation to examine the transportation and medical loans;
• the program has undergone changes over the years in response to financial pressures;
• policy options and recommendations have been and continue to be considered; and
• there has been a high level of interest in the program among stakeholder groups.

2.2 Evaluation Scope

The evaluation assessed the relevance, delivery and performance of the Immigration Loan Program, and was guided by the program logic model, which presents the expected immediate and intermediate outcomes:^{Footnote 29}

Immediate Outcomes

• Program development, management and governance effectively support program delivery and decision making.
• Individuals in need receive loans or contributions according to their need in order to cover costs associated with admissibility, transportation, landing fees and other eligible expenses related to settlement.
• Loan recipients are fully aware of and understand the meaning and nature of the loan agreement they have signed.
• Collection services are accessible, effective, and responsive to client needs, and facilitate loan repayment.

Intermediate Outcomes

• Loans contribute to the settlement of recipients.
• Loans are repaid in full and in a timely fashion.
• The advance from the Consolidated Revenue Fund is adequately replenished to sustain the Immigration Loan Program.

2.2.1 Evaluation Questions and Issues

The evaluation questions, organized by core issue,^{Footnote 30} are presented in Table 6.

2.2.2 Evaluation Timeframe

The timeframe for the evaluation is 2003 to 2012. It was chosen in recognition that the loan repayment period can extend up to 6 years after landing and in some cases, go beyond this period. In certain circumstances, however, a shorter timeframe was used, based on the line of evidence and data availability. Timeframes are specified throughout the report as applicable.

2.2.3 Evaluation Focus

The evaluation primarily focused on GARs and PSRs as they represented the vast majority of recipients (97.8%), and on transportation, admissibility and assistance loans, as they were the most frequently provided during the timeframe being evaluated. RPRF loans were not examined due to the very small number issued (6 between 2008 and 2012).

2.3 Data Collection Methods

The evaluation of the Immigration Loan Program included multiple lines of evidence to allow for a triangulation of information (see Table 7). A mix of quantitative and qualitative approaches was used, and an evidence matrix served as a guide throughout the process.

Where qualitative information is presented in the report, the scale shown in Table 8 is used. Note that, in some cases (i.e., where the number of respondents was too small or where the question yielded more descriptive information), the responses were not coded and a summary approach was used to analyze the information.

These lines of evidence, along with data collection instruments, are described in greater detail in the Technical Appendices.

2.4 Limitations

The evaluation contained a balance of qualitative and quantitative lines of evidence and allowed the triangulation of data; however, there are a few limitations that should be noted. These limitations were mitigated as described below, and therefore did not have a major impact on the evaluation findings.

• Various data systems have been used by CIC to record information on loans and loan recipients. As a result, some data that is only available on microfiche had to be manually re-entered, and a sampling approach was used to build the administrative dataset. In other instances where demographic information on loan recipients was required, the analysis was based on a sub-population of loans (2008 and 2012), representing 99.7% of the loans (a total of 25,858 accounts) issued during this period. Finally, as contributions data and loans data are recorded in separate systems, datasets were compiled and both systems were cross-referenced; business rules were developed.
• Loan information was not available in the landing data file and therefore could not be linked to the Longitudinal Immigration Database (IMDB). Since almost all refugees receive a loan, economic outcomes analysis was conducted on all GARs and PSRs as a proxy for the recipients.
• The survey of loan recipients could not be fully administered in all of the languages originally identified due to difficulties in finding surveyors who could speak Burmese and Somali. While the survey could not be administered in Burmese, it was administered in Somali using an online version.
• The response rate to the survey of SAHs was relatively low. Survey results were reported as qualitative evidence in conjunction with findings from other lines of evidence.

• Previous
• 1
• 2
• 3
• 4
• 5 (current)
• 6
• 7
• 8
• 9
• 10
• 11
• 12
• 13
• Next