Evaluation of Government Assisted Refugees (GAR) and Resettlement Assistance Program (RAP)

Appendix A: Evaluation framework

Evaluation questions Indicators Methods/Data sources Location
ProfileFootnote 27
 
  • Number and profile of GAR arrivals, including UPP, international appeals for protection (groups), JAS and blended cases, and trends over time
  • Profile of the needs of GARs arriving in Canada (pre- and post-IRPA)
  • Profile of RAP funding used on income support (averages by family size and P/T), service delivery, capacity building, loans forgiveness and other
  • Profile of RAP income support levels in comparison to P/T social assistance rates across Canada
  • Profile of RAP service delivery approaches across SPOs and development of a RAP typology, incl. aspects such as size of SPOs, geographic location, life skills training, enabling services, co-location with settlement services, client-focuses approaches, health support, blended approaches, etc.
  • Document review (Program documentation, research/documentation on refugee needs, stakeholder reports, cultural profiles from IOM, health profiles from HMB, Bhutanese needs assessments, client-centred pilot information)
  • Facts and Figures
  • Analysis of IMDB data and provincial statistics on social assistance rates
  • Analysis of financial data
  • Key informant interviews (Refugees Branch, OMC Branch, HMB, International and Intergovernmental Relations, IR, Regions/Local offices, UNHCR, DFAIT, CIDA, CCR, Amnesty International, other stakeholders) SPOs survey/interviews

Section 2.3.1 GARs profile

Section 2.3.2: GARs: SPOs clients

Section 4.5 RAP services

Section 4.6 Income support

Section 4.6.1

Loans

Relevance

1a) Is there a continued need to provide protection to refugees?

1b) Is there a continued need for RAP?

  • Number of refugees world-wide identified for resettlement
  • Number/percentage of refugees world-wide assisted through resettlement by other countries
  • Proportion of identified refugees (GARs and PSRs) assisted through resettlement in Canada and trends over time
  • Perceptions of CIC and partners (UNHCR, DFAIT, CIDA, PHAC, HRSDC and SPOs) on need for refugee protection and RAP
  • Profile of the needs of GARs arriving in Canada (pre- and post-IRPA)
  • Stakeholder perceptions regarding the use of resettlement as a durable solution
  • Document review (UNHCR statistics and reports, program documentation, research/documentation on refugee needs, speeches from the Minister, UNHCR Global Report on Resettlement, Medium Term Planning documents, program analysis on RAP, UNHCR news releases)
  • Facts and Figures
  • Key informant interviews (Refugees Branch, OMC Branch, International and Intergovernmental Relations, IR, Regions/Local offices, UNHCR, DFAIT, CIDA, PHAC, HRSDC, CCR, Amnesty International, other stakeholders)
  • SPOs survey/interviews

GAR: Section 3.1.1

RAP: Section 4.1.1

2) Are RAP and the GAR program consistent with departmental, government-wide and international protection priorities and commitments?

  • Alignment with CIC population priorities, targets, commitments
  • Alignment with the priorities and commitments of the Government of Canada and partner federal departments (DFAIT and CIDA) on the promotion of humanitarian objectives, peace and good governance
  • Alignment with priorities and commitments in the Agenda for Protection
  • Alignment with commitments in relevant international Conventions/Declarations/Agreements
  • Document review (IRPA, Agenda for Protection, UNHCR international appeals for protection, international Conventions/Declarations/Agreements, SFT, Budget, RPP, DPR, DFAIT and CIDA documentation related to priorities and commitments, CRC, CEDAW, Canadian Protection Statements at UNHCR ExCOM)
  • Key informant interviews (Refugees Branch, Strategic Policy and Priorities Branch, International and Intergovernmental Relations, UNHCR, DFAIT, CIDA)

 

3) Are RAP and the GAR program consistent with federal roles and responsibilities?

  • Alignment with legislative and federal obligations
  • Comparison of federal program to Quebec program
  • Perceptions of CIC and other stakeholders
  • Comparison of RAP to PSR assistance
  • Document review (IRPA, Constitution, Agenda for Protection and international Conventions/Declarations/Agreements, program documentation, documentation on Quebec program as available)
  • Key informant interviews (Refugees Branch, OMC Branch, International and Intergovernmental Relations, Regions, Provinces/Territories)
  • SPOs survey/interviews

GAR: Section 3.1.2

RAP: Section 4.1.2

Design and delivery

4) Are GAR selection, matching and processing efficient and effective?

  • Is CIC using the right design to select, match and process GARs?
  • Do SPOs have sufficient information to meet GAR needs upon arrival?
  • Are arrival patterns coordinated?
  • Are GARs being matched to communities appropriately?
  • Do GAR population priorities and targets consider settlement capacities at home?
  • Comparison of GAR and PSR application acceptance and refusal rates by visa office and overall (and reasons, if available)
  • GAR application processing times (and inventories) by visa office and overall
  • Cost per GAR application processed
  • Evidence of quality assurance in GAR application processing
  • Evidence of coordination within CIC and with IOM
  • Profile and comparison of different selection and processing approaches (e.g. individual versus group, targeting specific geographic regions, Quebec approach)
  • Comparison of arrival patterns across SPOs and refugee’s level of need over time
  • Reliability of information on arrival times (i.e. NATs)
  • Extent/quality/appropriateness of information provided to SPOs (incl. Supplemental Medical Form)
  • Gaps in information provided to SPOs
  • Perceptions of SPOs and P/Ts on coordination of GAR arrivals and capacity to meet their needs
  • Perceptions of GARs on quality of matching, arrival experience, and if applicable, reasons for secondary migration
  • Incidence of secondary migration
  • Document review (Program documentation, quality assurance reports, audits/reviews as available, HMB)
  • IR and OMC statistics
  • Analysis of IMDB, financial and iCAMS data (RAP/FOSS cube)
  • Key informant interviews (Refugees Branch, OMC Branch, Regions/Local offices, IR, CIC visa offices, SPOs, IOM, P/Ts)
  • GAR survey/focus groups
  • Analysis of HMB data (as available)
  • SPOs survey/interviews

Sections:

  • 3.2
  • 3.3
  • 3.4.1
  • 3.4.2
  • 4.2
  • 4.3
  • 4.5

5) Is RAP appropriate and sufficient for the needs of the GAR population arriving in Canada?

  • Are RAP income support levels appropriate and sufficient?
  • Does RAP offer the right services to GARs?
  • Are there any gaps in RAP service delivery?
  • Does RAP achieve comparable outcomes across Canada?
  • Profile of RAP spending breakdown
  • Profile of the needs of GARs arriving in Canada (pre- and post-IRPA)
  • Profile of RAP income support levels in comparison to P/T social assistance rates across Canada
  • Quantity and quality of RAP services provided to GARs
  • Perceptions of CIC, SPOs, P/Ts and GARs on appropriateness of resettlement assistance provided (incl. timeliness, accessibility, usefulness and client focus)
  • Evidence of gaps in resettlement service delivery
  • Perceptions of GARs that their immediate and essential (financial and service) needs have been met through RAP
  • Comparison of immediate outcomes of RAP recipients across Canada in relation to RAP service delivery and P/T income support profiles
  • Earnings, employment and social assistance rates among GARs
  • Document review (Program documentation, research/documentation on refugee needs, stakeholder reports, cultural profiles from IOM, health profiles from HMB, Bhutanese needs assessments, other research reports and policy analysis, UNHCR statistics and reports, Metropolis)
  • Analysis of IMDB data and provincial statistics on social assistance rates
  • Key informant interviews (Refugees Branch, OMC Branch, Regions/Local offices, P/Ts, HRSDC, CMHC, National Council of Welfare)
  • SPOs survey/interviews
  • GAR survey/focus groups
  • Comparative analysis of outcomes of RAP recipients by RAP service delivery approach and income support profile

Sections:

  • 4.4
  • 4.5
  • 4.6

6) Is resettlement policy and program development for GARs evidence-based, consultative and responsive to the diverse needs of refugees and communities?

  • Extent/appropriateness of stakeholder consultation
  • Evidence of using and addressing findings of consultation, research, performance measurement and evaluation in policy and program development
  • Partners’ and stakeholders’ perceptions on responsiveness/flexibility of policies and programs in meeting the diverse needs of refugees and communities
  • Document review (Program documentation
  • Key informant interviews (Refugees Branch, OMC Branch, International and Intergovernmental Relations, HMB, IR and Regions/Local offices, P/Ts, RAP WG, UNHCR-Geneva, DFAIT, CIDA, CCR, Amnesty International, other stakeholders)
  • SPOs survey/interviews

Section 6.2

6.2.1

Performance (effectiveness)

7) Are the immediate and essential needs of RAP recipients met through RAP?

  • Number/percentage of RAP recipients receiving RAP services:
    • Reception
    • Temporary housing
    • Support with urgent/emergent health needs
  • Perceptions of GARs on extent to which basic needs (e.g., food and weather-appropriate clothing) have been met
  • Perceptions of GARs, SPOs, and CIC on the appropriateness of resettlement assistance provided in meeting the immediate and essential financial and service needs of RAP recipients
  • Analysis of iCAMS data (RAP/FOSS cube)
  • Key informant interviews (Refugees Branch, OMC Branch, HMB, Regions/Local offices)
  • SPOs survey/interviews
  • GAR survey/focus groups

Section 4.4: Temporary housing

Section 4.5.1: Immediate and urgent needs

8a) Do GARs have the necessary knowledge, skills and means to live safely and independently?

8b) Are they linked to services they need to address issues as they emerge?

  • Income support level of GARs (incl. start-up and allowances)
  • Perceptions of GARs and SPOs on extent to which RAP has changed the level of knowledge and skills of GARs, incl.:
    • Financial knowledge and banking skills
    • Non-financial knowledge and life skills related to transportation, shopping, rights and responsibilities in Canada, cooking/appliances and using health and social services
  • Perceptions of GARs, SPOs and CIC on adequacy of RAP income support
  • Perceptions of GARs, SPOs and CIC on adequacy of permanent housing (do they have it, how long to find it and its quality, such as crowdedness)
  • Adequacy of food/use of food banks (food security) and use of charities/in-kind support
  • Evidence of links to mandatory services (support with completing applications for SIN card, health card, NCB and registering children in school)
  • Extent/appropriateness of links to other services (incl. IFH, child care, municipal housing, mental health, etc.) based on client need
  • Satisfaction/experience of GARs regarding links to services
  • Evidence of overseas orientation or language training
  • Document review (Income Support Study, Housing paper, Evaluation of COA program(
  • Analysis of financial and iCAMS data (RAP/FOSS cube)
  • Analysis of iCAMS (RAP/FOSS cube) and IMDB data
  • Key informant interviews (Refugees Branch, OMC Branch, Regions/Local offices, National Council of Welfare, CMHC)
  • SPOs survey/interviews
  • GAR survey/focus groups

Section 4.5.2

Section 4.5.3

Section: 4.6: Income support, housing, food banks

Section:3.4.3: COA

9) Do GARs obtain and benefit from CIC settlement services? If not, why?

  • Use of settlement services (ISAP, Host and LINC)
  • Time lag between use of resettlement and settlement services (as available)
  • Comparison of GARs using settlement services to overall GAR profile
  • Extent to which GARs feel that they have benefited from settlement services
  • Settlement outcomes of GARs (as available)
  • Perceptions of GARs, SPOs and CIC regarding success factors and barriers to using settlement services
  • Earnings, employment and social assistance rates among GARs
  • Level of integration among GARs (LSIC)
  • Literature review (Metropolis, etc.)
  • Analysis of IMDB, LSIC and iCAMS data (LINC, ISAP and Host data as available)
  • Key informant interviews (Refugees Branch, OMC Branch, Regions/Local offices)
  • SPOs survey/interviews
  • GAR survey/focus groups
  • Findings from Settlement Evaluations as available

Section 4.8

4.8.1: Language acquisition

4.8.2: Employment and education

10a) To what extent does CIC influence international protection policies through resettlement?

10b) Does CIC’s resettlement program leverage benefits for both selected refugees and those not resettled?

  • Extent of CIC’s international engagement related to resettlement
  • Other states’ and NGO perceptions of Canada’s influence related to resettlement
  • Evidence of CIC positions regarding resettlement reflected in international protection policies and in UNHCR Executive Committee Conclusions
  • Number of GARs protected (arrivals)
  • Evidence of leveraged benefits for other refugees
  • Document review (UNHCR statistics and reports, program documentation, Executive Committee protection statements, Canada’s statements at Executive Committee, Canada’s report on the Agenda for Protection, Mexico Resettlement Solidarity Plan of Action, UNHCR Global Appeal)
  • Facts and Figures
  • Key informant interviews (Refugees Branch, International and Intergovernmental Relations, WG on Resettlement, UNHCR, DFAIT, CIDA, CCR, Amnesty International, other stakeholders)

Section: 3.1.1

Performance (efficiency and economy)

11) Are there alternative RAP design and delivery options that would better facilitate the achievement of improved outcomes for GARs?

  • Best practices identified for resettlement programs in Quebec and other countries
  • Best practices identified through comparative analysis of RAP service delivery approaches
  • Literature review
  • Comparative analysis of GAR outcomes by RAP service delivery approach/case studies

Section 5.2

12) Are there approaches to GAR selection and processing that could lead to a more coordinated and efficient process?

  • Cost per GAR application processed (e.g. individual versus group, in specific geographic regions)
  • Best practices identified for selection and processing in other countries
  • Profile and comparison of different selection and processing approaches (e.g. individual versus group, targeting specific geographic regions, Quebec approach)
  • Perceptions of referral organizations on efficiency of GAR application processing and coordination of GAR departures (comparison of individual, UPP, group)
  • Perceptions of SPOs & P/Ts on coordination of GAR arrivals
  • Perception of GARs on quality of matching/arrival experiences
  • Literature review
  • Document review (UNHCR statistics and reports, program documentation, Welcome to Europe book)
  • Analysis of financial data
  • Key informant interviews (Refugees Branch, OMC Branch, IR, Matching Centre, UNHCR, Hebrew Immigrant Aid Society, visa offices, P/Ts)
  • SPOs survey/interviews
  • GAR survey/focus groups
  • Case study of group processing

Section 5.1

Page details

Date modified: