Evaluation of the Immigrant Settlement and Adaptation Program (ISAP)

Evaluation Issue	Evaluation Question
*Program Rationale* – *To what extent does the program remain relevant to the priorities of the Government of Canada and to the needs of newcomers?*	Is there a continuing need for federal government programming to provide the services offered? Are newcomers adequately prepared for the settlement, adaptation and integration experience prior to coming to Canada? Are the priorities and criteria for the programs appropriate for the range of clients served? Are there other services that would assist the integration of newcomers that are not included? Are the services offered complementary with other settlement/ resettlement programs (e.g. ISAP or Host, LINC and RAP) and with the components of these programs that are managed by provincial governments?
Program Delivery – Are the design and delivery of the program(s) appropriate?	Are program objectives and the roles and responsibilities of CIC (HQ, regional offices, local offices) and the SPOs, including reporting responsibilities, clearly defined and understood? Does the application process result in the most appropriate service providers being funded to deliver settlement services? To what extent have settlement programs been successfully promoted to newcomers and other stakeholders? Which mechanisms have been most effective? Are potential clients able to access settlement services in a timely fashion? To what extent is the program capable of serving newcomers in both official languages? To what extent do service providers have adequate tools and support for delivery of settlement services? To what extent do SPOs have the capacity to deliver effective services to newcomers? Is there an appropriate balance between consistency in delivery of services across CIC jurisdictions and flexibility to accommodate changing needs and regional priorities? Are there adequate financial controls in place to monitor program delivery? To what extent does CIC monitor, evaluate, report and use information on results, share best practices and use information to contribute to program improvements? To what extent do service providers collect and report program data? Do they have adequate tools and support to do this? To what extent have special projects funded under ISAP Stream B improved program delivery?
*Effectiveness and Efficiency – To what extent does the program use the most appropriate, efficient and cost-effective methods to meet its objectives?*	Is service delivery through external service provider organizations effective and efficient? What methods are used to assess client needs and are these effective? Are program funding allocations appropriate to meet the needs of the range of clients? Are there alternative approaches to service delivery?
*Success – To what extent has the program(s) been successful in achieving its desired outcomes?*	Are clients satisfied with the settlement services they receive and do they meet their needs? To what extent do ISAP needs assessments contribute to the program’s desired immediate outcomes? To what extent do ISAP information and referral services contribute to the program’s desired immediate outcomes? To what extent do ISAP translation and interpretation services all increased access for clients to mainstream services? To what extent do referrals to external organizations/agencies contribute to newcomer settlement and integration? To what extent do solution-focused counselling and subsequent referrals to other services contribute to newcomer settlement and integration? To what extent has ISAP increased awareness of the benefits of immigration and settlement among mainstream organizations? To what extent does ISAP service bridging contribute to the program’s desired immediate outcomes? To what extent have settlement services contributed to the intermediate outcomes of clients? Outcome: Increased ability to communicate Outcome: Improved ability to independently access community resources and services Outcome: Improved ability to meet personal goals, such as further education, employment or income improvement Outcome: Newcomers have satisfying paid work consistent with their skills Outcome: Participate in cultural, social and civic life in their communities To what extent do ISAP employment-related services contribute to the program’s desired immediate outcomes?

Program Rationale –
To what extent does the program remain relevant to the priorities of the Government of Canada and to the needs of newcomers?

Is there a continuing need for federal government programming to provide the services offered?
Are newcomers adequately prepared for the settlement, adaptation and integration experience prior to coming to Canada?
Are the priorities and criteria for the programs appropriate for the range of clients served? Are there other services that would assist the integration of newcomers that are not included?
Are the services offered complementary with other settlement/ resettlement programs (e.g. ISAP or Host, LINC and RAP) and with the components of these programs that are managed by provincial governments?

Program Delivery –
Are the design and delivery of the program(s) appropriate?

Are program objectives and the roles and responsibilities of CIC (HQ, regional offices, local offices) and the SPOs, including reporting responsibilities, clearly defined and understood?
Does the application process result in the most appropriate service providers being funded to deliver settlement services?
To what extent have settlement programs been successfully promoted to newcomers and other stakeholders? Which mechanisms have been most effective?
Are potential clients able to access settlement services in a timely fashion?
To what extent is the program capable of serving newcomers in both official languages?
To what extent do service providers have adequate tools and support for delivery of settlement services?
To what extent do SPOs have the capacity to deliver effective services to newcomers?
Is there an appropriate balance between consistency in delivery of services across CIC jurisdictions and flexibility to accommodate changing needs and regional priorities?
Are there adequate financial controls in place to monitor program delivery?
To what extent does CIC monitor, evaluate, report and use information on results, share best practices and use information to contribute to program improvements?
To what extent do service providers collect and report program data? Do they have adequate tools and support to do this?
To what extent have special projects funded under ISAP Stream B improved program delivery?

Effectiveness and Efficiency – To what extent does the program use the most appropriate, efficient and cost-effective methods to meet its objectives?

Is service delivery through external service provider organizations effective and efficient?
What methods are used to assess client needs and are these effective?
Are program funding allocations appropriate to meet the needs of the range of clients?
Are there alternative approaches to service delivery?

Success – To what extent has the program(s) been successful in achieving its desired outcomes?

Are clients satisfied with the settlement services they receive and do they meet their needs?
To what extent do ISAP needs assessments contribute to the program’s desired immediate outcomes?
To what extent do ISAP information and referral services contribute to the program’s desired immediate outcomes?
To what extent do ISAP translation and interpretation services all increased access for clients to mainstream services?
To what extent do referrals to external organizations/agencies contribute to newcomer settlement and integration?
To what extent do solution-focused counselling and subsequent referrals to other services contribute to newcomer settlement and integration?
To what extent has ISAP increased awareness of the benefits of immigration and settlement among mainstream organizations? To what extent does ISAP service bridging contribute to the program’s desired immediate outcomes?
To what extent have settlement services contributed to the intermediate outcomes of clients?

Outcome: Increased ability to communicate

Outcome: Improved ability to independently access community resources and services

Outcome: Improved ability to meet personal goals, such as further education, employment or income improvement

Outcome: Newcomers have satisfying paid work consistent with their skills

Outcome: Participate in cultural, social and civic life in their communities
To what extent do ISAP employment-related services contribute to the program’s desired immediate outcomes?

Search

Evaluation of the Immigrant Settlement and Adaptation Program (ISAP)

Appendix D: List of Evaluation Issues and Questions