ARCHIVED – Enhanced Language Training Initiative: Formative Evaluation
Executive Summary
This report presents the results of a formative evaluation of the Enhanced Language Training (ELT) Initiative (referred to as the “program”) carried out between November 2006 and July 2007 by Goss Gilroy Inc., on behalf of Citizenship and Immigration Canada (CIC).
Enhanced Language Training Initiative
The Enhanced Language Training (ELT) Initiative was launched as a pilot project in 2003/04 to develop and deliver higher levels of language training and job-specific language training, including labor market understanding and/or experience. ELT funds two different kinds of projects: development and delivery projects. Development projects can include the development of assessment tools, software tools, research, study guides and other learner supports, and curriculum. Delivery projects include two components: language training and bridge-to-work. In most cases, language training is delivered to attain Canadian Language Benchmarks (CLB) levels 7 to 10 (English/French), including job-specific language training. In smaller centers, the program funds language training at CLB levels 1 to 10, including job-specific language training. The second component, bridge-to-work, includes a variety of employment-related activities that can include, for example, orientation to the local labor market, assistance in finding employment in the immigrant’s field of specialty, mentoring, work placements, cultural orientation in the workplace and preparation for licensure exams and internships.
The program allocates $20M annually ($5M starting in 2003–04 and an additional $15M starting in 2004–05) for all provinces and territories outside Quebec [ note 1 ] and additional amounts specifically for Ontario as per the Canada-Ontario Immigration Agreement (COIA). ELT funds are available to cover costs related to child minding and transportation. ELT projects must include cost-sharing in the form of funds, in-kind contributions, services, tools or facilities. Eligible partners include provincial and territorial governments, municipalities, businesses, not-for-profit organizations, individuals, non-governmental organizations, community groups and colleges and universities, etc. [ note 2 ]
Originally, ELT management and delivery was centralized at Integration Branch, NHQ. Decentralization to the CIC regional offices began in Summer 2006 in Alberta and culminated in Spring 2007 in Ontario. The Governments of British Columbia (BC) and Manitoba have both signed cost-sharing settlement realignment agreements, which include ELT. Also, a cost-sharing contribution agreement was signed with the Government of Saskatchewan. In these three provinces, SPOs submit applications directly to the provincial government. In Alberta, the province participates in joint Calls for Proposals and joint selection of projects. Alberta SPOs then sign separate contribution agreements with CIC and Alberta.
ELT Projects and Participants
Since the program launch in 2003/04, CIC has funded 253 ELT projects through 140 unique SPOs. There were 2,488 ELT participants [ note 3 ] in Ontario, Alberta and the Atlantic provinces, the provinces in which CIC directly funds the program. Just over half were female (56%) and 43% were male. [ note 4 ] The largest group of participants were 35 to 44 years of age, which is consistent with the participants’ education profile. As would be expected, nearly three-quarters of participants reported having at least a university degree (just over one-third has a graduate or post-graduate degree). This is above average for the levels of education for immigrants generally. By far the largest percentage of participants (30%) comes from China – the next closest country of origin is India at 8%. Just over two-thirds of participants (69%) are permanent residents and 11% have applied for landed immigrant status. The most common occupation is engineering and the next most common occupational category is pharmacists, dieticians and nutritionists.
Findings
Findings from this formative evaluation, which are largely qualitative in nature, indicate that ELT is, in general, a successful initiative that meets the immediate needs of the target audience.
Rationale
The evaluation findings reflect the extent to which the ELT has achieved its expected outcomes. As a starting point, key informants and SPO representatives confirmed that labor market integration for newcomers remains a challenge and that there is a continuing role for the federal government in this area.
Outcomes
The development component produced a total of 103 ELT tools (e.g. curriculum, evaluation tools, research, promising practices, occupational terminology and other tools). External experts assessed the quality of the tools being used by these SPOs (some of which were developed through ELT development component and some not). Generally, the experts rated the tools very highly. The content and format and design of tools were rated very positively, and the process and generalizability of tools were rated as good. As a result, they likely contributed to the outcome of supporting and improving projects and programs.
The delivery component had specific expected outcomes with respect to preparing newcomers for the labor market by preparing them for licensure exams, improving language skills, developing an increased knowledge of, and experience in, the Canadian work environment, strengthening job finding skills, and establishing mentors/contacts and networks. These were expected to contribute to the longer-term outcome of participants finding employment commensurate with their skills and experience. The outcomes that SPOs appear to have been attained successfully are improving language skills and gaining knowledge/experience in the Canadian work environment – both through the in-class components and the work placements:
- ELT database information and feedback from the focus groups suggest that there has been an increase in language levels with respect to all four dimensions (listening, speaking, reading and writing).
- Two main components of the ELT projects – the in-class component and the bridge-to-work component – contributed to increased knowledge of, and experience in, the Canadian work environment. Evidence from the site visit case studies suggests that the focus of the language training, which occurs during the in-class component, is organized around the workplace and participants indicated that this contributed to preparing them for the Canadian work environment.
- ELT delivery projects are required to include at least one bridge-to-work component. Nearly three-quarters of the case study SPOs included work placements (unpaid, paid or partially paid) as the key bridge-to-work component. Although case studies suggest considerable gains in terms of increased knowledge of, and experience with, the Canadian work environment, some gains may not necessarily be related directly to increased commensurate employment. For example, some participants have returned for further education or, as a result of more realistic expectations of the requirements to enter the Canadian labor market, sought alternative, and perhaps more attainable, employment.
Since this was a formative evaluation, the methodology did not include an in-depth assessment of the extent to which ELT projects have contributed to the longer-term expected outcome of participants having employment commensurate with their skills and experience. The limited information from the ELT database indicates that for about one-quarter of the participants for whom information on commensurate employment is included in the database, 59% are reported to have secured employment commensurate with the employment they had before their arrival in Canada. Qualitative information about employment from the case studies suggests that in just over half the case studies, a large number of participants were employed after their participation in an ELT project.
ELT Administration
The evaluation found that, generally, relationships between CIC and provinces/ territories are strongest at the regional and local office level (particularly in cases where the CIC and provincial/territorial offices are co-located). Provincial/territorial representatives identified some challenges related to the management of ELT by NHQ: the lack of face-to-face meetings; high turnover among ELT staff; and poor communications regarding matters that would assist provinces/territories in their programming/support of ELT. SPO respondents generally felt that communications with CIC were adequate, although they voiced key frustrations with delays in negotiating contribution agreements, the lack of continuity of funding and the lack of coordination of programs for newcomers (including the various ELT projects) and a perceived lack of a long term vision of immigrant settlement.
CIC ELT Database
Rather than integrating the ELT data collection requirements into the existing CIC database for other settlement programs – iCAMS – the ELT program management decided to create a separate database for the ELT program. Although the ELT database allows for some participation information, it became clear during this evaluation that there were considerable limitations to the information available from the database:
- At the time of the evaluation, the database did not include information on all projects and/or participants. As a result of the different delivery structures in different provinces, data on projects and participants is not being collected in the same way in all provinces;
- There are limitations to, and inconsistencies in, the participant data that is available. This has occurred, in part, because CIC changed the data reporting format during the course of the program. These limitations include gaps in the individual participant data and limited information on participant outcomes;
- The lack of a fixed format for data entry means that considerable data cleaning is required before meaningful analysis can be undertaken; and
- Concerns related to the quality of the data for some variables.
Eligibility requirements
Findings from the evaluation indicated that a number of ELT clients were not eligible to participate in the program because of their immigration status (Canadian citizens and refugee claimants). Limited information was available to clarify this issue.
Dissemination of Tools
The quality of ELT tools and curricula is a strength of the initiative and the expert review indicated that many tools are generalizable to other occupations or settings. The findings indicate that there is a need for more sharing of these tools and more knowledge transfer within the initiative.
Work Placement as a Key Component
The flexibility of the program, which is viewed by stakeholders as one of its strengths, has resulted in a variety of delivery approaches. The evaluation found that some are more successful than others: the use of work placements, in particular, was considered a key success factor by program participants. The work placements appear to be more effective when specific resources are dedicated to finding and managing them. In addition, when work placements are not possible, the SPOs integrate explicit bridge-to-work activities into the program - this could include such things as mentorships and job shadowing.
Proactive Marketing of ELT Program
Some of the ELT-funded projects are “niche” projects, in that they focus on specific occupations – for example, financial service representatives for credit unions. Such projects, in a specific geographic area, are only able to take in a limited number of participants each year, as the number of participants who can be absorbed into the labour market in that area is limited. Thus, where these types of projects appear to be very successful, consideration should be given to actively marketing these approaches in other geographic areas.
Data Collection
The information provided by the projects and/or provinces for inclusion in the ELT database is inadequate for a quantitative assessment of the initiative outcomes. At the same time, SPOs have indicated that the data collection requirements are onerous and some have concerns about providing personal information about participants to CIC. An approach needs to be developed to ensure adequate information is collected to support a more quantitative assessment of participant’s outcomes for the summative evaluation.
_______________
[1] ELT excludes Quebec as all settlement services delivered in the province of Quebec are governed by the Canada-Quebec Accord, signed in 1991. The Quebec government has exclusive responsibility for offering to all immigrants in Quebec integration services comparable to those offered elsewhere, with financial compensation from the federal government.
[2] ELT CIC Staff Training Kit Ontario Region, updated June 9, 2006. Page 14.
[3] There are probably considerably more participants than reflected in the database. It is estimated that there are probably about 5,500 ELT participants in the provinces covered by ELT, some of which are missing because CIC has not yet completed the data entry for all project participants. As of July 2007, there were approximately 1,500 participant records from completed projects to be added to the database.
[4] Information was missing for 1% of the participants.
Page details
- Date modified: