Evaluation of HOST

Executive Summary

Context for the Evaluation

CIC’s mission is to build a stronger Canada by:

• Enabling the migration of temporary and permanent residents to meet the social, economic and cultural needs of communities across Canada;
• Contributing to the management of international migration, including refugee protection;
• Screening newcomers to help protect the health, safety and security of Canadians;
• Supporting the successful integration of newcomers; and
• Promoting Canadian citizenship.

In supporting the successful integration of newcomers, CIC funds and administers three settlement programs namely, the Host Program, the Immigrant Settlement and Adaptation Program (ISAP), and the Language Instruction for Newcomers to Canada (LINC) Program, as well as one resettlement program for refugees called the Resettlement Assistance Program (RAP). The 2003-2004 actual expenditures for these three settlement programs were $2.9M for Host, $36.7M for ISAP, and $92.7M for LINC. The 2003/04 expenditures for RAP were $40.4M.

The Host program is a volunteer-based matching program. The Host program aims to help immigrants overcome the stress of moving to a new country. Volunteers familiar with Canadian ways help newcomers learn about available services and how to use them, practice English and French, get contacts in their field of work, and participate in the community. At the same time, host Canadians learn about new cultures, other lands and different languages; they make new friends and they strengthen community life.

Scope and Objectives of the Evaluation

The evaluation of Host is part of the implementation of the evaluation component of the Contribution Accountability Framework (CAF) for Settlement and Resettlement Programs. The Host evaluation focused exclusively on services managed directly by CIC. The agreements signed between the Government of Canada and some provinces—Quebec, Manitoba and British Columbia—for the delivery of settlement services were not included. As well, Canadian Orientation Abroad (COA) will be evaluated separately.

The objective of the Host evaluation was to provide evidence to answer questions related to four main evaluation issues:

• Program Rationale – To what extent does the program remain relevant to the priorities of the Government of Canada and to the needs of newcomers?
• Program Delivery – Is the design and delivery of the program appropriate?
• Effectiveness and Efficiency – To what extent does the program use the most appropriate, efficient, and cost-effective methods to meet its objectives?
• Success – To what extent has the program been successful in achieving its desired outcomes?

Methodology Overview

The data collection for this evaluation spanned about six weeks, from July 19 to August 31, 2004. Over this time period, GGI conducted the following:

• A data and document review;
• 18 key informant interviews with CIC representatives, including six individuals at headquarters and 12 individuals at the regional or local level of CIC;
• Nine key informant interviews with SPO representatives;
• Two key informant interviews with stakeholders;
• Five focus groups with 51 Host clients held in Halifax (n=1), Toronto (n=2), Ottawa (n=1, in French) and Calgary (n=1);
• Four focus groups with 46 Host volunteers held in Halifax (n=1), Toronto (n=1), Ottawa (n=1, in French) and Calgary (n=1);
• A telephone survey of 29 service provider organizations delivering Host—representing a response rate of 88%; and
• Two innovative practice case studies: Business Mentoring, and Conversation Circles.

Overview of Findings and Conclusions

Rather than a traditional evaluation report organized around the main evaluation issues, a more strategic approach was taken. This approach presents the evaluation results by theme, or main finding. It is the expectation that the themes will better lend themselves to decision-making for the program. The main findings, therefore, are organized as follows:

• Appropriateness of the Current Delivery Model;
• Overall Success of the Program; and
• Adequacy of Capacity and Service Gaps.

Appropriateness of the Current Delivery Model

There is an identified need for continuing Government of Canada involvement in the funding and support of settlement programming; the federal government was reported to play a critical role in maintaining consistent and appropriate immigration priorities. In addition, Host is considered a highly relevant program that meets the needs and expectations of newcomers to a significant extent. Ongoing federal government support is viewed as critical to Host.

There was overall agreement that the current suite of settlement and resettlement services addresses key success factors of settlement and immigration. Seventy-three (73) percent of survey respondents agree that the overall suite of settlement and resettlement services is appropriate given the needs of newcomers, with 14 percent disagreeing. With respect to Host, the majority of survey respondents, 57 percent, agree that the criteria to access services under this program are appropriate, while 21 percent disagree.

Many key informants agree that Host priorities are appropriate given the level of resources allocated to this program. According to interview data, priorities are laid out in contribution agreements with SPOs and can differ by SPO within the same community or region. While some variation exists, government-assisted refugees (GARs) and other refugees are generally identified as high needs clients and are served on a priority basis.

The majority of key informants and survey respondents agreed that the program provides an appropriate balance between flexibility and consistency. In general, program flexibility of the Host program was viewed as a key strength. Flexibility allowed for different Host delivery models that could potentially address a range of client needs. Host delivery models include individual or group matching (e.g., conversation circles). Other Host delivery models include conversation circles, youth matches, and professional matches. However, some key informants noted that consistency could be enhanced through more sharing of information among service providers with respect to volunteer training and information resources, and best practices.

Local community-based service providers were generally considered to be the most appropriate delivery mechanism as these organizations are client focused; have access to community support and a volunteer network; and are well-positioned to respond to local needs. Therefore, the current model of settlement service delivery is most appropriate.

Overall Success of the Program

There is overall agreement that Host has positive impacts for those who access it. Host is considered effective in meeting newcomers’ needs and expectations to a significant extent. All lines of evidence strongly support significant positive impacts of the Host program most notably with respect to providing social support/friendship and expanding the newcomers’ social networks. A number of focus group participants noted that Host helped to reduce their feelings of stress and isolation by providing emotional support and friendship.

With respect to achievement of immediate outcomes, the evaluation found that Host provided newcomers with social support and friendship and engaged newcomers in the social networks of Hosts. Focus group clients reported that Host provided them with helpful guidance and advice in a variety of areas (e.g., job search, financial management).

Moreover, Host was reported to have many positive impacts on volunteers. The majority of survey respondents report that volunteers’ expectations and goals are met to a large or greater extent. Focus group volunteers were highly satisfied with the Host program and indicated that it met their expectations to a great extent. Volunteers reported that their involvement in Host increased awareness, understanding, and appreciation of other cultures. Many focus group volunteers reported that they have been involved in previous successful matches and plan to continue being a Host volunteer. A number of volunteers also noted that they have continued friendships with their matches. Some Host volunteers also noted broader positive impacts on the volunteers’ families and friends.

Many focus group clients report that involvement in Host increased their awareness of Canadian society and values. They noted that this helped them feel more confident in communicating with others outside their own ethno-cultural background. Host was also noted as contributing to a number of intermediate results such as improved access to services, expansion of social networks, improved communication skills, and improved ability to achieve personal goals. Focus group clients indicated that, to a great extent, Host helped improve their ability to access community resources and services. Clients also reported increased confidence in conducting daily activities (e.g., banking, shopping, using the transit system) as a result of Host activities.

With respect to communication skills, many focus group clients/volunteers reported that Host provides clients with increased opportunities to practice their language skills, particularly for those who were involved in Host conversation circles.

Many focus group clients/volunteers reported that they met their personal goals principally in the areas of education and language improvement, less so with respect to employment.

Limited access to Host hampers its potential reach and therefore its effectiveness. The Host program represents only a small portion (about 3 percent) of the national settlement/resettlement budget. While many key informants considered access to be good given the level of resources, lack of timely access for some applicants and lack of awareness were identified as barriers to the program. Access was also reported to vary somewhat across regions and by community. However, in many cases the demand for Host exceeded its capacity. The majority of survey respondents (66 percent) reported waiting lists for newcomers. Twenty-one percent of survey respondents reported waiting times of over 60 days. Focus group clients indicated that Host would be most beneficial within the first month of arrival to Canada.

Adequacy of Capacity and Service Gaps

All lines of inquiry strongly support the contention that many Host service providers lack the capacity to effectively manage the program or to improve its access. Capacity, at the organizational level, refers to the resources, knowledge, and processes employed by the organization such as staffing; program management; and infrastructure and financial resources.

Most key informants attributed the lack of service provider capacity to lack of resources, lack of awareness of Host among mainstream organizations, and gaps in support for volunteers and Host coordinators.

With respect to funding levels, the great majority of key informants identified the overall level of funding for Host as inadequate. It was also noted that the level of funding has not increased in the past eight years despite rising overhead costs and additional responsibilities (e.g. iCAMS). More specifically, there were a number of identified gaps in funding with respect to outreach and promotion, volunteer management, support for Host Coordinators, and reporting. The majority of survey respondents, 59 percent, reported that Host funding was inadequate to achieve the expected outcomes; only 21 percent agreed that funding was adequate in this respect. Almost half of survey respondents, 45 percent, reported that funding was not adequate to meet the requirements of the contribution agreement—24 percent agreed that they had adequate funding to deliver the requirements of the contribution agreement.

Many SPO key informants and survey respondents indicated moderate levels of satisfaction with the support and guidance provided by CIC. Fifty-two (52) percent of survey respondents indicated that, overall, they were satisfied with the tools and support provided by the CIC—with 24 percent dissatisfied. Notably, CIC guidance was given the highest adequacy rating by survey respondents. Lower adequacy ratings were given to training workshops funded or coordinated by CIC, and to dissemination of research findings and best practices.

Some CIC and SPO key informants reported that Host coordinators need enhanced opportunities for professional development and additional support in terms of professional development activities (e.g., attendance at conferences or workshops) and increased hours. According to some key informants, the Host Coordinator salary is not commensurate with the skills required for the job (e.g., marketing, outreach and promotional skills, group facilitation/training skills, volunteer management skills).

Many SPO key informants reported that they lacked resources for volunteer management. With respect to volunteer screening, police checks of volunteers were reported to be general practice—although not policy. All SPO survey respondents report that they have policies and guidelines regarding volunteer screening and monitoring and the majority of respondents report that these guidelines and policies are effective. However, some CIC key informants noted that volunteer screening and risk management practices of SPOs should be reviewed to ensure that the risks associated with volunteer screening are adequately mitigated.

Focus group volunteers indicated high satisfaction with orientation and the ongoing support they received from the service provider. However, there were some noted gaps and inconsistencies with respect to volunteer support. These include the need for consistent provision of the following types of information, training and support: local community resources and services information; information about newcomers’ countries of origin; overview of immigration programs and services; additional support and training for dealing with high needs clients (e.g., life skills training); training to provide support for improving newcomer language skills; multicultural/cultural sensitivity training; increased opportunities to interact and share information with other volunteers; and, enhanced opportunities for social activities with other volunteers and newcomers.

With respect to promotion, many key informants indicated that there is a lack of awareness of Host among mainstream community organizations and the general public, for the most part, in larger urban areas. Nonetheless, the demand for Host exceeds capacity in many cases. The great majority of key informants, however, cited good awareness of Host among other immigrant serving organizations. Key informants stated that there is insufficient promotion of Host due to lack of resources. There is also limited capacity to handle additional matches, according to interviewees.

Most key informants note that there is adequate information sharing between local CIC offices and the SPOs. The majority of survey respondents, 73 percent, agree that communication/information/feedback from the CIC is adequate or more than adequate. There is also some sharing of information between SPOs and the national level through designated working groups.

Service provider respondents indicated that provincial organizations such as the Alberta Association of Immigrant Serving Agencies (AAISA) contributed to communication between SPOs at the provincial level. However, some CIC and service provider key informants noted gaps with respect to sharing best practice information nationally and inconsistent sharing of best practice information among service providers.

CIC key informants generally indicated that reporting was appropriate and adequate with respect to monitoring finances and activities, although some cited the need for better tracking of outcomes-based information. SPO key informants reported duplication of efforts as they submit reports to the local and regional CIC and to iCAMS.

A key reporting issue is the lack of accessible information about clients accessing Host. While service providers submit monthly or quarterly reports to the local CICs this information is not sufficiently rolled up and disseminated to the regional and national CIC levels. SPOs are currently entering information into a national database–iCAMS. The system is still in the development stages and data from this system is not presently available.

Challenges

These conclusions and the recommendations derived from them must be put in the context faced by the program and those involved in delivering it. These challenges and their implications are outlined in this section.

• Volunteerism is on the decline. The Host program depends on volunteers for its success but Host faces competition with other community agencies for volunteers. In addition, there has been a general decline in the number of volunteers since the 1990s. For example, just over 6.5 million Canadians volunteered in 2000, compared to 7.5 million in 1997. With the exception of Prince Edward Island, volunteer rates declined in all provinces from 1997 to 2000. Efforts to improve access or to expand the program must be understood within this context. Different group Host models such as conversation circles attempt to expand the reach of Host while relying on fewer volunteers. Moreover, enhanced promotional activities for volunteers targeted at specific groups (e.g., university students, other immigrants) may yield good results.
• Different levels of services available in regions/communities. The evaluation revealed that there were some variations in levels of settlement and other services across regions and communities. For example, some areas reported very long waiting lists for LINC (e.g., Calgary, Saskatoon). There were also some noted variations in the availability of employment services for newcomers. This was reported to impact negatively on the Host program as newcomers may have increased expectations of other settlement programs, such as Host, with respect to improving language skills or obtaining employment. Where Host is utilized as the main venue to fulfill these expectations, the dilution of other Host objectives, such as the provision of social support and friendship, may result.

Recommendations and Management Response

The recommendations based on the findings of the national evaluation of Host are presented as follows.

1. Expand the Reach of Host to Increase the Number of Clients. While Host is considered to be very effective, it has limited client reach. It is recommended that Host be expanded to offer services to an increased number of eligible clients. Host could be expanded either through an increase in number of individual matches or through an increase in group-based Host models—such as conversation circles—or by adopting both strategies. However, the majority of key informants stated that there are insufficient resources to conduct additional recruitment or outreach activities or to manage additional matches. Therefore, appropriate resources should be allocated towards meeting this recommendation. An assessment of group-based models should also be conducted to better inform CIC about potential impacts, risks and resource issues (see Recommendation 2). Enhanced outreach could also be achieved through targeted promotion to ethnic communities and through strengthened linkages and partnerships with other immigrant serving and mainstream organizations.

Management Response

The Department agrees with this recommendation.

The Department recognizes the benefits of group activities such as the Host Conversation Circles. Conversation circles are mechanisms that derive the maximum benefit from existing volunteers and target a larger number of eligible clients. Newcomer needs are also addressed in a timely manner through this initiative.

Other potential areas of expansion of the Host program include youth Host and professional mentoring, with additional support. Partnerships with other key stakeholders would be essential components of these initiatives.

The first National Host Conference is scheduled to take place in February 2005. It will present opportunities for representatives of service provider organizations (SPOs) to network and to exchange ideas on the establishment of effective partnerships with other agencies and mainstream organizations (see Management Response 2 for further details).

The Department is committed to finding innovative ways to reach a broader population of eligible clients. An example of this commitment is Ontario Region’s initiative to fund a project promoting settlement programs within ethnic communities at places most frequented by newcomers, such as ethnic supermarkets, recreational centers and others.

In order to establish an identifiable look for the Host program, the Department recently developed promotional materials such as videos, DVDs, brochures and Host certificates. These tools are intended to support SPOs in their volunteer orientation and outreach activities

Time Frame

Spring 2005: The National Host Conference will provide an opportunity to network and share information. Proceedings will also be developed and disseminated.

Fall 2005: The National Host Working Group will examine the review of conversation circles, and draft national guidelines.

2. Assess Various Host Delivery Models and Disseminate Lessons Learned/ Best Practices. The case studies—business mentoring and conversation circles—illustrate that different Host delivery models have the potential to address a range of newcomer needs. It is recommended that CIC continue to assess various existing or potential innovative Host delivery models (e.g., conversational circles, business mentoring, youth matches, professional matches, GAR pilot project). Reviews of existing delivery models or pilot projects could assess the potential impacts, lessons learned/best practices, and potential risks. In addition, a review of resource requirements for these models should also be conducted. Lessons learned and best practices should be disseminated to service providers.

Management Response

The Department agrees to take action on this recommendation.

Initiatives such as conversation circles, business mentoring, and group services to government-assisted refugees (GARs) contribute to the Department’s commitment to examine alternative forms of settlement services that will accommodate a greater number of eligible clients.

One of the objectives of the National Host Conference (February 2005) will be to share innovative practices within the settlement community. Innovative service delivery models such as youth Host, conversation circles, and professional mentoring will be presented at the conference. Lessons learned and best practices will form part of the conference proceedings that will be distributed among the settlement sector.

The Department will develop a template to capture the various service delivery models offered by the SPOs. The information collected would become a useful reference point for service provider organizations and other stakeholders involved in the Host program.

The Department is also exploring the possibility of conducting an inventory of youth Host models. The inventory will be the first phase in the development of national guidelines for youth Host.

Time Frame

Spring 2005: The National Host Working Group will initiate the collection of various delivery models. Following the conference, a report including innovative practices will be distributed.

3. Develop strategies to enhance the capacity of Service Providers to effectively recruit, train and support volunteers as Hosts. Although volunteer Hosts are the core of the program, many key informants noted that resources were not adequate enough to effectively support volunteer management activities at present levels. If expansion of Host is realized, then resource issues will become more critical.

1. Enhance Support for Service Providers and Host Coordinators. In order to effectively support volunteer management activities, Host coordinators must be adequately supported. CIC HQs and Regions should consult with service providers to identify the resources, tools and support required to adequately support service providers and coordinators. It is also recommended that best practices, key tools and resources relevant to volunteer management—including guidelines relating to screening and risk management—be adequately disseminated to service providers.
2. Enhance Recruitment of Volunteers. In some areas, additional volunteers are required to meet the current demand for Host. If Host is expanded, recruitment activities will need additional enhancements. Recruitment of appropriate volunteers can be achieved through promotional activities and selection tools. Key informants noted that there was an insufficient awareness of the Host program among mainstream community organizations and the general public, particularly in larger urban areas. Branding of the Host program, either nationally or regionally, would help to increase overall awareness of this program and facilitate recruitment. Specific groups such as post-secondary students or other immigrants could also be targeted for recruitment. It is also recommended that CIC make available to service providers, selected best recruitment practices and tools by reviewing available volunteer recruitment resources.
3. Enhance Volunteer Training and Support. Many key informants noted that there were insufficient resources for training and ongoing development and support of volunteers. Interview and focus group data revealed some gaps and inconsistencies with respect to resources, training and support (e.g., opportunities for volunteers to share information with each other, volunteer appreciation activities, etc.) available to volunteers. It is recommended that CIC HQs and Regions consult with service providers to identify volunteer training and development gaps, and appropriate strategies and resources to address these gaps. It is also recommended that strategies be determined for addressing these gaps (e.g., how best to support volunteers – funds to HQs for national resources and tools or to the local development of professional activities).

Management Response

The Department agrees with the principle of this recommendation.

One of the objectives of the National Host Conference (February 2005) will be to share innovative practices and tools that support SPOs and volunteers. Through workshops and presentations, the conference will provide a positive learning environment for issues such as volunteer recruitment, screening and training, and to discuss options relating to effective volunteer management. The Host conference may be an opportunity to recognize publicly the contribution of volunteers to the program, and the volunteer management efforts to-date of SPOs.

The Department will initiate the development of a Host Manual in order to create a more consistent approach to service delivery and to support Host workers in their day-to-day activities. The manual will serve as a vehicle to share successful practices and to guide service providers in the implementation of such practices. Guidelines relating to the screening of volunteers, which is part of the broader volunteer management function, will be a component of this manual.

The Department continues to support the efforts of SPOs in their recognition of volunteers. The recently developed Host Volunteer Certificate is a tool used by SPOs to acknowledge the contribution of volunteers.

The Department will explore opportunities that will encourage SPOs to target volunteers such as students, seniors, and immigrants. The Department will also promote the distribution of promotional materials. Additional outreach is contingent upon additional resources.

The promotion of the Host program and its benefits to newcomers can be further explored through various communication mediums.

In consultation with service provider organizations, the Department recently developed promotional materials such as videos, DVDs, brochures and Host certificates. These tools are intended to support SPOs in their volunteer orientation and outreach activities. These products established a common look for the Host program. Future developments of the Host program will build on these products to support outreach activities of SPOs, subject to additional support.

Time Frame

Spring 2005: Following the conference, a report including innovative practices will be distributed.

4. Streamline, Reduce Duplication in Reporting Requirements for Host Service Providers. The evaluation found that there was duplication of effort as service providers were required to report to the local CIC offices and through iCAMS. Currently, service providers are required to enter data monthly into iCAMS, as well as submit monthly or quarterly—depending on the contribution agreement—activity reports. It is recommended that service providers only be required to enter data into iCAMS and that the written report requirement be dropped. This will only be feasible once reports are regularly being produced and made available to all levels of CIC—by individual SPO and in aggregate by city and region—and service provider—for the activities undertaken by their organization. It is further recommended that, where narrative reports are currently required, they continue to be submitted, but on a quarterly basis. These changes to the current reporting requirements would ease the funding pressures at service provider organizations and enable staff to spend more time delivering direct services.

Management Response

The Department agrees that efforts must be made to reduce duplication in reporting. The Department is aware that SPOs report not only to CIC but to other funders, in addition to the records they maintain for their own internal management.

The Department has already reduced one part of CIC duplicate data entry through a system interface for LINC services in Ontario, and special data entry arrangements in a small number of other situations.

 In addition, the Department has recently started a report consolidation process. The Department is consulting internally on the type of reports that can be generated using the data from the performance measurement system (iCAMS) in order to meet the monitoring requirements of CIC local offices. These new reports would replace some of the existing monthly and quarterly written activity—statistical reports submitted to local offices. However, they could not replace the qualitative and narrative data, or the data that is not currently collected through iCAMS.

As iCAMS is quantitative in nature, the written narrative report requirement remains an important tool to determine SPOs’ activities and unique initiatives, and to identify trends and changes in immigration patterns from the perspective of the SPO. Decisions to proceed with quarterly written reports instead of monthly must be made at the local or regional level. Integration Branch will recommend quarterly reporting requirements, while allowing regions the flexibility for alternate reporting time frames.

Time Frame

Reports resulting from the consolidation exercise are expected in spring 2005.

5. Provide More and Improved Information on Clients. There is an identified need for accessible information on clients. This would enable CIC to make more informed programming priorities and decisions. Specifically, this information could contribute to the allocation of resources, strategies for expanding reach, and tools and support for volunteer management processes, based on who is accessing which services and how often. There is also a need for outcome-related information.

Management Response

The Department agrees, in principle, with this recommendation.

The Department will have access to information on the demographic composition of client groups and service usage through the performance measurement system (iCAMS). This data will inform analyses for program improvements.

The Department plans to make some of these reports available to service providers. Where reports include client demographics, these will be displayed on an aggregate basis (no individuals are identifiable) and will enable service providers to view reports concerning only their clients.

The Department will need to secure ongoing funding for the regular maintenance of accountability activities. The maintenance of iCAMS would update the system and produce relevant reports.

The Department will explore the possibility of developing and piloting a national survey for immigrants to complete during their citizenship process. This survey would capture what settlement services immigrants accessed and which were found to be most beneficial in the settlement process.

Time Frame

iCAMS will be able to generate HOST reports by summer 2005. CIC is also developing a distribution strategy for these reports.

<< contents | next >>