Evaluation of HOST

Appendix A: Methodology

The following lines of evidence were employed for the evaluation of Host:

• Data and document review;
• Interviews with 18 CIC representatives;
• Interviews with 12 SPOs and two (2) stakeholders;
• Five focus groups with Host clients—including youth;
• Four focus groups with Host volunteers; and
• A telephone survey of 29 SPOs.

Data and Document Review

Where key informants supplied GGI with examples of materials or reports, these were reviewed in the context of developing a better sense of how the program was delivered in that region/community. websites were also reviewed including settlement.org and integration-net.

GGI also received some high level data from the LSICs and IMDB and this was used to develop an understanding of the context for the program.

Interviews with CIC HQs, Regions and Local Offices

The research team conducted interviews with 18 individuals—often in a group interview setting. Of those, six (6) individuals were interviewed at HQs and 12 were interviewed from the regions/local offices. Interviews were conducted in-person if the respondent was located in the National Capital Region or in a city visited as part of the focus groups (where possible). Otherwise, interviews were conducted over the telephone. Interviewees were invited to participate in their preferred official language.

Interviews with Service Providers and Stakeholder Organizations

GGI conducted a total of 12 service provider interviews and two (2) stakeholder interviews. Interviews with service providers differed from those conducted as part of the survey in that this line of evidence probed more deeply into issues of program relevance, design, overall success and alternatives.

As for the interviews with CIC, interviews conducted in Ottawa and in the sites visited for the focus groups were conducted in-person (where possible). The others were conducted by telephone. Respondents were invited to participate in the interview in the official language of their choice.

Focus Groups with Clients

A total of four (4) focus groups were undertaken with program clients. Focus groups were held in service provider locations with the exception of one Ottawa group conducted in the GGI boardroom. Focus groups were carried out as follows:

• One (1) in Ottawa (in French);
• Two (2) in Toronto;
• One (1) in Halifax; and
• One (1) in Calgary.

Up to two service providers were asked to assist in the recruitment of each focus group—each was paid $75 for each group of five (5) participants they recruited. They were asked to recruit a mix of clients. The selected clients should be:

• No longer accessing Host services as well as some clients who continue to receive services;
• Chosen from men and women;
• From different age groups;
• Both single individuals and immigrants with families;
• From different ethnic/racial backgrounds; and
• Government-assisted refugees (GARs) (just one or two per group).

In order to get a different perspective on the services offered, some focus groups had participants who spoke a language other than English or French. In those cases, interpreters were available for each non-official language speaker in the group.

In all, 10 participants were recruited for each group with the intention of having eight (8) show up for the discussion. Overall, groups were well-attended. Participants were offered an honorarium of $50 to offset any expenses they may have incurred to participate (e.g., child minding, transportation, parking).

GGI prepared a focus group moderator’s guide. This guide is semi-structured allowing the moderator to pose questions that flow from one issue to the next. The moderator made efforts to involve all participants in the discussion. The other evaluation team member attending the discussion was responsible for taking notes. Due to the fact that many newcomers are sensitive about the confidentiality of the information being provided, no audio or videotapes were produced.

Focus Groups with Volunteers

A total of four focus groups were undertaken with Host volunteers as follows:

• One (1) in Ottawa (in French);
• One (1) in Toronto;
• One (1) in Halifax; and,
• One (1) in Calgary.

As with the focus groups with clients, Host service providers were asked to recruit 10 participants. Up to two service providers were asked to assist in the recruitment of each focus group— each was paid $75 for each group of five (5) participants they recruited. Participants were offered an honorarium of $50 to offset any expenses they might have incurred to participate (e.g., child minding, transportation, parking). GGI prepared a semi-structured focus group moderator’s guide allowing the moderator to pose questions that flowed from one issue to the next.

Survey of Service Providers

A telephone survey of all service providers delivering Host was conducted to assess the ongoing relevance, success, design, and delivery of the programs. All current service providers—those who have signed a contribution agreement for the 2004/05 fiscal year—were contacted. GGI managed to complete 29 of the 33 names provided by CIC (representing a response rate of 88%).

Prior to the fieldwork commencing, a notification letter was sent to all potential respondents in the sample to outline the purposes of the study, the specific survey, and the time requirements.

Potential respondents were contacted by telephone to ascertain whether they were willing to participate in a phone interview. If they were willing to participate, the respondent was asked to complete the interview over the telephone at that time. If the time was not convenient, an appointment was arranged when the GGI interviewer could call them back to complete the interview.

All potential respondents were called up to five times before being considered a non-response. Service providers were called on different days and at different times during the business day (in their time zone) to ensure that respondents were not called back at the same time each day. All phone interviews were conducted using Computer Assisted Telephone Interviewing (CATI) systems. Interviews were conducted in the preferred official language of the respondent. Interviews lasted between 20 and 30 minutes, on average, for the Host component of the questionnaire.

Case Studies of Innovative Practices

Two case studies of innovative practices in Host delivery were conducted—and continue to be conducted in some cases. The following cases were selected:

• Business mentoring (Atlantic); and,
• Conversation circles (Ontario).

For each of the case studies, a document review and up to two interviews were conducted. For each, the factors that make the project/case an innovative practice and how it might be applicable to a wider context and in different jurisdictions were explored. A short, three to four page, case study write-up was prepared, including conclusions regarding wider applicability of the case (see Appendix C).

<< contents | previous | next >>