ARCHIVED – Summative Evaluation of the Metropolis Project Phase II: Knowledge Transfer Activities and Impacts

Appendix F: Distribution of findings based on familiarity with the Metropolis Project

Conclusion: Based on survey results, respondents who reported being ‘quite’ or ‘very familiar’ with Metropolis were, overall, more positive about their experience. They participated/used Metropolis research/activities to a greater extent than people who only had a limited familiarity with Metropolis. They also were more satisfied with their Metropolis experience.

Context/issue:

The extent to which potential Metropolis users were familiar with the Metropolis project may affect their use and perception of it. This is the reason why, in this section, a closer look is taken at how survey respondent perceptions and opinions are related to their familiarity with the Metropolis Project. People who are familiar with the Metropolis Project may have had a different Metropolis experience than the ones that are less familiar, which may lead to different perceptions and opinions.

However, before further discussing the results, it must be emphasized that knowledge transfer activities are supposed to be directed towards and reach all potential users and not only a sub-group of people who are highly familiar with the Project.

Also noteworthy, the familiarity question is subjective in nature. It reflects the perception of the respondents’ knowledge on what they believe the Metropolis Project is (ie: they may believe that the Center they know is Metropolis).

Profile of Potential Users (survey respondents):

Survey respondents that were identified as potential users were split as to their familiarity with the Metropolis project. About half of the respondents were either ‘quite’ or ‘very familiar’ with the Metropolis Project (47%) and a little over half were ‘not at all’ or ‘not very’ familiar with it (52%) [ Note 24 ].

Table 1: Familiarity with the Metropolis Project

Level of familiarity Frequency Percent
Not at all familiar 53 19.4%
Not very familiar 89 32.6%
Quite familiar 76 27.8%
Very familiar 52 19.0%
Do not know 3 1.1%
Total 273 100.0%

Even higher percentages reported not being familiar with the various Centres and the Secretariat, although respondents were somewhat more familiar with the Centre in the region where they are located (see Table 2).

Table 2: Familiarity with the Metropolis Centres and Secretariat*

Centre/
secretariat
Not familiar Familiar
All respondents Respondents located in the region, in question All respondents Respondents located in the region, in question
Secretariat 64% 58% 34% 40%
IM (Quebec region) 85% 72% 12% 28%
CERIS (Toronto region) 71% 65% 26% 35%
PMC (Prairie region) 81% 41% 16% 59%
RIIM (BC region) 80% 17% 18% 67%

* Not familiar includes those who indicated they were ‘not at all familiar’ or ‘not very familiar. Familiar includes those who indicated they were ‘very familiar’ or ‘quite familiar’.

Senior managers and those who identified themselves as policy advisors/analysts/officers were more likely to be familiar with Metropolis than the total survey population. As illustrated by Table 3, almost three quarters of senior management respondents (73.9%) and 61% of policy advisors/analysts or officers indicated they were ‘quite’ or ‘very familiar’ with Metropolis. On the other hand, Program/project advisors/analysts/officers were more likely to report being ‘not at all’ or ‘not very familiar’ with Metropolis.

Table 3: Position by familiarity with the Metropolis Project*

Position Not familiar (n=142) Familiar (n=128) Total (n=270)
 Senior Management (n=23) 21.6% 73.9% 100.0%
Other management (n=31) 48.4% 51.6% 100.0%
Program/Project Advisor, Analyst or Officer (n=102) 68.6% 31.4% 100.0%
Policy Advisor, Analyst or Officer (n=64) 39.1% 60.9% 100.0%
Research Advisor, Analyst or Officer (n=48) 52.1% 47.9% 100.0%
Other (n=2) 50.0% 50.0% 100.0%
Total (n=270) 52.6% 47.4% 100.0%

* Not familiar includes those who indicated they were ‘not at all familiar’ or ‘not very familiar. Familiar includes those who indicated they were ‘very familiar’ or ‘quite familiar’.

Similarly, those survey respondents who were mainly involved in research, or in policy development and/or advice, were more likely to be familiar with Metropolis, while those in Program design or delivery and operational/client services were less familiar with it.

Table 4: Type of work by familiarity with the Metropolis Project*

Type of work Not familiar (n=142) Familiar (n=128) Total (n=270)
Research (n=65) 41.5% 58.5% 100.0%
Policy development and/or advice (n=78) 29.5% 70.5% 100.0%
Program design or delivery (n=68) 64.7% 35.3% 100.0%
Operational / client services (n=44) 81.8% 18.2% 100.0%
Other (n=15) 80.0% 20.0% 100.0%
Total (n=270) 52.6% 47.4% 100.0%

* Not familiar includes those who indicated they were ‘not at all familiar’ or ‘not very familiar. Familiar includes those who indicated they were ‘very familiar’ or ‘quite familiar’.

Differences were also noted, in terms of familiarity with Metropolis, across the federal funding partner organizations included in the survey. More of CIC’s and PCH’s respondents reported being familiar with Metropolis, while the reverse stands for surveyed employees from HRSDC. Table 5 illustrates the level of familiarity with Metropolis of respondents from the various organizations.

Table 5: Department/Organization by familiarity with the Metropolis Project*

Department Not familiar (n=142) Familiar (n=128) Total (n=270)
PCH (n=29) 37.9% 62.1% 100.0%
CIC (n=116) 41.4% 58.6% 100.0%
HRSDC (n=105) 66.7% 33.3% 100.0%
Other (n=20) 65.0% 35.0% 100.0%
Total (n=270) 52.6% 47.4% 100.0%

* Not familiar includes those who indicated they were ‘not at all familiar’ or ‘not very familiar. Familiar includes those who indicated they were ‘very familiar’ or ‘quite familiar’.

Respondents in the NCR were in majority ‘quite’ or ‘very familiar’ with the Metropolis Project (53%), and survey respondents outside the NCR had a greater propensity to answer they were ‘not at all’ or ‘not very familiar’ with the Project (63%). The breakdown of the familiarity by location is illustrated in table 6.

Table 6: Region by familiarity with the Metropolis Project*

Location Not familiar (n=142) Familiar (n=128) Total (n=270)
NCR (n=180) 47.2% 52.8% 100.0%
Regions (n=90) 63.3% 36.7% 100.0%
Total (n=270) 52.6% 47.4% 100.0%

* Not familiar includes those who indicated they were ‘not at all familiar’ or ‘not very familiar. Familiar includes those who indicated they were ‘very familiar’ or ‘quite familiar’.

Use of Metropolis Products and Activities:

Survey results indicate that the use of Metropolis increases with familiarity with the project. Table 7 shows that on average people who report having a good familiarity with the Metropolis Project devote a higher percentage of their time reading/reviewing research materials and/or attending research dissemination activities relating to immigration, settlement and/or diversity issues on Metropolis products/activities (48%).

Table 7: Percentage of time devoted to Metropolis products/activities by familiarity with the Metropolis Project*

  Not familiar (n=55) Familiar (n=111) Total (n=166)
Mean 41.5% 48.4% 46.2%
Median 37.5% 50.0% 40.7%

* Not familiar includes those who indicated they were ‘not at all familiar’ or ‘not very familiar. Familiar includes those who indicated they were ‘very familiar’ or ‘quite familiar’.

Respondents who are more familiar with Metropolis are also using/participating to a greater extent each product/activities listed in Table 8. Although to a different degree, the most used or participated in products for both groups are Centre research reports or publications and Metropolis/Centers websites followed closely by magazines and policy syntheses. Note that both familiar and unfamiliar identified similar ranking of items.

Table 8: Use of Metropolis products/activities by familiarity with the Metropolis Project*

Used or participated in: Not familiar (n=142) Familiar (n=128) Total (n=270)
National or international conferences 16.9% 71.9% 43.0%
Centre retreats 4.9% 18.8% 11.5%
Centre workshops 7.7% 44.5% 25.2%
Centre research reports or publications (including Working Papers) 35.9% 89.1% 61.1%
Policy Syntheses (e.g., Policy Matters, Research Capsules) 25.4% 68.8% 45.9%
Metropolis Conversations 9.2% 51.6% 29.3%
Metropolis Presents 7.0% 46.9% 25.9%
Our Diverse Cities magazine 21.8% 71.1% 45.2%
Canadian Issues magazine 20.4% 65.6% 41.9%
Canadian Diversity magazine 23.9% 67.2% 44.4%
Journal of International Migration and Integration (JIMI) 16.2% 57.0% 35.6%
Special issues of other journals 9.2% 42.2% 24.8%
Metropolis and/or Centres’ websites 31.0% 88.3% 58.1%
Other 7.7% 9.4% 8.5%

* Not familiar includes those who indicated they were ‘not at all familiar’ or ‘not very familiar. Familiar includes those who indicated they were ‘very familiar’ or ‘quite familiar’.

Almost all respondents who were familiar with Metropolis used at least one Metropolis output (98%), and a research product (92%). In opposition, respondents who declared being less familiar with Metropolis reported lower use of Metropolis outputs; only 55% had used at least one of the outputs listed in the previous table, and at least one research product (45%). For both groups, the participation in Metropolis activities was lower than for research products, with 81% of the respondents familiar with Metropolis and 19% of the ones who were less familiar reporting the participation in at least one Metropolis activity (see table 9).

Table 9: Use of at least one of Metropolis product by familiarity with the Metropolis Project*

Participated/used at least one Metropolis: Not familiar (n=142) Familiar (n=128) Total (n=270)
Output (activity or research) 54.90% 98.40% 75.60%
Activity 19.00% 81.30% 48.50%
Research product 45.10% 92.20% 67.40%

* Not familiar includes those who indicated they were ‘not at all familiar’ or ‘not very familiar. Familiar includes those who indicated they were ‘very familiar’ or ‘quite familiar’.

As shown in Table 10, when using Metropolis research, both groups of respondents, those who were familiar and not familiar with Metropolis, reported they used it to keep up-to-date in their field although to a different extent (75% and 30% respectively). Many also used it to inform policy discussions (65% and 24%) and to inform program design and delivery (43% and 23%). However, half of the respondents who reported being not at all or not very familiar with Metropolis reported not using Metropolis research.

Table 10: Use made of Metropolis research by familiarity with the Metropolis Project*

Used Metropolis research: Not familiar (n=142) Familiar (n=128) Total (n=270)
To inform policy discussions 23.9% 64.8% 43.3%
To inform program design and delivery 23.2% 43.0% 32.6%
To inform and align research development 14.1% 39.1% 25.9%
To be up-to-date in your field 29.6% 75.0% 51.1%
For your own personal information 12.0% 29.7% 20.4%
Did not use Metropolis research 50.7% 2.3% 27.8%
Other 4.9% 5.5% 5.2%
Total 100% 100% 100%

* Not familiar includes those who indicated they were ‘not at all familiar’ or ‘not very familiar. Familiar includes those who indicated they were ‘very familiar’ or ‘quite familiar’.

Notwithstanding their familiarity with the Metropolis Project, the majority of survey respondents indicated that it was either ‘quite’ or ‘very important’ that Metropolis conducted pan-Canadian, comparative and longitudinal research.  However, respondents who were ‘quite’ or ‘very familiar’ with Metropolis reported this in higher numbers as indicated in table 11 below. Less important but again, worth mentioning is that the “unfamiliar” group reports higher ‘do not know’ frequencies.

table 11: Importance that Metropolis conducts pan-Canandian, comparative and longitudinal research by familiarity with the Metropolis Project*

Research types Level of importance Not familiar Familiar Total
Pan-Canadian research (n=217) Not at all or not very important 14.6% 13.2% 13.8%
Quite or very important 62.5% 83.5% 74.2%
Do not know 22.9% 3.3% 12.0%
Total 100% 100% 100%
Comparative research (n=216) Not at all or not very important 12.6% 7.4% 9.7%
Quite or very important 66.3% 90.1% 79.6%
Do not know 21.1% 2.5% 10.6%
Total 100% 100% 100%
Longitudinal research (n=214) Not at all or not very important 13.5% 10.2% 11.7%
Quite or very important 59.4% 88.1% 75.2%
Do not know 27.1% 1.7% 13.1%
Total 100% 100% 100%

* Not familiar includes those who indicated they were ‘not at all familiar’ or ‘not very familiar. Familiar includes those who indicated they were ‘very familiar’ or ‘quite familiar’.

Satisfaction With Metropolis:

In general, survey respondents who identified themselves as being familiar with Metropolis express greater satisfaction with it. For each of the products/activities they used/participated in they report in greater proportions that they believe it was ‘quite’ or ‘very useful’ to them (around 80 to 90% for most of the products). For this group, Centre research reports or publications and National or international conferences were perceived to be useful by 93% and 88% of the 114 and 92 respondents that reported using these products. Although more mixed, the results from survey respondents with little familiarity with Metropolis show that they generally perceived the products to be useful to them. For them, Centre research reports or publications appeared to be the most useful product (84,3% out of a total of 51 respondents), followed by policy syntheses (77,8% out of a total of 36 respondents). However, Metropolis Conversations and Metropolis Presents were less appreciated, with respectively 62% and 60% (out of 13 and 10 respondents who used these products) of this group reporting that these products were ‘not at all’ or ‘not very useful’. For both groups, the product that was identified as being the least useful was the Centre retreats, with 33% of the 33 respondents who were familiar with Metropolis and that participated in this activity and 86% of the 7 respondents with little familiarity with Metropolis answering that it was ‘not at all’ or ‘not very useful’. Table 12 reports on the perceived usefulness of Metropolis products and activities.

Table 12: Usefulness of Metropolis products/activities used/participated by familiarity with the Metropolis Project*

Usefulness of: Not familiar Familiar Total
Not useful Useful Not useful Useful Not useful Useful
National or international conferences (n=116) 29.2% 70.8% 12.0% 88.0% 15.5% 84.5%
Centre retreats (n=31) 85.7% 14.3% 33.3% 66.7% 45.2% 54.8%
Centre workshops (n=68) 54.5% 45.5% 17.5% 82.5% 23.5% 76.5%
Centre research reports or publications (including Working Papers) (n=165) 15.7% 84.3% 7.0% 93.0% 9.7% 90.3%
Policy Syntheses (e.g., Policy Matters, Research Capsules) (n=124) 22.2% 77.8% 13.6% 86.4% 16.1% 83.9%
Metropolis Conversations (n=79) 61.5% 38.5% 16.7% 83.3% 24.1% 75.9%
Metropolis Presents (n=70) 60.0% 40.0% 18.3% 81.7% 24.3% 75.7%
Our Diverse Cities magazine (n=122) 32.3% 67.7% 13.2% 86.8% 18.0% 82.0%
Canadian Issues magazine (n=113) 27.6% 72.4% 14.3% 85.7% 17.7% 82.3%
Canadian Diversity magazine (n=120) 32.4% 67.6% 16.3% 83.7% 20.8% 79.2%
Journal of International Migration and Integration (JIMI) (n=96) 30.4% 69.6% 17.8% 82.2% 20.8% 79.2%
Special issues of other journals (n=67) 53.8% 46.2% 24.1% 75.9% 29.9% 70.1%
Metropolis and/or Centres’ websites (n=157) 20.5% 79.5% 19.5% 80.5% 19.7% 80.3%
Other (n=23) 72.7% 27.3% 50.0% 50.0% 60.9% 39.1%

* Not familiar includes those who indicated they were ‘not at all familiar’ or ‘not very familiar. Familiar includes those who indicated they were ‘very familiar’ or ‘quite familiar’. Not useful includes those who indicated ‘not at all useful’ or ‘not very useful’. Useful includes those who indicated ‘quite useful’ or ‘very useful’.

Survey respondents who were ‘quite’ or ‘very familiar’ with Metropolis show greater satisfaction in regards to timeliness, relevance and overall quality of Metropolis research, as indicated in Table 13. However, the number of ‘do not know’ responses is much higher among those unfamiliar with Metropolis.

Table 13: Satisfaction with timelines, relevance and overall quality of Metropolis research by familiarity with the Metropolis Project*

Category of evaluation Level of satisfaction Not familiar Familiar Total
Timeliness (n=176) Very dissatisfied or dissatisfied 0.0% 3.6% 2.3%
Neither satisfied nor dissatisfied 22.7% 20.9% 21.6%
Satisfied or very satisfied 37.9% 70.9% 58.5%
Do not know 39.4% 4.5% 17.6%
Total 100% 100% 100%
Relevance (n=206) Very dissatisfied or dissatisfied 6.1% 4.9% 5.4%
Neither satisfied nor dissatisfied 20.7% 9.8% 14.1%
Satisfied or very satisfied 52.4% 82.9% 70.7%
Do not know 20.7% 2.4% 9.8%
Total 100% 100% 100%
Overall quality (n=200) Very dissatisfied or dissatisfied 4.9% 4.2% 4.5%
Neither satisfied nor dissatisfied 17.3% 8.4% 12.0%
Satisfied or very satisfied 53.1% 82.4% 70.5%
Do not know 24.7% 5.0% 13.0%
Total 100% 100% 100%

* Not familiar includes those who indicated they were ‘not at all familiar’ or ‘not very familiar. Familiar includes those who indicated they were ‘very familiar’ or ‘quite familiar’.

For both groups, this was the method that most respondents identified as Trends regarding the usefulness of various ways of providing departmental input to Metropolis on their department’s needs are reported in Table 14. Overall, respondents who had limited familiarity with Metropolis found the five methods ‘not at all’ or ‘not very useful’, while the respondents who were familiar with Metropolis found them ‘quite’ or ‘very useful’. For example, 89% and 88% of the 8 and 9 respondents with little familiarity with Metropolis answered that Centres’ Annual Retreats and Metropolis Interdepartmental Working Committee (IDC) were ‘not at all’ or ‘not very useful’ to them when providing their input, while 83% and 77% of the 44 and 29 respondents who were familiar with Metropolis answered that these same two products were ‘quite’ or ‘very useful’ to them. The only exception is for discussing priorities with individual Metropolis researchers, where the majority of respondents with little familiarity with Metropolis (62% out of 13 respondents) identified this method as being useful to them.

Table 14: Usefulness of methods to provide input to Metropolis on department’s research needs by familiarity with the Metropolis Project*

Usefulness of: Not familiar Familiar Total
Not useful Useful Not useful Useful Not useful Useful
Centres’ Annual Retreats (n=38) 88.9% 11.1% 17.2% 82.8% 34.2% 65.8%
Metropolis Interdepartmental
Working Committee (IDC) (n=52)
87.5% 12.5% 22.7% 77.3% 32.7% 67.3%
Centres’ workshops (n=60) 70.0% 30.0% 20.0% 80.0% 28.3% 71.7%
Discussed priorities with individual
Metropolis researchers (n=85)
38.5% 61.5% 15.3% 84.7% 18.8% 81.2%
Provided written input to a Metropolis
Centre(s) or the Metropolis Secretariat (n=45)
71.4% 28.6% 13.2% 86.8% 22.2% 77.8%
Provided input through a Centre’s
Board of Directors (n=31)
70.0% 30.0% 23.8% 76.2% 38.7% 61.3%
Other (n=16) 87.5% 12.5% 25.0% 75.0% 56.3% 43.8%

* Not familiar includes those who indicated they were ‘not at all familiar’ or ‘not very familiar. Familiar includes those who indicated they were ‘very familiar’ or ‘quite familiar’. Not useful includes those who indicated ‘not at all useful’ or ‘not very useful’. Useful includes those who indicated ‘quite useful’ or ‘very useful’.

Respondents who used at least one of the options listed above and were ‘quite’ or ‘very familiar’ with the Metropolis project were also ‘satisfied’ or ‘very satisfied’ with these processes for identifying and communicating policy information needs and priorities to Metropolis (67%). Respondents with limited familiarity with Metropolis were more split, with 31% of them who did not know and who appeared ‘neither satisfied nor dissatisfied’ with the processes. Table 15 reports on these findings.

Table 15: Satisfaction with the processes to identify and communicate policy information needs and priorities to Metropolis by familiarity with the Metropolis Project*

Level of satisfaction Not familiar (n=13) Familiar (n=92) Total (n=105)
Very dissatisfied or dissatisfied 23.1% 9.8% 11.4%
Neither satisfied nor dissatisfied 30.8% 20.7% 21.9%
Satisfied or very satisfied 15.4% 67.4% 61.0%
Do not know 30.8% 2.2% 5.7%
Total 100.0% 100.0% 100.0%

* Not familiar includes those who indicated they were ‘not at all familiar’ or ‘not very familiar. Familiar includes those who indicated they were ‘very familiar’ or ‘quite familiar’.

As indicated in Table 16, most (62% and 68%) of the 98 and 97 respondents who had limited familiarity with Metropolis did not have an opinion of the knowledge brokering role of Metropolis. However, over half of the 115 and 122 respondents who were familiar with Metropolis believed that the Metropolis Centres and the Secretariat often or always operated as an effective knowledge broker (50% and 58% respectively).

Table 16: Opinion as to the effectiveness of Metropolis in its knowledge brokering role by familiarity with Metropolis*

Metropolis Rate of effectiveness Not familiar Familiar Total
Centres (n=213) Never or rarely 15.3% 10.4% 12.7%
Sometimes 12.2% 17.4% 15.0%
Often or always 10.2% 50.4% 31.9%
Do not know 62.2% 21.7% 40.4%
Total 100.0% 100.0% 100.0%
Secretariat (n=219) Never or rarely 15.5% 9.8% 12.3%
Sometimes 6.2% 10.7% 8.7%
Often or always 10.3% 58.2% 37.0%
Do not know 68.0% 21.3% 42.0%
Total 100.0% 100.0% 100.0%

* Not familiar includes those who indicated they were ‘not at all familiar’ or ‘not very familiar. Familiar includes those who indicated they were ‘very familiar’ or ‘quite familiar’.

___________

24. There are 1% of the respondents that did not know to what extent they were familiar with the Metropolis project. The remaining analysis presented in this section will exclude those respondents who answered ‘Do not know’ on the familiarity question.

Page details

Date modified: