ARCHIVED – Summative Evaluation of the Metropolis Project Phase II: Knowledge Transfer Activities and Impacts

Appendix A: Evaluation Matrix

Knowledge transfer – policy outcomes

# Evaluation Issues Indicator Methodology
1 Have the Metropolis Centres conducted macro (incl. pan-Canadian), comparative and longitudinal studies to support policy development?
  • # & type of macro, comparative & longitudinal research studies
  • #, type & distribution/attendance of Metropolis products developed from this research
  • Stakeholder views re relative value of macro studies
  • Review of Centres’ Activity Reports
  • Case studies
  • Survey of potential (federal) users
  • Interviews with funding partners
2 Have the Metropolis Centres successfully integrated key policy issues identified by federal funding partners into their research plans?
  • Centres’ processes (formal and informal) to solicit input from federal funding partners on key policy issues (research topics, conference themes) that are of interest for them
  • # and type of federal funding partners engaged in these processes
  • Centres’ processes (formal and informal) to convey policy needs to research community
  • Departmental processes (formal and informal) to provide Centres with key policy research needs
  • Distribution of Centre projects in relation to 11 policy issue areas (# and value)
  • Nature of projects in 11 policy issue areas (research question linked to policy needs?)
  • Stakeholders views re processes and distribution
  • Document Review: Annual Reports; descriptions of funded projects, other Centre/ Secretariat reports
  • Review of Centres’ & Secretariat Activity Reports
  • Case studies
  • Survey of potential (federal) users
  • Interviews with funding partners

 

3 Is Metropolis producing research products that are relevant to government policy makers?
  • #, type & distribution/attendance of products directed to policy-makers
  • Stakeholder views re utility
  • Review of Centres’ & Secretariat Activity Reports
  • Case studies
  • Survey of potential (federal) users
  • Interviews with funding partners
4 Do federal policy-makers access Metropolis research? Do departments support the access and use of research in policy-making?
  • No. of policy-makers associated with Metropolis (including involvement in committees, distribution lists, conference attendees, etc.)
  • Departmental policies, procedures, directives re access and use of research
  • Usability of Metropolis products including websites
  • Stakeholder views
  • Document Review: policies, directives, etc.
  • Review of Centres’ & Secretariat Activity Rpts
  • Case studies
  • Survey of potential (federal) users
  • Interviews with funding partners
5 Has knowledge transferred from Metropolis informed and influenced government policy development? (This question, in conjunction with Q.4, will look at a spectrum of possible use by policy-makers.)
  • Examples of ways in which Metropolis research has informed and influenced policy-making
  • Activities related to policy development (presentations, background documents, briefs, MCs and TB submissions) that have been directly or indirectly influenced by Metropolis outputs
  • Case studies
  • Survey of potential users
  • Interviews with funding partners
6 Do the Centres, Secretariat and Federal Consortium operate as effective knowledge “brokers” in the knowledge transfer process? (A framework identifying the different possible roles & activities of brokers will need to be developed.)
  • Centres & Secretariat processes to link knowledge producers and users, including processes to develop research products for policy-makers
  • Stakeholder views
  • Review of Centres’ & Secretariat Activity Reports
  • Case studies
  • Survey of potential (federal) users
  • Interviews with funding partners

Page details

Date modified: