CIC Evaluation Policy

Table of Contents


1. Policy Objective

1.1  The objective of this policy is to ensure that Citizenship and Immigration Canada (CIC) has an effective and independent evaluation function.

2. Policy statement

2.1 CIC will maintain an effective evaluation function that plays a strategic role in providing the Department with objective, timely and evidence-based findings, conclusions, and recommendations on the relevance and performance of programs, policies and initiatives, including conducting evaluations of the main grants and contributions programs. Evaluation is an integral part of the policy process at CIC; the results of evaluation research inform ongoing policy development and program design.

3. Authority

3.1  This CIC Policy is carried out under the authority of the Treasury Board Secretariat Policy on Evaluation (2009), which seeks to:

  • Create a comprehensive and reliable base of evaluation evidence that is used to support policy and program improvement, expenditure management, Cabinet decision-making, and public reporting; and
  • Ensure that credible, timely, and neutral information on the ongoing relevance and performance of direct program spending is:
    • Available to Ministers, central agencies and deputy heads and used to support evidence-based decision-making on policy, expenditure management and program improvements; and
    • Available to Parliament and Canadians to support government accountability for results achieved by policies and programs.

4. Context

4.1 Evaluation supports:

  • CIC accountability to Parliament and Canadians, by helping the government to credibly report on the results achieved with resources invested in programs;
  • Decisions on resource allocation and reallocation to CIC programs, by assessing the ongoing relevance and performance of existing programs;
  • Managing for results, by informing the Deputy Minister about whether CIC programs are producing the outcomes they were designed to produce, and at an affordable cost; and
  • Policy and program improvements, by helping to identify lessons learned and best practices.

4.2 The Deputy Minister is responsible for CIC’s evaluation function.

4.3 This policy will enable CIC to meet the requirements of the Treasury Board Secretariat (TBS) Policy on Evaluation (2009), its accompanying Directive and Standard[note 1], and the Financial Administration Act, as well as other instruments (detailed in Sections 8.1 and 8.2), while meeting the unique needs of CIC.

5. Definitions

5.1 Definitions of terms used in this policy can be found in Annex A.

6. Roles and responsibilities

6.1 This policy outlines the governance and operation of the CIC evaluation function, and the associated roles and responsibilities of the Deputy Minister, the Head of Evaluation, and program managers. Roles and responsibilities of the CIC Departmental Evaluation Committee can be found in Annex B.

a. Deputy Minister

Consistent with the TBS Evaluation Policy, the Deputy Minister is responsible for:

  1. Establishing the evaluation function and designating the Head of Evaluation;
  2. Establishing the Departmental Evaluation Committee;
  3. Approving the Departmental Evaluation Plan;
  4. Approving, distributing, and using the results of evaluations;
  5. Ensuring the departmental implementation of performance measurement activities and availability of performance information and data; and
  6. Monitoring and reporting on compliance with the TBS Policy on Evaluation.

b. Head of Evaluation

The Head of Evaluation is responsible for:

  1. Overseeing the evaluation function and conduct of evaluations;
  2. Submitting and implementing the Departmental Evaluation Plan;
  3. Submitting and disseminating evaluation reports;
  4. Monitoring and reporting on performance measurement activities (including reviewing and approving all Performance Measurement Strategies, accountability and performance provisions within Cabinet documents, and the performance measurement framework embedded in the department’s Management, Resources, and Results Structure); and
  5. Submitting to the Departmental Evaluation Committee an annual report on the state of performance measurement of programs in support of evaluation.

c. Program Managers

CIC program managers are responsible for the management of their respective organizations and programs. These management responsibilities include the ongoing measurement and monitoring of performance and results achieved.

Under this Policy, program managers are responsible for:

  1. Consulting with the Head of Evaluation regarding accountability and performance measurement provisions to be included in Cabinet documents (Memoranda to Cabinet and submissions to the Treasury Board Secretariat), as well as the planning of evaluations;
  2. Consulting with the Head of Evaluation regarding performance measurement strategies, including performance measurement and evaluation plans for high-impact regulations, and related work for all new and ongoing direct program spending;
  3. Engaging the responsible departmental branches for the development and validation of performance measures and data sources;[note 2]
  4. Designing and implementing  performance measurement to effectively support evaluation;
  5. Monitoring, analyzing, and using performance data in support of program management and assessment;
  6. Ensuring that those conducting evaluability checks, Management Response and Action Plan follow ups, and evaluation studies have access to all departmental records, documents, and information deemed necessary to fulfill their responsibilities, as well as access to departmental staff that they wish to interview to obtain information necessary to complete their studies;
  7. For evaluations involving their program, reviewing and validating preliminary findings and conclusions for accuracy and completeness, in a timely manner;
  8. Ensuring the timely development and implementation of management responses and action plans; and
  9. Incorporating approved evaluation findings and management actions into priority-setting, planning, reporting, and decision-making processes.

7. Standards

7.1 Evaluation work (whether performed in-house or contracted) will be conducted in accordance with the Standard on Evaluation for the Government of Canada and the standards of the Canadian Evaluation Society,[note 3] and will reflect and uphold the principles and rules of conduct contained in the Values and Ethics Code for the Public Service.

8. References

 8.1 Relevant legislation/regulations 

8.2 Related policy instruments/publications

8.3 Related Professional Standards

9. Enquiries

Please direct enquiries about this policy to: 

Director General
Research and Evaluation
Citizenship and Immigration Canada
300 Slater Ave. West
4th Floor
Ottawa, ON
K1A 1L1
Tel.: (613) 954-6526
Fax: (613) 957-5936

10. Effective date

10.1 This policy takes effect on December 19, 2011, and replaces the 2007 CIC Evaluation Policy.

Annex A: Definitions[note 4]

Administrative aspect of major statutory spending:
The resources (both human and financial) used to deliver and administer major statutory spending.
Direct Program Spending:
refers to the portion of total budgetary spending that excludes public debt charges and major transfers to persons and to other levels of government. Direct program spending includes operating and capital spending and grants and contributions (as specified in the Public Accounts).
Effectiveness:
the extent to which a program is achieving expected outcomes.
Efficiency:
the extent to which resources are used such that a greater level of output is produced with the same level of input or, a lower level of input is used to produce the same level of output. The level of input and output could be increases or decreases in quantity, quality, or both.
Evaluation:
the systematic collection and analysis of evidence on the outcomes of programs to make judgments about their relevance, performance and alternative ways to achieve the same results.
Expenditure Management System (EMS):
is the framework that guides decisions on spending. It supports responsible and effective spending by the government through the identification and implementation of the Government’s spending plans to achieve its priorities within the fiscal limits established by the Budget. The EMS governs how money is spent, how much, on what and for what purpose. The overarching objectives of the government’s approach to expenditure management is to ensure that: aggregate fiscal discipline is maintained (fiscal credibility); all government resources are allocated to the Government’s priorities (management excellence in the allocation of resources); and programs are operationally efficient and effective (management excellence in the delivery of programs and services).
Major Statutory Spending:
refers to the portion of total budgetary spending that includes public debt charges, major transfers to individuals (e.g. elderly benefits, employment insurance benefits, children’s benefits), and major transfers to other levels of government (e.g. Federal transfers in support of health and other programs, fiscal arrangements, alternative payments for standing programs).
Performance:
the extent to which effectiveness, efficiency and economy are achieved by a program.
Performance measurement strategy:
the selection, development and ongoing use of performance measures for program management or decision-making.
Program of Grants and Contributions:
refers to the definition of "program" in section 42.1 of the Financial Administration Act, which defines program as a program of grants or contributions made to one or more recipients that are administered so as to achieve a common objective and for which spending authority is provided in an appropriation Act.
Relevance:
the extent to which a program addresses a demonstrable need, is appropriate to the federal government, and is responsive to the needs of Canadians.
Review of relevance and effectiveness:
Section 42.1 of the Financial Administration Act requires that every department shall conduct, every five years, a review of the relevance and effectiveness of each ongoing program of grants and contributions. This policy defines such a review as a form of evaluation.
Risk-based approach to determining evaluation approach and level of effort:
is a method for consideration of risk for the purposes of determining the evaluation approach for individual evaluations. Departments should determine, as required, the specific risk criteria relevant to their context. Specific risk criteria may include the size of the population that could be affected by non-performance of the program, the probability of non-performance, the severity of the consequences that could result, the materiality of the program and its importance to Canadians. Additional criteria could include the recentness and quality of the last evaluation and/or other studies, their findings, the extent of change experienced in the program’s environment, or other criteria.
Risk-based approach to planning coverage of direct program spending:
is a method for considering risk when planning the extent of evaluation coverage of direct program spending (excluding ongoing programs of grants and contributions) pending the full implementation of section 6.1.8 (a) of the Policy on Evaluation. Departments using a risk-based approach to planning coverage of direct program spending should determine the specific risk criteria relevant to their context that will inform their evaluation coverage and non-coverage choices. Specific risk criteria may include the size of the population(s) that could be affected by non-performance of their individual programs, the probability of non-performance, the severity of the consequences that could result, the materiality of their individual programs and their importance to Canadians. Additional criteria could include the recentness and quality of previous evaluations and/or other studies, their findings, the extent of change experienced in the environment of their individual programs, or other criteria.
Value for Money:
the extent to which a program demonstrates relevance and performance.

Annex B: Departmental Evaluation Committee Terms of Reference

Mandate

The Departmental Evaluation Committee (DEC) is a Citizenship and Immigration Canada governance committee chaired by the Deputy Minister. The Committee provides a forum for the presentation of evaluation terms of reference, reports, recommendations, and management responses, as well as of the implementation status of management response actions and the status of performance measurement within the Department. The Committee recommends for approval by the Deputy Minister evaluation terms of reference, reports, and annual plans.

The goals of the DEC are to support the evaluation function in the conduct of its work, and to support senior management in ensuring evaluations are available to contribute to evidence-based decision-making regarding the management of ongoing direct program spending and grants and contributions programs.

The Committee:

  • Provides advice and counsel to assist the Deputy Minister in discharging his/her responsibilities with respect to evaluation;
  • Reviews and recommends for approval by the Deputy Minister a five-year departmental evaluation plan in the final quarter of each fiscal year which:
    • Aligns with and supports the departmental Management, Resources, and Results Structure;
    • Supports the requirements of the Expenditure Management System, including strategic reviews;
    • Provides a schedule in which all direct program spending and ongoing programs of grants and contributions are evaluated every five years; and
    • Includes specific additional evaluations such as are requested by the Secretary of the Treasury Board following consultation with the affected Deputy Minister;
  • Reviews the year-to-year progress of the Evaluation Plan to ensure commitments are met and the adequacy of evaluation coverage;
  • Ensures that the Evaluation Division and CIC managers have the management support and  resources required to fulfill their obligations under this Policy;
  • Reviews and recommends for approval key elements of an evaluation, such as proposed Terms of Reference, and performance measurement strategies and performance measurement and evaluation plans as needed;
  • Reviews and recommends for approval evaluation reports and associated documents;
  • Recommends for approval action plans developed by CIC managers in response to evaluation findings;
  • Reviews program progress against approved action plans;
  • Ensures that the results of evaluations are incorporated into CIC priority-setting, planning and policy decision-making processes;
  • Ensures that ongoing performance measurement is implemented throughout the department to support the evaluation of programs;
  • Reviews the performance of the evaluation function, and  recommends action to address any weaknesses;
  • Reviews the CIC Evaluation Policy periodically and recommends necessary revisions; and
  • Addresses other issues as requested by the Head of Evaluation.

Committee membership

  • Deputy Minister (Chair)
  • Associate Deputy Minister
  • Chief Financial Officer
  • Assistant Deputy Minister, Strategic and Program Policy
  • Associate ADM, Strategic and Program Policy
  • Assistant Deputy Minister Operations
  • Associate ADM Operations
  • Assistant Deputy Minister, Corporate Services
  • One external member

Other senior departmental executives may be invited to participate or observe as necessary. Representatives from central agencies may be invited to participate or observe as necessary.

Secretariat: Research and Evaluation Branch

Roles and responsibilities of members

Chair

The Chair is responsible for:

  • Ensuring that the meeting runs according to the agenda in a timely manner.
  • Ensuring that all members have a chance to provide comments.
  • Providing the final decision regarding items presented to the DEC for approval, on recommendation of the other Committee members.

Members:

Members are responsible for:

  • Supporting the DEC in the joint fulfilment of the Committee’s mandate;
  • Referring discussion items to other departmental committees, as appropriate; and
  • Reporting back to their branch/sector management committees on all relevant topics.

Secretariat

The duties of the secretariat function are to:

  • Identify agenda items for discussion and approval that are well-timed for meaningful input, direction, and decision-making; and
  • Manage committee meeting agendas and records of decision to track action items and follow-up activities.

Governance

To the maximum extent possible, decisions will be made by consensus of the Members and the concurrence of the Chair. In the absence of consensus, final authority rests with the Chair.

Administration/Operations

Frequency and format

Meetings will take place on a regular basis, at least four times per year.

Records of Decision

The Records of Decision will be compiled by Evaluation Division staff and approved by the Committee members at the subsequent DEC meeting.

Terms of Reference

The Deputy Minister approves these Terms of Reference, and the Committee reviews them annually. Changes to the TOR are approved by the DM.

Attendance Protocol

  • In exceptional circumstances, members may send an alternate.
  • Observers are permitted at the discretion of the Chair.

Language

  • Meetings will be held in both official languages.

Distribution of Meeting Material

  • Materials for meetings will be produced by the Evaluation Division and sent to the Secretariat for distribution.
  • Materials will be distributed to DEC members one week prior to meetings.

Timelines

At least 5 business days prior to the meeting:

  • Meeting materials for the DEC’s consideration shall be produced by the Evaluation Division and circulated to members, to allow Committee members adequate time to review materials in advance of the meetings.

For further information, please contact the Director General, Research and Evaluation Branch.


Footnotes

Date Modified: