Audit of the Governance of the Federal Infrastructure Investment Program at Parks Canada

Final Report

January 26, 2017

Office of Internal Audit and Evaluation

Report submitted to the Parks Canada Audit Committee: January 26, 2017
Approved by the Agency CEO: February 10, 2017

Her Majesty the Queen of Canada, represented by
The Chief Executive Officer of Parks Canada, 2017

Catalogue #: R62-472/2017E-PDF
ISBN: 978-0-660-07575-4

Table of contents

Executive Summary

In 2014, the Government of Canada (GC) announced the investment of $2.6 billion to support the rehabilitation of built assets within national historic sites, national parks, and national marine conservation areas across the Parks Canada Agency (PCA) system over the next five years (2015-2020). In order to support the delivery of the Federal Infrastructure Investment Program (FIIP) on time, scope and budget, PCA had to develop and implement various organizational structures (branches, governance committees, etc.), to increase organizational capacity on several fronts and to establish new business processes adapted to this initiative.

The objective of the audit was to provide assurance to Senior Management that governance, risk management and control frameworks are in place to support effective management of the investment planning program, as well as the delivery of projects from an operational perspective. The audit focused on providing assurance that national frameworks with respect to 1) planning, risk management and organizational capacity and 2) monitoring and reporting related to Infrastructure Investment Program were in place and functioning effectively.

The audit focused specifically on the management framework for infrastructure projects funded through FIIP and excluded any projects financed via other sources of funds (Budget 2014, Conservation and Restoration fund, Centralized Asset Investment fund and Federal Contaminated Sites Action Plan etc.). It should be noted however that much of the framework applies to all of these kinds of projects.  The audit work was largely carried out between July 2015 and June 2016 although the team continued to review evidence provided by management through fall 2016.

The audit methodology included an examination of the relevant FIIP documentation, interviews and file review.

This audit conforms with the Internal Auditing Standards for the Government of Canada, as supported by the results of the quality assurance and improvement program.

Overall, Internal Audit (IA) is of the opinion that the program, the way it is currently structured and managed is generally well-controlled with some minor opportunities for improvement. The audit showed that PCA overhauled its Investment Planning structure, processes and tools to support the delivery of the FIIP initiative. Robust governance regime, program framework, review and approval processes, tools and guidance have been developed to support timely project management. Consideration has been given to optimizing stakeholders’ engagement. Opportunities for improvement reside in the refinement of the program and project risk management frameworks, definition and implementation of the project close-out process, oversight over financial coding of expenditures.

Table 1: Criteria assessment summary
Criteria Assessment
Effective governance and management structures have been established and implemented to support successful delivery of the FIIP. Minor improvements required
Effective processes have been established for the review of project submissions, prioritization and timely allocation of resources to support the delivery of FIIP program. Minor improvements required
An effective risk management framework has been established to systematically identify and manage strategic and operational risks that may preclude delivery of FIIP program in scope, time and budget. Moderate Improvements required
Effective business processes have been established to support the effective delivery of projects. Minor improvements required
Controls are in place to optimize stakeholder involvement (relationship) in timely and effective project delivery. Minor improvements required
Financial controls allow for optimal resource management and reporting. Moderate Improvements required
Departmental performance in relation to FIIP is monitored consistently and reported on an ongoing basis. Minor improvements required

The audit recommendations ranked in order of priority, based on the rating system in Appendix C, are shown below.

Table 2: Internal Audit Recommendations
Moderate priority
1. The Vice President Strategic Policy and Investment (VP SPI) should review the current risk management frameworks applied to both program and projects risks and implement a plan to ensure they include all expected elements (risk identification and ratings, mitigation documentation and reporting, assignment of risk owners and identification of risk tolerances) and that they are applied consistently and systematically across the investment program.  
3. The Chief Financial Officer (CFO) should review the situation with respect to the maintenance of project tracking codes in the financial system and establish appropriate mechanisms to ensure the integrity of the coding structure is maintained over time. 
4. The VP SPI should clarify and communicate how and when project close-out should be communicated to the financial community to ensure that the close-out process is applied in a timely manner in the financial system at the end of each project.
Low priority
2. The VP SPI should negotiate an invoicing convention with PSPC that would ensure that all the required data (detail about items charged and project tracking) is clearly indicated on invoices.

1 Introduction

This engagement was included in the Parks Canada Multi-Year Internal Audit Plan 2015-16 to 2017-18 as approved by the Chief Executive Officer (CEO) in April 2015. 

In 2014, the GC announced the investment of $2.6 billion[1] to support the rehabilitation of built assets within national historic sites, national parks, and national marine conservation areas across PCA’s system over the next five years. This historic investment supports the National Conservation Plan while promoting visitor experiences and making PCA’s infrastructure safer and more appealing to visitors. In order to support the delivery of this unprecedented program of work on time, scope and budget, PCA had to develop and implement various organizational structures (branches, governance committees etc.), to increase organizational capacity on several fronts and has established new business processes adapted to this initiative. By the end of 2019/2020, PCA is targeting an improvement of its entire built asset portfolio, aiming for an overall fair to good condition rating.

2 Focus of the Audit

The audit objective is to provide assurance to senior management that national frameworks for the investment program are in place and functioning with respect 1) planning, risk management, organizational capacity and 2) monitoring and reporting. 

Seven major criteria were developed for the audit (see the section About the Audit for more details). The major criteria are: 

  • Effective governance and management structures have been established and implemented to support successful delivery of the FIIP.
  • Effective processes have been established for the review of project submissions, prioritization and timely allocation of resources to support the delivery of FIIP.
  • An effective risk management framework has been established to systematically identify and manage strategic and operational risks that may preclude delivery of FIIP in scope, time and budget.
  • Effective business processes have been established to support the effective delivery of projects.
  • Controls are in place to optimize stakeholder involvement (relationship) in timely and effective project delivery.
  • Financial controls allow for optimal resource management and reporting.
  • Departmental performance in relation to FIIP is monitored consistently and reported on an ongoing basis.

The audit procedures included:

  • An in-depth review of the documents constituting the legal, policy and control framework;
  • A review of the documents used in the administration of the FIIP;
  • Interviews with various stakeholders involved in the delivery of the FIIP;
  • 125 RPAs and the documentation that supports them were reviewed and analyzed;
  • The creation of flow charts detailing the business processes.

Observations and recommendations included in this report are in accordance with the Office of Internal Audit and Evaluation (OIAE) Rating System described in Table 3:

Table 3: Audit Reporting Rating System
Red Unsatisfactory Controls are not functioning or are non-existent. Immediate management actions need to be taken to correct the situation.
Orange Significant Improvements Needed Controls in place are weak. Several major issues were noted that could jeopardize the accomplishment of program/operational objectives. Immediate management actions need to be taken to address the control deficiencies noted.
Yellow Moderate Improvements Needed Some controls are in place and functioning. However, important issues were noted and need to be addressed. These issues could impact on the achievement or not of program/operational objectives.
Blue Minor Improvements Needed Many of the controls are functioning as intended. However, some minor changes are necessary to make the control environment more effective and efficient.
Green Controlled Controls are functioning as intended and no additional actions are necessary at this time.

3 statement of assurance

This audit conforms with the Internal Auditing Standards for the Government of Canada, as supported by the results of the quality assurance and improvement program.



Brian Evans
Chief Audit and Evaluation Executive – Parks Canada Agency

4 Audit Opinion

Overall, IA is of the opinion that the program, the way it is currently structured and managed is generally well-controlled with some minor opportunities for improvement. The audit showed that PCA overhauled its Investment Planning structure, processes and tools to support the delivery of the FIIP initiative. Robust governance regime, program framework, review and approval processes, tools and guidance have been developed to support timely project management. Consideration has been given to optimizing stakeholders’ engagement. Opportunities for improvement reside in the refinement of the program and project risk management frameworks, definition and implementation of the project close-out process and oversight over financial coding of expenditures.

5 Observations and recommendations

5.1 Governance and Management Structures

Blue Minor improvements required Many of the controls are functioning as intended. However, some minor changes are necessary to improve the efficiency and effectiveness of the control environment.

We expected that the Agency would:

  1. Clearly define governance and management structures to ensure adequate oversight for the overall management of a program (i.e., provide strategic direction, ensure that actions are aligned with organizational priorities);
  2. Clearly define and communicate information on relevant roles and responsibilities;
  3. Clearly identify, document and consistently apply related delegated authorities.

5.1.1 Overall Governance and Management Structures

  • Overall approvals for Parks Canada Investment Plan and for monitoring progress are vested in the Agency Executive Management Committee (EMC). The committee itself lacks an overall terms of reference although aspects of its mandate with respect to Investment Planning are articulated in other documents (e.g., Investment Planning and Project Management Orientation Guide). Presentations on various aspects of the investment program are regularly made as part of the EMC Agenda and records of relevant decisions are recorded, although not always readily available on the Agency intranet.

  • To support the work of EMC, the Agency created an Investment Program Oversight Committee (IPOC). Terms of Reference (ToR) for the committee were developed and lay out its mandate, roles and responsibilities, and frequency of meetings, although they have not been kept up to date to reflect changes in the Agency since April 2016. IPOC’s roles and responsibilities were described in the Agency Orientation Guide and communicated by email to all staff.

  • The Agency’s audit committee has also performed an oversight role for the investment program. It is a standing item on the committee’s Agenda.

  • In addition to new or revised roles for various governance committees, the Agency created two new organizational units to provide the Agency’s staff with strategic directions, adequate guidance and tools, and to assume delivery responsibility for aspects of the program:
    • The Investment Planning and Reporting Branch (IPR), provides corporate functional leadership in the provision of investment planning at Parks Canada, as well as the development and communication of guidance and tools to support investment decisions. It oversees the project approval and monitoring processes to track progress on individual projects;
    • Asset Management and Project Delivery (AMPD) provides corporate functional leadership in asset/project management and project delivery services in support of Parks Canada’s built asset portfolio. It is also responsible for delivery of the program of work for highway and waterway assets, and provides project delivery services for heritage, visitor, townsite and, in some cases, supporting asset projects.

  • Chief Financial Officer Directorate (Financial Management Advisory/CFOD-IP), support Field Unit superintendents and National Office and their management teams with respect to all financial aspects of investment planning.
Conclusion

The Agency has quickly established adequate structures to support the investment program. These structures continue to change and evolve in response to various changes within the Agency (e.g., a reorganization of various reporting structures in the Agency in April 2016).

5.1.2 Roles and Responsibilities

  • Roles and responsibilities of the various groups involved in the FIIP (e.g., the structures referenced above), for Functional Experts (FE)[2] at National Office, and for Public Services and Procurement Canada (PSPC), are defined and documented in various ways. Definitions are found in several documents which for the most part have been shared to all PCA staff through emails.

  • The definitions of the roles, responsibilities and accountabilities in the documentation are usually at a high level with more detail available for VPs, Directors, and Managers. Definition of roles and responsibilities for operational staff is not as detailed.

  • At the program level, PSPC and PCA roles and responsibilities are defined, documented and communicated in a master agreement that lies down the basis of the collaboration between the two organisations.

  • At the project level, for any project managed by PSPC in the name of PCA, a project charter has to be drafted. Those documents are much more detailed in terms of the roles and responsibilities of the parties. The level of detail of the project charter is adjusted accordingly to the complexity of the project. These charters are drafted by PSPC and then reviewed and adjusted as necessary by Agency staff. 
Conclusion

We found that roles and responsibilities are adequately defined and communicated. The information is spread out between several documents/sources and that there is no one coherent go-to-document summarizing the information from all webpages and documents. We also noted that documentation is not always kept up-to-date and that sometimes documents refer to the individuals in the roles rather than job titles making it necessary to change the documentation when staff changes. 

5.1.3 Delegation of Authorities

  • Delegation of capital investment project authorities are clearly defined and communicated.[3]

  • We sampled 50 projects to test if the project approval/amendment was obtained from the right delegated authority. In all but three cases we confirmed that the approval was obtained at the appropriate level. In the three exceptions, approval was provided at the appropriate level but by an actor in the position. There was no record the actor had been formally delegated authority to sign on behalf of the incumbent.

  • We also observed four cases were in which an actor in a position formally recommended approval by IPOC for projects for which they were the sponsor. This can create the perception of conflict of interest. We were told that informally, when considering recommending a project, IPOC requires that project sponsors who are members of IPOC, recuse themselves from the discussion and recommendations. In principle, this should address conflicts of interest. We noted that the process is not formally documented in the committee’s terms of reference. 
Conclusion

Overall, we found that delegated authorities are well defined and documented. Compensating measures over the risk that Executive Director (ED), ADMP could be approving/recommending his own projects should be better explained to the project management community to prevent perception of conflict of interest.

5.2 Resource Allocation

Blue Minor improvements required Many of the controls are functioning as intended. However, some minor changes are necessary to improve the efficiency and effectiveness of the control environment.

We expected to find that effective processes have been established for the review and prioritization of project proposals, that FEs had been consulted in reviewing proposals, and that tools and processes had been put in place for ensuring reasonable cost estimates. We expected that resources would be allocated in a timely manner to support the delivery of FII program. Finally, we expected that a process would be in place to ensure sufficient internal capacity to support the program. 

5.2.1 Proposal Review Process

  • An Investment Program Framework in which selection criteria for project approval are defined and documented was put in place. Projects are first screened to determine if the project consists of deferred work (i.e., for the FII program). If so, additional screening based on defined criteria is applied. The specific criteria depends on the nature of project (i.e., whether it is a highway, waterway, or heritage/visitor experience asset).

  • Based on a review of 60 RPAs we found that all but one of the projects in our sample met the criteria of deferred work. There is uncertainty in this determination in part because the RPAs themselves provide limited information on the details of the work to be carried out and in part because some definitions for determining what is deferred work (e.g., asset, system, etc) are not clear. Details about this issue to be communicated to program officials through a management letter.

  • The Agency has introduced new tools (e.g., a new checklist for assessing projects in April 2016) which may help to provide more information and clarity for ensuring the project meets the criteria for deferred work.

  • We found that the project specific criteria were applied consistently in the sample of projects examined. We noted that other considerations beyond the formal criteria also played a part in recommending that projects proceed (e.g., Field Unit’s priorities and the quality of the RPA/feasibility of the projects at hand).

  • Additionally, we found that some, but not all, FEs were consulted during 2015 RPA submission round review (submission received in Oct 2015). Those who were consulted were not provided an adequate turnaround time (few days to a few weeks for hundreds of RPAs) to appropriately review all the projects. Management recognized this issue and was proposing a new process for the Oct 2016 project intake review. If implemented as intended, this new procedure and tool should ensure more timely consultations and documentation of recommendations by FEs.

  • With respect to costing we noted that Agency established a gating process for project approvals i.e., initial conceptual approval, followed by preliminary project approval and finally effective project approval. Costs estimates are expected to be more precise as projects move through the approval stages. The Agency has offered some non-mandatory training on project costing. Reference material (e.g., the new project management standard) contains links to GC costing tools, a section of various approaches to costing, and information on setting contingency amounts for various types of projects. There is also guidance on estimating return on investment for visitor related projects.

  • A request for proposal must also be accompanied by a check list prepared by a Finance and Administration Manager demonstrating that all relevant costs have been considered. Other than the review by finance and administration personal we did not find any evidence that project costs are subject to independent verification by technical experts outside the project team.

  • The Parks Canada 2016 project management standard also requires that project managers document planned vs actual expenditures at project close out and if the variance passes certain thresholds to provide an explanation. In principle this should contribute to better costing in the future.
Conclusion

Adequate processes have been established for the review and prioritization of project proposals. Where problems were identified in the first round of project proposals, management has taken steps to address the issues. 

5.2.2 Timeliness of Project Approvals

  • The normal cycle of project approval requires proponents to submit new or amended proposals to National Office at the beginning of October each year with decisions on funding approved in December (i.e., approval is approximately 3 months). This was not identified as a problem for project delivery.

  • In certain cases, project approvals or amendments may take less time. For example: 
    • Some new projects were submitted for approval outside the normal Oct. to Dec. time frame. In a sample of 10 of these projects, we found that on average they were approved in 16 days. With respect to this situation, we also noted that while some new project proposals were approved outside the normal cycle, others were logged but not submitted for approval until the normal cycle. We could not identify any criteria for directing how a project would be treated. 
    • Some already approved projects submitted amendments outside the normal cycle. Based on a sample of 20 projects, we found these amendments took on average 9 days for approval[4].
    • In the case of emergencies (natural disasters, sudden key infrastructure failures) we found that in a sample of five requests the approvals took an average of 12 days.
Conclusion

Processes have been established for the timely allocation of resources to support the delivery of the FIIP as part of the normal cycle of project approval. In the absence of clear standards for what is acceptable turnaround times for off-cycle approvals or amendments we cannot conclude on whether or not the observed times are reasonable. 

5.2.3 Human Resources Strategy

  • We found that the Agency formally assessed and documented, by means of an HR template, its requirements for various types of positions at specific locations. It used a national approach to allocate resources to funding these positions.

  • Following this, it developed and implemented a structured approach to increase capacity to deliver on the infrastructure program. This included assignment of executive leaders responsible for leading recruitment for various types of positions (i.e. PG, PM, EG, etc.); developing generic work statements of qualification; and running national omnibus competitions. Competitive staffing was used to create pools of qualified candidates, where possible, to address future demands in a timely manner. National coordination and logistical support was provided by the Office of the Chief Human Resources Directorate.

  • Following the initial staffing effort, a review of staffing requirements was again undertaken in March 2016. 

  • The Agency was also undertaking a second phase of HR analysis and planning focusing on employee development, employee retention and succession plans.
Conclusion

The Agency implemented and applied processes and controls to ensure sufficient capacity support delivery of the FIIP.  

5.3 Risk Management Framework

Yellow Moderate improvements required Some controls are in place and functioning. However, major issues were noted and need to be addressed. These issues could impact on the achievement of program/operational objectives.

With respect to risk management, at either the program or project level, we expected to find:

  • A documented process and methodology to identifying and prioritizing risks.
  • Documented processes to identify risk mitigations, as well as ensuring implementation, and monitoring of results.
  • That risk owners, from the appropriate level of management, are designated to ensure mitigation measures are being implemented as they should be.
  • Reports on the status of key risks, effectiveness of mitigation measures and residual risks should be made available to program and senior management.

5.3.1 Program Risk Management

Program risks are those that could affect the Agency`s ability to deliver the Investment plan on scope, in time and on budget. We found:

  • Over time, the Agency has identified various program level risks related to governance, internal and external capacity to deliver (i.e., evident in a number of plans, reports and analysis to support funding). In some cases the risks were rated based on likelihood and impact of the events, although it is not always clear how these ratings were made. Depending on the document we looked at, various mitigation strategies where identified.

  • The most extensive list of program level risks (i.e. 10) resulted from a June 2015 forum of senior management in the Agency. These risks where not prioritized for likelihood and impact. However, each of the risks was assigned to one or more senior managers as the risk owner, who together with task teams, identified mitigation strategies. The results of their analysis were presented to executive management in December 2015.

  • Ownership of most risks is assigned either formally (e.g., following the June 2015 senior management forum, to VP Strategic Planning and Investment in the case of asset management as a whole) or assumed as part of normal job responsibilities (e.g., various Vice Presidents assuming roles in developing capacity, tools and procedures related to topics such as environmental assessment, cultural resource impact assessment, visitor experience tools and processes, tools for aboriginal consultations).

  • We found evidence that reporting on some of the risks and mitigation measures has taken place (e.g., regular reporting to EMC on the results of the HR strategy noted above, as well as other aspects of investment plans and program progress) but could not find evidence of systematic reporting on other risks (e.g., potential lack of third party contractors to support the program of work).

  • Discussions of program risk rarely make reference to the concept of risk tolerance. The concept is embedded in some of the Agency`s practices, although not explicitly labeled as setting risk tolerance levels, (e.g., in the level of over programming the Agency will accept in a given year; in the contingency amounts for approved project budgets). Interestingly, lack of a clearly defined level of `reasonable risk tolerance’ was one of the 10 risks identified at the senior managers forum.

  • Finally, it should be noted that this audit is itself part of the risk control framework which assesses the adequacy of various mitigation measures put in place by management to manage some, but not all of, the previously identified risks (e.g., adequacy of program governance, timeliness of decision making, effective and timely recruiting of new staff, effective partnering with PSPC, and adequacy of communication strategies). 

5.3.2 Project Risk Management

Project risks are those that within scope, and/or the on-time or on-budget delivery of specific projects. 

  • In principle, project sponsors are to identify risks and mitigations when proposing projects for approval and to provide up-dated overall assessments of project risks in the Agency project tracking tool when needed.

  • The Agency has some tools for assessing the overall riskiness of a project including;
    • The Project Complexity and Risk Assessment Tool developed by Treasury Board Secretariat (TBS) which is mandatory for projects over $875K and;
    • A risk rating matrix developed in the Agency that helps to classify projects as having a low, medium or high risk. At the time of the audit field work, we could not find evidence that the latter tool had been widely disseminated.

  • We found that requests for approval of specific projects did not consistently complete the section in which risks and mitigation measures were to be identified.

  • We found that project managers’ overall project risk rating (i.e., low medium high) were consistently captured in the project tracking tool, although it is not always clear how managers arrive at these ratings.

  • IPR also developed a number of automated flags in the project tracking tool to identify discrepancies in various data fields that could indicate project risk (e.g., a project rated as low risk but which has recorded little expenditures at various points during the year). The flags were used to trigger follow-up contact with project managers to determine actual risks.

  • As part of its revised project management standard, the Agency developed a risk registry tool which will be mandatory for certain projects and encouraged for others. The tool is essentially a spreadsheet listing all project specific risks and mitigations. It also provides examples of common project risks. It had not yet been implemented at the time this report was written.

  • We found limited evidence of project level risk reporting to senior management. When reporting occurs it is almost exclusively based on an analysis of financial data (project funds and forecasts) and completeness of work. Reports by major classes of assets and per delivery mechanisms (i.e., the Agency vs. PSPC are common).

  • At the time of writing this report the Agency had begun work on more project based risk report at the request of the Audit Committee. It is intended that the report be used for senior management as well. 
Conclusion

In summary, we can identify many elements of what was expected in terms of a program and project risk identification, mitigation and reporting in the Agency and various tools and processes to support some aspects of risk management. However, a complete risk management framework is not applied in a consistent systematic fashion at either a program or project level. Some key concepts such as risk tolerance are not systematically defined and addressed across all risk areas. As a result, there is a potential for decision making to be based on incomplete risk information or that emerging risks are not detected and managed.

Recommendation 1

The VP SPI should review the current risk management frameworks applied to both program and project risks and implement a plan to ensure they include all expected elements (risk identification and ratings, mitigation documentation and reporting, assignment of risk owners and identification of risk tolerances) and that they are applied consistently and systematically across the investment program. 

Management Response

Agree: Management will review the current risk management frameworks at both the program and project level to ensure all expected elements are included and consistently applied. At the project level, the implementation of the risk register will contribute to an improved program risk monitoring and reporting framework. At the program level, the Investment Planning and Oversight Committee will review and update the ten risks identified by Senior Management (June 2015), including mitigation strategies, ownership and risk tolerance. The results of these initiatives will be incorporated into an overall risk management framework document. 

Target completion date: Dec 31, 2017

5.4 Business Processes

Blue Minor improvements required Many of the controls are functioning as intended. However, some minor changes are necessary to improve the efficiency and effectiveness of the control environment.

We the expected the Agency to have established effective business processes to support the delivery of projects including relevant guidance/tools and relevant orientation, training and knowledge sharing and communication protocols. 

5.4.1 Guidance, Tools and Templates

  • The Agency updated it 2011 PCA Project Management Standard (and associated tools and templates). The new standard was distributed in July 2016 to come into effect October 2016. The 2016 Standard documents the requirements for project documentation, functional, monitoring and reporting requirements. The standard was written to fit all types of PCA projects but does identify requirements specific to projects funded centrally under the Investment Program Management Framework.

  • The Agency has produced a number of tools and templates to support project managers in the delivery of PCA projects including:
    • A request for project approval (RPA) template and associated guidance;
    • Tools for tracking approved/amended requests for approvals (i.e., the RPA Tracking System or RTS) and for tracking progress on projects (i.e., the Milestone Reporting Tool or MRT);
    • Templates for developing project charters, project plans and required resources;
    • Checklists for project documentation and project performance assessment and;
    • Various guidance documents such as “Construction Site Roles and Responsibilities” and “General Guidelines for Estimating Contingency on Construction Projects”.

  • Specialized guidance was also developed for conducting environmental and cultural resource impacts assessments and for identifying visitor requirements in asset planning. 

5.4.2 Orientation, Training and Knowledge Sharing

  • Given the number of new staff in the Agency to support the investment program, an orientation portal for new employees was deployed on the intranet.

  • A site was also developed (i.e., the learning path) providing a list of mandatory training for all employees, supervisors, managers and executives. It also lists recommended training.

  • To support the investment program in particular, the Agency developed (November 2015) an Investment Planning and Project Management Orientation Guide. It was communicated to all PCA team members via email and the Intranet. This guide includes a variety of information and resources to help team members successfully plan and execute projects, and ensures integrated, as well as, consistent project planning and delivery across the country.

  • In addition, an Excel workbook highlighting PCA Investment Planning’s Learning requirements has been developed. It lists all mandatory and recommended training for team members involved in the FIIP, including training that is in the midst of being developed.

  • The Project Management Office (PMO) also provided a link to relevant project management courses offered at the Canada School of the Public Service. At the time of the writing of this report, PMO was planning its own specific courses for fall 2016 and winter 2017.

  • Finally, it was reported that project managers were sharing lessons learned during monthly national calls hosted by the Asset and Environmental Management Team. There were no records of the specific details of what was shared. 

5.4.3 Communications Protocol

  • An infrastructure specific section in the Corporate Communications Branch was created and is responsible for implementing an overall communications strategy in collaboration with field units.

  • Protocols for communication are in place and are made available to staff on the Intranet. The site includes tips, key messages, templates and processes, as well as Q&As. National Office reviews all communications material.

  • Internal communication, i.e. with field units or peers of other directorates, is conducted by the Investment Planning group on a daily basis (tele/video-conferences, calls, emails relative to data systems, etc). Conference calls for Field Unit Superintendents (FUS) and FEs involved in the program delivery are held on a regular and frequent basis to share information and knowledge about FIIP.
Conclusion

A wide variety of business processes supporting project management, orientation, training, knowledge sharing and communication have been developed and communicated. Additional tools and support structures continue to be developed. 

We did note that the various business processes are distributed across several sites on the Agency’s intranet, not always kept up to date with changing circumstances in the Agency, and not always linked in ways that make it easy to connect from one functional area or theme to another. 

5.5 Stakeholder Involvement

Blue Minor improvements required Many of the controls are functioning as intended. However, some minor changes are necessary to improve the efficiency and effectiveness of the control environment.

We expected the Agency to have processes in place to effectively engage other government organisations, such as PSPC and third party contractors, to support its program delivery.

5.5.1 PCA-PSPC Project Delivery Relationship

At the time of the audit it was estimated that approximately 40% to 50% of the total value of the Agency’s investment program would be delivered by PSPC. To manage this important relationship, the Agency:

    • Signed a master agreement to govern the business partnership between the two organisations.

    • Held quarterly meetings between senior executives (ADMs, VPs) of both organisations[5] to discuss/resolve any situations (at a strategic level) that could jeopardize the timely and efficient delivery of certain projects;

    • Held bi-weekly meetings between the operational management and key stakeholders of the program of both organisations to discuss/resolve key issues in the delivery of the program of work as well as challenging projects;

    • Participated in various forums, seminars as well as networking and partnership opportunities;

    • Obtained weekly a report on “ high risk ” projects sent by PSPC officials to Executive Director, AMPD;

    • Worked closely with PSPC to ensure the communication of the Agency’s program of work and ensure the definition of requirements to support PSPC planning of resources;

    • Created a dispute resolution procedure and appointed a senior executive in the Agency responsible for the process;

  • At the level of individual projects, the Agency set requirements in its 2016 PCA Project Management Standard that PSPC project charters contain detailed communication protocol between the project team members of the two organisations for most projects, as well as a description of the project team, project timelines, frequency of project team meetings, and milestones amongst other key parameters.

  • We found that regular meetings (frequency defined in project management plans) between the PCA project sponsors/teams and the regional PSPC official to discuss specific project delivery were taking place.

  • The Agency also reached an agreement with PSPC to use the Agency’s project number when billing for services. Despite this control, we found that 28% (11/40) of the tested PSPC invoices we examined did not contain sufficient information to track the expenditures to a specific project. We also noted that 47.5% (19/40) invoices did not provide details on the actual object for which PSPC was billing the Agency. 
Conclusion

Processes and controls to promote an effective business relationship with PSPC are generally in place. The only exception that we noted concerns the consistency with which PSPC invoices are linked to Agency projects and the level of detail needed for the Agency to adequately understand the costs. Lack of sufficient information can create inefficiencies in tracking project costs (i.e., additional work to identify missing information) and limits the Agency’s ability to exercise due diligence over spending. 

Recommendation 2

The VP SPI should negotiate an invoicing convention with PSPC that would ensure that all the required data (detail about items charged and project tracking) is clearly indicated on invoices.

Management Response

Agree: AMPD will collaborate with PSPC through the existing Federal Infrastructure Investment governance model to ensure all PCA required data is consistently recorded in all invoices.

Target completion date: Dec 31, 2017.

5.5.2 Promoting Awareness and Competition

  • In order to mitigate the risks resulting from a lack of available supplier (i.e., delays in starting work, cost overruns) we expected the Agency to take a proactive approach to make potential bidders aware of opportunities and encourage competition.

  • National Contracting Services encourages staff to communicate with local suppliers about the fact that PCA is about to release some tenders and invite them to consult www.buyandsell.ca or any other media through which the contracts could be posted. Potential bidders can also register as “interested bidders” in order to be noticed about posted tenders. In our interviews with onsite project managers we were told that they had all organised local site visits to raise awareness about the work to be done and answered questions from the potential bidders.

  • When staff are communicating with potential bidders, it is important that they do not reveal details about the potential contracts in order to avoid providing competitive advantage to particular suppliers. All PMs interviewed as part of the audit were aware of this restriction However, only one of them was able to provide supporting documentation (e.g., letters, advertisements, e-mails, and minutes of meetings) showing that their communication with suppliers respected this principle.

  • Another example, used to encourage competition was the organization of a vendor day for potential bidders for various waterways projects. This event was jointly attended by Agency and PSCP representatives. The approach has not been replicated at the time of the audit work but National Office Contracting services was looking for other opportunities. 
Conclusion

The Agency has put controls and processes in place to support vendor awareness about the work to be completed and encourage competition on its tenders.

5.6 Financial Controls

Yellow Moderate improvements required Some controls are in place and functioning. However, major issues were noted and need to be addressed. These issues could impact on the achievement of program/operational objectives.

We expected that a combination of preventative and detective controls would be in place to support optimal resource management and reporting.

5.6.1 Funds Allocation and Expenditure Monitoring

  • Assigning of budgets to projects in the financial system is done independently of the project approval process. The allocation of funds is triggered once an approved RPA is sent to the Chief Financial Officer Directorate Resource Management (CFOD-RM) in National Office. The CFOD-RM assigns the project budget to a business unit’s fund center.

  • Assigned funds are tracked in two ways. At the macro-level, all FII funds have a code identifying the source of the funding. Expenditures are tracked against the source of funds.

  • For tracking project expenses, a code for the project and sub-codes for each of the specific assets associated with the project are created in the financial system. The Agency has clearly communicated that this set of project codes is to be created by representatives of the CFOD, and personal outside the CFOD are not to change or modify the codes. However, there are no hard controls in the financial system that prevent changes, modifications or deletions to the cost coding structure because of the potential impacts such controls may have on operations.

  • Changing the code structure does not delete expenditures from the financial system or affect financial statements but it does change the record of expenditures assigned to the project.[6] At the time of the audit there was no monitoring to detect changes in the project code structure.

  • There is monitoring of expenditures of FII funds without a project code (i.e., called orphan transactions). When these are detected follow-up is made to correct the errors.

  • As noted above, funds for projects are allocated in the financial system to business units. Financial and administrative managers in the business unit are then allocate the funds to specific projects according to the approved RPA amounts in the STAR system. In order to detect if amounts assigned to the project in the financial system are correct, the CFOD conducts quarterly reviews of project budgets shown in the project tracking system against those in the financial system. The requirement for project sponsors to submit year end RPA’s to confirm/reprofile their funding requirements will also contribute to providing of assurance that the funds are expensed in compliance with what they were granted for.

  • In principle salary costs associated with projects should be identified at the beginning of the project in order to better monitor costs and expenditures. In practice we found that only a minority of business units allocate salary budgets against projects at the beginning of the year. This is because many staff are involved in the delivery of multiple projects and rarely are able to predict with any degree of accuracy the portion of their workload that will be associated to each project. As a result most salary costs are allocated to projects at year end. There is some monitoring of unassigned salary dollars throughout the year by National Office based on data in the project tracking system and evidence that salary costs are being allocated to projects at year end. 
Conclusion

High level financial controls over the investment program are generally in place. The one control weakness we identified concerned the ability of personnel to change or modify the project based coding structure in the financial system. Changes can result in an inaccurate record of project expenditures used for tracking purposes. 

Recommendation 3

The CFO should review the situation with respect to the maintenance of project tracking codes in the financial system and establish appropriate mechanisms to ensure the integrity of the coding structure is maintained over time. 

Management Response

Agree: CFOD will review the situation and analyse options by September 2017 in order to ensure the integrity of the project expenditures and coding structure in the financial system. Based on the results of the review and consultation with the financial community, new controls will be implemented no later than March 31, 2018. Meanwhile, compensating controls will be implemented by March 31, 2017 in order to ensure the integrity of the financial information in the financial system.

5.6.2 Project Close-out Process

Ideally project close-out procedures would be well defined and take place in a timely manner so that unused funds could be recovered quickly and reallocated to other priority areas. 

  • We found that at the time of the audit, project close-out procedures were not widely understood (i.e., four of the seven project managers we interviewed were not aware of the close-out process).

  • There is guidance on project close-out processes in the 2016 PCA project management standard. However, this is almost exclusively focused on close-out from a project management perspective (e.g., obtaining proper certificate and close-out papers) although it does include references to closing out the project in the RTS and MRT systems.

  • There no clear references to how other functions in the Agency which are impacted by project close-out should be informed. In particular, representatives from functions such as asset accounting, contracting and funds allocation need to be informed in order to de-commit unused funds in the financial system, start the process of capitalizing the relevant assets and recover unused funds for reallocation. 
Recommendation 4

The VP SPI should clarify and communicate how and when project close-out should be communicated to the financial community to ensure that the close-out process is applied in a timely manner in the financial system at the end of each project.

Management Response

Agree: VP SPI will clarify and communicate (e.g. Intranet etc.) the Project Management Standard project close-out process to the financial community. Additionally, Project Checklist sign offs will be amended to include business unit finance managers to further ensure timely notification and corresponding updating of the financial system.

Target completion date: Dec 31, 2017

5.7 Performance Reporting

Blue Minor improvements required Many of the controls are functioning as intended. However, some minor changes are necessary to improve the efficiency and effectiveness of the control environment.

We expected that central reporting requirements would be clearly defined and met based on complete, accurate and timely data. 

5.7.1 Reporting Requirements

  • The basis of internal reporting is the information captured in the RTS and MRT systems. Required fields in these systems are defined.

  • The actual form of reporting to senior management and the frequency which reporting is to take place are not defined.

  • During the period between October 2015 and July 2015 we identified 12 investment plan status tabled at EMC or slightly less than one every two weeks. When interviewed, respondents reported that they were expected to provide bi-weekly reports.

  • The form of the reporting to senior manager varies over the presentations. Typically, they present information on financial authorities, approved project funds and forecasts for the FIIP and the other programs that form part of the Investment Program (Budget 14, CAI, CoRe and FCSAP). There is some information on expenditures and progress in project completion usually at the level of classes of projects (e.g., highways, waterways and other projects). At the time of audit, there was little reporting on the status of specific projects.

  • In the case of external reporting, the only expectation for 2015 was for an opening report to be presented to TBS. This focused largely on investments in the three asset portfolios (Heritage & VE, Highways, and Waterways) and on the status of five individual projects that were considered of higher risk. 

5.7.2 Integrity of the Information Reported

  • We examined controls for ensuring the completeness, timeliness and accuracy of information in the RTS and MRT systems.

  • With respect to the RTS, the original information is received as an electronic template/form. The forms are converted to PDF files and retained in the system. Certain information from the forms is then manually entered into standard data fields in the system by personnel in National Office. The data entry is then validated by program analysts who are independent of the data entry process. Responsibility for the integrity of this data is aligned in one directorate in National Office.

  • Controlling the integrity of data in the MRT is more complex.

  • Part of the data in the MRT is imported directly from the RTS, and part it is uploaded from a data file from the financial system. The remainder consists in an input by project managers in business units. We did not examine the processes in place to ensure data in RTS or the finance system was correctly imported into MRT. Some monitoring to ensure consistency of data in MRT and the financial system was noted previously.

  • There are several processes and controls operating to ensure the integrity of the data in MRT: 
    • There is clear guidance on which fields in the database users are required to populate.
    • There is guidance on how to calculate values for fields where this is required (e.g., how to calculate the percentage of work completed).
    • Training on how to use the system has been provided. 
    • Filters are applied in MRT on the mandatory fields to identify any blank cells before any information is reported to decision makers.

  • National Office is also able to run a variety of reports to detect potential errors (i.e., comparing different data fields to identify inconsistencies, comparison of data at different dates, tests for unreasonable expressions, or unreasonable amounts).

  • When errors or anomalies are detected program analysts in National Office contact business units to resolve issues.

  • The complete system is updated every day to reflect ongoing changes in projects status. The financial data is exacted and updated on a regular schedule. There is no fixed schedule for project managers update their data. Typically there are calls to update all relevant data fields in advance of presentations to senior management. 
Conclusion

The current structure of controls over RTS and MRT is sufficient to provide reasonable assurance that that data being stored is accurate, complete and timely.

Appendix A. About the Audit

Authority

This engagement was included in the Parks Canada Multi-Year Internal Audit Plan 2015-2018 as approved by the Chief Executive Officer (CEO) approved in June 2015.

Objective

The audit was designed to provide assurance that national frameworks for the investment program are in place and functioning effectively with respect to 1) planning, risk management and organizational capacity and 2) monitoring and reporting. 

Scope and Approach

The audit focused specifically on the management framework for infrastructure projects funded through Federal Infrastructure Investment Program (FIIP) and excluded any projects finance via other sources of funds (Budget 2014, Conservation and Restoration fund, Centralized Asset Investment fund and Federal Contaminated Sites Action Plan etc.). It should be noted however that much of the framework applies to all such projects. 

The audit criteria were developed based on

  1. Review of relevant Federal Legislation and Regulations, policies and directives from the Treasury Board of Canada Secretariat (TBS) and PCA internal policies, directives, guidelines and procedures which are listed in Appendix A.
  2. a program risk assessment conducted by the audit team,
  3. the Investment Planning Framework developed by the Parks Canada Agency (PCA) and
  4. the Audit Criteria Related to the Management Accountability Framework[7].

In order to align the audit work carried out on the Federal Infrastructure Investment Program (FIIP) across the Government of Canada (GC), the audit team also considered some other government departments audit programs in the development of the Parks Canada audit program. The audit criteria were reviewed and approved by the former Vice-President, Strategic Policy and Investment, the Chief Financial Officer and the Chief Human Resources Officer as a suitable basis for assessing the Agency’s overall investment framework and process. 

The audit work was largely carried out between July 2015 and June 2016 although the team continued to review evidence provided by management through fall 2016.

Appendix B. Applicable Legislations and Policies

Acts and Regulations

  • Financial Administration Act
  • Federal Real Property and Federal Immovable Act
  • Parks Canada Agency Act

Treasury Board of Canada Secretariat

  • Integrated Risk Management Framework
  • Policy on Investment Planning - Assets and Acquired Services
  • Policy on Management of Real Property
  • Policy on the Management of Projects
  • Project Management Plan Guidance
  • Risk Management Policy
  • Standard for Organizational Project Management Capacity
  • Standard for Project Complexity and Risk

Parks Canada Agency

  • Parks Canada Investment Planning and Project Management Framework
  • Investment Program Framework
  • Investment Program Cycle
  • Investment Planning and Project Management Orientation Guide
  • Parks Canada Project Management Standard December 2011
  • 2015-16 Corporate Risk Profile
  • Parks Canada Directive on Impact Assessment
  • Aboriginal Consultation and Engagement in Investment Planning (guide)
  • Parks Canada Asset Management Directive
  • Statement of Visitor Requirements Asset Investment Planning Guide and Template
  • Cultural Resource Management (CRM) Policy

Appendix C. Glossary

AA
Aboriginal Affairs
AMPD
Asset Management and Program Delivery
CEO
Chief Executive Officer
CFOD
Chief Financial Officer Directorate
CRM
Cultural Resource Management
CRP
Corporate Risk Profile
DAC
Departmental Audit Committee
DW
Deferred work
ES
Environmental Services
EMC
Executive Management committee
F&A
Financial & Administration
FE
Functional Expert
FIIP
Federal Infrastructure Investment Program
FU
Field Unit
FUS
Field Unit Superintendents
GC
Government of Canada
IA
Internal Audit
IO
Internal Order
IP&PM
Investment Planning and Project Management
IPOC
Investment Program Oversight Committee
IPR
Investment Planning and Reporting
JV
Journal Vouchers
MRT
Milestone Reporting Tool
OIAE
Office of Internal Audit and Evaluation
PA
Portfolio Analyst
PCA
Parks Canada Agency
PCRA
Project Complexity and Risk Assessment
PCX
Executive
PDS
Project Delivery Services
PF
Parks Canada Places Framework
PM
Project Manager
PMO
Project Management Office
PMS
Project Management Standard
PoW
Program of Work
PSPC
Public Services and Procurement Canada
RoD
Records of Decision
RPA
Request for Project Approval
RTS
RPA Tracking System
SAP/STAR
Parks Canada Agency’s financial information management system
SFT
Salary Forecasting Tool
SoQs
Statement of Qualifications
TB Sub
Treasury Board Submission for the Investment Plan
TBS
Treasury Board of Canada Secretariat
TEC
Total Estimated Cost
ToR
Terms of Reference
VE
Visitor Experience
VP
Vice-President
VP SPI
VP, Strategic Planning and Investment

Appendix D. Recommendations Priority Rating System

Table 1 : Internal Audit Recommendations Priority Rating System
Priority Condition
High Management should initiate immediate action to address the comment.
1 Major internal control weakness
2 Major policy or procedure exceptions
3 Significant risk exposure
4 Major financial exceptions – loss, misstatement, errors, fraud
5 Significant law or regulatory violations
6 Significant potential opportunity – revenue, savings, efficiencies, improvements
Moderate Management should initiate timely action to address the comment.
1 Substantial internal control weakness
2 Substantial policy or procedure exceptions
3 Substantial risk exposure
4 Substantial financial exceptions – loss, misstatement, errors, fraud
5 Substantial law or regulatory violations
6 Substantial potential opportunity – revenue, savings, efficiencies, improvements
Low Management should initiate reasonable action to incorporate a plan to address the comment in the normal course of business.
1 Minor internal control weakness
2 Minor policy or procedure exceptions
3 Limited risk exposure
4 Minor financial exceptions – loss, misstatement, errors, fraud
5 Minor law or regulatory violations
6 Limited potential opportunity – revenue, savings, efficiencies, improvements

Appendix E. List of 10 FIIP risks Identified by Senior Management (June 2015)

  • Team might not remain motivated, healthy, productive and positive about the work they are doing at Parks Canada while implementing FIIP.
  • Culture of innovation in project delivery that includes a reasonable tolerance of risk might not be fostered.
  • Governance to support rapid in-year decision making might not be optimal.
  • Momentum moving towards long-term sustainability might not be maintained.
  • Investment Program might overtake the organizational identity and operations.
  • Industry might not be engaged or show interest for our projects.
  • Partnership with PSPC, both at a national and regional level, might not be aligned and/or effective.
  • Recruitment might not be effective and timely.
  • Onboarding might not be effective and timely.
  • Communication strategies might not be sufficient to communicate about the program effectively.

[1] In addition to the 391 M$ announced in Budget 2014.

[2] Cultural Resource Management, Visitor’s Experience, Environmental Impact Assessment, Aboriginal Affairs Secretariat etc.

[3] Executive management committee initially approves all projects no matter what the value and approves amendments and off cycle RPA’s over 5M$. IPOC approves off-cycle RPAs as well as amendments between 1M$ and 5M$. The Executive Director, Investment Planning can approve off cycle RPA’s and amendments under the value of 1M$.

[4] The approval of one sampled amendment was delayed until the in cycle intake which resulted in an approval delay of 95 days. This item was not considered in the average calculation.

[5] Representatives of other departments benefiting from FIIP funding are also present to those meetings.

[6] In the course of the audit work we identified one instance in which changes to the coding structure resulted in several million dollars in previous years’ expenditures was no longer linked to the project. 

[7] March 2011, Office of the Comptroller General of Canada, Treasury Board Secretariat.

Related links

Date modified :