Point of Sale (POS) Audit Report

Final Report
June 27, 2016

Office of Internal Audit and Evaluation

Report submitted to the Parks Canada Audit Committee on: June 2, 2016
Approved by the Agency CEO: August 29, 2016

Table of Contents

1 Executive Summary

The Point of Sale (POS) system was implemented across Parks Canada Agency (PCA) during the 2013 operating season in national parks, national historic sites and national marine conservation areas to facilitate and standardize the collection of revenue and financial information, which used to be collected through various systems (Vectron, manual systems, etc.). The POS system was also selected to help the Agency collect, store and manage business intelligence at different parks and sites. The system is mainly managed by an internal Parks Canada team, and the data are stored on Agency servers. The Agency also has support from a third party for system configuration and administration, and user support. As of November 2014, over 250 terminals were divided among 32 field units (FUs).

The audit objective was to evaluate the adequacy of the existing control framework (governance, internal controls, and risk management) to support the collection of financial and business intelligence through the POS system. The scope of the audit included a sample of national parks and national historic sites, ensuring representation of various types of Parks Canada operations. The data used for analysis dated between April 1 and November 30, 2014.

The audit methodology included a review of the legislative and policy framework documents (and other relevant documents) relating to the administration of the system, interviews with stakeholders involved in managing and using the system, the creation of flow charts, physical observation during site visits and a review of sample of transactions.

This audit conforms with the Internal Auditing Standards for the Government of Canada, as supported by the results of the quality assurance and improvement program.

The audit findings show that the control framework for the administration and use of the POS system and related business processes require improvements, particularly with respect to the updating of policies and other reference documents and the monitoring of system use and data input. Controls in various sections of the business processes, such as collecting business intelligence, adequate cash management and compliance with legislative and policy requirements, must be strengthened. For reporting purposes, the integration of data collected through the various collection methods would be desirable.

Table 1: Criteria assessment summary
Criteria Assessment
A Management Control Framework is in place to support the system effectively. Minor improvements needed
The information technology infrastructure that supports the system is adequate to ensure reliable data collection. Minor improvements needed
The data collected through the POS system is complete. Significant improvements required
The data collected through the POS system is accurate. Moderate improvements needed
The data collected through the POS system is available in a timely manner. Minor improvements needed

Below is the list of audit recommendations ranked in order of priority based on the rating system in Appendix C of this document.

Table 2: Summary of internal audit recommendations
High priority
1. The Chief Financial Officer should update the reference documents used for revenue management and financial transactions in the POS, have them approved by the appropriate delegated authority and communicate them to the FUs. Directive/standards should detail how compliance will be monitored and mechanisms to implement corrective actions when situations of non-compliance are observed.
3. The Vice-President (VP) External Relations and Visitor Experience (ERVE) should identify key POS data of interest to various stakeholders and facilitate access to system's information by automating the POS reporting function.
4. The VP ERVE should consider maximizing the integration of data from various sources, including but not limited to POS, Parks Canada Reservation System (PCRS), commercial sales, automated pass machines and POS stand-alone terminals into a single data output that would allow the production of more comprehensive reports.
5. The VP ERVE should develop a security strategy related to the administration of user accounts, the management of passwords and the assignment of POS system access permissions and privileges, and communicate it to the FUs.
7. The VP, Operations should document where forgone revenue from entry fees is likely to be a significant issue (i.e., parks or sites) and seek assurance that entry fees compliance/revenue recovery mechanisms have been developed and implemented where it is cost effective to do so.
10. The VP ERVE should develop and communicate direction to FUs describing the expectations with regard to the collection of business intelligence. Reports on compliance with the directives should be made available to VP Operations, on a regular basis to ensure that corrective actions are taken when necessary.
Moderate priority
2. The VP ERVE should develop and communicate to FUs a document outlining POS elements that are essential to the training of cashiers by the FUs.
6. VP ERVE should develop and communicate to field units clear direction on the utilization of the POS system functions to support management of operations (including but not limited to: inventory of merchandise, Discovery and seasonal passes, use of scanners, etc.). Analysis on the compliance to directives should be conducted and reports should be made available to VP Operations, on as needed basis, to ensure that corrective actions are taken when necessary.
Low priority
8. The VP ERVE should develop and implement an automated control that would limit the possibility of errors in terms of transaction quantities and/or amounts.
9. The Chief Financial Officer should reinforce the use of the auto-clearing function for deposit files in STAR to streamline the reconciliation process.

2 Introduction

Parks Canada Agency (PCA) has implemented a computer system to support the collection of revenue (mostly entry fees) and business intelligence in a safe, effective manner, providing quick and easy access to the information.

This audit was included in Parks Canada’s Multi-Year Internal Audit Plan 2014-15 to 2016-17, approved by the Agency’s Chief Executive Officer in June 2014.

3 Background

With 22,073,047 person visits to Parks Canada’s national parks, national historic sites and national marine conservation areas in the 2003–2004 fiscal year, Parks Canada concluded that it needed a modern system to effectively manage revenue collection, more specifically, entry fees to the parks and sites and allow for automated integration of financial data between the point of sale and financial systems (i.e., STAR). Originally, the Agency targeted use of a specific system (Vectron) in all sites with more than $75K in revenue. In response to evolving management needs (e.g., for more timely information regarding revenue, desire to capture various business intelligence as well as a National Audit of Operating Revenue: Camping, Entry and User Fees (2009), the Agency decided to acquire a new system, to be implemented at most of its revenue collecting locations.

A steering committee constituted of representatives from the External Relations and Visitor Experience (ERVE) Directorate, the Chief Financial Officer (CFO) Directorate, and the Chief Information Officer (CIO) was created to oversee selection and implementation of the new system. Rollout of the common POS system began over 2012 and 2013. As of November 2014, the system had been implemented in 32 FUs (over 250 terminals). Roll out is still ongoing (e.g., canals only began implementing the system in 2015).

The system is mainly administered by Parks Canada staff, and the data it gathers are stored on the Agency’s servers. The system configuration (programming) and user support for more complex situations are usually entrusted to a third party.

The system is not the only tool for revenue recording and management in the Agency. The other system include the Parks Canada (Campground) Reservation Service (PCRS), corporate account invoicing, some older Vectron machines still in use, automated pass sale machines and various manual systems.

Table 3 shows the income received by the various systems in place at Parks Canada Agency. For comparative purposes, data are presented for complete years 2014-2015 and 2015-2016 where the system was in use.[2]

Table 3: Revenue collected through PCA’s various systems
Fiscal year POS Parks Canada Reservation Service (PCRS) Other Total revenue[3]
2014–2015 revenue $68,326,237
(54%)		 $25,684,067
(20%)		 $33,407,517
(26%)		 $127,417,821
2015–2016 revenue $70,880,571
(51%)		 $39,714,747
(28%)		 $29,909,405
(21%)		 $140,504,726

4 Legislative and Policy Framework

The POS system has two main components: Revenue collection, is mainly overseen by PCA under the Directive on Revenue Comptrollership for User Fees and the Management of Revenue and Cash Standards. The second component of the POS system, business intelligence gathering, is supported by the Parks Canada RMS User Guide and the Parks Canada RMS User Guide for Visitor Information Collection. The Agency's operations also have to be conducted in compliance with various legislations, regulations and Treasury Board (TB) policies and directives governing cash management (cash handling and revenue reconciliation), the safeguarding of Agency assets, and information and technology management. Appendix A contains a list of the elements of the legislative and policy framework.

5 Objective and Scope

Audit Objective

The audit objective was to assess the adequacy of the control framework (governance, risk management, internal controls and monitoring) supporting the data collection through POS. Cash handling and safeguarding of assets, which are intrinsically linked with the POS system, were also included in the audit program.

Audit Scope

The scope of the audit was limited to the Parks Canada Point of sale system and included:

  • Accuracy, completeness, and timeliness of revenue and business intelligence data collected through the system;
  • Cash handling and safeguarding processes for revenue collected where point of sale terminals were in use;
  • Reconciliation of revenue data from the point of sale system in the Agency financial system (STAR)

The transactions selected for review occurred between April 1 and November 30, 2014.

The scope of the audit did not include revenue collected through other systems such as:

  • the Parks Canada Reservation System (PCRS);
  • debit/credit card terminal transactions managed by a third party.

The audit also did not look at other processes for gathering business intelligence data (e.g., surveys).

At the time the audit was launched, the POS system had not yet been implemented in the canals operations. Although it was being implemented as this report was being written, those sites were excluded from the audit procedures.

6 Methodology

The audit criteria were developed mainly on the basis of the Directive on Revenue Comptrollership for User Fees[4], and the Parks Canada RMS User Guide for Visitor Information Collection. The audit criteria developed by the internal audit group and approved by the Point of Sale Steering Committee can be grouped into five distinct categories:

  • A Management Control Framework is in place to support the system effectively;
  • The information technology infrastructure that supports the system is adequate to ensure reliable data collection;
  • The data collected through the POS system are complete;
  • The data collected through the POS system are accurate; and
  • The data collected through the POS system are available in a timely manner.

The applied audit processes included the following:

  • An in-depth review of the documents constituting the legal and control framework;
  • A review of the documents used in the management and operation of the POS system;
  • Interviews with employees involved in the administration and use of the POS system (Appendix D);
  • Seven site visits (Appendix D) where the interviews were conducted, the deposit and transaction files (529) were analyzed, locally developed reference documents to guide employees in using the POS system were gathered and where transactions were observed;
  • The reconciliation of deposit transactions recorded in STAR with supporting documentation;
  • Benchmarking with other revenue-collecting public organizations with similar mandates to that of Parks Canada (4); and
  • The creation of flow charts detailing the business processes.

NB: For ease of reading, the term “Field Unit” (FU) is used to refer to national parks and national historic sites.

Table 4: Internal audit report notation system
Red Unsatisfactory Controls are not functioning or are nonexistent. Immediate management actions need to be taken to correct the situation.
Orange Significant improvements required The controls in place are weak. Several major issues were noted that could jeopardize the accomplishment of program/operational objectives. Immediate management actions are needed to address the control deficiencies noted.
Yellow Moderate improvements needed Some controls are in place and functioning. However, major issues were noted and need to be addressed. These issues could impact on the achievement of program/operational objectives.
Blue Minor improvements needed Many of the controls are functioning as intended. However, some minor changes are necessary to make the control environment more effective and efficient.
Green Controlled Controls are functioning as intended and no additional actions are necessary at this time.

7 Statement of Assurance

The audit conforms with the Internal Auditing Standards for the Government of Canada, as supported by the results of the quality assurance and improvement program.



Brian Evans
Chief Audit and Evaluation Executive – Parks Canada Agency

8 Conclusion

The audit showed that the control framework around the administration and use of the POS system and related business processes requires improvement, particularly with respect to the updating of policies and other reference documents and the monitoring of system use and data input. Controls in various sections of the business processes, such as collecting business intelligence, adequate cash management and compliance with legislative and policy requirements, need to be reinforced. For reporting purposes, data collected through the various collection methods should be integrated.

9 Observations and Recommendations

9.1 A Management Control Framework is in place to support the system effectively

Blue Minor improvements needed Many of the controls are functioning as intended. However, some minor changes are necessary to make the control environment more effective and efficient.

A control framework is implemented by an organization to support its operations and ensure that employees carry out their duties efficiently and effectively. These are the key elements of an effective management control framework: clear governance, well-defined roles and responsibilities, effectively communicated instructions, appropriate tools, and control and monitoring measures.

9.1.1 Accountability

At the start of the audit, the development and implementation of the POS was overseen by a Steering Committee (SC), made up of the national POS team and representatives of the Chief Financial Officer Directorate and the Information Services Directorate. It was chaired by the Chief Social Science and its mandate was oversee the overall implementation of the system in parks and sites.

During the course of the audit, accountability for the ongoing implementation and operation of the system was assigned to the VP ERVE and the Visitor Experience Branch within the ERVE Directorate.

The POS Steering Committee continues to exist and maintain its mandate as an advisory group to the VP ERVE.

9.1.2 Roles and Responsibilities

Roles and responsibilities are assigned relatively consistently across the Agency among the key stakeholders. The national POS team is responsible for the overall operation and configuration of the system, its implementation and users support. Field Unit Superintendents are responsible for ensuring that that Agency direction on implementing the system requirements is carried out in their FU. System administrators in the FUs are responsible for day to day operation of the system (user accounts, price setting etc.) and visitor service attendants are responsible to collect revenues and enter data in POS system in compliance with policy requirements. Revenue clerks and the financial operations group (comptrollership) in Cornwall are responsible for the reconciliation of POS revenue deposits with electronic files received by PCA in STAR through the Receiver General government banking system.

Although the majority of roles and responsibilities are assigned in a standard manner and accepted by the various stakeholders, they are not formally documented. Consequently, responsibility for some controls, including tracking of refunds and cash discrepancies, is not clearly assigned and, as a result, sometimes this responsibility is not fulfilled at all. Conversely, in some instances, a duplication of tasks was observed for this type of control. All the FUs visited have remedied this shortcoming by developing local reference documents detailing the responsibilities of the various stakeholders.

A best practice in standardizing procedures is to formally and clearly document the roles and responsibilities in reference documents at the national level. That practice could be considered for the administration and use of the POS.

9.1.3 Policies, Directives and Other Reference Documents

The RMS-POS User Guide is the main reference for information on procedures for “management” functions (price setting, creation of user accounts, etc.) and “user” functions. This document is available on the intranet and has been widely communicated to the FUs. Knowledge of the guide was clearly demonstrated by users and we were able to verify that it was referred to in the operational manuals. Since this document contained little information on the procedure for collecting business intelligence, the Parks Canada RMS User Guide for Visitor Information Collection was developed to provide the missing information. While the latter document is also available on the intranet, it has not been sufficiently communicated to be widely integrated into operations. Only a minority of FUs were able to clearly demonstrate any familiarity with this guide, which contains key information for users. As indicated in section 9.3, the Agency’s operations would benefit from better communication of this document to the various stakeholders.

As part of the analysis leading up to the selection of the POS system as a revenue-collecting solution, the SC had requested a threat and risk analysis related to implementation of the POS. The third-party report recommended that the Agency establish a security policy associated with the POS, which would include directives on user account management, passwords, permissions and access privileges as a framework for system use. No such policy had been established when the audit procedures were being conducted. Although the audit team acknowledges that developing a specific security policy for the POS may constitute a duplication of efforts given the existing policies and directives on computer network use, there are currently no parameters for access controls in the framework. The main issues associated with this situation are outlined in section 9.2.

As for the guidance on cash handling and revenue-reconciliation processes, PCA has developed the User Fees and Revenue Management Policy, the Directive on Revenue Comptrollership for User Fees and the Management of Revenue and Cash Standards, and documented the business process (Manage Point of Sale Transactions). Although those documents are currently available on the Finance Group intranet site, only the User Fees and Revenue Management Policy can be enforced as it is the only piece of guidance that has been formally approved by the EMC. Since the others have not been given this approval, they have not been widely disseminated. Their application is therefore not enforced and/or monitored. This results in inconsistent practices from one field unit to another, particularly for the daily terminal close-outs, the transportation of deposits to the bank, the allocation of change funds and the time frames in which deposits should be reconciled.[5] Some Agency practices fail to comply with the TB Directive on Receipt, Deposit and Recording of Money.[6] Those practices also jeopardize the safeguarding of Agency assets (potential for administrative errors, theft and fraud).

9.1.4 Training

The national POS team’s preferred training method is to provide essential training for designated system administrators in the FUs, who are then responsible for training users. This training method has been in place since the implementation pilots and the deployment of the system across the Agency. At that time, system administrators were trained for all sites equipped with POS terminals. These training sessions are still available through WebEx or in person. A training session was developed for POS administrators regarding reporting functions. The national POS team is also available to provide customized training for FUs, upon request.

That methodology is considered adequate, given that training on the POS is part of the general training given to attendants. However, it was noted, that the POS topics included in the training sessions vary from one field unit to another because the training documents developed by individual FU are not all based on the same reference documents. As a result, not all the essential topics required for the appropriate use of the system is explained. Since not every employee has been trained to pay attention to certain details that are considered important, the integrity of the data could be affected.

A training document on revenue management and revenue reconciliation is available on the Chief Financial Officer Directorate intranet site. The document provides a narrative description, along with screenshots of the steps to follow, for a reconciliation. However, the audit procedures showed that only two of the fifteen revenue clerks interviewed were aware of this document. Because of the lack of knowledge about national training documents, the parks/sites have to create their own training material, causing inconsistency in the information given to employees. The efficiency of the process is also affected by revenue clerks sometimes implementing additional unnecessary controls or failing to use the automated functions (e.g. auto-clearing) appropriately. This results in delays with revenue reconciliation and non-compliance with TB requirements. The revenue information reported to Agency Management is underestimated and therefore not reliable for timely decision-making purposes.

9.1.5 Technical Support

The technical support for system administrators and users is mainly provided by internal Agency resources working on the national POS team. Technical support is available seven days a week, 18 hours a day.

FUs needing technical support outside the hours of availability of the national POS team and the third party have to contact the IT unit’s first line of service for technical support at the National Office, which can resolve minor issues or refer them to the appropriate authority (POS or third party) as soon as it becomes available.

Since some of the components of the POS are technically complex, PCA contracted with a third party (through which the Agency also acquires system licenses) for a given number of client support service hours. Administrators and users can therefore contact the third party directly to obtain answers to their questions or problems.

Ninety-four per cent of those interviewed were either very satisfied or satisfied of the quality and timeliness of responses to technical support requests. That information is corroborated by simulations carried out during site visits, demonstrating that the maximum response time was 45 minutes, while the service standard is 60 minutes.[7]

The national POS team documented the methods for solving recurrent or frequent problems on its intranet site, where all the information can be found, enabling FUs to solve their own technical problems, as needed.

An analysis was conducted on the number of technical referrals to the third party to determine the efficiency of referring to that resource. The results showed that PCA only uses a portion of the person-hours included in the contract. Since the third party is paid by the hour, the value added for the Agency in this contractual relationship is reasonable.

9.1.6 Reporting Function

The POS Reports utility produces an array of reports on various types of information. Although the FUs mainly use sales and attendance reports (with variable frequency), the Social Sciences group in the National Office also uses these data to produce reports for PCA senior management and market analyses for the FUs.

It was noted, however, that the administrators were unfamiliar with the system’s reporting capabilities, despite the information sessions on this subject provided by the national POS team. The field unit system administrators cannot therefore be fully benefitting from the system’s reporting capabilities. For example, access to information about certain transactions that are considered risky (cancellations, reprinting receipts, opening the cash drawer without a sale) is not easily available or requires extensive manipulations in the system. Administrators/supervisors therefore do not have quick and easy access to important information in identifying training needs and managing performance (e.g. cash discrepancies at the user level, collection of visitor information). Moreover, since there is a limited number of “HQ Manager” licenses, administrators who do not have one must wait until they are physically on a given site to obtain reports from the store in question. The POS has a “DB Mail” component for sending predefined reports at a programmed frequency to those who are interested in the management of POS stores. Lack of accessibility to key information that is available through POS complicates the enforcement of compliance to Agency’s directives. The internal audit issued Recommendation #4 to address this observation.[8]

The system also has an audit log, making it possible to track all user manipulations in the system. This control tool, which was made available by the third party under contract with the Agency, is unknown to key stakeholders in the administration and use of the POS.

POS system data are used to produce revenue and business intelligence reports at the organizational level that are presented to the EMC regularly. The audit procedures have determined that the data that have been entered beforehand in the POS are accurately reported through the reports module. As noted in the presentation material of these reports submitted to the EMC, the reported data contain significant limitations on the representativeness of operational activities taking place within the Agency as a whole. Thus, the revenue and business intelligence collected through the PCRS, automated pass machines, commercial group entry fees[9], sales recorded by stand-alone POS terminals[10] and all manual sales processed are not accounted for in the reports. According to the internal audit estimates, the lack of integration among the various systems, timely availability of some of the data and the poor frequency of the connection of stand-alone POS terminals to the Agency’s network resulted in an underestimation of approximately 574,000 data entries, reported to the EMC, which represents roughly $6.5M in the 2014–2015 fiscal exercise.[11] Those responsible for implementing Recommendation #4 of this report, should take these factors into account when developing their action plan, in order to present the most accurate and up-to-date information possible to the EMC.

Conclusion

Accountability for the POS system was formally assigned to the VP ERVE, and that information was disseminated across the Agency. The roles and responsibilities surrounding the administration and use of the POS system are well defined and applied, despite the fact that they are not fully documented. Comprehensive reference documents governing the management and the use of the system are in place but they need to be communicated more effectively, especially the Parks Canada RMS User Guide for Visitor Information Collection. As for the documents governing business processes related to POS, such as cash handling and revenue reconciliation, they need to be reviewed and receive EMC approval before being actively disseminated across the Agency. System user training is adequate, although it could benefit with standardization. The existing client support structure for administrators and users is adequate and enables problem resolution times within the field units’ expectations. The reporting function requires improvement, especially the integration of other data sources in reports to senior management and automation of the preparation and submission of reports containing key information to managers/supervisors.

Recommendations

  1. The Chief Financial Officer should update the reference documents used for revenue management and financial transactions in the POS, have them approved by the appropriate delegated authority and communicate them to the FUs. Directive/standards should detail how compliance will be monitored and mechanisms to implement corrective actions when situations of non-compliance are observed.

    Management response

    Agree: CFOD agrees to review, update and communicate the following documents over the course of fiscal year 2016/2017:

    1. Directive on Revenue Comptrollership for User Fees
    2. Management of Revenue and Cash Standards
    3. Accounts Receivable Directive

    The revised documents will specify:

    1. how monitoring will be done to ensure compliance; and
    2. corrective actions in cases of non-compliance.

    CFOD will obtain approval from the appropriate authority for its Directives/Standards.

  2. The VP ERVE should develop and communicate to FUs a document outlining POS elements that are essential to the training of cashiers by the FUs.

    Management response

    Agree: The VP ERVE will develop and communicate to Field Units a POS training checklist by May 31 2016 to ensure that all new and returning cashiers are adequately trained on the use of the national POS system before the start of each operating season.

  3. The VP ERVE should identify key POS data of interest to various stakeholders and facilitate access to system’s information by automating the POS reporting function.

    Management response

    Agree: The VP ERVE will explore the possibility of making POS reports available to various Parks Canada staff using an automated reporting function.

    In the short term, standardized reports will be prepared by the national POS team and made available to Visitor Experience managers at regular intervals during the 2016 operating season.

    The POS team will explore the technical feasibility and user-friendliness of automated reports by March 31 2017.

  4. The VP ERVE should consider maximizing the integration of data from various sources, including but not limited to POS, PCRS, commercial sales, automated pass machines and POS stand-alone terminals into a single data output that would allow the production of more comprehensive reports.

    Management response

    Agree:     The VP ERVE, in consultation with the CIO, will explore the costs and timelines related to the integration of data from the POS and PCRS systems by September 30 2016.

    The VP ERVE will also work to ensure that data from all point of sale systems is integrated as new systems (e.g. automated gates, pay and display) are implemented at Parks Canada facilities across the country.

9.2 The information technology infrastructure that supports the system is adequate to ensure reliable data collection

Blue Minor improvements needed Many of the controls are functioning as intended. However, some minor changes are necessary to make the control environment more effective and efficient.

9.2.1 User Account Management

Proper management of user accounts, passwords and permissions and privileges is an essential element of an effective control framework for any system. Since the POS is used to collect revenue, this type of control was expected to be in place to minimize opportunities for fraudulent transactions.

The Agency did not establish a security policy for the POS system, despite the recommendation (by the third party under contract with PCA at that time) as part of the threats and risks analysis, in 2011. The audit procedures demonstrated a minimal level of control in user account management.

The majority (95%) of individual users had unique user accounts associated with their names. Exceptions were observed in some sites that provided guidance to employees to share general user accounts. Those facilities adopt such practices mainly because they do not have enough POS terminals for the number of employees present at any given time (mainly in some visitor reception centres and historic sites). Consequently, the employees use a common user code to avoid having to exit and re-enter the system for every transaction. That practice makes it almost impossible to identify training needs, attribute cash discrepancies or dubious transactions to the person at fault or manage performance.

The absence of clear directives regarding the frequency of password changes is reflected in the control environment as only 35.8% (15/39) of the interviewed individuals indicated that they had changed the password initially assigned to them. The 15 system administrators/supervisors who were interviewed stated that employees are not given any directives on security features (upper case vs lower case letters, numbers vs letters, minimal number of characters, special characters, etc.) which their passwords should include. One of the attendants who were interviewed (1/27, or 3.8%) indicated having made his/her password more complex by including security features. Forty-one percent of the cashiers questioned indicated (and demonstrated on screen) that they knew the procedure for changing their password, while 58.6% were unable to do so, despite the fact that the procedure was explained in the POS help module.

The system has a control for freezing a user account after a certain number of unsuccessful access attempts, but the control has been deactivated. Only a notice pops up at the next successful access attempt, indicating the number of unsuccessful attempts.

Regarding the management of inactive accounts, neither the expectations nor the procedures are clearly defined. The practice was to erase user accounts that were no longer required (e.g., because the employee had left the Agency) and to reassign them to new employees. The audit procedures determined that this practice should be abolished because, when a user account is erased, all the historical transactional data associated with the cashier’s name are erased as well. It would therefore be very complicated to retrace who processed a specific transaction, which limits the possibility of recovery in case of fraud. At the time of writing this report, the national POS team was reviewing the user manuals and adding the appropriate procedure for managing accounts that are no longer required. This piece of information was also included in the training given to system administrators.

Nine of the fourteen (63.4%) system administrators interviewed indicated that they monitored accounts that were no longer required, but none of them uses the method that protects the integrity of the historical transactional data. Of the 681 user accounts analyzed, 51.5% were assigned to employees who were not in the PeopleSoft database on December 10, 2014. Of those 351 user accounts, only 15.3% had been properly de-activated in the system, which means that 294 accounts in the names of employees who were no longer working at the Agency at the time were still active.

The analysis of permissions and privileges of 700 user accounts showed that 52.9% were consistent with the day-to-day tasks[12] of the individuals in question and with the permissions and privileges included in the templates proposed for the various levels of system users. In four of the seven FUs visited, although the appropriate templates had been used to create the user accounts, administrator and manager permissions had been granted to employees who did not need that type of access to carry out their day-to-day tasks.

Similarly, one FU systematically granted attendants access to the “X” report at the end of the day.[13] This report discloses the amount to be deposited before the cash is counted. The process of closing out the cash with advance knowledge of the amount to be deposited bypasses a control, thus increasing the risk of theft. This practice was brought to the FU’s attention in September, but when writing the report in December, no action had been taken to correct the situation. The internal audit considers closing out the cash without advance knowledge of the deposit amount to be a reliable and appropriate process when applied under normal circumstances (where the cashier enters the amount of cash received for the day, then accesses the cash report).

9.2.2 Business Continuity, Asset Safeguard and Visitor Experience

The POS system is used daily and extensively by Agency employees. A breakdown in this system could have a considerable impact on revenue and data collection, site access, visitor safety, the visitor experience and the conduct of operations. Controls were therefore expected to be in place to minimize service interruptions and enhance system reliability. Compensatory measures were also expected to be in place to mitigate the impact in the event of service disruptions.

The interviews revealed that 55 of the 62 employees (88.7%) surveyed on this subject perceive the system as being very reliable and never or hardly ever experienced service interruptions. The main causes of service disruption in the POS are related to connectivity, power outages and defects in the terminal’s hard drive. Those factors are not inherent to the POS and may have the same effect on any other sales system (PCRS and/or others).

The system configuration ensures that the operation of the four servers is not vital to the day-to-day business since the POS terminals can operate in offline mode. In these situations, the data is stored on the device’s hard drive until it is transferred to the appropriate server via a network connection. That configuration allows for continuous use of the POS despite the remoteness and lack of access to a network connection in certain sites.

Fluctuations in electrical current at some sites altered the way the terminals operated. The national POS team purchased UPS devices to stabilize the current and serve as a backup battery in case of a power outage. These devices give the terminals sufficient battery life to operate until the current is restored (for most power outages).

System users were interviewed to determine the real impact of system service interruptions. Although the majority of users indicated that the system was very rarely unusable, 89% of respondents (34/38) stated that they did not think a temporary system outage would have a significant impact on the visitor experience (increased wait times at the cash, annoyance of being redirected to another revenue collection point). Four of the 38 interviewees indicated that they thought the service interruptions would have a significant impact on the collection of attendance data and information on visitors as they would stop collecting them. Otherwise, the data would be noted on paper and entered into the system later on. None of the individuals interviewed felt that a system outage would affect revenue collection since other revenue collection methods (PCRS, manual pass systems, etc.) can be used to collect entry fees. No evidence of documented plans in case of system breakdowns could be gathered through the audit procedures. The internal audit considers that such emergency plans would be an effective method to communicate expectations and guide attendants in case of a system breakdown.

Data are transferred from the “HQ Manager” server to an SQL server to reduce the risk of corruption of the system’s operational database. Backup copies of the “HQ Manager” are made on a shared drive created specifically for that purpose. The POS data are saved on servers behind a firewall administered by Shared Services Canada (SSC), which considerably reduces the risk of breaches and data loss. Backup copies of the entire database are made once a week at the Agency’s data centre while daily variances are saved at the same location every day. There is daily communication between SSC and Parks Canada personnel, and exception reports are submitted to PCA when the data are incorrectly copied. Data that are not uploaded on the national server on a given date remain at risk.[14] The risk of lost data represented $2,165,188 in revenue and 101,313 in park and site entries for the period covered by this audit.

A life cycle management plan for the terminals was developed to mitigate the risks of data loss and service breakdown that could occur as a result of the aging of the machines.

9.2.3 Optimization of POS Implementation and Use

It was expected that sites selected to be equipped with POS terminals would be chosen following a fact-based analysis to maximize the return on investment, while providing visitors with the most consistent experience possible across the Agency. We also expected that sites that generated a certain amount of revenue but did not have viable web connections would be equipped with stand-alone terminals. All system options that could bring value-added to Agency operations were also expected to be activated and the staff was expected to be properly trained to the benefit of the organization. Lastly, we expected that mobile terminals adjusted to the FUs’ needs would be available to meet those requirements and that this would be a reliable, effective option for collecting revenue and business intelligence.

The POS SC provided evidence demonstrating that analyses had been conducted to develop the strategy for allocating terminals across the Agency.

The audit procedures provided an understanding that all the sites that met the assignment criteria and had a reliable connection were connected to the PCA network. At sites that cannot be connected to the network, system implementation is maximized through the installation of stand-alone machines.[15]

The majority of functions relevant to Parks Canada operations are currently available in the system. Although these POS functions were rolled out, not all FUs use them, often because they do not know they exist. Of the three functions that the internal audit group identified as those that could provide the greatest efficiency gains, none are being used by over half the attendants interviewed.[16] The FUs are under no obligation to use these functions. However, the FUs that do not use them continue to maintain manual records (accounts receivable[17], inventories of merchandise and annual/season passes), which is less efficient. Moreover, manual records are considered to be less effective than automated controls in preventing fraud and theft.

All of the people interviewed (13) about the mobile revenue collection devices indicated that these devices were a viable option for specific needs, including compliance programs and mobile gates. The audit team noted, however, that this solution was not known in the field and that these devices could be beneficial for operations at certain sites. Awareness activities could increase knowledge about these devices and help certain FUs improve the effectiveness of their operations.

Conclusion

User accounts, passwords, permissions and privileges are not being administered so as to optimize the integrity of data and revenue collected through the POS system. Existing controls ensure the backup of data collected through the POS and mitigate the risk of service interruptions as well as their impact. Although most of the required functions in the POS have been rolled out, many FUs continue to use manual processes instead of using the system.

Recommendations

  1. The VP ERVE should develop a security strategy related to the administration of user accounts, the management of passwords and the assignment of POS system access permissions and privileges, and communicate it to the FUs.

    Management Response

    Agree: The VP ERVE, in consultation with the CIO, will develop, communicate and implement a security strategy related to the administration of user accounts, passwords and system access permission and privileges by June 30 2016.

  2. VP ERVE should develop and communicate to field units clear direction on the utilization of the POS system functions to support management of operations (including but not limited to: inventory of merchandise, Discovery and seasonal passes, use of scanners, etc.). Analysis on the compliance to directives should be conducted and reports should be made available to VP Operations, on as needed basis to ensure that corrective actions are taken when necessary.

    Management Response

    Partially Agree: The VP ERVE (POS Team) will schedule pre-season teleconferences with field units and liaise with them on a regular basis during the operating season to ensure that they are better informed of the various POS system functions and features that are available to support them in the management of their operations.

    In the short term (by June 30, 2016), the POS team will be enhancing the content of the POS Intranet page (new users manuals, etc.) and offer ongoing POS training to ensure that front line staff and supervisors are up to date on the features that are available to them.

    The VP ERVE disagrees with the recommendation that analysis on the compliance (related to this matter) is required since the use of some POS functions/features is optional and at the discretion of each Parks Canada location.

9.3 The data collected through the POS system are complete

Orange Significant improvements required The controls in place are weak. Several major issues were noted that could jeopardize the accomplishment of program/operational objectives. Immediate management actions are needed to address the control deficiencies noted.

9.4 The data collected through the POS system are accurate

Yellow Moderate improvements required Some controls are in place and functioning. However, major issues were noted and need to be addressed. These issues could impact on the achievement of program/operational objectives.

9.5 The data collected through the POS system are available in a timely manner

Blue Minor improvements required Many of the controls are functioning as intended. However, some minor changes are necessary to make the control environment more effective and efficient.

For ease of reading, criteria 9.3, 9.4 and 9.5 will be grouped in two broad categories representing the data types concerned: financial data, and, attendance data and information on visitors.

Controls Related to Financial Data

The POS system was implemented because the Agency needed to adopt a common methodology for collecting entry fees for collecting admission fees for its activities. The purpose of the system is to support the collection of all entry fees owed to PCA and to ensure that the information compiled is recorded in a way that allows for accurate and timely reporting. More specifically, it was expected that:

  • All revenue from entry fees owed to the Agency be collected and measures exist to detect visitors who neglect to pay entry fees;
  • Batch close-outs be carried out the end of each shift, at a minimum;
  • Cash deposits be made at a frequency as set out in the policy framework and that all financial data be transferred from the POS system to STAR as frequently as possible;
  • Revenue control measures be in place to safeguard cash under PCA’s responsibility;
  • Fees charged to visitors via the POS system are in compliance with the Parks Canada price list in the Canada Gazette;
  • Transactions that could affect the accuracy of the Agency’s revenue (returns, rebates, cancellations, etc.) be handled diligently;
  • Amounts declared as revenue be accurate;
  • The routing of deposits be adequately reflected throughout the revenue collection and reconciliation process (e.g., amounts recorded in the deposit log are exactly the same as those received in STAR and are eventually reconciled);
  • The revenue reconciliation process be effective and timely;
  • Complete and accurate reports on financial data be produced in a timely manner.
Total revenue collected and payment of entry fees

All the national parks we visited confirmed that there are ways for visitors to enter PCA facilities without paying admission fees.

All the field units we visited had mechanisms in place to educate visitors on the importance of paying fees (prevention controls) through various notices, signs, and pamphlets. The sites report use of reactive approaches when noting suspected instances of non-compliance through issuing “friendly” stressing the importance of paying and providing information on where to pay.[18] Two of the field units we visited had gone farther than this by implementing use of mobile gates where cars at specific locations are systematically stopped and asked for evidence of payment. When non-payment is identified park staff are able to collect payment, issue passes and record revenue immediately through use of mobile POS devices. In both cases, FU personal responsible for the programs reported that the revenue gains (i.e., approximately $621 000 from April 1 to December 31, 2015) from the interventions exceeded the costs of the compliance initiatives. We asked for documentation on the costs relative to the revenue gains but management was not able to provide this information.

The potential for additional, cost effective, revenue collection at the other sites in our sample could not been systematically assessed to determine the relevance and value added of additional detection and control mechanism.

At an Agency wide level it is clear that the amount of forgone revenue at some sites is likely to be minimal or non-existent (i.e., those with very low levels of visitation or controlled points of entry, as exist for example in many northern parks).

It is also clear that attempts to capture forgone revenue need to take account of costs of additional controls relative to the amount of revenue generated and that no one approach to capturing forgone revenue is appropriate to every situation (i.e., mobile gates are not a solution in every circumstance).

We also noted that the Agency lacks an overall coordinated strategy (or clear methodology) for assessing where and in what circumstances it makes sense to implement additional controls to capture forgone revenue.

Billing of corporate accounts

Sites in Eastern Canada[19] use the “on account” tender type in POS to record sales for which payment is not received at the time of the visit. The use of the “on account” tender type allows for timely recording of information in the system, unlike systems in which payments are collected in advance, manual or self-reported billing (entries recorded the day on which the payment is processed). However, regardless of the mechanism used to record the sale, the accounts receivable entry must be made manually in STAR. Current account receivables recording practices do not allow for the tracing of account receivable entries to related POS transactions. As a result, the audit team had to conclude that documentation in support of accounts receivables is inadequate for auditing purposes and that the manual processes currently in place for billing and collecting accounts receivables do not guarantee that all revenues owed to the Agency are invoiced. Key controls used to ensure minimal success of the accounts receivable function should include the development and communication of a clear and common accounts receivable recording process (should be considered in recommendation #1 of this report), the harmonization of client lists (POS and STAR), the centralization of control over modifications to POS client accounts, the implementation of centralized billing in STAR and a mandatory “client number” field.[20]

Manual cash management controls

The audit included a criterion intended to verify the adequacy of cash management practices in place to safeguard revenue collection through POS. For security reasons, the conclusions drawn from the audit procedures carried out as part of this criterion will be communicated by means of a letter to management.

There is currently no standard for keeping a deposit log nor for the information that such a log should include. As a result, the information recorded in the deposit logs in the majority of the locations that were visited did not contain sufficient information to allow for POS deposit reconciliation with STAR entries. Others did not maintain deposit logs at all. The deposit reconciliation process at a few facilities would not have enabled the audit team to track POS batches against deposits in STAR, particularly because some bank deposits include batches of several work shifts or terminals.

Issues related to the separation of duties were noted in three of the field units we visited. The three sites had systems whereby the person responsible for reconciling revenue in STAR also had access to cash as part of their duties. During conversations held subsequently to the site visits, each of the filed units demonstrated that they have a resolution plan.

Issues related to manual internal controls and the availability of reliable deposit information stem from the lack of a clearly communicated national directive regarding these aspects and a lack of monitoring of these controls. This could lead to mistakes, fraud and omissions that could result in losses for the Agency. This situation is covered by Recommendation #1 of this report.

Automated financial controls

The primary automated controls in the system are the amount limit per transaction, the reimbursement limit and the cash discrepancy limit. Following an audit follow-up report submitted to the POS Steering Committee in February 2015, the national POS team attempted to standardize the cash discrepancy limit for all sites managed by the Agency.[21] It nevertheless was unable to achieve this for all accounts, due to technical problems. The Directive on Revenue Comptrollership for User Fees does not provide details on acceptable limits for the other parameters.[22] The audit procedures revealed that some of these controls were tested during the system’s implementation phase and were abandoned because of the operational constraints they could cause. According to the internal audit team, it would be ideal to implement these preventive controls over certain transactions which involve a potential for fraud. Since implementing controls upstream has an impact on operations, the internal audit team recommends that a mechanism be developed to monitor these parameters. The mechanism should be included in centralized reporting procedures (Recommendation #3 of this report) and communicated by the national POS team. The field units should be responsible for monitoring and taking corrective measures.

The audit procedures also enabled us to determine that there is no control over the maximum number that can be entered in some fields, including the quantity and dollar value of the transaction[23]. One of the main causes of cash discrepancies is cashiers entering the wrong quantity in the field in question but collecting the right amount from visitors (since they have memorized the prices). Consequently, visitors receive a pass whose expiration date does not correspond to what they paid for. In addition, cashiers must take the time to understand the cash discrepancies created when they make this mistake. Although a control over the validity of quantity fields and total transaction fields can potentially extend the duration of transactions that include mistakes, it could make reconciling cash discrepancies more efficient at the end of the day. A control that is similar to the one used to limit the number of visitors associated with a transaction should be put in place to prevent cash discrepancies stemming from this type of administrative error.

The characteristics of certain types of receipts could lead to administrative errors, fraud or theft. In February 2015, these characteristics were communicated to the POS Steering Committee, which has agreed to take the action required to correct situations involving problematic receipts. The internal audit team was able to confirm that these changes are being made but were not complete when this report was being drafted.

Contrary to the popular belief that transactions could be cancelled only on the current day (71.4%), it was demonstrated that transactions from previous days could also be cancelled. If this type of transaction was done in the POS, sales data for previous days could be modified and the reconciliation process would be even more complicated. This observation was communicated to the system Steering Committee, which is making the changes to prevent this situation from occurring. Implementation of these changes was not complete by the time this report was drafted.

Support documentation for risky transactions

With regard to the compliance of documentation related to risky transactions,[24] as indicated in the Directive on Revenue Comptrollership for User Fees, we expected supporting documentation (receipts) to be included in envelopes containing batch close-out documents. We also expected reimbursement receipts to be signed by visitors[25] and the employee who handled the transaction. In addition, funds are to be repaid to visitors using the same payment method as the initial transaction and a copy of the original receipt should be attached. Refunds/rebates should be accompanied by supporting documents (coupon, rebate, promotional offer, etc.). Here are our findings to this effect:

  • 75.1% (245/321) of receipts were available on file for review;
  • 42.4% (104/245) had both required signatures (visitor and employee);
  • 57.6% (141/245) were not compliant, that is, one of the required signatures was missing;
    • 45.4% (64/141) had only the employee’s signature but no signature from a third party (visitor, supervisor or co-worker);
    • 7.8% (11/141) had only a third party’s signature (not the employee’s signature);
    • 46.8% (66/141) did not have any signature;
  • 83.9% of reimbursements were made using the initial payment method;
  • 33% (185) of reviewed transactions including rebates, refunds or other reductions had supporting documentation.

In addition, the site visits enabled us to determine that the field units hardly ever follow up on risky transactions.[26]

Material codes, eligible items and pricing

The current procedure for creating/modifying material codes in the system ensures coding integrity in the financial system and the harmonization of coding between STAR and the POS.

The internal audit team analyzed price lists taken from the POS and compared them with the price lists approved by the Parliament of Canada.[27] In the vast majority of cases, prices programmed in the system comply with approved prices. The main exceptions are due to the fact that some sites simply do not program prices for items they do not sell. The other exceptions were reported to the field units so that the appropriate corrective measures could be taken.

As well, because all items sold must be categorized under a material code before they can be billed and the management of material codes is centralized at National Headquarters, it is practically impossible for items that are not in the Canada Gazette to be sold using the POS system. Using existing and approved material codes to sell different products may be possible, but the audit scope and procedures could not have identified this.

Uploading of sales data in STAR and revenue reconciliation process

Deposit reconciliation practices in the financial system currently in place at the Agency prevent the organization from complying with the Directive on Receipt, Deposit and Recording of Money (section 6.1.6), according to which all cash receipts are to be reconciled the day after files are received from the Government Banking System.[28]

According to batch close-out analyses, 12 stores do not close out their batches on a daily basis. For the sites concerned, the field units we visited indicated that they closed out their batches less frequently because the sales amount did not justify all the administrative steps that the clerks need to complete to make a deposit. Although this practice does not comply with the Directive on Revenue Comptrollership for User Fees, the internal audit team feels that a risk-based approach to this aspect could result in efficiency gains in the reconciliation process, provided asset safeguarding controls are enhanced.

The national POS team developed a POS extraction tool, which creates purchase orders in STAR automatically when a POS system file is uploaded. However, manually creating billing documents for every single purchase order creates a bottleneck in the process. A remote revenue reconciliation pilot project intended to centralize accounts receivable management activities in Cornwall by 2017-2018 is currently under way. The Chief Financial Officer Branch feels that processes streamlined by this consolidation will enable the Agency to be more effective and meet the requirements of the directive. Until the transition is completed, the CFO Directorate plans to include in its monitoring activities[29] regular compliance audits of revenue reconciliation activities to reduce the amount of time required and aim for the parameters in the directive. Clear follow-up measures that include supporting evidence should also be considered. Should the consolidation of accounts receivable not have the anticipated effect, the CFO Directorate should plan to take steps to implement a collective reconciliation process.[30] Right now, financial data on revenue from STAR reported for decision making are under-estimated (as of November 30 2014, $5.9M[31] still had not been reconciled).[32]

The controls that ensure the integrity of sales data uploaded from the POS to STAR were also tested. The effectiveness of these controls was demonstrated; all purchase orders with errors were rejected and sent to STAR’s correction facility.

The simulations did not reveal any irregularities in the data that was uploaded. However, an analysis of the format of these validity controls showed that files taken from the POS in “.txt” format can be opened and modified by people who have access to them. For security purposes, the internal audit team’s recommendation to this effect will also be included in the letter to management.

The internal audit procedures showed a high level of compliance of electronic deposit files received through the Government Banking System. This part of the deposit management and reconciliation process is deemed to be impervious.

None of the people who are responsible for reconciling revenue at the field units we visited indicated that they used the auto-clearing feature in STAR, which automatically reconciles billing documents and deposit files without the need for manual entries. It would thus take less time to reconcile data. For the most part, people do not use this feature because they are not aware it exists. The internal audit team feels that it is a simple and effective method that speeds up the reconciliation cycle and recommend that it be used throughout the Agency.

Conclusion

The control framework in place does not lead us to conclude that the revenue data in the POS system are complete. While most automated controls are deemed to be effective (blind batch close-outs, limit on cash discrepancies, automated data transfers between the POS and STAR), some need to be tweaked (system access management, access to information on risky transactions). The main elements that could compromise the completeness and accuracy of revenue entered in the POS are manual cash manipulation controls, including the lack of support documents on monitoring deposits from start to finish. The data extracted from the POS tool, which are available in a timely manner, accurately reflect information that is entered. There are opportunities to use the system to save data collected with other methods, so as to produce more comprehensive reports on the Agency’s routine business. In addition, the financial data in STAR are not available in a timely manner because of the time taken to reconcile revenue.

Recommendations

  1. The VP, Operations should document where forgone revenue from entry fees is likely to be a significant issue (i.e., parks or sites) and seek assurance that entry fees compliance/revenue recovery mechanisms have been developed and implemented where it is cost effective to do so.

    Management Response

    Agree:     Field units will assess foregone entry fees as part of their annual planning. Furthermore, the Operations Directorate will work with functional experts (i.e., External Relations and Visitor Experience, and Finance) to implement compliance/revenue recovery mechanisms that take into account the operational realities of each field unit. Recognizing that there will be free entry in 2017, any developed recovery mechanisms determined to be operationally feasible will be implemented in 2018.

  2. The VP ERVE should develop and implement an automated control that will limit the possibility of errors in terms of transaction quantities and/or amounts.

    Management Response

    Agree:     The VP ERVE, in consultation with the CIO, will explore the costs, timelines and technical feasibility of developing and implementing additional automated controls to limit the possibility of errors related to transaction quantities and/or amounts by March 31 2017.

  3. The Chief Financial Officer should reinforce the use of the auto-clearing function for deposit files in STAR to streamline the reconciliation process.

    Management Response

    Agree: As part of the review of the directives and standards related to revenue management, the auto-clearing function will be assessed and its use will be reinforced when appropriate.

Business Intelligence

The other main factor that justifies the acquisition of a new admission fee management system is the opportunity for the Agency to gather business intelligence. It is in the Agency's interest to know the number of people who visit its parks and sites and to understand the composition of groups that come to reception desks. It is also worthwhile for the Agency to have data concerning visitor groups, their residency, their PCA facilities visitation habits and the language in which service was provided. These data, coupled with data from various other sources (PCRS, satisfaction surveys, etc.), are a wealth of business intelligence that make a substantial contribution to the Agency's communication activities and to the services it offers to its visitors. These data serve as the basis for reports on operational activities created by the Social Science group. The reports are intended for senior management and field unit managers for decision making purposes. The Agency was expected to have implemented proper controls to ensure it maximizes the return on its investment by collecting complete, accurate and timely data . More specifically, the internal audit team expected to find:

  • Consistent use of the system, as described in the reference documents;
  • Measures in place to maximize the collection of business intelligence while minimizing the impact of these practices on the visitor experience;
  • Communications sent to the field units regarding the importance of collecting business intelligence and a demonstration of the benefits of these data at all levels of the organization;
  • Communication of clear expectations in terms of the collection rates;
  • Elements in place that monitor the collection of business intelligence;
  • Controls in place to detect unconventional entries;
  • Mechanisms that automatically extract system data and upload them to other analysis systems;
  • Timely system updates to ensure the accuracy of collected data;
  • Timely business intelligence for reporting purposes, so as to inform conclusions drawn by the Agency.
Compliance of attendance data recording

The audit team examined 318 transactions to confirm whether they had been processed as indicated in the User Guide for Visitor Information Collection. The transactions (regular sale of day passes) were recorded in compliance with the guide in over 90% of cases (individual billing of items, proper use of group material codes, sale of grouped items and processing of national initiatives).

However, park entries were accounted for in only 29.6% (8/27) of transactions, when season or annual passes were sold, which leads us to underestimated number of entries.[33] In addition, the audit team noted that subsequent visits by pass holders are not consistently recorded in the POS. Fifty percent of visitor services attendants indicated that when annual/season pass holders came through the gates and stopped, they recorded it in the POS.[34] The number of unrecorded repeat visits is even more difficult to estimate, since these pass holders generally use the bypass lanes and/or do not stop at the gates.[35]

We were unable to assess the recording of visitors who do not pay (children under the age of six, group leaders, bus drivers) against a sufficient number of transactions so as to draw conclusions. However, 42% of clerks indicated that they recorded these entries in the system.

When visitor services attendants sell entry passes, they have to record the number of individuals in the car and the following characteristics: adults, teenagers, seniors and children (under the age of six). The Agency uses this information directly to analyze the composition of groups who visit its facilities and indirectly in its communications strategies. Two of the field units we visited have given instructions to their clerks to enter everyone (in vehicles) as adults[36], contrary to the Agency’s directives listed in the reference documents. Intelligence about the composition of visitor groups at these field units are therefore quite unreliable, which has an impact on the Agency’s data in general.

Communicating the importance of data collection

According to information gathered by the Social Science group, the reason for collecting business intelligence and its importance to the Agency were communicated in training given to administrators when the system was implemented. The audit team was unable to find any official communication sent to convey the importance of gathering such information. Although the User Guide for Visitor Information Collection is available on the intranet and contains information on the subject, the field units are relatively unfamiliar with it. According to 37.5% of system administrators/supervisors (9/24), they had received information from National Office on the importance of collecting intelligence or reports produced with data collected by the POS. In addition, 25% of system administrators/supervisors stated that they had received instructions on the proper way to collect this intelligence. On this note, the results of analyses of data collected through the POS only started being sent to the field units as recently as June 2014. Based on the audit results, we conclude that the reason why this data is gathered, how they are analyzed and how the organization can benefit are vague notions for many key people who administer and use the system. Consequently, the need to improve business intelligence collection is not reinforced on a regular basis.

Use of data in local and national attendance reports

Postal codes gathered and entered in the system enable the Agency to better understand where visitors are from and, as a result, better target its awareness activities and communication efforts. To this end, collected raw data are analyzed using software that requires at least 1,000 postal code entries.[37] The statistical basis for requiring 1,000 postal codes could not be demonstrated. However, the greater the amount of data in the analysis software, the better it is able to provide accurate results on where these visitors are from, which facilitates the targeting of promotional activities and improves their effectiveness. Understanding this business intelligence also allows the Agency to adjust its operations in line with visitor needs. The data analysis showed that major improvements could be made to business intelligence collection to help the Agency’s external relations groups achieve their objectives.[38]

Gathering visitor information and transaction ratio

The RMS User Guide for Visitor Information Collection highly recommends the collection of business intelligence on each transaction.[39]The internal audit team was unable to find any documentation demonstrating that analyses have been conducted to develop a strategy to optimize the collection of data while reconciling the best possible visitor experience. Therefore, factors such as seasonality, volume of transactions and type of facilities as well as acceptable situations that could justify a decrease of the transaction ratio, have not been taken into consideration in the expectations communicated in the Guide.

At the technical level, the transaction ratios can vary from 1/1 to 1/100 transactions. The ability to set these ratios has been delegated to the FUs in order to ensure flexibility with the system. Given that the parameters surrounding the administration of the ratios is documented in a user guide and not in a policy or directive, the frequency of the collection of information on visitors is not imposed or monitored. Four of the seven field units we visited decided to reduce how often the visitor information entry screen is displayed. No supporting documents were provided to demonstrate that the approval process for changing the transaction ratio was followed as per the User Guide for Visitor Information Collection. There is no monitoring activity carried out on the transaction ratios fixed by the FUs[40]; however, the frequency of the collection business intelligence can be calculated by dividing the number of sales transactions containing material codes associated with entry fees, by the number of postal codes/country or origin collected. In consideration of all these factors, the information collection rates on visitors varies from 100% to 0% depending on the FU.

There are also ways to circumvent the dialog box that is displayed to collect visitor information. The clerks have the option to simply cancel the dialog box to avoid having to enter information. They may also check “Refused/Don’t know”. Analyses carried out in 2014 on data gathered in 2013[41] showed that many sites resorted to these tactics instead of collecting the required business intelligence.

The audit team noted some best practices whereby questions asked to obtain the expected business intelligence had been integrated into conversations between the visitor service attendants and the visitors. The sites where these practices have been incorporated into client service protocols collected more information from clients. Some scripts and sample questions are also available in the RMS User Guide for Visitor Information Collection.

Conclusion

The Agency does not make full use of the benefits of the POS system. Automated controls exist, but there are opportunities to improve the general control framework for the collection of business intelligence. Although data gathered are largely accurate and available by means of the timely reporting utility, governance issues, activity monitoring, instructions given to the field units and weaknesses of some automated controls prevent the Agency from fully benefiting from the POS.

Recommendations

  1. The VP ERVE should develop and communicate direction to FUs describing the expectations with regard to the collection of business intelligence. Reports on compliance with the directives should be made available to VP Operations, on a regular basis to ensure that corrective actions are taken when necessary.

    Management Response

    Agree: The VP ERVE, in consultation with the Chief Social Scientist and the Director of Visitor Experience, will develop and communicate direction to the Senior VP of Operations (and regional Executive Directors) related to the expectations and benefits associated with the collection of demographic information on visitors via the national POS system by May 31 2016.

    The POS team, in collaboration with Social Science specialists, will explore a mechanism to track performance related to the collection of visitor information to relevant staff during the operating season.

Appendix A: Applicable Legislation, Policies and Directives

Acts and regulations

  • Parks Canada Agency Act
  • Financial Administration Act
  • Receipt and Deposit of Public Money Regulations, 1997
  • Accountable Advances Regulations

Treasury Board policies, directives, guidelines and standards

  • Directive on Management of Information Technology
  • Policy on Government Security
  • Directive on Accountable Advances
  • Directive on Losses of Money or Property
  • Directive on Receipt, Deposit and Recording of Money
  • Guideline on Accountable Advances
  • Operational Security Standard on Physical Security
  • Operational Security Standard: Management of Information Technology Security (MITS)
  • Policy on Internal Control

Communications Security Establishment documents

  • Clearing and Declassifying Electronic Data Storage Devices

Parks Canada Agency policies, directives, guidelines and standards

  • Directive on Revenue Comptrollership for User Fees
  • Management of Revenue and Cash Standards
  • User Fees and Revenue Management Policy

Other documents

  • Parks Canada Agency Risk Profile 2014-2015

Appendix B: Glossary

CEO:
Chief Executive Officer
CFO:
Chief Financial Officer
CFOB:
Chief Financial Officer Branch
CIO:
Chief Information Officer
PCRS:
Parks Canada Reservation System
EMC:
Executive Management Committee
ERVE:
External Relations and Visitor Experience Directorate
FU:
Field unit
PCA:
Parks Canada Agency
POS:
Point of Sale system
PWGSC:
Public Works and Government Services Canada
SC:
Steering Committee
SSC:
Shared Services Canada
STAR:
Parks Canada Agency’s financial information management system
TBS:
Treasury Board Secretariat
VP:
Vice-President

Appendix C: Recommendation Prioritization System

Table 5: Internal audit recommendation prioritization system
Priority Condition
High Management should initiate immediate action to address the comment.
1 Major internal control weakness
2 Major policy or procedure exceptions
3 Major risk exposure
4 Major financial exceptions – loss, misstatement, errors, fraud
5 Major law or regulatory violations
6 Major potential opportunity – revenue, savings, efficiencies and improvements
Moderate Management should initiate timely action to address the comment.
1 Substantial internal control weakness
2 Substantial policy or procedure exceptions
3 Substantial risk exposure
4 Substantial financial exceptions – loss, misstatement, errors, fraud
5 Substantial law or regulatory violations
6 Substantial potential opportunity – revenue, savings, efficiencies and improvements
Low Management should initiate reasonable action to incorporate a plan to address the comment in the normal course of business.
1 Minor internal control weakness
2 Minor policy or procedure exceptions
3 Limited risk exposure
4 Minor financial exceptions – loss, misstatement, errors, fraud
5 Minor law or regulatory violations
6 Limited potential opportunity – revenue, savings, efficiencies and improvements

Appendix D: List of Interviews Conducted and Field Units Visited

Table 6: Summary of interviews as part of the POS system audit
Employees involved in administering and using the POS at the national level 14
Finance and Administration managers in the field units 7
Staff responsible for POS administration in the field units (system administrators/supervisors) 26
Front-line staff using the POS on a daily basis 36
Staff responsible for revenue management and reconciliation activities in the field units 15
External stakeholder 1

Table 7: List of field units visited as part of the POS system audit

  • Banff Field Unit (east gate, visitor centre, Cave and Basin, administrative office)
  • Hot Springs Enterprise Unit (Banff, Radium)
  • Kootenay-Yoho-Lake Louise Field Unit (Lake Louise visitor centre, west gate)
  • Jasper Field Unit (east, west and south gates, visitor centre and administrative office)
  • Southwest Ontario Field Office (Point Pelee National Park, Fort George)
  • New Brunswick South Field Unit (Fundy National Park)
  • Cape Breton Island Field Unit (west gate, Ingonish and Chéticamp visitor centres, Alexander Graham Bell NHS, Fortress of Louisbourg NHS)
  • Prince Edward Island Field Unit

[2] Data are presented for comparative purposes of revenue collection by the different systems. These are unverified data for which no audit procedure was performed to validate the accuracy and/or completeness.

[3] Of this amount, $2,255,274 (2014-2015) and $3,640,449 (2015-2016) were administrative charges paid to the third party who administers the PCRS for Parks Canada.

[4] The internal audit group acknowledges that the Directive is not an official document because it has not been formally approved by the EMC. However, it is the only internal document on which to base the audit work and determine the expectations for internal controls for handling and safeguarding cash.

[5] There are other elements on this list which will be detailed in the management letter.

[6] Subject discussed in greater detail in Section 9.3 of this report.

[7] Service standard with the third party, as documented in the contract agreement.

[8] The list of elements to be considered as part of this report initiative will be communicated in a management letter.

[9] Especially in the Western FUs, the Eastern FUs having integrated the POS into their billing process.

[10] Significant delays were noted between stand-alone POS terminal connections and the server. As at October 6, 2015. 82 days for Eastern stores and 92 days for Western stores, on average, in 2015.

[11] Around 10% of total sales made through the POS.

[12] According to the description of duties associated with their position titles in the PeopleSoft system.

[13] This type of situation was also found in anecdotal cases in other FUs visited, usually because of changes in responsibilities related to acting positions.

[14] Estimated from data uploaded in late 2014 as 2015 data had not yet been uploaded.

[15] Sales and attendance data from 2014/04/01 to 2014/11/30 collected in stand-alone terminals: $3,476,999 (5.6% of total Agency sales), corresponding to 212,297 visitor entries (4.9% of total entries).

[16] Namely, use of the “accounts receivable” payment method, inventory management (merchandise and Discovery Passes) and use of scanners.

[17] “Accounts receivable” function discussed in Section 9.3.

[18] Training on use of friendly and formal warnings aimed at influencing a variety of visitor behaviours is part of the Agency’s overall quality visitor experience training program.

[19] Stores in Western Canada barely use the “accounts receivable” payment method; they use alternative mechanisms instead. Audit findings related to these mechanisms will be communicated to the CFOD through a management letter.

[20] The field should also include a validation control against the client number list.

[21] By mutual agreement among the various parties involved, the limit was set at an amount that is not in line with the Directive on Revenue Comptrollership for User Fees.

[22] This observation is addressed by Recommendation #1.

[23] As indicated in the section concerning information on visitors, the POS includes a control over the number of people who visit facilities.

[24] 321 reimbursement transactions and 185 rebate transactions were chosen.

[25] A signature from a supervisor, co-worker or any other observer was also accepted.

[26] Two field units reported that they carried out these follow-ups. Only one was able to clearly demonstrate this with supporting documents.

[27] In the Canada Gazette.

[28] The Agency also does not comply with section 6.1.7 of the Directive on Receipt, Deposit and Recording of Money, according to which supporting documents are to be available to trace any transaction from its inception to the final outcome.

[29] Account Verification Framework (draft) and monthly follow-up of pending revenue accounts.

[30] Process whereby deposits associated with many batches (day/week/month) are grouped in the purchase orders.

[31] Of the $128M Agency’s total revenues.

[32] For the POS, PCRS and other systems.

[33] Given the number of annual and season passes sold during the audit period and the collection rate when transactions were observed, the number of visitors was underestimated by 877,872 people, taking into consideration the hypothesis that they stayed for only one day. Taking into account the proportion of clerks who indicated they recorded these entries, we arrive at an underestimation of 627,052 visits.

[34] All the terminals examined had a shortcut key to record these entries.

[35] Most of the field units have traffic counters to manage this. However, the counters do not provide information that is as accurate as what can be entered into the POS, especially the duration of the stay, the composition of groups and where they come from.

[36] This practice was also noted in a third field unit, but it was not generalized; the internal audit team was unable to find probative evidence demonstrating that this practice stemmed from a directive.

[37] We were unable to demonstrate which analysis required this number.

[38] Compliance rate of 60% for Eastern stores and 7.3% for Western stores. Overall Agency compliance of 24%. Data from April 1 to September 30, 2015.

[39] The internal audit team acknowledges that at some facilities, and at certain times of the year, it may not be desirable to collect information for each transaction. These scenarios are exceptions, however.

[40] The national POS team is unable to examine this information by store in the system.

[41] Analysis carried out by the PCA Social Science group.

Related links

Date modified :