Evaluation of the CFIA's Meat Programs
Final Report

Executive Summary

The Meat Programs are a long-standing component of the Canadian Food Inspection Agency's (CFIA) food safety programming. They are in place to protect the safety of domestic, imported and exported meat and meat products.

The Meat Programs use a considerable portion of the CFIA's annual resources. From fiscal year 2010-11 to 2014-15, they accounted for between 30 and 36 per cent of the Agency's overall expenditures. In 2014-15, the budget for the CFIA Meat Programs was $301 million.

The evaluation focuses on the CFIA Meat Programs between 2009 and 2014, with some data from 2014-15 included. It addresses 16 questions related to the relevance and performance of the four components of the programs:

• domestically produced and distributed products
• imported products
• exported products
• recalls

In addition to the main elements of the Meat Programs, the evaluation examines the three activities for which the CFIA received additional funding:

• Daily Shift Inspection Presence
• Inspection Verification Office
• Listeria

Key Findings and Recommendations

Key Findings

Overall, the evaluation found the Meat Programs are ensuring the safety of meat products to a very high degree–both products distributed in Canada and exported products.

All components of the program were found to be effective. Specifically:

• The requirements and associated procedures for plant operations, in particular the requirement for the development, implementation and maintenance of Hazard Analysis Critical Control Points (HACCP) plans;
• the system for slaughter inspections (i.e. inspections that take place up to the start of the carcass chilling process);
• the system for processing inspections (i.e. inspections that take place after the carcass chilling process);
• the system for the sampling and testing of domestically produced meat products;
• the system for ensuring the safety of imported products, which involves (i) assessments that the meat safety systems of countries from which meat products originate or were processed and the relevant establishments in those countries are equivalent to Canada's and (ii) import inspections;
• the system to ensure the safety and compliance of exported products; and
• the food safety recall process.

The evaluation also examined the rationale for the Meat Programs and found that:

• the Meat Programs address a legitimate need, and it is important that the Meat Programs be delivered by the federal government;
• the Meat Programs are consistent with federal government and CFIA priorities;
• the roles of CFIA, other federal partners, industry, and the provinces in delivering the Meat Programs are clearly laid out, accepted, and appropriate; and
• the delivery process for the Meat Programs as currently designed was found to be effective, though with some areas of inefficiency.

Possible improvements were identified in the delivery of Compliance Verification System (CVS) inspection tasks, performance measurement, and training materials for slaughter inspectors. In particular, it was determined that a more risk-based approach to inspections would provide an opportunity to better target inspection resources to areas of highest risk.

Recommendations

The evaluation recommends the following changes to improve program effectiveness and increase program efficiency:

1. For those HACCP plans marked by inspectors as unacceptable, the CFIA should clarify which ones represent serious issues.
2. To improve meat program efficiency, there should be less emphasis on carcass by carcass inspections in slaughter inspections done by the CFIA.
3. In import inspections, organoleptic (sensory) inspections should be reduced or eliminated.
4. The CFIA should coordinate its performance measurement reporting for its meat programs to provide comprehensive results reporting.
5. The CFIA should increase the degree to which CVS tasks are based on risk, including the historical compliance record.
6. The CFIA should examine opportunities to provide increased flexibility in the delivery of Daily Shift Inspection Presence requirements.

1.0 Introduction

1.1 Background to the evaluation

The evaluation of the CFIA Meat Programs was identified in the CFIA's 2013-14 Evaluation Plan. The evaluation was conducted in accordance with the Treasury Board Policy on Evaluation (2009) and its supporting Directive and Standard^Footnote 1. The evaluation's main objective was to address the research questions outlined by Treasury Board related to relevance, effectiveness and economy and efficiency^Footnote 2.

In addition, because the CFIA was in the midst of a major change initiative at the time of this study (called "Agency Transformation"), the evaluation attempted to identify lessons learned from the recent operations of the Meat Programs that might be useful in informing the change initiative.

The evaluation was carried out between January 2014 and autumn 2016. It focuses on the CFIA Meat Programs between 2009 and 2014, including some data from 2014-15. It addresses 16 questions (see Annex J) related to the relevance and performance of the four components of the programs:

• domestically produced and distributed products
• imported products
• exported products
• recalls

The assessment of the export component relied heavily on the assessment of the domestic component, since CFIA activities to ensure meat safety are the same in both cases (except for a few additional requirements for export to some countries).

The assessment of the recall process relied heavily on three recent reports:

• Report of the Independent Investigator into the 2008 Listeriosis Outbreak, Sheila Weatherill, July, 2009 (Weatherill Report);^Footnote 3
• Independent Review of XL Foods Inc. Beef Recall 2012, May, 2013 (XL Report), and:
• 2013 Fall Report of the Auditor General of Canada, Chapter 4 – Canada's Food Recall System, November, 2013

In addition to the main elements of the Meat Programs, the evaluation examines the three activities for which the CFIA received additional funding: Inspection Verification Office, Listeria and Daily Shift Inspection Presence.

2.0 Program Profile

Program Profile^Footnote 4

2.1 Meat Programs

The following is a high-level overview of the Meat Programs. Due to the technical complexities of the Meat Programs, a detailed description of activities has been prepared separate from the evaluation report, and can be found in Annex A.

The Meat Programs are a long-standing component of the CFIA's food safety programming. They are in place to protect the safety of domestic, imported and exported meat and meat products.

The Meat Programs use a considerable portion of the Agency's annual resources. From fiscal year 2010-11 to 2014-15, they accounted for between 30 and 36 per cent of the Agency's overall expenditures. During the same period, inspection and surveillance accounted for between 74 and 80 per cent of total Meat Programs expenditures. In 2014-15, the budget for the CFIA Meat Programs was $301 million. A table of expenditure data is included in Annex G.

Pursuant to section 11 of the Canadian Food Inspection Agency Act, the CFIA is responsible for the administration and enforcement of various statutes including the Meat Inspection Act. As part of those responsibilities, the CFIA verifies that federally registered establishments,^Footnote 5 the operators of those establishments and the meat products prepared in those establishments comply with the requirements of the Meat Inspection Act and Meat Inspection Regulations. The CFIA also verifies that meat imported into Canada originates from countries and establishments whose meat inspection systems were approved and that the meat products meet Canadian standards.

The Meat Inspection Act and Meat Inspection Regulations address all areas of the Agency's Meat Programs, including imports, exports, slaughter, registration of establishments, licencing of operators and animal and meat inspection.

Activities carried out within the Meat Programs include:

• Registering of establishments and licensing of operators
• Inspecting registered establishments
• Carrying out inspections and testing related to slaughter and processing
• Reviewing the meat safety systems of countries exporting meat products to Canada
• Inspecting imported meat products
• Certifying meat products for export
• Following the protocol for "high visibility issues"^Footnote 6 when required, including a range of activities related to incident management and recalls.

The majority of Meat Programs activities are delivered by the CFIA's regional offices, which are comprised primarily of Operations Branch staff. A number of other branches within the CFIA, including Science, Policy and Programs, and Communications and Public Affairs, also have responsibilities related to the Meat Programs.

2.2 Inspection Verification Office

In January 2013, the CFIA established the Inspection Verification Office (IVO) as a pilot initiative to provide assurance on whether inspectors are doing the right things and whether they are doing them correctly. The IVO is responsible for verifying meat inspection activities at a number of chosen establishments through spot checks conducted by the Inspection Verification Teams (IVTs) with very short notice.

2.3 Listeria

Following the Maple Leaf listeria outbreak in 2008, and the 2009 publication of the Weatherill Report, the CFIA started to enhance its listeria policies and inspection activities. The Agency's funding was enhanced starting in 2009 to increase its focus on listeria by hiring more inspectors, enhancing training, improving information technology for inspectors, updating policies, directives and manuals, improving test methods, conducting more tests and enhancing its public communications.

3.0 Study Methodologies

The study methodologies are described below. Conclusions were drawn from multiple lines of evidence, where possible. Limitations and mitigation strategies are found in Annex B.

3.1 Document and data review

More than 100 documents and data sources were reviewed during the course of the evaluation. Documents reviewed included:

• Acts, regulations and guidelines governing the Meat Programs
• Past evaluations and audits
• CFIA and federal government priorities related to meat safety
• Roles and responsibilities of CFIA and other government departments and agencies, provincial governments, and industry
• Descriptions of CVS tasks
• Past surveys of stakeholder groups and submissions of stakeholder groups
• Reports of review panels of three recall investigations
• Those related to the organization and operation of the Meat Programs
• Numerous Canadian and international documents dealing with program operational and performance issues (e.g., magazine and newsletter articles)

The evaluation also examined various data related to sampling and testing, recalls, corrective actions requests (CARs), HACCP reviews, delivery of CVS tasks, import inspections, export point-of-entry violations, and program performance measurement.

3.2 Interviews

Sixty-seven interviews were conducted, involving a total of 96 people. These numbers do not include follow-up interviews to obtain clarifications or additional information, or interviews conducted during the planning phase of the evaluation. The 96 interviewees included:

• 49 CFIA employees
• 18 officials from other government departments and agencies
• 18 industry officials
• seven provincial government officials
• four academia

This was a "purposive" sample of interviewees. In other words, interviewees were chosen based on their knowledge of the specific evaluation questions.^Footnote 7 The Advisory Committee for the evaluation also provided a number of suggestions regarding potential interviewees.

In reporting the results of the interviews we have used the following terminology:

• "a few" = one per cent (if >= three respondents) through 25 per cent of respondent groups
• "some" = 26 per cent through 50 per cent of respondent groups
• "most" = 51 per cent through 75 per cent of respondent groups
• "a large majority" = 76 per cent through 100 per cent of respondent groups

Interview guides are contained in Annex C.

3.3 Focus groups

Three half-day focus groups were held in Guelph (11 participants, including some from the Toronto regional office), Abbotsford (nine participants), and Montreal (nine participants). Participants included a mix of Inspection Managers, Supervisors, and Supervising Veterinarians, whose selection was mainly determined by the availability of people to participate.

The goal was to get participants' opinions about the effectiveness of the delivery process for the CFIA's meat inspection system and possible improvements. The main topics discussed were:

• The availability of resources and adequacy of competencies for carrying out required meat inspection tasks;
• The adequacy of procedures to ensure that products and operations comply with Canadian requirements; and
• The governance of the meat inspection process.

Discussions were wide ranging, and there was no attempt to get consensus.

These visits also included three tours of slaughter and processing plants (two in Toronto and one in Abbotsford) and supplementary interviews of CFIA Area officials.

3.4 Survey of inspectors

The evaluation team administered a web-based survey of meat and poultry inspectors across the CFIA's four Areas. The goal was to gain their perspectives as front-line employees of the Meat Programs. Invitees included:

• meat processing inspectors
• poultry processing inspectors
• meat slaughter inspectors
• poultry slaughter inspectors

Specifically, 1,086 individuals were invited to participate in the survey between June 15, 2015 and July 13, 2015.^Footnote 8 A total of 610 people completed the survey, representing a 57 per cent response rate. This was a relatively high rate for e-mail surveys of organization employees, according to KPMG^Footnote 9's experience, where similar surveys have previously been under 40 per cent.^Footnote 10

A copy of the survey questionnaire can be found in Annex C.

3.5 Expert panel

The evaluation team convened a panel of seven experts from Canadian, U.S. and Australian sources to provide informed opinions about the safety of meat products distributed in Canada. The members included:

• One academic who is an ex-US Food Safety Inspection Service official (the Chair)
• Two industry representatives
• Two academic food safety experts
• One representative of the Australian Ministry of Agriculture
• One CFIA official^Footnote 11

The individuals who served on the panel are listed in Annex D.

These individuals are recognized as credible experts in the field of meat inspection and, in particular, are knowledgeable about:

• procedures used for inspections and approvals of plant operations
• procedures used for product inspections, sampling, and testing
• procedures used to ensure the safety of meat products imported into Canada (equivalency determination and import inspections)
• scientific criteria for ensuring the safety of meat products

Three conference calls were held to discuss the various research questions. Prior to each call, panel members received background information about the discussion questions. A member of the evaluation team served as the secretary. The draft notes of each panel meeting were circulated to the Chair for review following the meeting, and the draft summary notes on the three conference calls were circulated to all the panel members for their review.

3.6 International comparison study

The evaluation team collected information on the meat safety programs of five countries–the United States (US), Australia, New Zealand, Denmark and Finland–as well as the European Union (EU). The evaluation team reviewed documents and information available on the web sites of relevant government departments and agencies, and consulted officials from each of the comparison countries.

3.7 Financial analysis

For this component of the evaluation, the team reviewed available data and interviewed senior CFIA officials regarding the costs and revenues associated with the Meat Programs. The purpose was to determine if there are less costly ways for CFIA to achieve the intended results of the Meat Programs.

4.0 Program Relevance

4.1 Q 1: The need for the Meat Programs

The evaluation found there is a demonstrated need for the Meat Programs. In Canada, the Food and Drugs Act is the foundation of Canada's food safety system. Jointly administered by Health Canada and the CFIA, this Act requires that all food sold in the country be fit for human consumption. While the federal government in Canada has the primary role for food safety oversight, in practice, meat products produced and sold within provincial boundaries are generally a provincial responsibility.

Internationally, the Codex Alimentarius Commission (Codex) provides guidance for national food safety standards. The Food and Agriculture Organization of the United Nations and the World Health Organization created Codex in 1963 to develop harmonized international food standards. Adherence to Codex standards and related texts is intended for use by national governments.^Footnote 12

One of the guiding principles of the CFIA's Food Program Framework is that it be consistent with international obligations and standard setting bodies (e.g., Codex).^Footnote 13 The Framework provides the foundations upon which the CFIA's food program is built and outlines the authorities and obligations of the CFIA and of regulated parties.

This approach is similar in virtually all other developed countries. The international comparison study found that oversight of food safety is the responsibility of government in all five of the countries reviewed. The federal government typically has the overall responsibility; however, in some cases, state/provincial governments also have a role, most notably in Australia.

• Australia: The national government is responsible for ensuring the production of safe meat for export, but the state governments are responsible for ensuring the safety of domestically distributed products.^Footnote 14
• US: The national government has overall responsibility, but the state governments have responsibility in cases of intrastate commerce and cases in which the state participates in the Cooperative Interstate Shipment program.
• New Zealand, Denmark, and Finland: The national governments have full oversight responsibility.

The evaluation also included reviews of meat safety oversight responsibilities in six other countries that were not part of the international comparison study (United Kingdom, France, Italy, Russia, Sweden and Germany). Industry oversight is the responsibility of government in all the countries reviewed, and in all but one case (Germany), it is the responsibility of the national government.

The Food and Drugs Act (1985) and Food and Drug Regulations establish requirements and standards for the safety and nutritional quality of foods sold in Canada.

The Meat Inspection Act (1985) covers all components of the Meat Programs: the import and export of meat products and interprovincial trade in meat products, the registration of establishments, the licencing of operators in those establishments, the inspection of animals and meat products in registered establishments, as well as the standards for those establishments and for animals slaughtered and meat products prepared in those establishments.

The Meat Inspection Regulations (1990) requires that every operator that prepares meat products develop, implement and maintain the required prerequisite programs, HACCP plans and other control programs as set out in the Food Safety Enhancement Program Manual and the Manual of Procedures.

The CFIA Act (1997) sets out the responsibility of the Agency to administer and enforce, among other legislation, the Meat Inspection Act and Regulations.

The Consumer Packaging and Labelling Act (1985) establishes rules related to packaging and labelling requirements for products permitted to be sold in Canada.

The Safe Food for Canadians Act (2012) modernizes and consolidates the Fish Inspection Act, the Canada Agricultural Products Act, the Meat Inspection Act, and the food provisions of the Consumer Packaging and Labelling Act. The Regulations were not finalized at the time this report was written. The Safe Food for Canadians Act:^Footnote 15

• makes food as safe as possible for Canadian families
• protects consumers by targeting unsafe practices
• implements tougher penalties for activities that put health and safety at risk
• provides better control over imports
• institutes a more consistent inspection regime across all food commodities^Footnote 16
• strengthens food traceability

Canadians are generally confident in Canada's food safety system, with only a minority expressing concern. According to a report by Ipsos Reid, Food Safety: Canadians' Awareness, Attitudes and Behaviours (2012-13), Canadians are moderately to extremely confident in Canada's food safety system (90 per cent). Forty-one per cent of Canadians say recalls make them feel more confident in the system working. The caveat is that of those surveyed only 18 per cent are familiar with the food inspection system. Canadians have more confidence in Canadian organizations (65 per cent) to contain the spread of contaminated food than systems outside of Canada (19 per cent).

A large majority of interviewees indicated Canadian consumers are generally satisfied with the safety of meat products. They noted that although foodborne illness outbreaks and recalls can temporarily reduce satisfaction and confidence in the system, it bounces back.

There was also a common sentiment on two other fronts. First, a large majority of interviewees acknowledged Canadians tend to trust the federal government in a meat safety oversight role more than industry. Interviewees were not asked about the provincial government's role. A few indicated that Canadians tend to believe industry won't always take the appropriate actions, as they are profit-driven. As such, the federal government's oversight role is seen to be critical. Second, some interviewees acknowledged Canadians probably don't truly understand the meat inspection system, which stakeholders are involved, or the ways in which they are involved (as documented by Ipsos Reid). Ultimately, they are only concerned the meat they consume is safe.

This finding is discussed in more detail in sections 5.4 and 5.5, but highlights are summarized below. This relates to the two components of the imported products program (See Annex A for more details):

• Equivalency determinations
• The import inspection program

Equivalency determinations

The international comparison study found, in the case of Australia and New Zealand, the only site visits involved in equivalency determinations are bovine spongiform encephalopathy (BSE) food safety assessments. Otherwise, industry is responsible for determining equivalency. In Europe, the EU is responsible for equivalency determinations (not member countries). EU equivalency assessments are a desk exercise and do not involve site visits.

The consensus opinion of the expert panel was there can be more reliance on the Foreign Competent Authority in the exporting country where there is a sufficient level of trust in that country's meat safety system. The panel viewed this as a more risk-based approach, thereby directing more on-site investigations to countries with poorer food safety records.

Import inspections

In Australia, import inspections mainly involve visual inspections and "label checking" to verify the exporting country's inspection and certification system and the exporting establishment have been approved. Only foods classified as "risk foods" are inspected and tested.

New Zealand has no import inspection system, although there is a minimal monitoring program. As one New Zealand official said: "Why would we have import inspections of products from countries whose systems we trust?"

4.2 Q 2: Consistency of the program with federal government priorities and CFIA's mandate and priorities

Food safety and support for trade in food products are top priorities for the federal government, and are reflected in a number of budget commitments.

In the CFIA 2015-16 Report and Plans and Priorities, the Minister of Health stated: "I am pleased to report that food safety continues to be a priority of the Government of Canada…The government will continue to strive to ensure the CFIA remains a world-class regulator in the food safety, animal health, and plant resources protection sectors."

The 2014 federal budget committed:

• $390 million over four years for food safety, and the Budget Plan noted that: "Since 2006 the government has taken significant action to support Canadian consumers…by improving product and food safety"
• $154 million over five years to strengthen CFIA's food safety program: "Funding will support the hiring of more than 200 new inspectors and other staff, the development of programs to minimize food safety risks, and an enhanced capacity to prevent unsafe food imports from entering Canada, offering better protection for consumers."
• $31 million to establish a Food Safety Information Network to enable "food safety data to be compiled, analyzed, and shared in real time, allowing for more rapid detection of, and response to, food safety hazards."

In 2011, the government allocated $100 million over five years to strengthen food inspection. This funding was invested in inspector training, additional science capacity, and electronic tools to support the work of front-line inspectors.

In 2009-10 the CFIA began receiving funds to enhance its listeria inspections which supported Heath Canada's new listeria policy, including the addition of 70 inspectors, plus another 20 starting in 2011 for non-meat listeria inspections. Inspection activities related to listeria are discussed below in section 5.2.

Government statements accompanying the introduction of the Safe Food for Canadians Act also highlighted the government's priority for food safety.

A number of the documents discussed above also note the importance of food safety for trade.

For example, the 2014 federal budget states: "The volume of global trade in agricultural and agri-food products has increased significantly over the last decade and is projected to continue to grow. To respond to this changing environment…the government is making a significant investment to continue strengthening our food safety system."

Other federal government measures to strengthen trade in food products include:

• the establishment of the Canada-US Regulatory Cooperation Council^Footnote 17
• the allocation of more than $77 million over the years 2010-11 through 2015-16 for the adoption of daily shift inspection presence (to support the export of Canadian meat products to the United States)

The Meat Programs clearly align with the CFIA's strategic outcome of "a safe and accessible food supply and plant and animal resource base." The CFIA's 2015-16 Report on Plans and Priorities states:

"Mitigating risks to food safety is the CFIA's highest priority. Safeguarding the health and well-being of Canada's people, environment, and economy is the driving force behind the design and development of the CFIA's programs. The CFIA, in collaboration and partnership with industry, consumers, universities, and federal, provincial and municipal organizations, continues to work towards protecting Canadians from preventable health risks related to food and zoonotic diseases.

The CFIA supports Canadian agriculture and the ability of agri-food businesses to enter domestic and global markets and compete successfully therein. To support this objective, the CFIA develops and enforces regulatory and program frameworks for imports and exports that meet both Canadian and international requirements."

The CFIA's priority for food safety is echoed in many Agency statements, for example "First and foremost, food safety is the top priority for CFIA."^Footnote 18 Not surprisingly, this priority is reflected in the Agency's budget. About a third of the Agency's program funding budget is dedicated to the Meat Programs.

A large majority of interviewees agreed the objectives of the Meat Programs are aligned to both the priorities of CFIA and the federal government as a whole. As one interviewee said: "Ultimately, the Meat Programs are in place for two purposes: to ensure meat is safe for human consumption and to ensure Canada's meat industry is complying with foreign government import requirements, as a means of enhancing Canada's economic well-being."

4.3 Q 3: Appropriate roles and responsibilities of CFIA, other federal partners, industry, and the provinces

The responsibilities of the various parties are as follows:

CFIA: The CFIA is responsible for ensuring the meat slaughter and processing industry adheres to all federal food safety legislation. The legislation outlining CFIA's responsibilities is described in section 4.1.

Health Canada: Health Canada is responsible for establishing policies and standards for the safety and nutritional quality of all foods sold in Canada. The department exercises this mandate under the authority of the Food and Drugs Act and the Department of Health Act. The department's regulatory mandate is outlined in the Food and Drug Regulations. Health Canada is also responsible for conducting health risk assessments at the request of the CFIA. See Annex A section V for further detail.

Public Health Agency of Canada (PHAC): PHAC assists the Minister of Health in carrying out the Minister's powers, duties, and functions in relation to public health. It is specifically responsible for coordinating national enteric (gastrointestinal) disease surveillance activities and for taking the lead role during multi-jurisdictional foodborne illness outbreaks. The main Acts outlining these responsibilities are the Public Health Agency of Canada Act, the Human Pathogens and Toxins Act, and the Quarantine Act. See Annex A section V for further detail.

Provinces: Constitutionally, the provinces and territories have primary jurisdiction for foods manufactured and sold within their province/territory. However, some provisions of federal legislation such as the Food and Drugs Act and its Regulations and the Safe Food for Canadians Act and its Regulations^Footnote 19 relating to labelling and food safety may also apply.^Footnote 20 As such, the CFIA and the provinces and territories share jurisdiction for the oversight of intra-provincial manufacturers in such cases. Hence, the CFIA consults with its provincial and territorial counterparts as part of the design and development of risk management strategies for manufacturers who trade solely within a province.

Industry: Industry is responsible for producing safe food, by having processes in place to identify and effectively manage identified hazards and marketing food products in a truthful manner.^Footnote 21

CFIA: Inspector awareness of their roles and responsibilities is continuously assessed by the Agency's Inspection Verification Office, as discussed under Finding 5.3. These risk-based verifications serve as an ongoing reminder to inspectors and management of their specific inspection task responsibilities. Concerns raised by inspectors during this evaluation about inspection tasks were largely limited to the opportunities to improve their effectiveness.

Federal partners: As noted above, CFIA's main federal partners in delivering the Meat Programs are Health Canada and PHAC. Other federal partners include Agriculture and Agri-Food Canada, Global Affairs Canada (the former Department of Foreign Affairs and Trade Development), and the Canada Border Services Agency. All interviewees agreed there are generally good relations among all these parties and all parties fully accept their responsibilities related to the Meat Programs.

Provinces: Representatives from three provinces were interviewed and they all had clear understanding and acceptance of provincial roles and responsibilities. All of the eight interviewees who addressed relations with the provinces agreed relations are "fairly good" overall, and are particularly good with certain provinces (Quebec and Ontario were identified in this regard). However, six of these interviewees noted there are some challenges in provincial relationships, particularly with regard to what they view as adequate collaboration.

Industry: All 12 respondents who addressed relations with industry said these relations are good, in general. Eleven of the 12 respondents agreed industry has a high degree of acceptance of their responsibilities.

Five of the 18 industry interviewees responded to the question: should the role of industry in ensuring safe-to-consume meat products be increased or decreased? Most didn't explicitly answer the question, but all five made comments supportive of industry doing more meat safety and quality control activities and CFIA carrying out an oversight role. Examples of comments included:

• "CFIA's predecessor was the police of industry, because industry couldn't police itself… You have bigger players now… There has been a shift in industry – no one is more interested in the safety of products than industry. They don't need to rely on CFIA to tell them what is right… Ultimately it's their brand that is out there."
• "The Canadian model where there is a high degree of cooperation gets better results than if you take an adversarial type of approach…The best example is HACCP in this regard…. Introduced voluntarily, then made mandatory because industry believed it was a good practice."

Five CFIA respondents addressed the role of industry, and all were supportive of industry taking on increasing responsibility. For example:

• "The current meat safety system puts too much onus on CFIA, when, actually, meat safety is the responsibility of the producers, with CFIA having an oversight responsibility. There could be a shift to more company responsibility without compromising food safety."

Until 2013, when it was disbanded, the Label Recipe Registration Unit was responsible for reviewing submissions for meat product labels, reviewing more than 15,000 in its last full calendar year of operation, in 2012. The function was removed from the Agency's role as part of the Government of Canada's Red Tape Reduction initiative. The intent was to "create a level playing field since pre-market registration is not required for other sectors of the food industry."^Footnote 22 Some industry representatives raised concern that the dissolution of the unit provides more opportunity for unscrupulous companies to use false and misleading labels. Guidelines for label requirements are posted on the CFIA's website, and there is some inspection conducted at retail establishments. The CFIA continues to work with industry groups to resolve a couple of outstanding labelling issues, including the use of the term "natural" when using curing agents and the use of the Product of Canada label for meat products from imported animals.^Footnote 23

Extensive consultations have been held with the industry regarding the proposed regulations for the Safe Food for Canadians Act. These regulations intend to make it clearer that companies–not the CFIA–are responsible for the safety of their products. There have been no objections from industry to this approach.^Footnote 24

5.0 Program Effectiveness

5.1 Q 4: Effectiveness of requirements for plant operations

Note: The term "satisfactory" means the associated regulatory and policy goals are met. In this case, for example, satisfactory means potential biological, chemical, and food safety hazards are prevented or reduced to an acceptable level (where "acceptable" is defined by Canadian food safety criteria, for example: Finished raw ground beef products should not contain detectable levels of E. coli 0157).

The consensus opinion of the expert panel was: "The requirements for domestic plant operations are satisfactory for providing a reasonable level of assurance that products produced in plants meeting these requirements satisfy Canadian meat safety criteria."

A large majority of the interviewees who commented on this issue indicated, in their opinion, the implementation of HACCP plans is effective for ensuring food safety. A representative sample of these comments include the following:

• "HACCP is very effective. This process gives industry responsibility for meat safety. CFIA as regulator verifies compliance. As we move away from hands-on inspection, there is more emphasis on things like HACCP." [CFIA]
• "What CFIA does is to review the HACCP plan on an ongoing basis to ensure companies are in compliance and that the plan is robust. …It is working well." [Industry]
• "The process is efficient and CFIA is doing a good job of reviewing HACCP plans." [Academic]
• "Since HACCP has been in place, the microbiological quality of meat has been improved quite a bit." [AAFC]

The Modernized Poultry Inspection Program was initiated by the CFIA in the early 2000s. It was the first Meat Programs area to:

• shift from prescriptive regulatory requirements to objective performance standards;
• transition from hands-on inspection to audit-based verification activities; and
• have industry assume responsibility for detecting and handling defective carcasses under continuous government oversight.^Footnote 25

Other Meat Programs areas are currently exploring options for modernizing their inspection regimes in a similar fashion to be more scientific and risk based, though both pork and beef slaughter inspection remains hands-on. The term "hands-on" refers to carcass-by-carcass inspection requiring manipulation, such as incision or palpation. The CFIA Meat Hygiene Manual of Procedures still refers to specific numbers of inspectors required based on slaughter plant line speeds, though this was under review at the time of this report's writing. Options being explored for improvements to slaughter inspections are part of the Agency's Modernized Slaughter Inspection Program, which is discussed in Finding 5.2 and 15.1.

Some interviewees commented on these changes:

• "In meat processing, HACCP is even more critical than it is in slaughter, but CFIA is still focused on traditional inspection tools."
• "There should be more effort up front on looking at company systems, particularly HACCP…and less effort on product inspections at the end."

Additional supporting evidence for this finding is indirect. As discussed in Annex A, the meat safety system for domestically produced products consists of two parts – Part 1: requirements for plant operations (the subject of this section), and Part 2: requirements for product inspections, sampling, and testing (addressed in section 5.2). In the remainder of the report, we provide considerable evidence that the Canadian meat safety system overall is performing very well and that this is not due to catching a lot of unsafe meat in Part 2; so it would be impossible for the system overall to be performing well if Part 1 were not functioning well.

Therefore, any evidence that the overall system is performing well supports finding 4.1. Examples include:

• More than 95 per cent of CFIA's microbiological testing results for meat products are satisfactory.
• The recent ranking of food safety performance by 17 Organisation for Economic Co-operation and Development (OECD) countries by the Conference Board of Canada ranked Canada's performance first.^Footnote 26
• Canadian exports of meat products are relatively unrestricted by foreign governments to which we export, including the US, which has a very demanding meat safety system. There are a very small number of port of entry violations (discussed in section 5.7).

The evaluation team reviewed 758 records of inspection results for company implementation of HACCP plans that CFIA inspectors had rated unacceptable. This represented 43 per cent of all the plans inspectors reviewed over the study period (2009-14). We found three quarters (578) were clearly minor issues, which we categorized as simple file verification issues. Examples include:

• "The dates when changes were implemented and reassessed were not included."
• "There are missing minutes of HACCP plan review meetings."
• "There is no signature and date of the HACCP Coordinator verifying that changes have been implemented effectively."

The one quarter (180) had issues more significant than just administrative errors, which included:

• "The only hazard considered is chemical. The CFIA Hazard database states these products would present biological, physical and chemical hazards."
• "For Example, potential cross-contamination hazard of Allergen (soy, wheat) from burger production is not indicated in Processing Room."

The Agency currently assesses implementation plans as either acceptable or not. The variation in the types of unacceptable issues, ranging from minor administrative tasks to more serious omissions, is not categorized by the Agency to identify which are of significant concern.

US: All operators must implement approved HACCP systems, which are verified by inspectors on an on-going basis.

Australia: Australia is a partial counterexample. All meat processing plants have to implement HACCP plans, but this is not required for slaughter plants.

New Zealand: Requires HACCP plans for all meat plants (both processing and slaughter), and these are periodically verified by the government.

EU: All plants in EU countries are required to implement approved HACCP systems. The EU also carries out some HACCP system verification activities.

Denmark and Finland: In accordance with EU requirements, all plants are required to implement HACCP systems, and these are periodically verified by government auditors.

The Food Safety Enhancement Program (FSEP) manual (section 3.3) specifically requires company validations of critical control points. The CFIA's FSEP, based on the principles of the HACCP system, specifies the requirements for an effective HACCP system and outlines the process for HACCP recognition by the CFIA.^Footnote 28

Although the expert panel did not review the FSEP manual requirement or its application in detail, there was a consensus opinion that this is a very important area: "The panel recognized the importance of CFIA's efforts to ensure that companies validate the reliability of the controls for critical control points when implementing HACCP plans." It was noted, in some countries, a big flaw in the implementation of HACCP plans is the weak (or no) validation of critical control points.

Five interviewees volunteered comments regarding the importance of validations, even though interviewees were just asked to comment on CFIA's activities related to reviews of HACCP systems and not about company validations.

In the international comparison study, information was available on company validation requirements for two countries, New Zealand and Finland. Both of these specifically required company validations of each Critical Control Point measure.

5.2 Q 5: Effectiveness of the systems for inspections, sampling, and testing

The term "slaughter inspections" includes both ante-mortem and post-mortem inspections up to the carcass chilling process. See Annex A for a description of these activities.

The expert panel spent a considerable amount of time discussing the Meat Programs processes for slaughter inspections and concluded these processes are satisfactory. As noted in Finding 4.1, the term "satisfactory" means that the associated regulatory and policy goals are met. In addition to this consensus opinion, the panel was of the opinion that the system for slaughter inspections can be made more effective and efficient. This is discussed further under finding 5.2 below, as is the direction the CFIA is taking to address these opportunities for improvement, including those related to the Modernized Slaughter Inspection Program.

In the survey of inspectors, approximately two-thirds of slaughter inspectors were of the opinion current slaughter inspection procedures are effective for ensuring food safety. There was some indication from the survey that inspectors with five years or less experience were more likely to have this opinion than those with more than five years of experience.

In addition, more than two-thirds agreed these procedures are effective for ensuring compliance with international export requirements. While the survey did not explore the reasons why about one-third felt that slaughter inspections were ineffective, seven respondents suggested making tasks more risk based. This was also raised by focus group participants, as outlined under Finding 5.3.

In the interviews, most respondents agreed with the following statement: "The activities related to domestic meat inspection, sampling, and testing are effective and efficient in contributing to the safety of meat products."

The expert panel concluded: "There should be less emphasis on carcass by carcass inspections, and slaughter inspections should be based on a systems approach that would involve measurements of the overall effectiveness of the slaughter process. Instead of relying totally on visual and hands-on inspection activities, there should be more effort on monitoring outcomes, including microbiological results. It would be better to have a process for detections of defects at the end of the slaughter process by accredited inspectors."

Individual panel members' comments included the following:

• "The veterinary inspections are adequate, but there should be decreased emphasis on the visual aspects and more on scientific aspects."
• "While some carcass by carcass inspections may be desirable, perhaps this could be done on a statistical sampling basis (as is the case in the Modernized Poultry Inspection Program)."

Reference was made to studies in which modernized techniques for slaughter inspections (use of sensors and computers) have led to better results than when inspectors were used.^Footnote 29 Reference was also made to evidence, both in Australia and the US, showing hands-on inspection has led to cross contamination of carcasses through the physical inspection process.^Footnote 30

As noted in Findings 4.1, the Modernized Slaughter Inspection Program is moving the Agency to more scientific and risk-based inspection models, transitioning from hands-on to more audit-based verification activities. To support this, the Agency is updating disposition policies, which provide the direction necessary to support the disposition of carcasses and their parts when abnormalities have been detected on post-mortem inspection.

The outlined criteria differentiate between quality issues (operator's responsibility) and food safety issues (under CFIA control), and have been drafted to facilitate the harmonization of dispositions across the country. The aim is that this policy will be the foundation to launch an associated pilot in 2017-18.

A long-term pilot, related to an independent risk assessment on the incision of mandibular lymph nodes in hogs, will entertain the move toward visual inspection of mandibular lymph nodes in hogs when upstream information demonstrates that food safety risks are sufficiently controlled on-farm.

The Modernized Slaughter Inspection Program is also looking at similar opportunities in beef production (e.g., research into the removal of cheek incisions to check for cysticercus bovis). Beyond the need to demonstrate the scientific basis of these changes, the Agency is concurrently working with several key trading partners to communicate these intentions. At a minimum, the program is intended to ensure there will be the same level of food safety control as currently in place, and that it remains aligned with Codex.

In the survey of inspectors, two-thirds of respondents felt current slaughter inspection procedures are effective for ensuring food safety; however, 30 per cent disagreed with this. Thirty-one percent also felt these procedures are not efficient for ensuring food safety. Therefore, while the survey results support the finding that the procedures are satisfactory (finding 5.1), they also support the finding that there is opportunity for improvement in both effectiveness and efficiency.

By "processing inspections" in this sub-section we mean CFIA inspections to verify that company requirements during product processing are fully and correctly carried out. Examples include: verifying the sanitary condition of equipment, verifying the company's monitoring and validation procedures for cooked RTE meat products; and verifying the company's examination program for boneless meat products. Sometimes these activities are called "CVS inspections," although CVS also covers aspects of slaughter inspections.

Evidence that the system is satisfactory

The consensus opinion of the expert panel was that "the system for processing inspections is satisfactory."

This opinion was reinforced in the survey of inspectors, in which 71 per cent of processing inspectors agreed CVS is an effective measure for ensuring food safety. Sixty-three per cent were of the opinion that it is also effective for ensuring compliance with international export requirements.

As noted above, most interviewees agreed with the following statement: "The activities related to domestic meat inspection, sampling, and testing are effective and efficient in contributing to the safety of meat products." Examples of comments made by senior CFIA officials in reference to processing inspections included:

• "There will always be a need for the independent verification of what industry does. It is important for CFIA to maintain third party independence and verify food safety through inspection or testing or both."
• "The pathogen reduction initiative…is a three-year study to determine baseline levels of salmonella, listeria, and E. coli 0157. The trends have been positive."

The following table shows delivery of required CVS inspection tasks averaged 88 per cent over the study period.

Finally, the inspection system for meat processing plants was found to be acceptable by foreign country audits and assessments during the study period.

Evidence of challenges in implementing CVS

While 71 per cent of survey respondents felt CVS is an effective system for ensuring food safety, 29 per cent disagreed with this. Thirty per cent also felt the system for processing inspections is not efficient for ensuring food safety.

Therefore, while the survey results support the finding that the system is satisfactory, they also support the finding that there is opportunity for improvement in both effectiveness and efficiency.

Challenges associated with implementing CVS were brought up in each of the three focus groups. Some representative comments included:

• "The frequency of some inspections is too high; this limits flexibility."
• "CVS tasks should be more risk-based, with frequencies determined on the basis of the relative risks of various plants (based on their history and the products they produce)."
• "There is no system to prioritize tasks when time is not available."

One CFIA staff member echoed these comments:

• "Regarding CVS, there are 156 tasks that have to be carried out over the course of the year. This is way too many. The inspectors are not really thinking now, just carrying out tasks on a rote basis, because of the overly prescriptive nature of CVS and the Manual of Procedures."

It is important to note that not all tasks are done at every establishment, many being specific to the establishment type.

Inspection Verification Office

The Inspection Verification Office (IVO) was created following the release of the XL Report, which found, among other things, that the food safety program responsibilities were not consistently met. At the time of the XL recall, the Agency was designing a replacement for its Quality Management System. The resulting IVO addressed recommendation five from the XL Report: "The CFIA must enforce its oversight responsibilities at the plant."

The IVO verifies meat inspection activities at establishments through spot checks conducted by the Inspection Verification Teams (IVTs) with very short notice. The establishments are selected based on a detailed risk analysis, which determines the targeted areas of a selected establishment's preventive control plan that IVTs will examine. These spot checks are designed to determine if inspection staff are adequately ensuring that preventive control plans are being appropriately implemented with the use of the Compliance Verification System.

Upon completion of the 2013 IVO pilot, the CFIA's Audit Directorate was asked to conduct a review, which concluded that the IVO would provide the VP of Operations with enhanced oversight of the Agency's inspection function. In 2014, the IVO was formally developed into a program, with a full complement of 30 Verification Officers in place by the beginning of the 2015-16 fiscal year.

The hiring of the full complement of 30 inspectors was completed later than originally planned which contributed to an overestimation of travel costs, which was the largest cause of the near $3 million Operating and Maintenance budget variance. See Annex G for the IVO Budget and Expenditure 2013-15 Table.

The first-year results of the IVO reveal the need to improve consistency of inspector application of CVS, particularly in the input of verification task codes and activity codes, including the recording of daily presence.

The results of the 2014-15 program implementation year come from the 58 verifications conducted of meat establishments (one per establishment). While the Inspection Verification Reports provide significant detail on the findings, they do not assess their significance or potential impact. This was an issue of concern raised in focus groups, along with the non-representative nature of the inspection results. However, most of the interviewees who were aware of the IVO were supportive of its intent and design. (Details on verification results are included in Annex E.)

All the inspection reports include recommendations. A Management Review and Action Plan (MRAP) was created to address each, and, as of July 2015, out of the 62 MRAPs (includes four fish plant inspections), the Areas had identified 36 as complete, 25 partially complete and one redundant as the establishment had since closed.

This is closely related to the previous finding, which provides evidence regarding the need for efficiency improvements and the inflexibility of CVS.

The consensus opinion of the expert panel was: "The system for processing inspections can be improved by making it more risk-based, with relatively more inspection effort devoted to high risk processes. In other words, the required company inspection tasks and CFIA verifications are sufficient, but probably not all necessary – i.e., there are probably too many tasks, and the specification of inspection tasks that need to be done should be based more on risk."

Interviewees were not specifically asked about this, but a few of them expressed essentially the same opinion:

• "There is acknowledgement throughout the governance structure that the Agency spends too much on meat inspection. It is not proportionate to the level of risk." [CFIA]
• "This [food safety] is not the main reason for the continued heavy role of CFIA in meat product inspections and testing – it's because of export requirements." [CFIA]
• "Meat is over-inspected, and other commodities are under-inspected." [Industry]
• "In CVS there are numerous tasks to be applied by inspectors, only a fraction of which impact on food safety; so it is too complex a system." [Academic]

The CFIA has developed plans to adjust the frequency of some CVS tasks in accordance with risks and to focus more on outcomes. This is discussed under Finding 15.1.

Another component of meat inspections that isn't risk-based is the Daily Shift Inspection Presence (DSIP) requirement. This is discussed in finding 10.1 and 15.1.

As well, the opportunity for a much more risk-based approach to slaughter inspection is discussed in Finding 5.2.

Inspections based more on risk could lead to an adjustment in the number of meat inspectors relative to inspectors for other food commodities. As of March 31, 2015, of 2,696 CFIA field inspection staff working with food, 1,657 worked with meat products. This represents 61 per cent of food inspection staff, though foodborne illness outbreaks from meat represent only 31 per cent of all foodborne illness outbreaks, according to a U.S. study of outbreaks from 2004 - 13.^Footnote 31 An earlier study of outbreaks from 1998-2008 found meat accounting for 22% of outbreaks and more than 29% of deaths.^Footnote 32 See Finding 16.2 for a discussion of relative inspection costs by commodity.

A study by PHAC found meat accounted for 26 per cent of foodborne illness outbreaks, but the study sample was far less statistically significant using only 19 meat related food borne illness outbreaks vs 4,589 in the U.S. study.^Footnote 33

Canadian food safety criteria are developed and continuously updated by Health Canada, based on extensive scientific evidence.

The panel did not have a consensus opinion regarding the adequacy of current testing procedures for microbiological contaminants and chemical residues.

Microbiological Sampling and Testing

The expert panel as a whole did not comment on the amount of industry microbiological sampling and testing, but some opinions were expressed that the amount of industry sampling and testing as currently designed, is not sustainable in the long run. One reason put forward was, because current sampling focuses on pathogens at the end-product stage, the number of organisms that will have to be tested for will continue to grow.^Footnote 34 Another concern was increasing sophistication of testing equipment may lead to an increased demand for microbiological tests.

Both companies and the CFIA conduct sampling and testing. The purpose of the company testing is to confirm that their control programs, as specified in their HACCP plans, are achieving their intended results. CFIA testing is intended to validate the company's sampling and testing. Both testing programs include microbiological testing of both food contact surfaces and products. Companies carry out many times more microbiological tests than the CFIA.

Company requirements for sampling and testing are outlined in detail in CFIA's Meat Hygiene Manual of Procedures, especially Chapter 4 and its annexes. In this study we reviewed the sampling and microbiological testing programs of three companies^Footnote 35 and compared them with sampling and testing performed by the CFIA. The data for one recent 12-month period were as follows:

*For the extra-large beef establishment used in the table above, the CFIA sample frequency is 36 times in a year. The establishment is required to sample every production lot of pre-cursor material (which is used to produce finished raw ground beef products). The number of samples tested is influenced by the establishment's lot definition and the production volume. In the case above, the particular establishment referenced tested 21,197 samples in 2015.

The table below shows the number of CFIA microbiological tests for all meat pathogens per year during the last four years of the study period, plus 2014-15, and the per cent of products that tested satisfactory each year.

Statistically there is a high probability that the actual percentages of products in each year that are satisfactory is very close to these percentages, even though this sampling may not fully meet all the requirements for statistical rigour.^Footnote 36 The satisfactory percentages in the table above do not indicate that the remaining tests were all unsatisfactory. Those not registered as satisfactory also include "investigative", which includes, among other issues, the presence of indicator organisms, which do not cause illness, but are important to track and investigate.^Footnote 37 These averaged 1.7 per cent of the tested samples in the table above.

Some panel members expressed the opinion that there should be increased focus on determining factors that influence end-product testing results (i.e., greater sampling earlier in the production process to determine the presence of causative factors that lead to unsatisfactory testing results at the end product stage). In other words, having a greater focus on the "hot spots" in plants (i.e., places where microbiological contamination is more likely to occur). This relates to a more risk-based approach, raised earlier in this report, and discussed in more detail under Finding 15.1.

The expert panel reviewed the following summaries of microbiological criteria:

• Policy on listeria monocytogenes in ready-to-eat foods
• Guidance document on E. coli-0157 in raw beef
• Other microbiological criteria – relating to deboned poultry product, sausage, etc.

The consensus opinion of the panel was:

• "The listeria criteria are adequate."
• "The E. coli-0157 criteria are adequate."
• "The guidance document outlining other criteria is good."

Most interviewees felt current sampling and testing procedures are adequate. According to our interviews, microbiological sampling for most meat products is conducted at about 90 per cent of the planned targets, though actual figures for 2010-15 average 82 per cent. See Annex F for details.

Chemical Residue Sampling and Testing

The CFIA's National Chemical Residue Monitoring Program conducts tests on meat samples for pesticides, veterinary drugs, metals, elements and other contaminants. Codex guidelines aid in the design and implementation of the Agency's program. These guidelines provide comprehensive descriptions of the approaches used in the development of sampling plans. It is recognized no government agency can routinely monitor for all residue/commodity combinations, and it is not necessary to do so in order to protect consumers.

The CFIA's sampling activities are carried out in accordance with Codex principles and guidelines and maximum residue limits consistent with Codex principles are used for chemical residue monitoring. In addition, Canada's monitoring of domestic food production has been judged to demonstrate the equivalence of Canada's residue control system with that of trading partners, including the United States and European Union. Canadian monitoring activities are also in line with World Trade Organization obligations.

The percent of satisfactory test results for chemical residues testing is generally much higher than that for microbiological. For example, in 2012-13, 99.8 per cent of the 3,695 meat samples tested for pesticide residues were compliant with regulations. For veterinary drugs, 98 per cent of the 10,241 samples of meat products were in compliance.^Footnote 38

U.S. FSIS Approval of CFIA Sampling and Testing

In 2012, the U.S. Food Safety Inspection Service (FSIS) carried out an equivalency audit of Canadian meat slaughter and processing operations. The audit found the Canadian meat safety system to be equivalent to the US system, including microbiological sampling and testing procedures for:

• Livestock carcasses for generic E. coli
• E. coli-0157:H7 in ground beef and veal
• Salmonella in raw products

In 2014, FSIS carried out another equivalency audit of Canadian meat slaughter and processing operations. The Canadian meat safety system was again found to be equivalent to the US system. All areas of concern were addressed to the satisfaction of the US authorities.

The CFIA Pathogen Reduction Initiative

In 2009, the CFIA initiated the Pathogen Reduction Initiative to establish baselines for the prevalence of Salmonella and Campylobacter in broiler chicken; E. coli O157:H7 - verotoxigenic E. coli in ground beef and trim; and Listeria monocytogenes in ready-to-eat meat. With the support of $2.02 million from Agriculture and Agri-Food Canada's AgriFlexibility Fund, the CFIA conducted a national microbiological baseline study - on Salmonella and Campylobacter in broiler chickens in 2012-13. The results of this baseline study will provide prevalence and concentrations of pathogens and indicator organisms, which will allow the CFIA and its stakeholders to set performance targets and better evaluate the effectiveness of pathogen reduction strategies in future. The data will also provide benchmarks against which industry can measure the effectiveness of their HACCP programs and/or intervention measures. CFIA also conducted two microbiological baseline study pilots on beef carcasses and RTE foods in 2015.

Listeria

From 2009-15, the CFIA received $92.1 million to support additional listeria meat inspection. Another $15.1 million was provided in 2015-16. See Annex G for the Listeria Budget and Expenditures Table.

The activities funded were largely implemented as planned, with the exception of some delay and overestimation of the number of inspectors and the related training, which represents most of the variance shown in the table in Annex G.

As of 2010-11, the CFIA hired 70 full time inspectors to cover the amount of cumulative time required to conduct the new listeria inspection requirements, formally outlined in Health Canada's 2011 "Policy on Listeria monocytogenes in Ready-to-Eat Foods." The CFIA's manuals and policies were updated thereafter. The primary changes were new end-product compliance criteria, categorization of types of food where listeria can occur and increased testing requirements, particularly the addition of food contact surface testing.

Overall, CFIA Listeria sampling increased threefold from 2009-14, while the prevalence of listeria that was found remained the same, even going back to 2006. This indicates that the pre-2009 sampling intensity may have been adequate because even these large increases in sampling did not discover a larger prevalence of Listeria monocytogenes. The increased sampling was critical for providing greater assurance that the prevalence figures were accurate by decreasing the margins of error for the degree to which the samples were an accurate representation of all RTE meat products produced in the country.^Footnote 39

Listeria Samples Tested

Description for Diagram - Bar chart for Listeria Samples Tested

This bar chart shows the distribution of listeria samples tested per fiscal year, from fiscal year 2006-07 to 2013-14.

On the top of the figure is the following title: "Listeria Samples Tested".

The horizontal axis identifies each fiscal year. From left to right, they are:

• 2006-07
• 2007-08
• 2008-09
• 2009-10
• 2010-11
• 2011-12
• 2012-13
• 2013-14

The vertical axis represents numbers. From top to bottom the numbers are:

• 7000
• 6000
• 5000
• 4000
• 3000
• 2000
• 1000
• 0

There are 8 bars in this chart, one for each fiscal year. The chart shows that:

• 2216 Listeria samples were tested in 2006-07
• 2149 Listeria samples were tested in 2007-08
• 2401 Listeria samples were tested in 2008-09
• 5617 Listeria samples were tested in 2009-10
• 6500 Listeria samples were tested in 2010-11
• 4833 Listeria samples were tested in 2011-12
• 4722 Listeria samples were tested in 2012-13
• 4570 Listeria samples were tested in 2013-14

Other Listeria monocytogenes enhancements are outlined in Annex E.

This evaluation also reviewed the non-meat inspection activities funded by an additional $17.4 million from 2011-15.^Footnote 40 The funding was received through the Food Safety Modernization Initiative.

The CFIA hired 20 new inspectors to support the focus on high-risk areas, developed new training material and trained inspectors, updated manuals, policies and guidance documents and implemented new environmental sampling plans. It has also validated new laboratory methodologies for Listeria in non-meat commodities, developed guidance material for industry, and developed a standard operating procedure with HC and PHAC about requests from provincial/municipal/territorial agencies for laboratory testing.

Summary Assessment of CFIA's Sampling and Testing

In reviewing the expert panel's overall positive assessment, the continued equivalency with the U.S., the work of the Pathogen Reduction Initiative, the enhancements to listeria testing and following a review of program criteria and principles, the evaluation team concluded that, overall, the sampling and testing programs appear well designed.

5.3 Q 6: Effectiveness of recall processes

There were 307 meat product recalls during the five-year study period plus the 2014-15 fiscal year – 270 recalls of domestically produced products and 37 recalls of imported products. Note, as outlined in the text box below, there are primary and secondary recalls, the latter being additional recalls linked to the primary event. The numbers used in this report are limited to primary recalls only. Recalls during the study period represent about one recall per 91 million kg of domestically produced products and one recall per 389 million kg of imported products, although the data for domestic and imported products are not strictly comparable (discussed in section 5.5). The three main hazards leading to recalls were:

• Allergens – 39 per cent of recalls
• Extraneous materials – 18 per cent of recalls
• Microbial hazards – 37 per cent
  • Listeria – 17 per cent of recalls
  • E. coli 0157 – nine per cent
  • Other microbial hazards – 11 per cent

As shown in the graph below (recalls per year), the number of recalls during this timeframe doubled from 32 to 64 from 2009-10 to 2014-15. Following the revision of the listeria policy and implementation of new sampling and testing requirements after the 2008 listeriosis outbreak (Maple Leaf Foods), the frequency of listeria-driven recalls declined from 14 to seven. Recalls resulting from E. coli-O157:H7 have remained relatively static over this time period.

Meat Product Recalls, with proportion of Listeria and E. coli 2009-15

Description for Diagram - Line Graph – Meat Product Recalls

This figure is a line graph with the numbers of meat product recalls per fiscal year, and the proportion of recalls related to E. coli and to Listeria.

The horizontal axis identifies each fiscal year. From left to right, they are:

2009-10
2010-11
2011-12
2012-13
2013-14
2014-15

The vertical axis represents the number of recalls. From top to bottom the numbers are:

70
60
50
40
30
20
10
0

There are three lines across this graph, orange representing E. coli O157:H7, blue representing Listeria and green representing the grand total of meat product recalls.

The orange line displays the following information for E. coli O157:H7:

3 recalls for fiscal year 2009-10
5 recalls for fiscal year 2010-11
7 recalls for fiscal year 2011-12
6 recalls for fiscal year 2012-13
4 recalls for fiscal year 2013-14
4 recalls for fiscal year 2014-15

The blue line displays the following information for Listeria:

14 recalls for fiscal year 2009-10
10 recalls for fiscal year 2010-11
9 recalls for fiscal year 2011-12
7 recalls for fiscal year 2012-13
4 recalls for fiscal year 2013-14
7 recalls for fiscal year 2014-15

The green line displays the following information for the grand total of meat product recalls:

32 recalls for fiscal year 2009-10
50 recalls for fiscal year 2010-11
51 recalls for fiscal year 2011-12
55 recalls for fiscal year 2012-13
55 recalls for fiscal year 2013-14
64 recalls for fiscal year 2014-15

There are many possible triggers for recalls. The data from 2009-10 to 2014-15 show the majority of the 307 recalls during that timeframe were client-initiated (34 per cent), the result of a positive CFIA facility or product inspection (20 per cent), or the result of a positive CFIA lab sample (14 per cent).

In 2011, the CFIA carried out an evaluation of measures implemented by CFIA, PHAC, and Health Canada (2009-11) to improve inter-departmental and inter-jurisdictional interfaces during food safety incidents.^Footnote 42 The findings of that report included:

• Appropriate governance structures are in place to provide oversight to CFIA, PHAC, and Health Canada in connection with foodborne illness outbreaks.
• Roles and responsibilities are clearly documented and understood by all the federal partners.
• The decision making process employed during the emergency management of foodborne illness outbreaks is effective, and
• Overall, the federal-provincial-territorial emergency management of foodborne illness outbreaks has improved markedly.

A large majority of the interviewees in this evaluation agreed the recall process is effective. Most of the comments dealt with specific aspects of the process, for example:

• "XL demonstrated that there is a need for much better and more timely information from the producers. There have been big changes in this regard."
• "There is now an inter-departmental committee for managing outbreaks…Also, there is a federal-provincial agreement on the process for closing outbreaks."

This evaluation included a review of the three reports noted in the footnote and an analysis of the implementation status of all the recommendations contained in these reports, which confirms this finding.

The major changes to recall processes the Agency has reported are:

• Procedures for better sharing of distribution and recall information during the various stages of recall activities
• Enhanced procedures for Health Risk Assessments
• Better recording and documentation of decision making

Two recommendations of the Weatherill Report were not fully implemented:

• The CFIA President should stay in position for the full five years as noted in the CFIA Act
• The Office of Food Safety and Recall (OFSR) should report directly to the Office of the President

Regarding the first recommendation, the Agency noted the appointment of Deputy Heads is the prerogative of the Prime Minster on the advice of the Clerk of the Privy Council Office. On the second recommendation, the Agency undertook a review of governance in relation to the OFSR and then committed to ensuring that the Office of the President be fully engaged in emerging food safety issues, through daily status briefings.

5.4 Q 7: Effectiveness of the procedures for equivalency determinations

The expert panel reviewed the system for equivalency determinations, and came to the following consensus opinions:

• "The current system for equivalency determinations is satisfactory.
• The system can be made more efficient:
  1. It can be made more risk-based – i.e., involving different risk levels for different countries and, as a result, different intensities of equivalency investigations; and
  2. There can be fewer on-site investigations, with more reliance on the Foreign Competent Authority in the exporting country where there is a sufficient level of trust in that country's meat safety system."

A large majority of the interviewees agreed the system used for equivalency determinations is satisfactory. Most of the comments dealt with specific aspects of the process, for example:

• "Canada is less prescriptive than the US in equivalency assessments – CFIA is more tolerant of different approaches as long as the outcomes are satisfactory."
• "The equivalency process is good – based on CODEX and international standards."
• "I am a huge fan of equivalency. Countries don't need to do the same thing as long as they achieve the same outcomes."

Somewhat contrary to the view of the expert panel, some CFIA interviewees indicated their concern with insufficient follow-up audits (site visits) of foreign countries:

• "Canada's auditing program is less active than that of the US…We could do more auditing."
• "The frequency and intensity of follow-up audits is weak."
• "Canada rarely does sufficient auditing of foreign operations to ensure that these operations are, in fact, equivalent to those of Canada."

As discussed in section 4.1, the international comparison study found that in the case of Australia and New Zealand, the only site visits involved in equivalency determinations are BSE safety assessments. However, it should be noted these countries are not big importers of meat products. The US conducts audits of foreign country facilities regularly, including in Canada.

In Europe, the EU is responsible for equivalency assessments. Equivalency determinations are table-top exercises for EU member countries, but involve on-site audits for non-EU countries. The EU is a major importer of meat products.^Footnote 44

5.5 Q 8: Effectiveness of the Import Inspection System

The expert panel did not comment on whether the import inspection system is satisfactory. However, they did comment on how it could be made more effective. As discussed in Annex A section II.2, the vast majority of import inspections involve organoleptic inspections, which the panel identified as being of low utility, as discussed under Finding 8.2 below.

Approximately nine per cent of imported meat products are subject to microbiological inspections. The graph below shows the results of microbiological tests of domestic products and imported products (excluding canned meat) over the latter four years of the study period.^Footnote 45

Satisfactory Microbial Samples 2010-14

Description for Diagram - Line Graph – Satisfactory Microbial Samples

This figure is a line graph with the percentages of satisfactory microbial samples from microbiological tests of domestic products and imported products (excluding canned meat) from fiscal year 2010-11 to 2013-14.

The horizontal axis identifies each fiscal year. From left to right, they are:

2010-11
2011-12
2012-13
2013-14

The vertical axis represents the percentage of samples satisfactory. From top to bottom the numbers are:

99.0%
98.0%
97.0%
96.0%
95.0%
94.0%
93.0%
92.0%

There are two lines across this graph, blue representing Domestic Meat Products and orange representing Imported Meat Products (No Canned Meat).

The blue line displays the following information for Domestic Meat Products:

96.4% satisfactory samples for fiscal year 2010-11
95.0% satisfactory samples for fiscal year 2011-12
96.3% satisfactory samples for fiscal year 2012-13
97.7% satisfactory samples for fiscal year 2013-14

The orange line displays the following information for Imported Meat Products (No Canned Meat):

97.2% satisfactory samples for fiscal year 2010-11
97.1% satisfactory samples for fiscal year 2011-12
98.2% satisfactory samples for fiscal year 2012-13
94.5% satisfactory samples for fiscal year 2013-14

The test results for imported products (excluding canned meat) are 0.5 per cent higher (i.e., better) on average over this period than the test results for domestic products.^Footnote 46

This finding does not relate directly to the sufficiency of the import inspection system, but it does relate to the safety of imported meat products. Another indicator of the sufficiency of the import inspection system is the data regarding recalls of domestic versus imported products. We reported in section 5.3 that, over the study period, there has been about one recall per 91 million kg of domestically produced products and one recall per 389 million kg of imported products.^Footnote 47 This represents about 4.3 times as many recalls per kg for domestic products compared with imported products, which suggests the import inspection process is more than satisfactory. As noted in Finding 6.1, recalls as an indicator of imported food safety have significant caveats, and do not in and of themselves provide conclusive evidence on food safety.

There are two other significant qualifications that should be made regarding this comparison:

• As noted above, when comparing domestic to imported products, the differences in product mixes have to be taken into consideration. Domestically, entire cows are processed and turned into different foods – ground beef, steak, roast beef deli meats, and so on. Over two-thirds of Imports are ready to eat (RTE) meat products.
• The treatment of domestic and imported products that are found to be unsafe (i.e., contain unsafe levels of pathogens) differs – domestic products can be sent for additional cooking and are thereby made acceptable, whereas imported products are often recalled or destroyed.

Without even looking at any numbers, one would expect imported products to be safer (per kg) than domestic products, because this is in one way an "apples to oranges" comparison:

• In the case of imported products, we are looking at products that have been through Canada's import inspection system, in addition to the meat inspection system of their country of origin (which has been determined to be equivalent to Canada's meat inspection system)
• In the case of domestic products we are just looking at products that have been through Canada's meat inspection system

In other words, if one accepts that Canada's and the foreign country's systems are equivalent, then products imported into Canada go through an extra level of inspection.

Even if one has doubts about the rigour of Canada's equivalency determination process (see interview comments in section 5.4), 86 per cent of imported meat products originate from the US,^Footnote 48 whose meat inspection system is generally agreed to be at least as thorough as Canada's. Therefore, when US imports that are inspected by the CFIA^Footnote 49 are considered, this involves comparing products that have met Canadian standards (Canadian domestic products) with those that have met Canadian equivalent standards (in the US) and also have "passed" Canada's import inspection system (imported US products).

Even more convincing, 93 per cent of meat imports originate from the US, Australia, and New Zealand,^Footnote 50 all of whose meat safety systems have been assessed as equivalent to Canada's. This raises the question of the utility of import inspections, at least for countries with lower risk meat safety systems.^Footnote 51 This is discussed further below.

Only 17 interviewees responded to the question regarding the effectiveness of the import inspection system, and only 10 of these provided a direct response to the question (others commented on particular features of the system). Six felt the import inspection system is sufficient, because it is just the last step in the process:

• "Import inspection is sufficient because it is just one hurdle among many – It's just the final verification; there are other necessary steps before this. The first step is the equivalency of the foreign country's meat safety system. This is regularly audited and is the basis for everything else."
• "Rather than increasing import inspections, it's much more effective to work with the other foreign competent authorities to have confidence in other country systems."

The four people who thought the system is ineffective cited the limited amount of microbiological testing, especially in comparison to the import testing done by the US.

Two other people who thought the system is ineffective were not counted in this total because their rationale was based on an inappropriate comparison – e.g., "We check domestic products in more detail [than we check imported products]." As discussed above, the checking Canada does of domestic products should be compared with the checking the exporting country does of their domestic products. Canada's import inspections are additional to that.

The consensus conclusions of the expert panel were:

• "Organoleptic inspections are not useful from a food safety perspective."
• "The import inspection system should involve some microbiological and chemical testing of samples."^Footnote 52

One of the panel members elaborated as follows: "The current system should be improved by having some internationally standardized measures that relate to food safety based on microbiological and chemical testing. You need to do testing in order to be able to draw conclusions regarding food safety."

As noted previously, organoleptic inspections comprise the largest component of import inspection. While some interviewees and panel members expressed the opinion organoleptic inspections do have some value as a "wholesomeness check" (e.g., as a check of whether spoilage has occurred), their low utility related to food safety was a noted concern. A number of scientific studies have presented concerns about the value of organoleptic inspections in identifying microbial contamination.^Footnote 53

As discussed above, the data indicates that the utility of import inspections is questionable for countries whose meat safety systems Canada trusts. This would include the US, Australia, and New Zealand, and any other countries which CFIA is confident have lower risk meat safety systems. For other countries (i.e. those which have a poorer meat safety history, in spite of CFIA having certified their system as equivalent), the opinions of the expert panel indicate that there is some value in microbiological and chemical residue testing.

Note the CFIA's current testing regime is risk based, including its auditing of foreign country systems and establishments. As of February 2015, there were 41 countries whose meat inspection systems were considered sufficiently equivalent to export meat products to Canada. Sufficiently equivalent is the minimum requirement, and there will always be some countries that have stronger food safety regimes than others. Limited import inspections is the approach taken by all the countries investigated in the international review, except for the US:

• Australia: No import inspections other than label checking (verifying that the exporting country's inspection and certification system along with establishments operating under that system have been approved by Food Standards Australia and New Zealand and the Australia Department of Agriculture);
• New Zealand: No import inspection system per se. Liability is on the exporting country and the importer for due diligence that products comply with New Zealand standards and requirements.
• Denmark and Finland: Import inspections mostly limited to products from non-EU member countries.

5.6 Q 9: To what extent are meat products distributed in Canada safe for human consumption?

This question is addressed in two steps:

• Do meat products distributed is Canada meet Canadian food safety criteria?
• Do Canadian food safety criteria ensure a high degree of food safety?

The evidence for this finding has been presented under previous evaluation questions, as summarized below:

Meat products produced in Canadian plants:

• Finding 4.1: The requirements and associated procedures for plant operations (HACCP systems) are satisfactory.^Footnote 54
• Finding 5.1: The system for slaughter inspections is satisfactory.
• Finding 5.3: The system for processing inspections is satisfactory.
• Finding 5.5: CFIA's sampling and testing is satisfactory.

Imported meat products:

• Finding 7.1: The system used for equivalency determinations is satisfactory.
• Finding 8.1: The import inspection system is satisfactory.

Recalls:

• Finding 6.1: Recall processes have been effective in removing unsafe products from the marketplace.

As noted in Annex A section V, Canadian food safety criteria are developed and continuously updated by Health Canada, based on extensive scientific evidence.

Section 5.5 discussed the adequacy of the Agency's sampling and testing programs for microbiological contaminants and chemical residues. The expert panel's overall conclusion was positive, noting particular satisfaction with the criteria for listeria and E. coli guidance documents, design and delivery.

The panel emphasized, in addition to scientific criteria, the role of public education in food safety is important.

The evaluation has already presented considerable additional evidence supporting the high degree of meat safety in Canada, including:

• More than 95 per cent of CFIA's microbiological testing results for meat products are satisfactory
• Canadian exports of meat products are relatively unrestricted by foreign governments to which we export, including the US, which has a very demanding meat safety system. There are a very small number of port of entry violations (discussed in section 5.7)

Some interviewees suggested meat safety criteria could be altered to greatly improve the safety of meat products, if regulatory approval for meat irradiation was granted. Canada already approves the irradiation of potatoes, onions, flour and spices, and many other countries irradiate meat. Health Canada is currently holding consultations on such a regulatory change for some meat products. Some groups think that meat irradiation might discourage strict adherence to good manufacturing practices,^Footnote 55 including at least one Canadian meat association. Recognition of such risks can be used by the CFIA when designing its inspection oversight of those meat operations using irradiation.

Finally, all interviewees who commented on the safety of meat products distributed in Canada were positive. As noted in other sections of this report, a number of people presented the argument that meat in Canada is over-inspected relative to what is required for food safety, which supports the suggestion the Meat Programs move to more risk- and outcome-based approached (as discussed under Finding 5.1).

5.7 Q 10: Effectiveness of the program to ensure the compliance of export products

Exported meat products form a major part of Canada's meat production and distribution enterprise, amounting to about 1.5 billion kg annually, with a dollar value exceeding $5 billion.^Footnote 56 For comparison purposes, domestic meat production totals about 4.1 billion kg annually, 2.7 times more than exported meat. Meat export volume for pork and beef by key major recipient country is presented in Annex I.^Footnote 57

The second part of this finding ("…safe for human consumption.") follows from previous findings. This is because, as discussed in section 2.4, products intended for export are subject to the same systems and procedures as those produced for domestic distribution. They are often subject to additional certification and testing procedures in order to meet the requirements of importing country. Therefore, the statement that exported meat products are safe for human consumption follows directly from the findings summarized in section 5.6 regarding the safety of domestically produced meat products for human consumption.

The first part of the finding ("Exported meat products comply with the requirements of importing countries…") is supported by data on port of entry (POE) violations by exported meat products. These data are provided in the following tables for the four major receiving countries.

Based on this data, between 2012-13 and 2014-2015, there has been an average of one POE violation for every 154 million kg of meat exported to the four major receiving countries.

The interviewees who commented on the export system (14 from the CFIA, five from industry) were unanimous in endorsing the effectiveness of the meat export program for ensuring safety and compliance. For example, one interviewee said: "It is a labour-intensive process, but meat exports from Canada represent huge numbers, and almost all products reach their destination without any problems."

However, over half these people commented on the inefficiency of the process. These comments mainly related to the extra procedures that have to be undertaken to satisfy the requirements of some importing countries. These are seen as being an inefficient use of resources (for which there is very little cost recovery) that does not contribute to food safety. For example:

• "Some countries have requirements over and above internationally accepted CODEX principles…sometimes [in other countries] science is overridden by public perceptions or politics."
• "Some countries require establishment-by-establishment certification [when system certification should be sufficient]."

Industry comments on this issue dealt with the desirability of simplifying export requirements from their perspective—particularly, avoiding unnecessary border testing by the U.S.

Canada dropped its inspection of U.S. meats to just one-in-10 loads, following signing of the Canada-U.S. Free Trade Agreement in the late 1980s. The U.S. initially followed suit, but then returned to inspecting every load. Furthermore, while Canada has 125 CFIA registered establishments where meat products imported from the U.S. may be inspected, the U.S. only has 10. All shipments of meat exported to the U.S. must be delivered to one of those 10 for inspection. The Canadian Meat Council recently reiterated its frustration at the cost to Canada's meat industry and lack of reciprocity.^Footnote 59

The CFIA's 2014 Evaluation of Daily Shift Inspection Presence concluded that "DSIP is clearly not cost-effective from a purely food-safety perspective. A risk-based approach to inspection would be more cost-effective and in-line with the principles of inspection modernization. However, DSIP is required to maintain access to the U.S. market."

See the Evaluation of Daily Shift Inspection Presence for details on this initiative and its costs.^Footnote 60

5.8 Q 11: Stakeholder Satisfaction

Note: This finding may or may not apply to the provinces and territories; the expectations of the provinces and territories were only minimally investigated.

Twenty-nine of the interviewees responded to the question about whether or not the Meat Programs are meeting stakeholder expectations. Twenty-three of these people answered this question directly, and the other six discussed specific aspects of certain relationships, such as CFIA challenges in satisfying company requirements for work shift arrangements in overtime situations.

All but four of these 23 respondents (four of seven provincial representatives) responded positively to the question, indicating expectations of major stakeholders are being met; however, some responses only dealt with specific stakeholder groups.

With respect to industry's expectations, most interviewees explicitly stated the CFIA-industry relationship is functioning well. Several expressed positive opinions regarding specific initiatives (e.g. the implementation of discussion platforms to resolve issues and for planned modernization initiatives). No CFIA or industry representative expressed a negative opinion regarding the CFIA-industry relationship. See also the discussion below regarding industry consultations related to Agency Transformation.

With respect to expectations of other government departments and agencies, all CFIA respondents and all respondents from Health Canada, PHAC, and AAFC said the relationship between the federal partners–and/or the relationship between their department and CFIA–works well overall. Some interviewees from AAFC noted recent challenges resulting from the CFIA reorganizations and coordination with market access issues. Industry interviewees were generally very positive when commenting on the support for market access provided by the CFIA.

Regarding expectations of consumers, industry officials who commented on this, including some involved in surveys of consumers, stated that consumers are largely satisfied with the outcomes of the Meat Programs. See also the discussion below regarding the Ipsos Reid survey.

Regarding expectations of provinces, while CFIA Head Office officials were positive about relations with the provinces, four of the seven provincial representatives interviewed were not, noting an opportunity for improved collaboration.

Additional information regarding the expectations of two of these groups follows:

Expectations of industry: By the time this report was written, there were two comprehensive rounds of consultations with industry across the country regarding Agency Transformation and the associated new draft regulations, which give the companies more flexibility but also make it clearer that the companies, not CFIA, are responsible for food safety. (Industry has always been responsible, but with a greater emphasis on their systems for food safety, including preventive control plans, it is clearer that they are responsible.) CFIA officials report that there have been no significant objections from industry to the planned approach.

Expectations of consumers and the public: See the discussion in section 4.1 regarding the report by Ipsos Reid, Food Safety: Canadians Awareness, Attitudes, and Behaviours. As noted in that report, Canadians are moderately to extremely confident in Canada's food safety system, and a large majority are generally satisfied with the safety of meat products.

6.0 Program Delivery

6.1 Q 12: Effectiveness of performance measurement

Supporting evidence

In this report the term "performance measurement system" is used as follows:

A performance measurement system for a program is a set of measures that provides, on an ongoing basis, a description and understanding of what is being done and what is being accomplished by the program.

Some important implications of this definition are:

• Regarding "what is being accomplished," a performance measurement system must have measures of the key intended outcomes of the program (the program objectives) that illustrate the extent to which these are being achieved.
• Regarding "what is being done," a performance measurement system must have some measures of the activities that are being carried out and the outputs that are being produced as a result of the activities – not only to describe what is being done but also in order to be able to understand the outcome measures. (For example, if the actual program outcomes do not meet expectations, the activity and output measures may help to explain why this is the case.)

As an example of the latter point, one output measure for the Meat Programs is the amount of completed sampling relative to plan, and if this measure were significantly lower than the planned amount, then this might help to explain why an outcome measure like safety of the food supply is less than desired.

Based on the CFIA's annual Departmental Performance Report, the performance measurement system for the Meat Programs is based on three measures of outcomes:

• Percentage of inspected federally registered meat and poultry establishments in compliance with federal regulations
• Percentage of tested domestic meat and poultry products in compliance with federal regulations
• Percentage of tested imported meat and poultry products in compliance with federal regulations.

This reflects a best practice in that the system is based on outcome measures, and they are measures of the achievements of the Meat Programs as a whole. This is the foundation for much of the CFIA's Performance Measurement Framework, used to report annually in its Departmental Performance Reports. However, this system is quite limited because:

• It does not contain any measures of antecedent activities or outputs, such as a measure of the number and quality of completed establishment licensing and inspection tasks, which is an antecedent output for the first outcome measure
• It does not contain any measures of important higher level outcomes, such as a measure of the safety of the food supply

For the study period, there was a Senior Management Committee (SMC) Program Dashboard used for internal performance reporting, though this was limited to output data and never intended to report on outcomes. The Dashboard contains unaudited information used by SMC for general guidance, information, and discussion purposes. It reports on Food as a whole, Animal Health, and Plant Protection, but it lacks specifics related to meat products. Six items are discussed for each of the three areas reported on:

• Inspection Outputs
• Laboratory Analysis
• Export Certificates
• Import Transactions
• Food Safety & Recall Incidents
• Enforcement Penalties

In the interviews, nine of the 14 CFIA officials who responded to the question on performance measurement commented on the effectiveness of the performance measurement system and/or the dashboard.^Footnote 61 All but one noted a need for improvement.

There are also a number of performance measures collected and reported on by various groups in the Agency supporting the Meat Programs. For example, Science Branch collects lab results and the Enforcement Office collects information about penalties and Corrective Action Requests. However, these efforts are not coordinated.

During the study period, a performance measurement framework for the Meat Programs was developed. It identifies a number of performance measures, which are mainly measures of activities, with some output and outcome indicators related to imports and exports.

In addition, a comprehensive logic model was prepared for the CFIA's overall food safety activities. This would have been suitable for use as the basis for a performance measurement system, but it has not been used for this purpose.

At the time this study was completed, a single performance measurement system for the Food Program had not yet been developed.^Footnote 62

The results of the international comparison study are summarized below.

U.S.: The performance measurement system is similar to Canada's, with three high-level performance measures for the Food Safety Inspection Service (FSIS):

• Number of illnesses from products regulated by FSIS
• Per cent of establishments passing verification testing
• Per cent of slaughter establishments with effective humane handling systems

Australia: There is no overall performance measurement system for the Australian meat safety program. There is a system for monitoring the performance of individual plants involved in export, called the Product Hygiene Indicator, or PHI (in Australia the federal government is only responsible for plants that export). The PHI was a unique example in this evaluation's comparison study of other countries where the private sector openly shared their internal inspection results. This is seen by the Australian government as a critical information source to help identify areas for improvement. This contains measures such as:

• Compliance with Australian regulations
• Compliance with importing country requirements
• The overall management capability of the plant

New Zealand: There is no overall performance measurement system for the New Zealand meat safety program. Data are collected for some of the individual sub-programs – e.g., levels of pesticide residues and chemical contaminants found in samples, levels of harmful micro-organisms found in samples.

European Union: The individual member states are responsible for performance measurement.

Denmark: Denmark appears to have an effective performance measurement system. There are 25 key performance measures related to outcomes, risk-based controls (e.g. achieving the target number of inspections), business processes, and human resources. Outcome measures related to food safety include reduced incidence of human illness caused by certain pathogens, as well as compliance measures. There is a monthly performance measurement report.

Finland: There is no overall performance measurement system for Finland's meat safety program. Performance indicators are just compiled on a plant by plant basis.

6.2 Q 13: Effectiveness of the governance and delivery process

Many aspects of the governance and delivery of the meat programs have already been discussed. No significant issues of concern were noted. However, two other areas of program redesign should also be considered in reviewing governance and delivery processes.

In 2015, there was a shift in responsibilities between Policy and Programs Branch and Operations Branch, which involved moving all operational resources to Operations and focusing PPB on policy and intended Agency outcomes. For example, PPB identifies high risk and high impact areas, and Operations develops the detailed operational plans for addressing these (e.g., inspection frequencies), based on available resources and required verification levels.

In addition, in 2013, the Agency established 16 Centres of Operational Guidance and Expertise "to provide industry and CFIA inspectors with better access to information and advice."^Footnote 63 Three of these relate specifically to the Meat Programs: red meat slaughter (Calgary), poultry slaughter (St. Hyacinthe), and processed meat and poultry (St. Hyacinthe).

Interviews were conducted during the initial implementation of "Agency Transformation", which involved organizational and governance changes. As a result, interviewees tended to comment more on these changes than on past organizational structure and governance processes.

Only a few people commented on the re-alignment of the Operations Branch, but those who did were positive, for example: "In the new organizational structure the responsibility of Operations will be the interpretation of guidance from PPB. They will be responsible for the "how" of implementation."

Regarding the Centres of Operational Guidance and Expertise, although these were intended to provide guidance to both industry and CFIA inspectors,^Footnote 64 guidance was only being provided to CFIA staff. This was of significant concern to industry leaders who were interviewed.

CFIA feedback on the Centres of Expertise was mainly gathered through the regional focus groups. Although consensus was not sought, the views expressed were almost uniformly negative. For example, the following are representative comments from each of the three focus groups:

• "The main problems with this system are big delays and inadequate knowledge in the Centres of Expertise."
• "There is now no access to expertise for advice–even for a simple question the answer may take weeks."

In the survey of inspectors, 39 per cent of the respondents either disagreed or strongly disagreed with the following statement: "There are systems within CFIA for obtaining timely guidance and approvals." Almost half (47 per cent) of meat processing inspectors either disagreed or strongly disagreed with this statement.

The implementation of the Agency's new designs for the provision of guidance and expertise through the Centres continued to evolve through the study period and beyond, and included plans to create a separate office to answer industry questions.

A change in program design, which moved meat processing into the new single food program, occurred at the end of the evaluation's study period. While this was not included in the evaluation's scope, some interviewees responded to questions on governance and program delivery by expressing confusion over the changes, as is expected in the early stages of any large-scale organizational transformation.

The evaluation did not review the relevance or effectiveness of internal CFIA governance committees, including the Food Business Line Committee, the Policy and Program Management Committee, or the Senior Management Committee as they related to meat program issues. No interviewee, focus group participant or survey respondent raised any concerns regarding the role or performance of these committees.

6.3 Q 14: Adequacy of staff resources and competencies

There are no federal or international rules or guidelines identifying a specific number of required inspectors to assure food safety requirements are met in an establishment. One exception to this is the Meat Hygiene Manual of Procedures which includes prescribed numbers of slaughter inspectors per work station.

However, the Modernized Slaughter Inspection Program is moving away from prescriptive to more outcome and risk-based inspection models, as noted in Findings 4.1 and 15.1.

For purposes of this evaluation, adequacy is assessed as sufficiency of operational resources to complete the required tasks.

There was no evidence of any critical challenges to completing required meat inspection tasks. Additionally, no foreign audits or equivalency assessments reviewed revealed any concerns about the adequacy of the number of CFIA meat inspectors.

There was some evidence that not all tasks currently required are necessary, and not all are necessary at the current required frequency. The plan to make CVS tasks more risk- and outcome-based (outlined under Finding 15.1) should provide significant opportunity to re-focus inspectors' work on areas of highest risk.

The interview question regarding the adequacy of Meat Programs staffing resources did not specifically address inspection resources. Nevertheless, no interviewees from CFIA Headquarters indicated staffing was a problem. Two typical comments were:

• "Staffing levels are about a four, on a one-to-five scale. We don't skip work (i.e., task
• "Do we have enough people to ensure food safety? Yes we do…but we can be more efficient."

The sentiment was different in the regions, where some expressed concern there were not enough inspectors. Each of the three focus groups expressed the view that there has been a decrease in inspectors over the study period to carry out traditional inspection duties.

The following shows the number of meat inspection staff over most of the study period, based on available data. These figures include both meat inspectors and supporting staff, such as those providing technical expertise on meat inspection systems or veterinary oversight.

Based on these numbers, it appears meat inspection staff have remained relatively constant over the study period, with small fluctuations of two-to-four per cent in certain years. These numbers are not evidence of resource adequacy, without context around the number of facilities and tasks.

In particular, the number of facilities fluctuates many times over the course of any given year, and each facility requires a different number of inspectors, based on the type and size of operation, and the number of shifts. For example, from 2010-15, the number of meat processing establishments dropped from 613 to 507.

A few significant changes to meat inspector numbers occurred during the evaluation study period. As noted in the discussion on enhancements to Listeria inspections, 70 new meat inspectors were hired to cover the increased sampling requirements. These inspectors were hired in the 2010-11 fiscal year.

In the 2010-11 fiscal year, the Agency also hired an estimated 100 new meat inspectors to address increased inspection times needed to address the DSIP requirement.

Two years later, in 2013, responsibility for provincial meat inspection was returned to the provinces of British Columbia, Manitoba, and Saskatchewan. These activities had been delivered by the CFIA for a number of years on a limited cost-recovery basis, and were transferred back to the provinces as a cost saving measure. These were the only provinces where the CFIA provided this inspection service. This removed the requirement for 68 meat inspector positions. Many of these individuals remained with the Agency, working in other areas aside from meat inspection.

While this amounts to a net increase of approximately 100 new meat inspection positions, for the reasons noted above, notably the constant fluctuation in the number of registered establishments, the effect on the total number of meat inspectors is not discernable.

In 2010, a PriceWaterhouseCoopers (PWC) study supported the CFIA's calculation for the required number of meat inspectors to deliver the meat inspection program.^Footnote 65 The PWC study addressed a recommendation of the Weatherill Report for an audit of meat inspection resource requirements.

In the survey of inspectors, 61 out of 258 slaughter inspection staff responded to an open-ended question about how slaughter inspections could be made more effective. The most common suggestion (made by 20 people) related to increasing the number of slaughter inspectors. The second most common suggestion (made by 19 people) related to decreasing the amount of inspection done by industry (and presumably increase the amount done by CFIA).

The slaughter modernization initiative may reduce the number of required CVS tasks, allowing for a greater focus on areas of highest risk and impact.

Training for CFIA meat inspectors was greatly enhanced following the release of the Weatherill Report and the implementation of Health Canada's Policy on Listeria monocytogenes in Ready-to-Eat Foods (2011). In response to these events, the CFIA implemented its National Meat Processing School, as well as training programming related to inspection for Listeria.

Meat training programs are subjected to regular evaluations to ensure they remain relevant and effective. They are also audited by the U.S. FSIS and Government Accountability Office. To date, the results have all been positive.

In the survey of inspectors, more than 80 per cent of the respondents said training has adequately prepared them for their current role. The following is a breakdown of inspectors who agreed with this statement:

• Meat processing inspectors – 81 %
• Poultry processing inspectors – 85 %
• Meat slaughter inspectors – 77 %
• Poultry slaughter inspectors – 85 %

In addition, more than 90 per cent said they have the knowledge and skills to successfully carry out their responsibilities. The following is a breakdown of inspectors who agreed with this statement:

• Meat processing inspectors – 91 %
• Poultry processing inspectors – 91 %
• Meal slaughter inspectors – 96 %
• Poultry slaughter inspectors – 93 %

Interview responses concerning training and competencies were generally positive, except several people noted there are challenges with turnover. The following are some representative comments:

• "CFIA has a very well trained and very capable inspectorate. Slaughter is a challenge, however…there is lots of turnover."
• "The competencies of inspectors has increased greatly over the years, but the turnover rate has been high."

In the focus groups, comments regarding the quality of available training were also positive; however, there were many comments about the difficulties associated with freeing up inspectors for training. For example:

• "New hires supposedly have to attend a mandatory six week preparatory course at the Meat School during their first year, but that is not possible."
• "The problem is the operational constraints. Often it is impossible for people to attend training before they are put into operations – it's necessary to get them on the job right away."
• "There isn't enough time for even the bare bones training – it requires six-to-nine weeks plus a six week prep course; then there is training in CVS."

With regard to training materials, 77 % of survey respondents rated these as "up to date with the current requirements of my role." However, the breakdown of inspectors who agreed with this statement indicates significantly more poultry processing inspectors agreed:

• Meat processing inspectors – 78 %
• Poultry processing inspectors – 91 %
• Meat slaughter inspectors – 68 %
• Poultry slaughter inspectors – 77 %

The CFIA's training division confirmed red meat slaughter training materials needed to be updated.

Finally, when survey respondents were asked "How could training have better prepared you for the work associated with your current role?" only two answers received more than 15% of the possible responses:

• 16 % of meat slaughter inspectors said more on the job training, such as job shadowing, would have been helpful
• 18 % of meat slaughter inspectors said more specific subject matter related to their current role would have been helpful

7.0 Economy and Efficiency

7.1 Q 15: Potential for more cost-effective delivery processes for the Meat Programs

The evaluation identifies four possible changes to the Meat Programs delivery process that could increase cost-effectiveness:

1. Less emphasis on carcass-by-carcass slaughter inspections and more on outcome measurement
2. More risk-based CVS tasks
3. Reduced site visits for equivalency determinations
4. Reduced organoleptic inspections in import inspections

The following table lists which data sources identified these possible changes. The column "Agency Modernization Working Papers" is included to identify those improvements already under consideration.

Brackets (X) refer to indirect evidence

Details on each of the four Possible Changes in the table above are provided here:

Possible Change 1
"Less emphasis on carcass by carcass slaughter inspections and more on outcome measurement"

See section 5.2, Finding 5.2. The expert panel provided considerable detail regarding this point.

The Modernized Slaughter Inspection Program, described in Finding 5.2, is both risk and outcome based, and moves the Agency's inspection regime away from prescriptive regulatory requirements towards maintenance of objective performance standards. The resulting reduction in CVS task requirements would provide the CFIA with significant opportunity to move inspection support to other commodities whose record of causing food borne illness outbreaks are comparable to that of meat, as discussed below under Finding 16.2.

Possible Change 2
"More risk-based CVS tasks"

See section 5.2, Findings 5.3 and 5.4. The CFIA was developing an integrated risk management approach at the time of this report's writing, including an Establishment-based Risk Assessment (ERA) model that assesses all food facility risk based on 15 criteria.^Footnote 66 It considers:

• Inherent risk factors, or risks associated with a specific food commodity, operation, or manufacturing process (e.g., the type of product, production volume, and the target population, such as residents of nursing homes or infants who are fed formula);
• Mitigation factors, or measures or strategies applied by a regulated party to further reduce the risk posed by a specific hazard, and to ultimately reduce the likelihood of the occurrence of a food-safety issue (e.g., putting in place an internationally recognized preventive control plan, having an employee on-site responsible for quality assurance, and introducing specific preventive processes); and
• Compliance factors, or establishment track records, which refer to both historical and current data collected by the CFIA on a regulated party's ability to successfully implement its preventive control plans, mitigation strategies, and compliance with regulatory requirements (e.g., the inspection results, the history of enforcement actions, and the food safety related recalls of class 1, 2 and 3).

The ERA model is intended to allow the CFIA to apply greater precision to the way it designs programs, prioritizes its oversight activities, allocates resources, and drafts annual work plans. It should also mean a more consistent, predictable, and transparent processes for regulated parties.

An outcome-based inspection proposal has been under development that redesigns the current approach to CVS in federally registered meat establishments. It is intended to focus on the outcome of the regulated party's mandatory food safety control programs, e.g., HACCP, which aligns with the requirements of the preventive control plan as outlined in the new integrated Agency Inspection Model. This Outcome-Based Inspection Proposal is to remove the prescribed task frequency for 70 of the current 156 CVS meat tasks so that delivery becomes "as required." The inspector would be required to complete a thorough monthly inspection of the establishment to assess the outcome of the regulated party's food safety programs. The results of the monthly inspection would act as an indicator of the outcome of the regulated party's system, e.g., if no deviations are observed, it suggests the food safety programs are likely working and none of the 70 tasks would need to be delivered at that time. If a deviation or non-compliance is observed during the monthly inspection, it would indicate a weakness in the regulated party's food safety programs and the applicable task will then be required to be delivered by the inspector. This approach focuses an inspector's time on assessing and documenting non-compliance(s) based on the observed indicators, instead of conducting all tasks at a prescribed frequency.

Examples of the 70 CVS tasks with an "as required" frequency would be inspections of: outside property; lighting; the pest control programs; ventilation; waste and inedible disposal; employees facilities; hand-washing stations and sanitizing installations, and; building design and construction. Other tasks would maintain a minimum prescribed frequency when the tasks are considered trade sensitive (e.g., Specified Risk Material, U.S. Food Safety Inspection Service export requirement) or when the outcome cannot be fully assessed through visual observation (e.g., recall program). The remaining CVS tasks (156-70 = 86 tasks) would continue to be delivered at prescribed frequencies. Examples of CVS tasks with a prescribed frequency are: pre-operational sanitation on-site – RTE; pre-shipment requirements, and; humane receiving and handling of animals (all species).

If all the risk-based and outcome-based criteria outlined in the two Possible Changes above were applied, significant resources could be made available for higher risk activities.

Possible Change 3
"Reduced site visits for equivalency determinations"

See section 5.4, Finding 7.1 and section 5.5, Finding 8.2. The evident duplication of some of the CFIA's import inspection activities with those of the exporting country presents opportunities for reduced site visits in those countries with strong food safety records.

Possible Change 4
"Reduced organoleptic inspections in import inspections"

See section 5.5, Finding 8.1. Both the expert panel and the practices of other countries indicate the low utility of organoleptic inspections. In addition, the data review provides evidence that the current amount of import inspection effort is more than necessary.

7.2 Q 16: Are there less costly ways for CFIA to achieve the intended results of the Meat Programs?

The evaluation calculated it costs approximately $3 million per year to implement DSIP at federally registered plants that are not eligible to export to the U.S.^Footnote 67 These costs could be reduced without affecting domestic food safety or Canada's ability to export to the U.S.

As described in Section 2.4 and Finding 10.1, DSIP was implemented to support continued meat exports to the U.S. This requirement is applied to non-exporting meat plants in Canada to ensure inspection consistency. As well, the U.S. requires domestic establishments that do not export but supply meat products to exporting establishments to meet DSIP requirements.

DSIP is not a risk-based system, not based on food safety, nor in line with inspection modernization. However, removing DSIP from some non-exporting plants could have a negative impact on public perceptions of meat safety in Canada.^Footnote 68 An example is a March 31, 2015 CBC article "Meat sold in Canada not inspected as well as exports: union; Food safety experts disagree that fewer inspectors increases risk of contamination."^Footnote 69 The perception that some domestic meat product is not inspected as well as meat for export is based on the misperception that having an inspector present daily provides greater oversight than having an inspector show up less frequently, but for longer periods. The Evaluation of Daily Shift Inspection Presence^Footnote 70 concluded there is no evidence to support this. There is also some evidence daily inspection is less effective, in part because of the greater predictability of daily versus intermittent and irregular inspection presence.

As noted in Finding 15.1, some of the possible changes presented would lead to cost reductions, and none would decrease program effectiveness.

The following, additional possible program changes were studied in relation to this evaluation question.

Greater use of third parties: It was determined that third parties are already being used to a great extent, for example for beef grading and lab analyses done by accredited private labs. The main factor inhibiting the greater use of third parties is international agreements which require government inspectors to perform certain activities.

Increases to cost recovery rates: The CFIA is in the process of reviewing and updating all of its service standards and user fees. Prior to amending the CFIA Fees Notice, the Agency will be considering:

• the mix of public and private benefits
• economic impacts on service recipients
• international comparisons
• feedback provided by stakeholders

One such consideration is the U.S. Food Safety Inspection Service practice of only charging for inspections that are requested by industry for periods outside of regular business hours. Over the past five years, CFIA cost-recovery revenues from federally registered meat establishments have averaged $16 million per year, or about seven per cent of the cost of the Meat Programs.

Reducing the cost to the CFIA of exporting to the U.S. This is discussed below.

The three major CFIA costs of exporting to the U.S., over and above Canada's normal meat inspection processes, are:

• The cost of Daily Shift Inspection Presence
• The cost of pre-shipment reviews
• The cost of increased testing for Listeria

The costs of the latter two are relatively minor – approximately $0.5 million per year each – and, in any case, they are necessary in order for exports to the US to continue,^Footnote 71 and the cost of DSIP is addressed above.

The evaluation included a review of studies related to the cost of foodborne illnesses caused by pathogens with known linkages to meat products. In the studies reviewed, the estimated annual costs of these illnesses in Canada are in the neighbourhood of $1 billion.^Footnote 72 The annual cost of the Meat Programs over the study period was approximately $260 million.

However, there are a number of qualifications to these studies:

1. They are based on foodborne illnesses due to pathogens "linked" to meat products, which is not the same as foodborne illnesses "attributable" to meat products. For example, Listeria can be found in fruit, vegetables, dairy products, and other food products. Similarly, contaminated eggs are a major source of illnesses linked to Salmonella.
2. A common finding among these studies was it was not always possible to identify the cause of the contamination that led to the illnesses. While it is possible products that led to illness where contaminated during production, it is also possible they were contaminated afterwards, due to improper handling by retailers, consumers, restaurants, etc.
3. Although "cost per case of illness" is the basis of the conclusions of these studies, it is the costs of deaths resulting from illnesses that have the greatest effect on the overall estimates. For example, Listeria, which has a high mortality rate,^Footnote 73 is just one of 11 pathogens examined in the CFIA study referenced above; however, it accounts for 32 per cent of the total estimated annual costs of illnesses. More problematic is the fact that, although costs of illnesses that do not result in death are straightforward to estimate (mainly healthcare costs and costs of lost productivity), this is not true of the cost of a death (i.e., the value a life). Countless articles have been written by economists on how to determine the "statistical value of life," but it remains contentious.^Footnote 74 However, the Treasury Board of Canada Secretariat Canadian Cost Benefit Analysis Guide; Regulatory Proposals references studies by the US Environmental Protection Agency and others to provide a 2015 adjusted figure of $7.4 million for what is termed the value of a statistical life.^Footnote 75
4. These studies are based on the costs of illnesses, but they do not include the costs of trade restrictions when pathogens are identified in Canadian meat products. These costs can be significant. For example, the detection of bovine spongiform encephalopathy (BSE) in Canada in 2003 led to a closure of borders for Canadian beef in many international markets,^Footnote 76 and beef exports are on the order of $2 billion per year.^Footnote 77

The implication of qualifications 1 and 2 is that the conclusions of these studies are an over-estimate of the economic benefits of the Meat Programs. The implication of qualification 4 is that they are an under-estimate. The implication of qualification 3 is unclear. Nevertheless, the difference between the benefits indicated by the studies ($1 billion) versus the cost of the Meat Programs ($260 million) is large enough to justify the finding.

If this finding is true for the Meat Programs, it is certainly true for other CFIA commodity food safety programs. For example, in a major study carried out recently at the U.S. Centers for Disease Control,^Footnote 78 more illnesses were attributed to plant commodities than animal commodities, and 46 per cent of illnesses were attributed to produce. More illnesses were attributed to leafy vegetables than to any other specific commodity. Illnesses associated with leafy vegetables were the second-most frequent cause of hospitalizations and the fifth-most frequent cause of death. These findings indicate hazards in fruit and vegetable products may be as much, or more of a concern than those in meat products. The annual cost of CFIA's Fruit and Vegetable Programs during the study period was only about 12 per cent of the cost of the Meat Programs.

8.0 Conclusions and Recommendations

8.1 Conclusions

Overall, the findings of the evaluation are very positive. The Meat Programs are ensuring the safety of meat products to a very high degree–both products distributed in Canada and exported products.

All components of the program were found to be effective. This includes:

• requirements and associated procedures for plant operation–in particular, the requirement for developing, implementing and maintaining HACCP plans;
• the system for slaughter inspections
• the system for processing inspections
• the system for sampling and testing domestically produced meat products
• the system for ensuring the safety of imported products (equivalency determinations and import inspections)
• the system for ensuring safety and compliance of exported products
• the recall process

The evaluation also examined the rationale for the Meat Programs and found:

• the Meat Programs address a legitimate need, and it is important that the Meat Programs be delivered by the federal government
• the Meat Programs are consistent with federal government and CFIA priorities
• roles of the CFIA, other federal partners, industry, and the provinces are clearly laid out, accepted, and appropriate

The delivery process for the Meat Programs was found to be effective, though with some areas of inefficiency

Possible improvements were identified in the delivery of Compliance Verification System (CVS) inspection tasks, performance measurement, and training materials for slaughter inspectors.

8.2 Recommendations

The evaluation recommends the following changes to improve program effectiveness and increase program efficiency:

1. For those HACCP plans marked by inspectors as unacceptable, the CFIA should clarify which ones represent serious issues.
2. To improve meat program efficiency, there should be less emphasis on carcass by carcass inspections in slaughter inspections done by the CFIA.
3. In import inspections, organoleptic (sensory) inspections should be reduced or eliminated.
4. The CFIA should coordinate its performance measurement reporting for its meat programs to provide comprehensive results reporting.
5. The CFIA should increase the degree to which CVS tasks are based on risk, including the historical compliance record.
6. The CFIA should examine opportunities to provide increased flexibility in the delivery of Daily Shift Inspection Presence requirements.

Annex A: Logic Models, Results Chains and Program Profile

The following logic model and results chain were constructed by the evaluation team and used to focus interviews and focus groups. These are constructed around the following four key functions: Imports, Exports, Domestic and Recalls. Only the domestic results chain is included here as it was the only one used during the focus groups. The other results chains are available upon request, but were not validated during the evaluation process.

Meat Programs Logic Model

Description for Diagram - Meat Programs Logic Model

This diagram describes the activities and outcomes of Meat Programs. On the left side of the diagram is a legend. The legend has the following text written in it from top to bottom:

• Strategic Outcomes
• Ultimate Outcomes
• Intermediate Outcomes
• Immediate Outcomes
• Outputs
• Activities

The logic model is organized the same as the legend with "Strategic Outcomes" at the top and "Activities" at the bottom.

Activities:

The Activities section has seven rectangles with blue borders, organized horizontally in one row. The seven rectangles contain the following titles from left to right:

• 1.1 Country system reviews and negotiations
• 1.2 Imported products inspected and samples tested
• 1.3 Engage in recall and consumer protection activities
• 1.4 Domestic products inspected and samples tested
• 1.5 Establishment licensing and inspection tasks
• 1.6 Engage in trade negotiations
• 1.7 Export certification

Above the Activities section is the Outputs section. There are arrows linking boxes in the Activities section to boxes in the Outputs section. Across the arrows is the following text: "Stakeholder Engagement".

Outputs:

The Outputs section has seven rectangles with blue borders, organized horizontally in one row. The seven rectangles contain the following titles from left to right:

• 2.1 Foreign establishment approvals and product certifications
• 2.2 Completed inspections and tests of imported products
• 2.3 Completed recall activities
• 2.4 Completed inspections and tests of domestic products
• 2.5 Completed establishment licensing and inspection tasks
• 2.6 Export agreements signed
• 2.7 Export certificates issued

There is an arrow pointing upwards from the Activities box "1.1 Country system reviews and negotiations" to the Outputs box "2.1 Foreign establishment approvals and product certifications".

There is an arrow pointing upwards from the Activities box "1.2 Imported products inspected and samples tested" to the Outputs box "2.2 Completed inspections and tests of imported products".

There is an arrow pointing upwards from the Activities box "1.3 Engage in recall and consumer protection activities" to the Outputs box "2.3 Completed recall activities".

There is an arrow pointing upwards from the Activities box "1.4 Domestic products inspected and samples tested" to the Outputs box "2.4 Completed inspections and tests of domestic products".

There is an arrow pointing upwards from the Activities box "1.5 Establishment licensing and inspection tasks" to the Outputs box "2.5 Completed establishment licensing and inspection tasks".

There is an arrow pointing upwards from the Activities box "1.6 Engage in trade negotiations" to the Outputs box "2.6 Export agreements signed".

There is an arrow pointing upwards from the Activities box "1.7 Export certification" to the Outputs box "2.7 Export certificates issued".

Above the Outputs section is the Immediate Outcomes section. There are arrows linking boxes in the Outputs section to boxes in the Immediate Outcomes section. Across the arrows is the following text: "Stakeholders aware of food safety requirements".

Immediate Outcomes:

The immediate outcomes section has four rectangles with blue borders, organized horizontally in one row. The four rectangles contain the following titles from left to right:

• 3.1 Imported products and operations of foreign establishment are compliant with Canadian requirements to enable access to Canadian market
• 3.2 Effective removal of unsafe products from the marketplace
• 3.3 Domestic products and operations of domestic establishments are compliant with Canadian requirements
• 3.4 International market accessible to Canadian exporters

There is an arrow pointing upwards from the Outputs boxes "2.1 Foreign establishment approvals and product certification" and "2.2 Completed inspections and tests of imported products" to the Immediate Outcomes box "3.1 Imported products and operations of foreign establishment are compliant with Canadian requirements to enable access to Canadian market".

There is an arrow pointing upwards from the Outputs box "2.3 Completed recall activities" to the Immediate Outcomes box "3.2 Effective removal of unsafe products from the marketplace".

There is an arrow pointing upwards from the Outputs boxes "2.4 Completed inspections and tests of domestic products" and "2.5 Completed establishment licensing and inspection tasks" to the Immediate Outcomes box "3.3 Domestic products and operations of domestic establishments are compliant with Canadian requirements".

There is an arrow pointing upwards from the Outputs boxes "2.6 Export agreements signed" and "2.7 Export certificates issued" to the Immediate Outcomes box "3.4 International market accessible to Canadian exporters".

Above the Immediate Outcomes section is the Intermediate Outcomes section. There are arrows linking boxes in the Immediate Outcomes section to boxes in the Intermediate Outcomes section. Across the arrows is the following text: "Stakeholders supportive of food safety requirements".

Intermediate Outcomes:

The Intermediate Outcomes section has two rectangles with blue borders, organized horizontally in one row. The two rectangles contain the following titles from left to right:

• Federal food safety criteria are met
• Canadian interests are reflected in science-based regulations, standards and arrangements of international regulatory bodies

There is an arrow pointing upwards from the Immediate Outcomes box "3.1 Imported products and operations of foreign establishment are compliant with Canadian requirements to enable access to Canadian market" to the Intermediate Outcomes box "4.1 Federal food safety criteria are met".

There is an arrow pointing upwards from the Immediate Outcomes box "3.2 Effective removal of unsafe products from the marketplace" to both Intermediate Outcomes boxes: "4.1 Federal food safety criteria are met" and "4.2 Canadian interests are reflected in science-based regulations, standards and arrangements of international regulatory bodies".

There is an arrow pointing upwards from the Immediate Outcomes box "3.3 Domestic products and operations of domestic establishments are compliant with Canadian requirements" to the Intermediate Outcomes box "4.1 Federal food safety criteria are met".

There is an arrow pointing upwards from the Immediate Outcomes box "3.4 International market accessible to Canadian exporters" to the Intermediate Outcomes box "4.2 Canadian interests are reflected in science-based regulations, standards and arrangements of international regulatory bodies".

Above the Intermediate Outcomes section is the Ultimate Outcomes section. There are arrows linking boxes in the Intermediate Outcomes section to boxes in the Ultimate Outcomes section.

Ultimate Outcomes:

The Ultimate Outcomes section has two rectangles with blue borders, organized horizontally in one row. The two rectangles contain the following titles from left to right:

• 5.1 Domestic and imported products are fairly labelled, safe and suitable for human consumption - Acceptance of domestic and foreign food safety systems
• 5.2 A competitive and market-oriented meat industry (sector competitiveness)

There is an arrow pointing upwards from the Intermediate Outcomes box "4.1 Federal food safety criteria are met" to the Ultimate Outcomes box "5.1 Domestic and imported products are fairly labelled, safe and suitable for human consumption - Acceptance of domestic and foreign food safety systems".

There is an arrow pointing upwards from the Intermediate Outcomes box "4.2 Canadian interests are reflected in science-based regulations, standards and arrangements of international regulatory bodies" to the Ultimate Outcomes box "5.2 A competitive and market-oriented meat industry (sector competitiveness)".

Above the Ultimate Outcomes section is the Strategic Outcomes section. There are arrows linking boxes in the Ultimate Outcomes section to boxes in the Strategic Outcomes section.

Strategic Outcomes:

The Strategic Outcomes section is at the top of the diagram, and has two rectangles with blue borders, organized horizontally in one row. The two rectangles contain the following titles from left to right:

• A safe and accessible meat supply and animal resource base
• A strengthened Canadian economy

There is an arrow pointing upwards from the Ultimate Outcomes box "5.1 Domestic and imported products are fairly labelled, safe and suitable for human consumption - Acceptance of domestic and foreign food safety systems" to the Strategic Outcomes box "6.1 A safe and accessible meat supply and animal resource base".

There is an arrow pointing upwards from the Ultimate Outcomes box "5.2 A competitive and market-oriented meat industry (sector competitiveness)" to the Strategic Outcomes box "6.2 A strengthened Canadian economy".

Meat Programs Results Chain

Description for Diagram - Meat Programs Results Chain

This diagram describes the assumptions/external factors that are necessary for the Meat Programs "results chain" to take place. More specifically, such assumptions would give the conditions for program "activities" to ultimately result in the program's "strategic outcome".

On the top of the diagram is the title: "Domestic". On the left side of the diagram is a legend. The legend has the following text written in it from top to bottom:

• Strategic Outcomes
• Ultimate Outcomes
• Intermediate Outcomes
• Immediate Outcomes
• Outputs
• Activities

Moving right from the legend there is a column with the title "Results Chain". Below the title there are six rectangles with blue borders organized vertically. Each rectangle corresponds to the legend on its left side. The six rectangles contain the following titles from top to bottom:

• A safe meat supply and animal resource base
• Domestic products are fairly labelled and safe for human consumption; Acceptance of domestic food safety systems
• Federal food safety criteria are met
• Domestic products and operations of domestic establishments are compliant with Canadian requirements
• Domestic establishments licensing and inspection tasks and product inspections and sampling and testing are completed
• Domestic products inspected and samples tested; Domestic establishment licensing inspection tasks

There are five blue arrows pointing upwards linking each rectangle to the one on the top. To the left of each arrow there are the words "if" next to the base, and "then" next to the top of the arrow.

Moving right there is a column with the title "Assumptions/External Factors". Below the title there are five rectangles with blue dotted borders organized vertically. There is an arrow coming from each rectangle and pointing left to the space in between rectangles in the column on the left. The five rectangles contain the following titles from top to bottom:

• Economic, marketplace, social, cultural, technological, environmental and political conditions enable marketplace outcomes; Existence of adequate and appropriate monitoring schemes and systems
• Federal food safety criteria are adequate to ensure food is safe for human consumption
• Canadian requirements are appropriately designed to ensure that, if they are met, the products allowed to be sold in the marketplace satisfy federal food safety criteria
• Domestic products inspection and testing and sampling procedures and requirements are appropriately designed and properly implemented; Domestic establishment operators are aware of the requirements and procedures for carrying out inspection and certification activities and are committed to the process; Domestic operators are willing to provide inspectors with the required and appropriate documentation needed for the inspection
• Availability of resources and adequate competencies of CFIA inspectors to carry out activities related to the inspection of domestic meat products and domestic systems and establishments; Stakeholder (e.g. Associations) support.

Program Profile

The objectives of the Meat Programs that are described in CFIA documentation^Footnote 79 are to ensure:

1. registration, licensing, and operators' control programs produce, pack, and store, in a hygienic way, safe and suitable meat products, fairly labelled for the intent to use in Canadian and international markets (and as necessary, initiate an effective recall of meat products from the market);
2. ante mortem and post mortem examination and inspection of animals and meat products allow sick or otherwise unfit animals and adulterated or otherwise unsuitable meat products to be identified, removed, segregated, and disposed appropriately;
3. standards and identification of edible and non-edible meat products are met;
4. food animals are handled, slaughtered, or killed under humane conditions; and
5. meat products are imported under equivalent Canadian standards.

I. Domestically produced and distributed products

I.1 Requirements for plant operations

CFIA requirements for domestic plant operations are based on the principles of HACCP systems developed by the Codex Alimentarius Commission (Codex). HACCP is an internationally recognized, science-based food safety system, designed to prevent, reduce or eliminate potential biological, chemical, and physical food safety hazards.

A HACCP system includes the following:

• Prerequisite programs – basic conditions and activities that are necessary to maintain a hygienic environment and good manufacturing practices throughout the establishment;
• A HACCP plan – a written document outlining procedures for controlling hazards associated within specific processes within the establishment; and
• Validation documentation of control measures and required maintenance and reassessment procedures.

The development and implementation of a HACCP plan is the responsibility of the operator of a federally registered establishment. The responsibilities of the CFIA include:

• Formally recognizing the HACCP plans of licenced operators of federally registered establishments, and;
• Verifying the implementation, effectiveness, and maintenance of the HACCP system for each operator on an on-going basis, and, at a minimum once every two years.

Details on these requirements are included in Annex H.

I.2 Requirements for inspections, sampling, and testing

We have divided the description of inspections into two parts:

a) Slaughter inspections

These are inspections that take place up to the start of the carcass chilling process. Their purpose is to ensure animals are suitable for slaughter and the derived carcasses and parts are fit for human consumption. The activities are either done directly by a CFIA employee, or in the case of "shared inspection programs," done by the operator of the registered establishment under CFIA oversight.^Footnote 80 The shared inspection program is used in all poultry slaughter facilities, where it is called the Modernized Poultry Inspection Program, but has not been implemented in other meat programs (beef, pork, etc).

ai) Pre-slaughter inspections ("ante mortem inspections" or "screening") are intended to identify animals that show deviations from normal behaviour or appearance that could render the carcass unfit for human consumption and to identify contaminated animals.

aii) Post-slaughter inspections ("post-mortem inspections") of red meat carcasses are generally based on the examination of heads and their lymph nodes, thoracic and abdominal viscera and their lymph nodes, and the exposed parts of the carcass. Post-slaughter inspections of chickens include checking that presentation standards are met (e.g., viscera parts not missing, no visceral parts left in the cavity) and examining for defects (e.g., dark coloured carcasses, viscera defects, cavity defects). CFIA inspectors work alongside plant staff on a full time basis in the plants.

b) Processing inspections

This category includes verification of all requirements that are not associated with slaughter inspections, including everything after the carcass chilling process. The vast majority of processing inspections are carried out by the company. Company inspectors are responsible for assessing compliance with critical control point requirements, prerequisite program requirements, and other program requirements.

In order to verify that these inspections are fully and correctly carried out, CFIA inspectors conduct verification activities using the Compliance Verification System (CVS), which is a standardized risk-based inspection tool that lays out verification tasks with detailed procedures and guidance for inspectors on the verification process. CVS is also used for some slaughter inspection activities.

II. Imported products

II.1 Equivalency determinations

Before a foreign country is permitted to export meat products to Canada, the country's foreign competent authority (FCA) for meat inspection must be deemed to have a system at least equivalent to that of Canada.

The CFIA evaluates foreign meat inspection and certification systems for equivalency through a system of document reviews and on-site assessments. The system approval may cover all species of food animals or it may be limited to the species of interest at the time of application.

The approved foreign country meat inspection systems, as well as the establishments approved for export to Canada, are periodically re-assessed to ensure that the approval continues to be granted only to the countries that continue to have and enforce meat inspection systems equivalent to Canada.

II.2 The import inspection program

The import inspection program consists of port of entry control and several types of inspections.

Point of entry control

All shipments of meat products being imported to Canada must be accompanied by a valid Official Meat Inspection Certificate (OMIC) for export of meat products to Canada, issued by the FCA of the exporting country. In addition to specifying the country of origin of the meat products, the OMIC lists the slaughter, processing and exporting establishments involved in the production and export to Canada. The Canadian Border Services Agency (CBSA) refers all shipments of meat products to the CFIA for decision before allowing products to enter Canada, unless they are considered "in transit" (i.e., just passing through Canada).

Inspection of imported meat shipments

There are three types of import inspections:

• Visual inspections
• Organoleptic inspections
• Microbiological inspections.

Visual inspections: Upon arrival in the establishment that is registered for inspection purposes, the CFIA inspector checks for the presence of seals on transport containers and verifies numbers against those on the OMIC. The CFIA Inspector also checks acceptability of the labelling and marking and verifies the product being presented for inspection is the same and of the same quantity as is certified.

Organoleptic inspections: The term "organoleptic inspection" means a physical examination of a representative number of sample units (e.g., cartons, carcasses, combo bins), drawn at the end of the visual inspection, using the senses of touch, smell, sight to determine the wholesomeness and cleanliness of a meat product.

The program for the organoleptic inspection of meat products imported from the US varies from the program for meat products imported from all other countries (referred to as "offshore"), because of agreements made following the 1988 conclusion of the US-Canada Free Trade Agreement. Also, the intensity of required inspections for all countries varies with the number of shipments that have been previously successfully exported to Canada by the exporting establishment, as illustrated in the table below.

Microbiological inspections: Approximately nine per cent of all imported meat product loads receive microbiological inspection, which involve laboratory testing to confirm the absence of harmful pathogens (above acceptable levels).

III. Exported products

III.1 Requirements for plant operations

Products intended for export are produced in plants governed by the same CFIA regulations and guidelines as products intended for domestic distribution – in particular, the requirements related to HACCP systems (see section 2.2.1). It should be noted that, in addition to meeting Canadian requirements, export-oriented products must be produced in plants that meet the operating requirements of the importing countries. In most cases, the plant operating requirements of the importing country are similar to Canadian requirements.^Footnote 81 Certain countries, however, have special requirements that are either in addition to, or are different from, Canadian requirements – e.g., related to construction, hygiene, or transport of meat products.

III.2 Requirements for inspections, sampling, and testing

As with plant operations, export-oriented products are subject to the same inspection processes as domestically distributed products (see section I.2). And similarly, the inspection procedures also have to meet the inspection requirements of the importing countries. A particular case in point is the United States.

U.S. meat and poultry legislation requires inspection of meat establishments on a continuous basis. This is referred to as "daily shift inspection presence" (DSIP) and means that establishments operating two shifts 24 hours would be inspected twice daily.^Footnote 82 In 2010, Canada adopted DSIP at all federally registered meat processing establishments to meet U.S. standards for the import of processed meat products, thus ensuring continued exports of Canadian meat to the U.S.^Footnote 83

All exported products are certified by the CFIA as having been prepared according to the requirements of the importing country.

IV. Recalls

Several triggers can start a food safety investigation, which could lead to a food recall. These include an illness outbreak, findings from inspections or food tests, consumer complaints, and recalls in other countries.

The food safety investigation and recall process involves four steps.

1. Food safety investigation -- Food safety investigations are complex and involve several essential steps to determine if a food product represents a potential health risk and if a recall is required. Investigations are carried out on-site using a trace-back/trace-forward approach.
2. Health risk assessment -- If a potential health risk is identified, the CFIA may submit a formal request for a health risk assessment to Health Canada. The purpose is to determine the level of health risk a specific food represents for the Canadian population.
3. Recall Process -- It is the responsibility of the company to remove potentially unsafe food from sale or distribution. The CFIA's role is to inform the public, oversee implementation of the recall, and verify that industry has removed recalled products from store shelves.
4. Follow-up -- After recalled product is removed from the marketplace, the CFIA continues to work with the processor, manufacturer or importer to ensure that any problems that led to the recall are resolved.

V. Role of Other Government Departments

The role of other federal government departments related to the meat programs lies primarily with Health Canada and the Public Health Agency of Canada.

Health Canada

The primary responsibility of Health Canada is to establish food safety standards via the Food and Drugs Act. They also set standards and policy for microbiological pathogens, chemical contaminants, additives, etc. Canadian food safety criteria are developed and continuously updated by Health Canada, based on extensive scientific evidence.

Health Canada conducts Health Risk Assessments (HRAs) at the request of CFIA, which then uses these HRAs to make decisions on food recalls as part of its responsibility for enforcing food-related standards. HRAs involve determining if the presence of a certain substance or micro-organism in food (e.g., chemical contaminant, natural toxin, allergen, unproved food additive, bacteria, virus, or parasite) poses a health risk to consumers.

Other responsibilities of Health Canada that have a tangential relationship to the Meat Programs include:

• Health Canada's pre-market assessment of industry food submissions. (An industry food submission is a request to Health Canada from an importer or manufacturer to market a new food product in Canada.)
• Health Canada's Veterinary Drugs Program, which has a number of collaborative arrangements with CFIA, some of which relate to meat and poultry (e.g., the Program is responsible for setting standards regarding veterinary drug residues in food).

Public Health Agency of Canada (PHAC)

PHAC's main responsibility related to the Meat Programs is to collaborate with CFIA in relation to human illness outbreaks that may be linked to food safety. PHAC collaborates in the management of outbreak investigations related to human illness. HC, CFIA and PHAC respond to outbreaks through the Foodborne Illness Outbreak Response Protocol (FIORP) with the establishment of an Outbreak Investigation Coordinating Committee. The FIORP outlines the role of each federal department and agency in an outbreak situation, as well as the roles of the provinces and territories. It also lays out provisions regarding information sharing and communications.

More general activities of PHAC related to food safety include national and international surveillance (e.g., antimicrobial resistance and enteric illness surveillance), and laboratory and epidemiological investigations.

VI. Organization of the Meat Programs

The following is a brief description of the organization of the Meat Programs. (This refers to the study period. As discussed in section 6.2, there have been some changes as a result of Agency Transformation.)

The Meat Programs are headquartered in Ottawa and are headed by the Director, Meat Programs Division, in the Policy and Programs Branch (PPB), but also by various offices in the Operations and Science Branches. In the PPB, there are three National Managers reporting to the Director, one each for, National Meat Inspection Procedures, International Programs, and Program Planning and Performance Management. The first two of these are responsible for program and policy development in their respective areas of responsibility.

Reporting to the National Manager, National Meat Inspection Procedures, are specialists in each of the relevant areas – slaughter, meat processing, food microbiology, and so on. Reporting to the National Manger, International Programs, are specialists in export programs and import programs.

The Meat Programs are delivered by CFIA's 18 administrative regions,^Footnote 84 and each Regional Director is responsible for the program in his/her region. Reporting to the Regional Director are Inspection Managers, Food Processing Supervisors and Regional Veterinary Officers.^Footnote 85 The overall responsibility for program delivery within each area lies with the Area Executive Director, Operations.

Annex B: Evaluation Methodology Limitations and Mitigation Strategies

The foundation of the evaluation questions were based on the Treasury Board's 2009 Directive on Evaluation:

Annex C: Examples of Interview Guides, plus the Survey Questionnaire

Available upon request.

Annex D: Members of the Expert Panel

Note: Numbers in brackets indicate number of conference calls attended. Individual roles listed are their roles at the time of the operation of the panel.

Panel members

Bonnie Buntain [3], Panel Chairperson, Retired Professor of Public Health, University of Calgary and former Chief Public Health Veterinarian, US Food Safety Inspection Service (currently serving as Acting Professor, University of Arizona)

Richard Arsenault [3], Executive Director, Domestic Food Safety Systems and Meat Hygiene Directorate, CFIA

Peter Stein [3], Corporate Director, Quality Assurance and Food Safety, Piller's Fine Foods

Michael Doyle [2], Professor of Food Microbiology and Director, Center for Food Safety, University of Georgia

Mark Klassen [2], Director of Technical Services, Canadian Cattlemen's Association (also provided written comments on the subjects of the call he missed)

Baden Pearse [2], Field Operations Manager, Exports Division, Australia Department of Agriculture

Glen Edmunds [1, and also one introductory meeting accompanying Baden Pearse], Senior Scientist, Exports Division, Australia Department of Agriculture

Lynn McMullen [3], Professor of Food and Nutritional Science, University of Alberta

Ex-officio attendees

Eric Seraphim [3], Evaluation Manager, CFIA (and manager of the evaluation of the CFIA Meat Programs)

Doug Williams [3], Senior Associate, KPMG (Panel Secretary and manager of the work carried out by KPMG on the evaluation)

Jason Russo [1], Manager, KPMG (principal analyst on the KPMG research team)

Annex E: Additional Findings: Inspection Verification and Listeria Enhancements

Available upon request.

Annex F: Microbial Sampling: Planned vs Actual

Available upon request.

Annex G: CFIA Meat Program Budget Tables

Available upon request.

Annex H: Responsibilities of the CFIA Related to HACCP

Prerequisite programs

There are seven categories of prerequisite programs, which include such things as: Premises (e.g., the building must be protected against potential sources of external contaminants), Personnel (e.g., requirements for training in food hygiene), Sanitation and Pest Control, and Recall Procedures.

HACCP Plans

The heart of a HACCP plan is identifying critical control points (CCPs) – points or steps in the production process where the process needs to be specified in detail or an action needs to be taken in order to prevent or eliminate a food safety hazard or reduce it to an acceptable level. CCPs are identified following a detailed analysis of potential hazards in the establishment that are not adequately controlled by prerequisite programs. For example, the cooking step is considered a CCP because control measures are necessary to deal with the potential hazard of pathogens surviving the cooking process.

Critical limits are specified for each CCP. Critical limits are criteria that separate acceptability from unacceptability. These parameters, if properly maintained, ensure the safety of the product. For example, critical limits at the cooking stage could include the specific time and temperature for cooking the product. The HACCP plan also includes monitoring procedures for each CCP to make sure the process continues to operate within the critical limits.

Finally, HACCP plans have to include validation procedures – methods, procedures, sampling, tests, and other evaluations (in addition to monitoring) to determine whether a control measure at a CCP is operating as intended.

CFIA HACCP system reviews

In order to verify that the operators' HACCP Systems meets the regulatory requirements, CFIA inspectors conduct verification activities using the Compliance Verification System (CVS), which is a standardized risk-based inspection tool that lays out verification tasks with detailed procedures and guidance for inspectors on the verification process.

Daily, CFIA inspectors verify the implementation and effectiveness of the operators' HACCP Systems as per defined national frequency which may vary from daily to once every two years depending on the risk associated with the verification task and historical compliance records.

Once every two years and whenever an operator submits a new HACCP plan (following any significant changes) or follow-up is required after a food safety recall, a CFIA team that is led by an Food Safety Enhancement Program Specialist verifies that the operator's HACCP System:

• is designed to effectively control food safety hazards;
• meets the Food Safety Enhancement Program (FSEP) manual and program requirements, and;
• is reassessed to ensure food safety hazards remain under control.

The in-depth review tasks are designed to focus on specific hazards of concern. These tasks are conducted in situations such as repetitive unsatisfactory test results indicating there may be a health risk associated with a product, or where there is confirmed illness/death cases related to food.

Annex I: Meat export data

Available upon request.

Annex J: Research questions addressed in the evaluation

Available upon request.