Archived - Evaluation of Administrative Monetary Penalties

This page has been archived

Information identified as archived is provided for reference, research or record-keeping purposes. It is not subject to the Government of Canada Web Standards and has not been altered or updated since it was archived. Please contact us to request a format other than those available.

Management Response and Action Plan

April 2012

Executive Summary
List of Abbreviations
Preamble
Introduction to the Evaluation of AMPs
1.0 Need
2.0 Alignment with Government Priorities
3.0 Alignment with Federal Roles and Responsibilities
4.0 Achievement of Expected Outcomes
5.0 Demonstration of Efficiency and Economy
6.0 Conclusions, Good Practices and Recommendations
Annex A: AMPs Results Logic – Expected vs. Actual
Annex B: Evaluation Framework, Methodology and Limitations
Annex C: Bibliography
Annex D: Interview Guides

Executive Summary

Administrative Monetary Penalties (AMPs) are intended to address non-compliance in areas in which corrective action requests and simple warnings are not enough, but where prosecution, seizure, or license suspension or revocation are considered too harsh. AMPs at the Canadian Food Inspection Agency (CFIA) are currently applied to select areas of the Health of Animals Act (HAA) and Health of Animals Regulations (HAR) and of the Plant Protection Act and Plant Protection Regulations.

Senior management in the Operations Branch, and the Programs and Policy Branch of the CFIA requested an evaluation of AMPs to assess their effectiveness and efficiency, and provide guidance on the merits of expanding their application to other acts and regulations, as the AMPs Act currently allows. The evaluation assessed the relevance and performance of AMPs, conducting literature, document, file and case reviews, and analysis, in addition to interviews. The scope of the study included the application of AMPs at the CFIA from 2000 to 2010.

Conclusions

AMPs are an effective compliance tool that has been successfully employed at the CFIA as an appropriately-scaled response for cases in which prosecutions would be excessively time consuming and costly.

Effective Implementation

Existing inefficiencies and obstacles to the effective application of the AMPs tool will be amplified with any expanded use of AMPs for other acts and regulations. Unclear legislative and regulatory wording, leading to uncertainty in interpretation, has contributed to increased risk of appeal. As well, the collection of evidence without regard to risk of appeal limits capacity for reprioritization of resources.

The lack of clear objectives, systematic measurement systems, and periodic reviews, which would have permitted systematic learning from the early application of AMPs, has limited improvements in the use of the tool, and reduced certainty about its value. Additionally, the lack of reporting capability has limited the opportunity to build learning into delivery and to provide managers with information on the issues causing the greatest challenges for inspectors and investigators.

The use of inspectors for less complex and lower risk of appeal cases would allow for additional capacity if AMPs were expanded.

Recommendation 1

The VP, Operations Branch, should establish a plan to build learning into the delivery of current AMPs for their ongoing development, and for any expansion of the use of the tool.

The plan should include the following elements:

a strategy to calibrate the level of investigation according to the risk of appeal;
a performance measurement strategy; and
a plan to incorporate on-going performance measurement in the development of training of and communication with inspectors and investigators across areas.

Strategic and Appropriate Use

There was limited integration of the AMPs tool into the CFIA's compliance strategies and little evidence of linkages between Programs and Policy Branch, and Operations Branch that would allow program and enforcement staff to identify problem areas early on, and adapt enforcement strategies according to the specifics of each case.

Key conditions, demonstrated internationally and at the CFIA, affect the success of administrative monetary penalty instruments (e.g., clarity of objectives, legislative wording, and inspection and enforcement roles). Consistency in understanding of the roles and obligations of all players, from regulated parties to government staff is also essential.

Recommendation 2

The VP, Operations, should work with Policy and Programs Branch, with guidance from legal services, to ensure that all existing AMPs are operating under the key conditions identified, and that AMPs processes, guidelines and strategies are included in program specific enforcement strategies, where relevant.

List of Abbreviations

ALRC: Australian Law Reform Commission
AMPs: Administrative Monetary Penalties
CART: Canada Agricultural Review Tribunal
CDSR: Cabinet Directive on Streamlining Regulations
CFIA: Canadian Food Inspection Agency
CLT: Certified Livestock Transport
EIS: Enforcement and Investigation Services
FCA: Federal Court of Appeal
HAA: Health of Animals Act
HAR: Health of Animals Regulations
ID: Identification
IT: Information Technology
NETS: National Enforcement Tracking System
PPA: Plant Protection Act
PPR: Plant Protection Regulations
RIAS: Regulatory Impact Analysis Statement
SCC: Supreme Court of Canada
US: United States
VP: Vice-President

Preamble

Introduction to the Evaluation of AMPs

The CFIA's Operations Branch, and Policy and Programs Branch requested that an evaluation of AMPs at the CFIA be carried out by the Audit, Evaluation and Risk Oversight (AERO) Branch. The request was part of a review of regulatory priorities which included consideration of the potential usefulness of expanding the application of the AMPs regulations to other Agency legislation.

Scope

The evaluation assessed the relevance and performance of AMPs at the CFIA since its introduction in 2000 through to 2010. The approach was to determine what works for whom under what conditions, focusing on how AMPs achieves outcomes in a given situation or context, with the objective of identifying areas where the use of this mechanism can be improved within the context of CFIA regulatory control and in consideration of any possible expansion of the use of AMPs to other program areas in the CFIA.

Methodology

Using interviews, a sample survey and file, document and literature review and analysis, the evaluation sought to assess the following:

the extent to which AMPs continue to address a demonstrable need and are responsive to the needs of Canadians
the linkages between AMPs objectives and (i) federal government priorities (e.g. CDSR) and (ii) departmental strategic outcomes (e.g. food safety and quality)
the role and responsibilities of the federal government in delivering the program
progress toward expected outcomes (including immediate, intermediate and ultimate outcomes) with reference to performance targets and reach, program design, including the linkage and contribution of outputs to outcomes
of resource utilization in relation to the production of outputs and progress toward expected outcomes

Details on the evaluation framework, methodology and study limitations are provided in Annex B.

How AMPs Work

The CFIA introduced its Administrative Monetary Penalties (AMPs) program in 2000, with partial implementation of the traveler stream, applying them in a select few airports. Over the years, the AMPs program has expanded to its current scope, which allows for the issuance of penalties or warnings at border points (administered by the Canada Border Services Agency) and within Canada (referred to as Commercial AMPs) for violations of scheduled sections of the Health of Animals Act, Health of Animals Regulations, Plant Protection Act and Plant Protection Regulations.

Under the Agriculture and Agri-Food Administrative Monetary Penalties Act and Regulations, the CFIA may issue an AMP as an enforcement measure to encourage compliance with the Health of Animals Act and the Plant Protection Act and their associated regulations. Its application typically begins with CFIA inspectors identifying possible violations of the relevant legislation and providing this information to CFIA investigators who may then issue an AMP with warning or with penalty.

Penalties range from $500 to $15,000, depending on the nature and gravity of the offence and the history of the offender.^Footnote 1 If an individual or a company receives a Notice of Violation, they may request a review of the facts. The review is made by the federal Minister of Agriculture and Agri-Food Canada or the Canada Agricultural Review Tribunal. In this report, reviews are also referred to as appeals.^Footnote 2

This report is organized around the evaluation questions as outlined in the evaluation framework, which is provided in Annex B.

Introduction

Sections from 1.1 to 5.3 below present the findings by evaluation question as originally developed for the evaluation framework. Section 6 provides the overall conclusions and recommendations.^Footnote 3

1.0 Need

1.1 What Needs are AMPs Supposed to Address?

AMPs are expected to fill a need for flexible sanctioning as part of the mid-range sanction^Footnote 4 of the compliance pyramid^Footnote 5.

Figure 1: Compliance Pyramid. Description follows. — **Figure 1: Compliance Pyramid^Footnote 6**

AMPs add a level of penalty based in administrative law before criminal law^Footnote 7 is invoked. This is discussed in some detail in M. Prabhu's 2010 doctoral thesis from the Faculty of Law at the University of Ottawa:^Footnote 8

"Using criminal law for even minor breaches of the Regulation has created unease and agitation among eminent scholars and regulation specialists, especially when a great many breaches are minor and overburden criminal courts…"

Furthermore, AMPs decriminalizes relatively minor transgressions. The distinction here between criminal and civil law is explored in J. Coffee's 1992 Yale Law Journal article: "Paradigms Lost: The Blurring of the Criminal and Civil Law Models - And What Can Be done About It":

I would suggest that in its characteristic operation, the civil law "prices", while the criminal law "sanctions'. … I would argue that the criminal law should be reserved to prohibiting conduct that society believes lacks any social utility, while civil penalties should be used to deter (or "price") many forms of misbehaviour (for example, negligence) where the regulated activity has positive social utility but is imposing externalities on others^Footnote 9.

1.2 AMPs at the CFIA

The original rationale for the use of AMPs is not fully articulated in any single document. A 2002 Regulatory Impact Analysis Statement (RIAS) suggests benefits in terms of cost effectiveness,^Footnote 10 using figures from the Province of Ontario Attorney General's Office regulatory prosecution costs, estimated at $13,000 per prosecution. The 2002 RIAS suggests that AMPs represented an appropriately-scaled response needed for cases where prosecution would be too time consuming and costly (RIAS for Cattle ID, 2002). The development of an AMP program for the CFIA began in the early 1990s, before the Agency was created. According to interviewees, prior to the establishment of AMPs, there was a belief among those involved with its implementation at the CFIA that AMPs would encourage compliance without prosecution, and that their use would improve efficiency – interviewees confirmed that this belief persists at the CFIA.

A link between AMPs and increased compliance was expressed in the 1997 RIAS: "The experience in the United States, where administrative monetary penalties are used extensively, is that compliance rates will go up considerably because of the deterrent impact of AMPs." AMPs were initially applied in 2000 to travelers entering Canada at airports, an application subsequently moved to the Canada Border Services Agency. However, interviews also suggest that there was concern from the beginning that AMPs might not work.

This caution on the part of various parties may help explain why AMPs were then applied to the Health of Animals Act and Health of Animals Regulations and to the Plant Protection Act and Plant Protection Regulations in 2002, but not applied as anticipated (i.e. within one year) to all the other acts included in the AMPs Act.^Footnote 11

1.3 Who is Affected?

Those affected by AMPs include the following:

Those subject to the Health of Animals Act (HAA) and Health of Animals Regulations (HAR), and the Plant Protection Act (PPA) and Plant Protection Regulations. (PPR)
Several key links of the food value chain are affected, especially animal transportation under HAA and HAR, and imported products and transported products for PPA and PPR.
General public through enhanced food safety and animal welfare
Agriculture sector through improved trade and consumer confidence

1.4 What Has the Reaction Been to Date?

So far, the industry reaction to the issuance of AMPs by the CFIA has been mild, as noted by interviewees.^Footnote 12 AMPs were not of very high interest to most of the industry associations contacted for this evaluation, though the most affected groups, as represented by association staff, note problems with consistency (for example, uncertainty regarding when AMPs are triggered), timing and fairness (such as the need for consistent treatment and consistent payment of penalties).

There are some opportunities to work with industry associations such as cattle, poultry and swine associations, which have taken steps to clarify animal care responsibilities.^Footnote 13 These associations and others have supported the evolving Certified Livestock Transport (CLT) program, which is modeled on the US Transport Quality Assurance program and may begin setting some de facto animal transportation standards for Canada. Some processors have expressed an interest in using only certified transporters in future.

2.0 Alignment with Government Priorities

AMPs address the Cabinet Directive on Streamlining Regulations (CDSR): "…Departments and Agencies are to demonstrate that the regulatory response is proportional to the degree and type of risk." (CDSR, April 1, 2007).

Following from this notion of the need for a proportionate response, AMPs are an appropriately-scaled response for cases in which prosecutions would be excessively time consuming and costly. (RIAS for Cattle ID)

3.0 Alignment with Federal Roles and Responsibilities

The issuance of AMPs appears to be aligned with overall federal roles and responsibilities. However, among CFIA stakeholders, there are different perspectives of how AMPs work. The available information suggests that not all CFIA investigators and inspectors involved with AMPs understand them or their use in a consistent manner and, for this reason, application is inconsistent. There are also some differences in the perceptions among some CFIA stakeholders regarding the appropriate role for legal counsel in AMPs, especially concerning the appeal process.

3.1 Roles of Key Stakeholders

The roles of key stakeholders varied according to area,^Footnote 14 though there were some common elements:

Inspectors are always the "front end" of the process, initially engaging the regulated party and obtaining inspection manager approval before having an AMPs file forwarded to investigation staff in most cases.
Investigators are engaged in most AMPs processes to conduct more detailed investigative procedures once a suspected violation subject to an AMP has occurred. In Ontario, in limited circumstances, some inspectors are authorized to issue AMPs directly, but they are still subject to review and approval. While 50 inspectors were trained to issue AMPs, only about five continue to do so.
Program management had limited connection to the application and operation of AMPs, though they were consulted occasionally on specific problematic applications. They also had little systematic feedback, other than receiving some complaints, regarding their effectiveness.
Legal counsel represents the CFIA when an AMP is appealed to the Canada Agricultural Review Tribunal (CART).
The CART hears reviews (appeals) of AMPs and renders decisions which have a bearing on AMPs.

3.2 Differences in Understanding and Interpretation across Stakeholders

Inspectors

The majority of investigators and inspectors interviewed noted significant variation in the understanding of the roles of inspectors, even within areas. Investigation staff suggested that many inspectors, given their history of mainly issuing non-sanctioning notices such as Corrective Action Requests as opposed to Notices of Violation, did not see their job as being part of an enforcement system. For this reason, investigation staff noted that some inspectors did not issue AMPs or sufficiently collect and organize data and evidence in support of AMPs.

Of eight responsibilities for inspectors listed in the CFIA Compliance and Enforcement Operational Policy, seven were identified in interviews as not always being an accurate reflection of roles and activities when it comes to AMPs.

**Inspectors**
Inspector Roles Outlined In Compliance And Enforcement Operational Policy	Inconsistencies And Variations From Expectations
1 carrying out inspections to verify, assess and monitor compliance with the law	not always clear to inspectors how much or what type of information should be collected
2 communicating with regulated parties regarding the legislative requirements, how compliance will be assessed and what will be considered non-compliance	inspector consistency in understanding the legislative requirements and in communicating those requirements
3 reviewing options for preventive and corrective actions and explaining legislative requirements to regulated parties	consistency of understanding and communication with regulated parties;
4 directing that corrective measures be taken to address non-compliance that can be immediately corrected	Notices of Violation can take months to issue – even for warnings
5 preparing non-compliance reports	level of information collection (documentation) in preparation for possible appeals
6 making comprehensive notes, completing inspection reports	level of information collection (documentation) in preparation for possible appeals
7 giving evidence in court and at reviews before the AMPs Review Tribunal	legal counsel appear in court or in front of the Canada Agricultural Review Tribunal (CART) when investigators may often be sufficient.

Another concern expressed by CFIA interviewees was with the excessive internal (CFIA) requirements for the collection and documentation of evidence. The main concern expressed by industry association representatives was with regard to the consistency of AMPs application and the long time it takes to have AMPs issued.

Investigators

Note that most (22) of the interviews were conducted with Enforcement and Investigation Services staff. There were fewer problems noted with the roles of investigators than with the roles of inspectors, though only four inspectors were interviewed. Notwithstanding this point, four of 13 functions were noted as not always being an accurate reflection of roles and activities when it comes to AMPs.

**Investigators**
Inspector Roles Outlined In Compliance And Enforcement Operational Policy	Inconsistencies And Variations From Expectations
1 review and prepare briefs of evidence recommending prosecution	the level of evidence gathering and evidence to demonstrate violation (e.g. undue suffering, section 138 (2)(a) HAR)
2 prepare Administrative Monetary Penalty files	despite stated intentions to move to prosecution after a few AMPs, some violators appear to receive repeated AMPs before CFIA moves on to prosecute (This raised questions regarding the quality of evidence, strength of penalties, costs, and understanding of escalation policy by CFIA staff)
3 liaise with CFIA Legal Services and the Public Prosecution Service of Canada with respect to investigations and prosecutions	lack of training of investigators and inspectors
4 deliver enforcement training to CFIA staff	That this is the appropriate role for EIS head office

3.3 Roles and Responsibilities Alignment

Legal Counsel

In the case of AMPs, legal counsel play a service role, representing the CFIA before the CART for all reviews, as well as before the Federal Court of Appeal for reviews of CART decisions.

Perspectives on the appropriate role of legal counsel differed. Some respondents suggested that the intent of AMPs was to have a system simple enough that legal counsel would not have to be involved to represent the CFIA before the CART. They also thought that the CFIA may be represented by a CFIA Enforcement and Investigation (EIS) officer at CART appeals, without legal counsel present, which was the original intent.

Interviewees also noted that some appeal cases had become more complex than originally envisaged. According to interviewees, the CART was intended to provide a simple and speedy administrative process, and not one requiring lawyers and large briefs. Others valued the support of legal counsel to interpret difficult clauses and to ensure that the CFIA made appropriate use of evidence.

Canada Agricultural Review Tribunal (CART)

A number of interviewees noted that because CART does not provide a speedy summary process, the CART process may not be aligned with the intent of AMPs. The CART typically takes from four to six months to reach a judgment.

Program Managers

Program managers were found to have a very limited role in terms of where, how and to what extent AMPs are being applied in their program areas. The extent of program involvement appears to be when they are consulted on specific AMP applications, at the discretion of investigators. Furthermore, their only information on AMPs in terms of system feedback appears to come from prominent complaints and anecdotal information. In addition to creating a gap in basic results knowledge, this means that, when considering operational decisions such as license renewals, program managers do not have consistent, if any information on whether and to what extent companies have received AMP warnings or penalties.

4.0 Achievement of Expected Outcomes

In general, despite a lack of detailed evidence, due to a paucity of reliable cost data and lack of performance targets and benchmarks, it appears that AMPs have succeeded in terms of providing the CFIA with the option to use a sanction which is less expensive or punitive than prosecution or licence revocation. Most AMPs are paid and most AMP sanctions are upheld upon appeal. Evidence suggests that on these terms, AMPs are more successful than criminal prosecutions.

Notwithstanding some overall positive achievements in terms of certain metrics, AMPs were found to vary in application consistency, in turn causing uncertainty across stakeholder communities. A more detailed systems review of AMPs in terms of how it operated as compared to the original theory is shown in Annex A.

Data such as an analysis of HAA and HAR cases taken from before, and then after the introduction of the AMPs option shows that on the whole, despite its problems, AMPs can serve as a legitimate and cost effective part of the compliance and enforcement toolkit for the CFIA. Scholars and interviewees identified a number of key factors and conditions important to the effectiveness of AMPs, outlined at the end of this section.

4.1 AMPs Actual Implementation Has Varied from What Was Planned

**The following table summarizes how AMPs have varied as compared to expectations.**
AMPs Expected Result (RIAS 2002)	Observed Results
Emphasizing compliance vs. punitive action	The penalties for AMPs can be higher than in prosecutions. Furthermore the Federal Court of Appeal commented that AMPs import some punitive elements of penal law. (Doyon decision 2009).
Provides for immediate enforcement and immediate corrective action	Significant delays in issuance of AMPs were noted. It sometimes took two months or more just to issue a warning, and often more than a year to issue a penalty.^Footnote 15 Records and interviews show the process has become more complicated, partially due to interpretations upon appeal which have called into question such elements as "undue suffering" leading to the imposition of a number of criteria by the CART against which the facts must be assessed to determine if the violation occurred.
Provides for the use of negotiated solutions to non-compliance for commercial violations	Records and interviews suggest other compliance options are not exercised. Compliance agreements, an option which allows for a violator to apply the monetary penalty directly to make a corrective action (for example, fix a faulty trailer) are almost never used. Some interviewees expressed a concern that compliance agreements are overly complex and will not be entered into in good faith.

There were also variations found in some common perceptions among interviewees about how AMPs work. These misperceptions have been referred to by some as myths, which are dispelled below.

**Myth/Reality**
Myth	Reality
Many AMPs are appealed	Relatively few AMPs are appealed, and most for a few specific violations. According to the evidence available, fewer AMPs are appealed than prosecutions.^Footnote 16 Roughly 13% of AMPs are appealed (to both the CART and the Minister), and these appeals are concentrated in a few sections of the regulations. There are over 400 sections (including subsections) of the two CFIA acts and their regulations that outline possible violations where AMPs could have been applied over the time periods examined: 2000-2010. Only 135 have been used to issue an AMP with penalty. AMPs that were appealed represent 58 of the 135 sections. Three (3) of those 58 refer to "undue suffering" and represent 37% of all the appeals, or 144 out of 393 appeals. Another 2 of those 135 involve animal transport and tagging, and have had 86 AMPs issued, representing another 22%. A total of 59% of appeals come from just five animal transport sections.
Many appeals result in reversals	Most AMPs decisions are upheld: available data suggests more AMPs decisions are upheld than prosecution decisions.^Footnote 17 Roughly 73% of AMPs are upheld on appeal.
Many violators do not pay AMPs	Roughly 10% of AMPs are unpaid^Footnote 18 (Source: National Accounts Receivable)

A lack of clarity in the wording of some of the infractions has caused significant challenges in the interpretation of their meaning, particularly in the appeal process, particularly those related to "undue suffering". As noted in the table above, three of those sections account for 37% of all appeals. What is "undue" is not always interpreted the same way by all parties. Note that the Doyon decision suggests specific tests for "undue suffering", but even these are subject to interpretation for the particularities of individual cases.

4.2 How AMPs Work in Different Circumstances

According to interviews and records, AMPs work better when there is clarity in the legislative and regulatory wording, when there are controlled inspection conditions, when there are few stakeholders, and when a regulated community generally wants to be part of a firm and stable marketplace, and is generally delivering medium to higher-end products. The problem players, prone to recurring violation, are often the so-called marginal actors. Interviews and file reviews pointed to dealers in cull animals and spent hens that collect animals to be slaughtered for marginal value attached.

AMPs do not work well when they are seen as a "cost of doing business", that is, where the AMP is a small cost of the load (e.g., chickens), and the system is geared for continuous production.

4.3 Achievement of Expected Results

The following factors limit the identification of strict causality related to the expected results:

deterrence is inherently hard to measure;
lack of performance measures. Specific expectations and targets were not set, even though AMPs were, in the opinion of some interviewees, developed as a pilot initiative. For example, there were no targets for a decrease in prosecutions due to the application of AMPs;
AMPs are not explicitly considered in the development and management of program business line plans – further reducing the opportunity to set and monitor expected results;
lack of collection of key data such as the use of employee time for key tasks such as evidence collection; and
lack of a database capable of producing reports and collecting or analyzing statistics

It is expected that the increase in the maximum penalty amounts, which were more than doubled in October 2010, will increase the deterrence effect of AMPs. No evidence was available at this early stage to assess this degree to which this expectation is being realized.

However, there is sufficient evidence to make a number of conclusions about the achievement of results for AMPs, as outlined below. As noted above, the 1997 RIAS referred to increased compliance in the United States and it stated that AMPs would allow the CFIA to increase enforcement activities: "…Since AMPs are not as time consuming as prosecutions, it will allow the CFIA to increase its enforcement activities." No evidence is cited with these claims in the RIAS.

Analysis in the previously referenced 2010 Prabhu doctoral thesis suggests that the number of individuals prosecuted for offences under Health of Animals Act and Regulations dropped from 125 in the five years preceding 2001, to 66 in the five years from 2001 to 2006. Note that there is correlation, but no confirmed attribution in this analysis, that is, although this shows that prosecutions declined following the application of AMPs as was intended, there are other possible causes of a decrease in prosecutions that have not been investigated, such as changes in policies, investigation practices and resources.

Prabhu's thesis analysis also points to a strong decline in the number of failed prosecutions over the five years after the introduction of AMPs. This may indicate that with fewer prosecutions there were higher quality prosecutions and a higher success rate, but may also suggest that AMPs were previously applied to cases that were relatively weak as criminal prosecutions. He also notes that the success rate for AMPs in terms of violations being upheld was much higher than what was experienced by enforcement officers in criminal courts.

4.4 Integrity of Implementation

The level of AMP issuance varies significantly by area. This is partially due to the nature of the regulated communities (e.g. some areas have large cattle volumes leading to more cattle ID AMPs), but is also due to differing area practices about when and how to issue AMPs. Across areas, Area managers employ different strategies, and different degrees of thoroughness in evidence collection.

Notwithstanding differences in context and conditions, the level of AMPs investigation and evidence gathering varies extensively. The documentation process is sometimes performed at a high level even when a Notice of Violation is issued without penalty (i.e., a warning^Footnote 19). Investigators have 15 days to prepare briefs for any appeal requested, and appeals can only be requested within 30 days of AMP issuance.^Footnote 20 This limitation to the time to produce an evidence brief has led most investigation managers to require that full briefs be prepared for all AMPs at the outset, in case they are appealed – despite the fact that only 13 percent are appealed and most of the appeals are for a few specific types of violation, as noted in the previous table.

Some interviewees suggested that the CFIA is reluctant to escalate from AMPs to other compliance tools when parties repeat the same violations. However, clear evidence shows that the track records of regulated parties, and of inspection and investigation efforts, are consistently used to determine CFIA actions to achieve compliance and enforcement. The ability to use this history to determine the course of action is limited to the degree that this information is accessible.

The National Enforcement Tracking System (NETS), the primary CFIA Enforcement and Investigation Services database, does not have the capacity to provide consistent tracking information and has no reporting capabilities. Its significant limitations have led to inconsistent data entering which in turn has led to the use of numerous Excel files in each office across the country keeping duplicate information, little of which is linked.

4.5 Some Key Principles to Guide AMPs Application

Scholars^Footnote 21 have noted that AMPs are appropriate when the following elements are present, and their observations are supported by the interviews conducted for this evaluation:

a large volume of cases is likely to be processed annually (that is, many transactions are being inspected);
the regulator had stronger sanctions but the monetary penalties could be used to moderate a harsher response;
speedy adjudication to the enforcement scheme is important;
specialized knowledge (for example, technical expertise) and agency expertise in the resolution of disputed issues was needed;
issues of law are rare;
consistency of outcome was important; and
there is a likelihood that an agency or group of agencies will establish an impartial forum in which cases can be efficiently and fairly decided.

Interviewees made the following suggestions of key factors affecting AMPs success:

The regulated party has a high level of commitment to the basic intent of the legislation, so there is a low level of willful non-compliance. For example, associations and companies are willing to work with the CFIA to ensure compliance;
The regulations contain clear language defining violations;
There are controlled inspection conditions. For example, the inspection takes place in a regulated public facility vs. a remote privately run location;
Transactions are not complex;
Inspectors and investigators share an understanding of what constitutes a violation;
Inspectors and all other regulator program staff share a commitment to the promotion of regulatory compliance;
There is a consistent interpretation of the legal responsibilities of all concerned parties, and of the burden of proof and of evidence;
Significant proportion of commercial transaction "value" represented by AMP (i.e. the cost of the AMP vs. the value of the shipment); and,
Regulated parties believe that enforcement actions will be upheld.

5.0 Demonstration of Efficiency and Economy

The evidence to date, while not allowing for a precise estimation, suggests that AMPs can be cost-effective (that is, provide economy and efficiency) in the right circumstances. Among interviewees, views were split in terms of whether a summary conviction ticketing scheme would be a better alternative.^Footnote 22 Views also varied in terms of how and to what extent evidence collection and appeals preparation and delivery could be streamlined. However, the evidence below suggests that there is an opportunity to improve the economy, efficiency and cost-effectiveness of AMPs.

5.1 AMPs vs. Alternatives

Some interviewees suggested that the CFIA should also seek alternatives to AMPs. In cases where remedies and sanctions such as detention or seizure of shipments were readily available and generally practiced, the contention was made that AMPs might not be necessary. Interviewees were split as to whether a summary conviction ticketing regime would work better than AMPs. Some pointed to Health Canada's Contraventions Act as an example because it allows for a determination of a violation, or a contravention, similar to a parking or speeding ticket, and appeals go before a special contraventions court. Others felt that this would only be relevant for certain types of AMPs that were straightforward, and that for complex AMPs, such as animal transport cases, a full investigation would be required.

5.2 More Cost-effective Ways of Achieving the Same Results

Cost-effectiveness is impossible to calculate precisely due to lack of strong data. However, it was an issue of interest to most interviewees. Many of the points raised about cost-effectiveness by interviewees related either to legal and administrative process improvements, or to training, though some related to the need for more strategic use of these instruments.

Efficiencies through legal and administrative process improvements

A number of interviewees expressed concern about the timeliness of issuing AMPs. The regulations provide up to six months to issue a minor AMP and up to two years for more serious AMPs. Beyond assuring the investigation staff that a violation has taken place, most evidence is prepared in case an AMP is appealed, as discussed earlier.

There are significant costs related to investigation, both in CFIA resources and in the frustration of AMP recipients for a violation that, after a year or more, they may no longer recall. The time elapsed may lead to repetitions of the violation before the AMP is issued, whereas a speedy issuance could deter additional violations. A number of interviewees noted the opportunity for cost savings if level of investigation were based on the risk of an appeal.^Footnote 23

For legal process improvements, many interviewees discussed the use of lawyers to represent the CFIA before the CART. Most considered this representation was necessary only when cases are complex. It was noted that the original intent when AMPs was implemented at the CFIA was for lawyers to provide some guidance to investigators for handling appeals, but lawyers have always represented the CFIA before the CART. Some have pointed to the use of lawyers by the CFIA at the CART as a reason for excessive investigation. Conversely, a reduction in the use of legal counsel might not improve cost-effectiveness for complex cases.

Improving payment compliance was raised by many interviewees, despite the fact that the vast majority of AMPs penalties are collected and only about 10 percent (by number and 14% by value) are not. A number of interviewees noted that when penalties are not collected, word spreads to others and reduces the credibility of the system. The CFIA has been working with the Canada Revenue Agency to obtain some of the non-payments.

Need for greater training of inspectors

More training of inspectors was a common suggestion to improve the efficiency and effectiveness of AMPs. The CFIA's Ontario investigators developed a training program for inspectors, but this has resulted in only about five inspectors issuing AMPs, although 50 were trained. Many interviewees noted that any increase in the use of AMPs would be feasible only if inspectors were trained to issue them. The grouping of AMPs into simple vs. complex types was considered key for identifying which AMPs could be issued by inspectors.

Interviewees also recommended inspector training to increase awareness of alternative enforcement options. A number of investigators believe that many inspectors are not using their full range of enforcement options. For example, few inspectors use their authority to seize, confiscate and destroy property. For animal transport issues, they can also immobilize vehicles, as well as seize vehicles, but this is seldom done.

Strategic Use

Program manager interviewees noted a need to better integrate the use of AMPs into business line strategies. The suggestion was made by CFIA interviewees that AMPs should be considered part of staged initiatives and targeted blitzes. As a minimum, program managers interviewed expressed a desire to be more involved in the planning and monitoring of AMPs applications. For example, some managers are responsible for licensing companies, without any awareness of the companies' compliance history, such as its receipt of AMPs. Some managers develop inspection strategies based on risk, without awareness of non-compliance information that AMPs investigations have uncovered. Increased program manager investment could help in operational decisions such as company license renewals, as well as in strategic and tactical program management decisions regarding compliance

5.3 What Would Happen Without AMPs?

Every interviewee noted that without AMPs, the CFIA would be deprived of a key tool for compliance, and more prosecutions would be required. Interviewees consistently noted that prosecutions could never replace the number of AMPs issued and as such compliance would decrease.

Interviewees also noted that AMPs provide "early warnings" about program non-compliance. For example, when a number of AMPs have been issued for a particular situation, such as a particular vegetable consistently arriving at the border contaminated and therefore leading to the imposition of AMPs, tracking of the use of AMPs provides the CFIA with an awareness of a possible need to consider increased certification requirements, such as the addition of a phytosanitary certification in addition to regular certification.

6.0 Conclusions, Good Practices and Recommendations

While AMPs have shown some success in some conditions, they have not lived up to expectations in terms of speed, cost, and likely deterrence effect. This appears to be due to uncertainties in certain legislative and regulatory wording and the complicated multi-party nature of certain inspected enterprises such as animal transport. The nature of certain parts of the regulated sector, for example, certain chronic non-compliers that do not want or cannot afford to comply, and differing policies on evidence collection compound the difficulties in effective application of the AMPs tool.

6.1 The Process - Build Learning Into Delivery

Conclusions

AMPs at the CFIA have not grown beyond the Health of Animals Act and Plant Protection Act over the past decade. The CFIA never established the type of clear objectives, systematic measurement systems and key review point milestones needed to permit systematic learning from the early applications, which would have facilitated improvements in the use of the tool and an increased certainty about their value. This knowledge and capacity gap may have contributed to myths about the process which, in turn, have led to ineffective practices such as a lack of delegation and excessive evidence collection.

Good Practices

AMPs require an effective database, monitoring, evaluation and reporting system to ensure that CFIA management is more aware of the changing compliance conditions across all areas and ensure that investigators have ready access to the history of violations. This history is one of the criteria for setting the value of the penalty, requiring an assessment of past violations. Database reporting would also provide information on the types of AMPs causing inspectors and investigators the greatest challenges which would assist in the maintenance of a categorization of AMPs by risk levels.

Categorization of AMPs by complexity and risk of appeal would provide an opportunity to gauge the comprehensiveness of evidence collection required. Regular communication between AMPs Areas would allow for an effective and regular sharing of good practices and lessons learned.

Recommendation 1

The VP, Operations Branch, should establish a plan to build learning into the delivery of current AMPs for their ongoing development and for any expansion of the use of the tool.

The plan should include the following elements:

a strategy to calibrate the level of investigation according to the risk of appeal;
a performance measurement strategy; and
a plan to incorporate on-going performance measurement in the development of training of and communication with inspectors and investigators across areas.

6.2 Strategy - An Integrated Approach to Compliance

Conclusions

Both the theory and the practical wisdom regarding the use of AMPs show that they need to be considered one instrument of policy among many, to be applied as part of an integrated compliance strategy. AMPs work best when integrated into a considered and consistently applied compliance strategy which clarifies the roles of inspectors and investigators, as well as legal counsel, review processes, communications and affected industry associations. There was little evidence of program strategies including any consideration of the different contexts and conditions where AMPs could be best applied. In addition, the determination of the potential role of inspection staff for the issuance of many of less complex AMPs is essential for any expansion of the application of AMPs to other Acts.

Good Practices

Integrating the needs of all players could provide a common understanding of the operations challenges, risks and problems facing specific food sectors and value chain segments. If connected to program (business line) strategies, AMPs could be better targeted and streamlined to address distinct compliance risks. Linkages of information between Programs and Policy Branch, and Operations Branch would allow program and enforcement staff to identify problem areas early on, and to adapt enforcement strategies according to the specifics of each case. For example, repeated violations in a particular area would be identified for review, possible education of stakeholders, or revisions of government policies.

6.3 Implementation - Setting the Right Conditions

Conclusions

Internationally and at the CFIA, it is apparent that there are key conditions which affect the success of this instrument. These include elements such as clarity of objectives, legislative wording, and inspection and enforcement roles. Consistency in understanding of the roles and obligations of all players, from regulated parties to government staff is also essential. Some of the key problems with AMPs at the CFIA relate to legislative wording of a few sections of relevant Acts (see section 4.1). A more complete list of conditions and principles is contained in section 4.5. Further suggestions for improvement are contained in section 5.2.

Good Practices

An assessment of all current and potential regulations and acts (as listed in the AMPs Act) to determine the degree to they meet the conditions and principles outlined in this study, would provide the CFIA with strong direction for choosing which parts of acts and regulations are most appropriate for applying AMPs. Such a determination would be most effective if developed cooperatively with operational and program staff, along with guidance from legal services.

Should any expansion of AMPs be considered, it should be assessed against these key conditions and principles for their success.

Recommendation 2

Annex A: AMPs Results Logic – Expected vs. Actual

There are various theories about the gaps which AMPs can address, and about the context appropriate for use of AMPs (see CFIA AMPs Evaluation Framework, July 2010). These can serve as background for understanding the results logic flow.

Figure A.1 starts with the perceived gaps and contextual factors then outlines the theory of AMPs use to promote compliance and food safety.

Figure A.1: The Perceived Gaps, Context and Results Logic of Administrative Monetary Penalties. Description follows. — **Figure A.1: The Perceived Gaps, Context and Results Logic of AMPs**

Given the perceived gaps, contextual factors and basic results logic shown in Figure A.1 above, the theory can be broken down into a series of premises:

CFIA Operations Branch undertakes a systematic process of responsive compliance management activities which follows a systematic process. The process starts with consultation and information. As noted in Figure 1, this persuasion phase may escalate to warning, then to penalty, then to prosecution or licence revocation. In general, the framework of detection, response, enforcement, assessment and modification can be seen (the "DREAM" model, Baldwin and Black 2007)^Footnote 24 can be seen throughout
Verification: the clear verification of non-compliance.
Regulated parties' responses render clear information to allow the CFIA to make a definitive choice regarding the most appropriate response to non-compliance.
Notices of violation are defined clearly and unambiguously to key responsible parties.
Notices of violations serve to foster respect for the process by virtue of regulated parties appropriately paying their penalties – or if not, proceed to step 6.
Nonpayment triggers appropriate collection actions which may include steps 7 and 8.
If a request for a review is made to either the Review Tribunal or the Minister, the investigator must assemble the case file within 15 days. The case file is used by the CFIA in presentation of their witnesses and evidence.
Tribunals (or the Minister) deliver clear, logical, evidence-based decisions which constructively impact compliance (while respecting the rights of all parties).
Regulated parties understand their violation and are provided with (or have) the knowledge, abilities, skills and aspirations to comply with requirements.
All these elements combine to improve compliance, foster even further stewardship actions (i.e., actions beyond simple compliance to support desired outcomes) and lead to benefits for Canadians.

The following chart, Figure A.2, as read left to right, describes the perceived need for AMPs, some key contextual factors affecting their delivery and the results logic of the inspection, investigation and appeals processes in terms of an integrated 'system.' The 'comment' boxes represent study observations where conditions and practices have apparently affected the system.

Figure A.2: The Perceived Need, Context, Results Logic and Findings re: Administrative Monetary Penalties at Canadian Food Inspection Agency. Description follows. — **Figure A.2: The Perceived Need, Context, Results Logic and Findings re: AMPs at CFIA**

Upon review, the following observations can be made with regard to this logic.

A review of AMPs violations shows that the problematic cases such as cases appealed, and decisions not upheld on appeal tended to relate to specific situations of animal transport. In these situations, the violators might have been prone to non-compliance in both will (that is, a lack of desire to comply) and capacity (the economics of the enterprise made compliance unattractive – or, more to the point – make the potential cost of non-compliance relatively attractive or worth the risk). Low-value animals being transported by broker groups appear to be a high-risk group.
The complexity versus simplicity of the transaction, location, context and culture of inspection and subsequently investigation engagement appears to have an effect on compliance success. For example, complicated transportation situations involving ambiguities in the law such as the idea of what constitutes "undue suffering" and who has custody of an animal during the time the suffering occurred, and what inspection or investigation conditions constitute "undue suffering", tend to be problematic.
The engagement or lack of engagement of associations in creating the support conditions for compliance, through information, education and "peer pressure", appear to make a difference. Groups without a strongly committed association appear to exhibit more problems.
The history or culture around inspection, in terms of whether inspectors were accustomed to issuing actual penalties, as compared to Corrective Action Requests, was claimed by several interviewees as a contributor to compliance or non-compliance behaviours on the part of the regulators and regulatees.
Relating to points 2 and 4 (and "downstream" appeal decisions as "precedence" see 7 and 8) it can take a long time to issue Notices of Violation. This reduces the ability of CFIA to reinforce messages and to create deterrence.
There were significant differences in approaches to AMPs issuance across areas. This is partly due to differences in context and conditions, related to inspection and enforcement philosophy and culture, but it is also apparently due to differences in interpretation.
Legal interpretations raise the evidence bar in the minds of many investigation staff. The evidentiary bar cannot be raised because it is set by statute, but interviews and file reviews suggest that evidence gathering, even when no Notice of Violation is issued, has increased over the years. Investigators attribute this to the nature of tribunals, the Federal Court of Appeal and Ministerial Reviews which appear to work against the principle of absolute liability (violations simply need to be demonstrated without intent and due diligence is not a defence) as opposed to the more common "strict" liability (where a due diligence defence is available). It would appear that the application of absolute liability has been distasteful to the legal communities involved in appeals (such as the Tribunal Chair, the Federal Court of Appeal, those responding in Ministerial Reviews) and that this has been argued to have resulted in decisions which have rendered more stringent the elements to be proven and therefore increased evidence collection to levels approaching those for prosecution in many cases. The level of information and evidence to be collected in both inspection and investigation, in order to withstand appeal challenge, has been interpreted as significant.
Due to the preceding points, there is a lack of consistency, certainty, speed and efficiency in the investigation and serving of AMPs. There may be the makings of a vicious circle in the system, as uncertainty leads to more evidence collection, and potential decision inconsistency which lead to delays reducing deterrence effectiveness and reducing regulatee trust and cooperation which lead to less compliance, more challenges and potentially more uncertainties, inconsistencies and more inefficiencies.

Annex B: Evaluation Framework, Methodology and Limitations

The July 2010 evaluation framework and plan showed that there was insufficient available data for detailed cost-benefit or cost-effectiveness analysis, and that there was a need to understand how (and to what extent) AMPs work, for whom, and in what conditions. The study included AMPs at the CFIA from its inception and addressed core evaluation questions of relevance and performance.

The approach of the evaluation was guided by an emerging TBS guideline related to assessing the contribution of programs to expected results in complex environments, based on the work of John Mayne and Ray Pawson, including the key evaluation question: What works for whom in what circumstances and in what respects, and how?^Footnote 25 This is also known as a "Realist" evaluation approach, which addresses outcomes by assessing, to the extent possible given the state of existing evidence, the extent that objectives were being achieved in different contexts and conditions. This approach allowed the report to identify the key principles and conditions to the success of AMPs. A realist interpretation of TBS compulsory issues suggests the questions as shown in the table below, which was included in the evaluation framework.

**Core TBS Evaluation Issues Interpreted to Address "Realist" CFIA Management Needs - Relevance**
	Core TBS Directive Evaluation Issues	Realist Evaluation Questions
Issue #1: Continued Need for program	Assessment of the extent to which AMPs continue to address a demonstrable need and are responsive to the needs of Canadians	What needs does AMPs appear to be organized to address? Why was AMPs initiated at CFIA? Who is served by the AMPs program? What has been the reaction to date from regulated parties?
Issue #2: Alignment with Government Priorities	Assessment of the linkages between AMPs objectives and (i) federal government priorities (e.g. CDSR) and (ii) departmental strategic outcomes (i.e. Food Safety and Quality)	Are there differences in the understanding of the "theory" of AMPs? Under what assumptions and circumstances does it align with federal government priorities (e.g. CDSR) and the strategic outcomes of departments? (i.e. Food Safety and Quality)
Issue #3: Alignment with Federal Roles and Responsibilities	Assessment of the role and responsibilities for the federal government in delivering the program	What is the understanding of the roles of key actors in the application of AMPs? Are there differences in stakeholder understanding and interpretation of those roles? How well aligned are roles and responsibilities?

**Core TBS Evaluation Issues Interpreted to Address "Realist" CFIA Management Needs - Performance (effectiveness, efficiency and economy)**
	Core TBS Directive Evaluation Issues	Realist Evaluation Questions
Issue #4: Achievement of Expected Outcomes	Assessment of progress toward expected outcomes (incl. immediate, intermediate and ultimate outcomes) with reference to performance targets and reach, program design, including the linkage and contribution of outputs to outcomes	Does the program theory for AMPs tend to bend in actual usage? How? Why? How do AMPs work in different contexts and circumstances? Does the theory fare better with particular individuals, interpersonal relations, sectors and situations? What are the best practices for given situations? What are some key principles to guide AMPs application? Integrity of the implementation chain – are AMPs elements applied consistently and cumulatively? Why or why not? How can / should this be addressed?
Issue #5: Demonstration of Efficiency and Economy	Assessment of resource utilization in relation to the production of outputs and progress toward expected outcomes	How does AMPs compare in terms of what it delivers to alternatives (e.g. prosecutions)? Are there more cost-effective ways of achieving the same results? What would happen without AMPs?

An evaluation advisory committee, consisting of CFIA executives from Operations Branch, Policy and Programs Branch and Corporate Management Branch, plus ex officio members from Legal Services and Health Canada, provided strategic and technical advice to the Head of Evaluation and approved the evaluation framework and final draft report. An Executive Director from the Operations Branch co-chaired the committee with the Head of Evaluation. The questions shown in the table above, included in the framework, were approved by the Committee, and were used to guide the evaluation, collect data and report findings, as seen in the organization of this report.

Study limitations were reviewed and addressed in the evaluation framework phase of the study. They can be summarized as follows:

Details on costs and details of AMPs for all incumbent activities are not readily available on a consistent pan-Canadian basis. This means that accurate estimates of the breadth and depth of coverage of AMPs was not possible. This also greatly hindered the ability to pursue a design which would have involved a generalizable sample or detailed costing estimates.
Roles, responsibilities and processes for delivering AMPs have been documented in various ways across areas and across CFIA groups. Furthermore, limited discussions regarding the CFIA's rationale for AMPs and its fundamental policy for AMPs were found. This limitation means that the evaluation had to rely heavily on respondent memory and third party documentation to glean even fundamental expectations regarding the use of AMPs at the CFIA.
Not all of the stakeholder groups were found to know much about AMPs. This limited the ability of the study team to garner knowledgeable viewpoints from a diverse set of stakeholders.

Given these three fundamental limitations, the study team chose to adopt a "generative" approach to knowledge production, starting with an early understanding of the theory of change related to AMPs, then testing and adjusting it as knowledge accumulated over the course of the study. The accumulation of knowledge in this fashion provides generative learning about what works for whom in what conditions and why.

Data collection was undertaken from July 2010 to July 2011 and consisted of the following:

Consultation with National Office, four areas and representatives of 10 industry associations. Over 60 personal interviews and consultations were conducted including:
- 22 Enforcement and Investigation Services (EIS) staff
- 4 CFIA inspectors delivering AMPs
- 2 CFIA accounts receivable personnel
- 1 CFIA regulatory policy official
- 4 CFIA administrative and IT support staff
- 8 CFIA policy and program managers and specialists
- The chair of the Canada Agricultural Review Tribunal (CART)
- 12 industry association representatives
Content analysis of CFIA case files, guides, training materials etc.
Database analysis
Reference to tribunal and court decisions
Sample survey of poultry processing sector
Content analysis of special studies, reviews and academic publications

Enforcement and Investigation Services (EIS) staff and the industry associations were identified at the outset as the key informants for this evaluation, representing those administering and issuing AMPs, and those who represent those receiving them, respectively. The greatest interview concentration was placed on the office of EIS. A total of 48% of EIS staff were interviewed (22 out of 46 at time of selection). Industry associations were chosen by the significance of their members' involvement with AMPs, i.e., those with high numbers of AMPs issued. The NETS database was not capable of providing specific percentages of AMPs issued by association representation. However, file reviews, interviews and reviews of the types of AMPs issued provided strong evidence for choosing those key association interviewed. Both the program and policy staff represented an important focus of the evaluation interview program.

The survey was a sample undertaken with one of the key industry associations, which distributed the questions to its members and received responses from, according to the association, "the more significant industry players". No statistical significance could be ascertained due to the small sample size, although most of the individual responses were comprehensive and supported this report's findings.

The following evaluation matrix was included in the planning document for the evaluation, or evaluation framework, approved by the evaluation's advisory committee.

Issues, Indicators, Sources and Collection Approaches

***Issue #1*** Continued Need for program
Issue	Indicators (Metrics / Evidence)	Source	Collection Approaches
1.1 Why were AMPs created? (What needs have AMPs addressed?)	Documentary descriptions	Originating policy documents Decision records Academic articles Reviews of AMPs (Prabhu)	Content analysis of documents
1.2 Why was AMPs initiated at CFIA?	Decision records Recalled reasons by key officers (interviews, emails, records of decision, RIAS)	Originating policy documents Decision records	Content analysis of documents
1.3 Who is served by the AMPs program?	Opinions expressed by key stakeholders Documentary descriptions	CFIA, CART and industry Originating policy documents	Interviews Content analysis of documents
1.4 What has been the reaction to date from regulated parties?	Self assessed reactions from regulated community Observed levels of compliance and appeals (see issue #4)	Industry sector representatives Other stakeholders CFIA NETS Documented reactions in incident reports	Interviews Content analysis of database Content analysis of case files

***Issue #2*** Alignment with Government Priorities
Issue	Indicators (Metrics / Evidence)	Source	Collection Approaches
2.1 Are there differences in the understanding of the theory of AMPs?	Observed differences in stated understanding of the use of AMPs across stakeholders	CFIA viewpoints Descriptive documents	Interviews with CFIA Content analysis
2.2 Under what assumptions and circumstances does it align with a) Federal government priorities (e.g. CDSR) and b) the strategic outcomes of the department (Food Safety and Quality)	Degree of consistency between observed AMPs use and: CDSR (i.e. streamlining regulatory processes and optimizing burden) Food Safety and Quality objectives What about animal health, including humane treatment?	NETS and CFIA documents CFIA, CART and other stakeholders	Content analysis of documents Interviews

***Issue #3*** Alignment with Federal Roles and Responsibilities
Issue	Indicators (Metrics / Evidence)	Source	Collection Approaches
3.1 What is the understanding of roles of key actors in application of AMPs?	Perceptions of roles by key stakeholders	CFIA, CART sector regulatee representatives	Interviews
3.2 Are there differences in stakeholder understanding and interpretations of those roles? Are we talking about differences within certain stakeholder groups or between types of stakeholders?	Perceptions of roles by key stakeholders	CFIA, CART sector regulatee representatives	Interviews
3.3 How well aligned are roles and responsibilities?	Observed degree of consistency with government roles and responsibilities	CFIA regulatory documents re: policy and process	Content analysis

***Issue #4*** Achievement of Expected Outcomes
Issue	Indicators (Metrics / Evidence)	Source	Collection Approaches
4.1 Does the theory for AMPs tend to bend in actual usage? How and why?	Evidence of theory in documents vs. Evidence of actual usage of AMPs	CFIA decision records Reviews of AMPs (Prabhu) NETS database Legal decisions CART decisions Academic articles Audit and Review working papers and reports Observed use by CFIA	Content analysis of files and literature (synthesis) Content analysis of documents (and synthesis) Interviews with CFIA and experts
4.2 How do AMPs work in different contexts and circumstances?	# of AMPs by region, Act, sector (other?) % payment compliance % appeals Process time Level of repeat AMPs Changes to compliance level (% of those not in compliance returning to compliance)	NETS database Reviews of AMPs (Prabhu) Legal decisions CART decisions Audit and Review reports including legal reviews Observed use by CFIA Reviews of AMPs (Prabhu)	Content analysis of documents (and synthesis) Interviews with key officers and experts
4.3 Do AMPs achieve expected results with certain individuals, interpersonal relations, sectors and situations?	Level of compliance (reduced non-compliance) % of those not in compliance returning to compliance Number of infractions % decreased enforcement $/costs^{Table Note t1} [All of the above considered by Authorizing Act, region and groups.]	CFIA Regulatee observations CART officials Audit and Review reports including legal reviews Review of AMPs (Prabhu)	Interviews with CFIA, regulatee representatives, CART officials and experts Content analysis of reviews (and synthesis)
4.4 What are some key principles to guide AMPs application? (e.g. should some of the principles discussed by legal analysts such as Goldschmidt and Priest be applied to the use of AMPs at CFIA?)	Demonstrated success rate associated with key factors	CFIA employee observations Audit and Review reports including legal reviews Academic articles Reviews of AMPs (Prabhu) Review observations Experts	Interviews with CFIA Content analysis of reviews and literature Interviews with other stakeholders (CART, Industry, courts etc.)
4.5 Integrity of the implementation chain - are AMPs elements applied consistently and cumulatively? Why or why not? How can / should this be addressed?	Described situation for AMPs in cases Level of AMPs per volume by region, act, sector (other) Observed level of overturned AMPs and reasons for decisions by region, Act, sector (other) and what body overturns them at what point in the justice system? Observed level of consistency in application of AMPs (decision processes, workflows) across regions, Acts, sector (other)	NETS CART, Ministerial and court decisions Descriptive documents CFIA employee recall Internal audit working papers	Content analysis of database Content analysis of decision records Content analysis of documents Interviews with CFIA Content analysis of internal audit working papers

Table Notes

Table Note t1

Note that a full cost-benefit study is not feasible at this time.

Return to table note t1 referrer

***Issue #5*** Demonstration of Efficiency and Economy
Issue	Indicators (Metrics / Evidence)	Source	Collection Approaches
5.1 How does AMPs compare, in terms of what they deliver, to alternatives? (e.g. regular ticketing, prosecutions, other)	Level of similarity or difference observed between AMPs and other instruments. Within Canada? GoC? International?	CFIA , CART, court officials and others Review of AMPs (Prabhu)	Interviews with key stakeholders Content analysis of reviews (and synthesis)
5.2 Are there more cost effective ways of achieving the same results? (e.g. in combination with other instruments)	Comparison of level of observed compliance and compliance change in AMPs usage as compared to the use of other instruments as noted at CFIA and elsewhere Level of perceived effectiveness compared to other instruments General level of costs associated with use of AMPs as compared to other approach	NETS Reviews of compliance Regulatory reviews Review of AMPs (Prabhu) Experts Cost / expenditure data	Content analysis (and synthesis) Interviews Content analysis of documents CFIA cost data
5.3 What would happen without AMPs?	Level of perception of AMPs impact as compared to use of other instruments Observed conclusions of reviews	CFIA officers, CART, Court, Experts Reviews of AMPs (Prabhu)	Interviews with key stakeholders Content analysis of documents / reviews

Annex C: Bibliography

Academic / Industry

Ayres, Ian, and Braithwaite, John, Responsive Regulation: Transcending the Deregulation Debate, New York: Oxford University Press, 1992. p35

Baldwin, Robert, and Black, Julia, Really Responsive Regulation, LSE Law, Society and Economy Working Papers 15, London School of Economics and Political Science, 2007

Coffee, John, Paradigms Lost: The Blurring of Criminal and Civil Law Models - And What Can Be Done About It, The Yale Law Journal 101, no. 8 (1992): 1875-1893, As cited in Australian Law Reform Commission Background Paper 7, supra note at 12

Goldschmidt, Harvey. An Evaluation of the Present and Potential use of Civil Monetary Penalties as a Sanction by Federal Administrative Agencies, in 2 Recommendations and Reports of the Administrative Conference of the United States, 1972

Mayne, John, Contribution Analysis: Addressing Cause and Effect in Forss, Marra and Schwartz editors Evaluating the Complex Transactions Publishers, 2011

Pawson, Ray, Evidence Based Policy: A realist Perspective, SAGE, 2006

Prahbu, Mohan. Efficacy of Administrative Monetary Penalties in compelling compliance with Federal Agri-Food Statutes, Faculty of Law at the University of Ottawa, 2010

Tait, Amanda. The Use of Administrative Monetary Penalties in Consumer Protection. Ottawa, The Public Interest Advocacy Centre, 2007

World Society for the Protection of Animals, Curb the Cruelty: Canada's farm animal transport system in need of repair, 2010 http://www.wspa.ca/ati/CurbtheCrueltyReport.pdf

Certified Livestock Transport (CLT) training program http://www.livestocktransport.ca/

Government of Canada

Canadian Food Inspection Agency, Compliance and Enforcement Operational Policy (2010) http://www.inspection.gc.ca/english/agen/transp/comp/pole.shtml

Canadian Food Inspection Agency, Enforcement and Compliance Policy (1999)

Cabinet directive on streamlining regulation. Ottawa: Government of Canada, 2007 http://www.tbs-sct.gc.ca/ri-qr/directive/directive00-eng.asp

Priest, Margot, Penalties and Enforcement Techniques that Promote Compliance: International Best Practices, Prepared for the CMC Enforcement Best Practices Research Group, Industry Canada, Office of Consumer Affairs, 2009

Legal/Regulatory

"Canada Agricultural Review Tribunal" Decisions and Reports http://cart-crac.gc.ca/CART-CRAC/display-afficher.do?id=1277928993362&lang=eng

Doyon v. Canada (Attorney General), 2009 FCA 152

R. v. Jarvis 2002 SCC 73

R. v. Ling 2002 SCC 74

"Proposed Regulations" Agriculture and Agri-Food Administrative Monetary Penalties Regulations, Regulatory Impact Analysis Statement, Canada Gazette Part 1 131, August 9, 1997 http://www.collectionscanada.gc.ca/databases/canada-gazette/093/001060-119.01-e.php?document_id_nbr=1572&f=p&PHPSESSID=fun3k5cvdm2dg535aktcjrl9d1

"Regulations Amending the Agriculture and Agri-Food Administrative Monetary Regulations" Regulatory Impact Analysis Statement, Canada Gazette Part II 136, no. 11 May 2002, http://www.gazette.gc.ca/archives/p2/2002/2002-05-22/pdf/g2-13611.pdf

Annex D: Interview Guides

AMPs Interview Questions

What has been your experience with AMPs? (time, types, what Acts, general roles played)
How have regulated parties responded to AMPs implementation over time? Why do you say that? (Reference specific case examples and/or data analysis performed on compliance rates, payments, appeals etc.)
In your view, how consistent is AMPs with CFIA's mandate?
What are the key roles to be played in AMPs delivery by important actors? Please name each key actor and their role.
Are there differences in stakeholder understanding and interpretations of those roles, including CFIA employees? What are these?
How have AMPs been used over time? Has this changed? How?
From your experience with AMPs - do they work differently for the different Acts and situations? How? Why?
How well have AMPs worked in general and in specific areas? (individuals, interpersonal relations, sectors and situations)
In general and in your view, where do AMPs work best? Why? Do you have recommendations regarding where, with whom and how to use AMPs?
How do AMPs compare, in terms of what they deliver, to alternatives like regular ticketing, prosecutions, other? Why ? How does the comparison differ by area of application? (How do AMPs compare in each key area of context ?- i.e different Acts and different process situations.)
Are there more cost effective ways of achieving the same results? (e.g. in combination with other instruments) Why?
What would happen without AMPs? In your opinion, if AMPs were stopped tomorrow - what difference would it make?

AMPs Regional Interview Questions

What has been your experience with AMPs? (time, types, what Acts, general roles played)
Why were AMPs used in your situation? (Needs addressed, what was the basic intent? )
How are AMPs applied in your Region? (Refer to background materials and previous write-ups where available.) How does this differ - to your knowledge - from other groups? Why?
How have regulated parties responded to AMPs implementation over time? Why do you say that? (Reference specific case examples and/or data analysis performed on compliance rates, payments, appeals etc)
How consistently is the intent of AMPs and its mechanisms understood? Why do you say that?
In your view, how consistent is AMPs with CFIA's overall intent in your area? (i.e. the Key Acts you support)
What are the key roles to be played in AMPs delivery by important actors? Please name each key actor and their role
Are there differences in stakeholder understanding and interpretations of those roles? What are these?
Are roles and responsibilities - as laid out in your situation - appropriately aligned? Why do you say that?
How have AMPs been used over time? Has this changed? How?
From your experience with AMPs - do they work differently for the different Acts you apply and situations you face? How? Why?
How well have AMPs worked in general and in specific areas? (Probe for differences with certain individuals, interpersonal relations, sectors and situations.)
In general and in your view, where do AMPs work best? Why? Do you have recommendations regarding where, with whom and how to use AMPs?
How do AMPs compare, in terms of what they deliver, to alternatives like regular ticketing, prosecutions, other? Why ? How does the comparison differ by area of application? (How do AMPs compare in each key area of context ?- i.e. different Acts and different process situations.)
Are there more cost effective ways of achieving the same results? (e.g. in combination with other instruments) Why?
What would happen without AMPs? In your opinion, if AMPs were stopped tomorrow - what difference would it make?
What improvement should be made to NETS to make it into a sufficient information management system to support AMPs?

Footnotes

Footnote 1

These amounts represent increases implemented in October 2010.

Return to footnote 1 referrer

Footnote 2

Parts of this description of AMPs are taken directly from the AMPs website, as of March 16, 2011:
http://www.inspection.gc.ca/english/agen/transp/comp/pene.shtml

Return to footnote 2 referrer

Footnote 3

Information on the Evaluation Framework is included in Annex B.

Return to footnote 3 referrer

Footnote 4

Goldschmidt, "An Evaluation of the Present and Potential use of Civil Monetary Penalties as a Sanction by Federal Administrative Agencies" in 2 Recommendations and Reports of the Administrative Conference of the United States (1972) 896.

Return to footnote 4 referrer

Footnote 5

Ayres, Ian and Braithwaite, John, Responsive Regulation: Transcending the Deregulation Debate, Oxford University Press, 1992, p. 35.

Return to footnote 5 referrer

Footnote 6

License revocation and suspension are not always the most severe enforcement tools. Depending on the sector and circumstance, for example, prosecution can be more severe for some.

Return to footnote 6 referrer

Footnote 7

CFIA offences are not "criminal" offences but rather "regulatory" offences.

Return to footnote 7 referrer

Footnote 8

Prabhu, Mohan (2010) Efficacy of Administrative Monetary Penalties in compelling compliance with Federal Agi-Food Statutes, mimeo Ch 1, p. 2.

Return to footnote 8 referrer

Footnote 9

The Yale Law Journal (1992) 101 (5); as cited in ALRC Background Paper 7, supra note at 12.

Return to footnote 9 referrer

Footnote 10

To determine costs, the RIAS used a combination of oral hearing costs of $1,400 and the Province of Ontario's prosecution costs, estimated at $13,000: The RIAS estimated that if 40 prosecutions could be avoided, then the costs of the Canada Agricultural Review Tribunal ($460,000 annually) would be covered. Note that no citations are included in the RIAS.

Return to footnote 10 referrer

Footnote 11

CFIA's original intention in 1998 was to phase AMPs into all its programs over the course of the first year, starting with meat inspection, feeds, eggs and animal transport and, after six months of evaluation, into the remainder of CFIA programs. The AMPs Act lists nine acts to which AMPs can be applied, including that for seeds, feeds, fertilizers, meat inspection, plant protection, pesticides and agricultural products.

Return to footnote 11 referrer

Footnote 12

Twelve industry association representatives were interviewed, including members of the Canadian Cattlemen's Association, Canadian Beef Breeds Council, Western Stock Grower's Association, Canadian Poultry and Egg Processors Council, Chicken Farmers of Canada and the Canadian Pork Council, among others.

Return to footnote 12 referrer

Footnote 13

See section 4.1 for the significance of animal transport AMPs.

Return to footnote 13 referrer

Footnote 14

CFIA divides Canada into four areas and 18 regions.

Return to footnote 14 referrer

Footnote 15

AMPs for minor violations must be administered within six months of the violation whereas AMPs for serious and very serious violations must be administered within two years.

Return to footnote 15 referrer

Footnote 16

Prabhu, Op. cit.

Return to footnote 16 referrer

Footnote 17

Ibid.

Return to footnote 17 referrer

Footnote 18

2009-10 CFIA accounts receivable data. Efforts are underway to find ways of collecting these overdue penalties, as noted in section 5.2.

Return to footnote 18 referrer

Footnote 19

Warnings can also be appealed.

Return to footnote 19 referrer

Footnote 20

These time periods are as per the legislation. There are two means of appealing an AMP: a Ministerial Review, or a review by the Canadian Agricultural Review Tribunal.

Return to footnote 20 referrer

Footnote 21

Op. cit.: Goldschmidt and Prabhu. See also Tait, Amanda, The Use of Administrative Monetary Penalties in Consumer Protection, 2007, The Public Interest Advocacy Centre.

Return to footnote 21 referrer

Footnote 22

Summary conviction offences are defined in many legal reference sources (e.g., http://legalglossary.ca/dictionary and http://www.irwinlaw.com/cold/term/307) as crimes which are considered less serious and which are handled by the courts with the simplest procedures. The most common comparison raised by respondents was provincial tickets for traffic violations or hunting out of season.

Return to footnote 22 referrer

Footnote 23

The evaluation found that while the investigation must always result in sufficient evidence to meet the Minister's burden of proof, often the investigation was more in depth than necessary.

Return to footnote 23 referrer

Footnote 24

Robert and Black, Julia, Really Responsive Regulation (December 2007). LSE Legal Studies Working Paper No. 15/2007. The CFIA process involves conducting inspections activities to verify compliance (i.e. inspect, sample, audit, assess). There may then be an opportunity for the regulated party to take corrective action and the most appropriate response to address the non-compliance is determined. The Compliance and Enforcement Policy sets out the continuum that is used within the CFIA.

Return to footnote 24 referrer

Footnote 25

Pawson, Ray, Evidence Based Policy: A Realist Perspective, Sage, 2006, and Mayne, John, paper on complex evaluation, to be published 2012.

Return to footnote 25 referrer

Archived - Evaluation of Administrative Monetary Penalties

This page has been archived

Table of Contents

Executive Summary

Conclusions

Effective Implementation

Recommendation 1

Strategic and Appropriate Use

Recommendation 2

List of Abbreviations

Preamble

Introduction to the Evaluation of AMPs

Scope

Methodology

How AMPs Work

Introduction

1.0 Need

1.1 What Needs are AMPs Supposed to Address?

1.2 AMPs at the CFIA

1.3 Who is Affected?

1.4 What Has the Reaction Been to Date?

2.0 Alignment with Government Priorities

3.0 Alignment with Federal Roles and Responsibilities

3.1 Roles of Key Stakeholders

3.2 Differences in Understanding and Interpretation across Stakeholders

Inspectors

Investigators

3.3 Roles and Responsibilities Alignment

Legal Counsel

Canada Agricultural Review Tribunal (CART)

Program Managers

4.0 Achievement of Expected Outcomes

4.1 AMPs Actual Implementation Has Varied from What Was Planned

4.2 How AMPs Work in Different Circumstances

4.3 Achievement of Expected Results

4.4 Integrity of Implementation

4.5 Some Key Principles to Guide AMPs Application

5.0 Demonstration of Efficiency and Economy

5.1 AMPs vs. Alternatives

5.2 More Cost-effective Ways of Achieving the Same Results

Efficiencies through legal and administrative process improvements

Need for greater training of inspectors

Strategic Use

5.3 What Would Happen Without AMPs?

6.0 Conclusions, Good Practices and Recommendations

6.1 The Process - Build Learning Into Delivery

Conclusions

Good Practices

Recommendation 1

6.2 Strategy - An Integrated Approach to Compliance

Conclusions

Good Practices

6.3 Implementation - Setting the Right Conditions

Conclusions

Good Practices

Recommendation 2

Annex A: AMPs Results Logic – Expected vs. Actual

Annex B: Evaluation Framework, Methodology and Limitations

Issues, Indicators, Sources and Collection Approaches

Annex C: Bibliography

Academic / Industry

Government of Canada

Legal/Regulatory

Annex D: Interview Guides

AMPs Interview Questions

AMPs Regional Interview Questions