Final report: Evaluation of the Canadian Food Inspection Agency's digitization of service delivery through the Digital Service Delivery Platform

On this page

• Executive summary
• Background
• Context
• Overview and scope
• Methodology
• Observations and recommendations
  • Governance
  • User experience
  • Funding
• Acknowledgements
• Annexes
  • Annex 1: Methodology
  • Annex 2: Limitations and mitigations

Executive summary

The Digital Service Delivery Platform (DSDP) is made up of an internal (Microsoft Dynamics 365 Customer Relationship Management (CRM)) and an external (My CFIA) portal for providing service delivery. Before DSDP, most applications for Canadian Food Inspection Agency (CFIA) services were offered only by fax, mail, or in person at a CFIA office.

The evaluation assessed whether the CFIA's use of DSDP improved the service experience and operations in accordance with the Treasury Board Policy on Service and Digital. The evaluation also examined user experience through inspector interviews and a survey, a review of four selected digitized services, and data previously collected by the CFIA from external clients.

The evaluation recommended that the CFIA clarify and document accountabilities, roles, and responsibilities for digital service delivery. Following the establishment of clear authorities, it is recommended that a strategic plan and implementation roadmap detailing how the CFIA will digitize its services, including improvements to the user interface for DSDP, be created. Training should also be updated in coordination with planned releases.

The evaluation found that 52% of CFIA staff were not satisfied with DSDP. Comprehensive data on the satisfaction of external clients with My CFIA was not available. Overall, the evaluation team was not able to conclude whether the CFIA's digital initiatives delivered through DSDP improved service experience and operations in accordance with the Treasury Board Policy on Service and Digital.

Background

The DSDP is an integrated technology platform that uses multiple commercial off-the-shelf based technologies that are employed by both government and industry. DSDP refers to both an internal (Microsoft Dynamics 365 CRM) and an external (My CFIA) portal for providing service delivery. Originally, the project to digitize service delivery at the CFIA was called Electronic Service Delivery Platform (ESDP). The ESDP project started in 2011 and was completed in 2018. From 2018 onwards, the initiative became known as DSDP.

Before 2017, most applications for CFIA services were offered only by fax, mail, or in person at a CFIA office. In January 2017, the CFIA launched My CFIA as an external facing interface on DSDP. Regulated parties, industry, and the general public can use My CFIA to access available services online. My CFIA provides a single, integrated e-Services platform for the CFIA, supporting all client interaction points. The My CFIA portal allows industry to conduct business transactions (for example, licence application) online.

DSDP is a tool that gives the agency staff the ability to issue export certificates and permissions (such as licences, permits, and registrations), capture inspection data, document compliance decisions, complete technical reviews, generate inspection reports, and assess and manage risk.

DSDP is the CFIA's largest single digitization initiative, with a current and planned scope to encompass most of the CFIA's external transactional regulatory permission services like licences, as well as much of the internal inspection-related case management. The CFIA also has invested in other technologies and applications outside DSDP that were not examined as part of this evaluation.

Since fiscal year (FY) 2011 to 2012, the CFIA has received over $120 million in funding to support the digitization of service delivery. Funding commenced in 2011 for the initiation and planning of ESDP as part of the Plan to Modernize Canada's Food Safety Inspection. This was followed by subsequent funding in 2015 for the implementation and close-out phases of the ESDP project, which was completed in 2018.

The original objectives of ESDP were to:

• give industry access to improved services
• give CFIA staff access to better information
• provide the agency a better understanding of the risk environment
• enable sharing of information more effectively with trading partners

ESDP was anticipated to include the registration of all external entities, permissions, export certificates, internal work tasking, inspection, and enforcement activities. The CFIA was provided the expenditure authorities to undertake ESDP at a substantive cost estimate of $52.1 million. Between 2011 and 2018, the CFIA accessed up to $49.4 million from the fiscal framework to complete this project.

Context

DSDP was conceived and designed as a platform to support Food, Plant Health, and Animal Health business lines. It was built and launched in 2018 to 2019, in the context of the Safe Food for Canadians Act and regulations.

This has 2 major implications:

1. the internal users of the system have primarily been from the Food business line
2. those users were adopting DSDP at a time when they were also undergoing a shift from prescriptive regulations to an outcome based approach, significantly increasing the complexity of the change management context

As external services have expanded, more internal users from outside of Operations Branch are accessing the system. For example, Policy and Programs Branch specialists who work on pre-market approvals, and International Affairs Branch staff who support export certifications.

Overview and scope

The CFIA's digitization efforts seek to deliver end-to-end digital services and information that is easy to find and understand. The CFIA's digitization efforts also aim to increase industry access to online service options and facilitate access to international markets.

Evaluation objective

To determine whether the CFIA's digital initiative through DSDP improved service experience and operations in accordance with the Treasury Board Policy on Service and Digital.

Evaluation scope

The evaluation focused on governance and accountability of the digitization of service delivery through DSDP as it relates to strategy and implementation. It also focused on internal user experience to deliver services digitally. Finally, the evaluation focused on activities to progress the digitization of service delivery at the CFIA from April 2018 to June 2023.

Use of findings

The findings of this evaluation will support the agency's onboarding and management of services. The findings will also help to identify where the digitization initiative through DSDP is working well and where there are opportunities for improvement.

Methodology

The mixed method research design for this evaluation incorporated seven lines of evidence using multiple data collection and analysis methods. The evaluation was guided by the following questions:

• 1. Does the CFIA have effective governance processes in place to support the successful implementation and oversight of digitization initiatives and outcomes?
• 2. To what extent are the CFIA's digital transformation initiatives improving service experience and operations?
• 3. How did the CFIA spend planned funds to achieve the outcomes?

The observations presented in this evaluation are supported by data collected between 2022 and 2023 from various sources, including:

• an internal user survey with 795 responses
• demonstrations
• four case studies
• interviews with CFIA employees and employees from other government departments that use Microsoft Dynamics CRM
• a document review
• a review of relevant financial data

Innovation, Business and Service Development Branch (IBSDB) engaged peer developers to do a deeper dive on the initial findings.

There are numerous internal and external stakeholders involved in the digitization of service delivery. Conducting interviews with all stakeholders was not possible. The evaluation design used a purposeful sample for interviews, meaning interviewees were not randomly selected. The evaluation team did face some difficulty and delays in identifying documents and staff with essential information for this evaluation.

The CFIA has numerous activities related to the digitization of service delivery. Evaluating all aspects of these activities was not feasible. Therefore, the evaluation team focused on four services as case studies representing the three business lines:

• Safe Food for Canadians Licence
• Certificate of Free Sale
• Domestic Movement Certificate
• Export of Live Bovine Certificate

Annex 1 details the full methodology, while Annex 2 details the full limitations and mitigations.

Observations and recommendations

• Governance
• User experience
• Funding

Governance

The digitization of CFIA's service delivery through DSDP is currently composed of two distinct parts:

• projects that received specified funding
• the rest of the initiative to digitize services via DSDP

The e-Certification and Digitization Project (e-Cert) has a clearly defined executive sponsor (Vice President (VP), International Affairs Branch), and business sponsors (Executive Directors from Operations and International Affairs Branches) responsible for the project, and the VP level e-Certification and Digitization Senior Project Advisory Committee (VP SPAC).

Separately, the rest of the initiative lacks a business owner or sponsor. It is unclear from interviews and document review whether IBSDB or Operations Branch is the lead or has the authority to prioritize the onboarding of services or functions to DSDP.

In the period examined by this evaluation there were many different governance bodies responsible for digitization of service delivery through DSDP, including the Strategic Priorities Oversight Committee in 2019, the Digital Services Steering Committee in 2021, and the Intake Triage Committee in late 2022.

The rest of the digitization initiative leverages the capabilities that were built in the platform to achieve the objectives of the e-Cert project. Additionally, product development work being done to advance the e-Cert project is identified as an agency priority, with its own dedicated resources in IBSDB. This is how the agency is currently advancing digital service delivery in the absence of a strategic plan.

The impact of the absence of clearly defined governance and accountabilities was evident from the scoping interviews with senior executives conducted by the evaluation team in spring 2022.

As a result of unclear governance at both the tactical and strategic levels, where the E-cert project meets the rest of the DSDP initiative and within the DSDP initiative itself, the digitization of service delivery has met with delays in achieving its objectives. Planning and interdependencies were not analyzed in advance, resulting in project changes that impacted resource allocation and release dates. Additionally, without a single point of governance, items relating to the digitization initiative were reported to multiple governance committees where the evaluation team observed limited discussions before items were approved and also limited guidance relating to roles and responsibilities.

An almost 100% turnover of senior executives since 2019 was cited in interviews as being a compounding factor leading to difficulties with creating and following plans at the operational and strategic levels of the digitization initiative.

From the case studies, the evaluation team observed that without an agency-level strategy, the business lines and branches have interdependencies and conflicting priorities for digitizing services that are not being governed efficiently or effectively.

The evaluation team expected to see a clearly identified individual or group responsible for digitization of service delivery at the CFIA, including for DSDP. Over the years the responsibility has changed many times and still remains unclear.

One of the objectives of the Policy on Service and Digital (3.2.1) is that integrated decision making is supported by enterprise governance, planning and reporting. There is no evidence to support that the CFIA is meeting this objective.

It is important to differentiate between funded projects that are under CFIA authority and the rest of the digitization initiative, as governance of the former is more robust.

The evaluation team expected to see a clear plan for how the agency would achieve the original goal of 100% of services available digitally. There appears to be no overall strategic plan due to a lack of accountability and responsibility for the digitization of services overall.

While there are many interdependencies as the business lines transform their current business processes to digital, ultimately there needs to be a strategic plan whereby all services are prioritized to achieve the agency's goal. For example, plant health business line developed a plan to digitalize their business in 2022, but an overall plan for the agency is required to align and prioritize multiple plans.

As of July 2023, prioritization was being done by IBSDB's Development and Operations Group with input from Operations Branch and the business lines.

User experience

• Use of DSDP
• User-centric design
• Feedback
• Training
• Support

Use of DSDP

In order to evaluate whether DSDP improved service experience and operations in accordance with the Treasury Board Policy on Service and Digital, the evaluation team accessed the DSDP training environment, read and watched training materials, interviewed staff and management in operational areas, administered an internal user survey, and received advice from external Information Technology (IT) specialists familiar with the use of Microsoft Dynamics CRM.

The evaluation team heard from inspectors interviewed that they did not enjoy using DSDP. The interviewees gave their satisfaction with the platform an overall rating of 5.5 out of 10. As a result of this small (33) convenience sample group, the evaluation team decided to conduct a wider survey to approximately 3,000 internal users and 795 responses were received. The evaluation working group, composed of staff from all relevant areas of the agency, provided input into the survey design and testing.

From the survey, the evaluation team was able to conclude that internal user experience is not satisfactory and does not lead to an improvement in operational effectiveness. The vast majority of users (78%) reported that they do not input data into DSDP while on site conducting an inspection, but rather take notes on paper to later enter when they are back at a CFIA office. Consequently, there is a duplication of effort and extra time spent to complete data entry. When ESDP was originally being developed, the intention was to equip inspectors with tablets to have easy access to relevant information (for example, inspection procedures, compliance history) on site, and to be able to record findings and upload evidence collected into the inspection case file. Only 17% of respondents agreed that DSDP does what it was promised to do.

In addition, the evaluation team received demonstrations and attended an inspection in order to better understand the difficulties that inspectors experienced. During these hands-on experiences, the evaluation team observed that while not impossible, it would be extremely difficult to carry a tablet during an inspection due to hand washing and sanitizing requirements. It can also be very noisy on site and there can be significant changes in temperature, making it difficult to focus on the numerous free-text fields required by DSDP for the completion of an inspection. Additionally, internet connections are not always available or reliable on site.

Text version – Impact of DSDP on day to day work

Ability to retrieve information

• Positive impact (4-5): 33%
• Negative impact (1-2): 49%
• No impact (3): 18%

Workload

• Positive impact (4-5): 22%
• Negative impact (1-2): 53%
• No impact (3): 25%

Time it takes to perform my tasks or duties

• Positive impact (4-5): 21%
• Negative impact (1-2): 60%
• No impact (3): 19%

Approximately half of respondents stated that they were not satisfied with DSDP and that using the platform had a negative impact on the time it takes to perform their tasks or duties. Corresponding data on the satisfaction of external clients with My CFIA was not available, making it difficult to assess how well the CFIA is meeting the expected results of the policy.

User-centric design

In fall 2021, a decision was made by the e-Certification and Digitization Senior Project Advisory Committee (VP SPAC) to prioritize the implementation of critical functionalities of the platform over digitizing export certificates. However, there remain general improvements to the platform as of spring 2023 that could be made to enhance internal user experience, such as more auto-fill/auto-complete fields, automated data validation to prevent user errors with data entry, and upgrading to the cloud-based version of the Microsoft Dynamics CRM to access the newest functionality.

It's critical that the internal user experience be improved in addition to adding new services. The data that is exported from DSDP is only as good as what users input. There is a risk that with internal users finding data entry onerous that the resulting data may not be accurate and complete.

80% of internal users also expressed that they have to use other tools in order to complete their work. This a source of frustration with 46% of respondents stating that this results in a duplication of work thereby decreasing efficiency. This issue will be complex to resolve as it does not only involve design improvements to the platform but also increased training and guidance to internal users and change management to encourage a culture shift to accepting the agency's way forward.

Text version - Sprint 4 week cycle

Sprint 4 week cycle: Platform updates take place during development sprints, which are four-week cycles when maintenance and development activities are addressed by the IBSDB development team.

Steps to complete a sprint cycle:

• Idea
• Planning, prioritization, and scheduling
• Development
• Test and acceptance
• Deployment

The evaluation team noted that user experience is not isolated to platform design and interface. Rather, the work thats required before digitizing a business process by Operations is crucial to identify resourcing and budget requirements for IBSDB. The Plant Transformation Team shared their best practices as part of case study interviews where they identified having regular engagement with horizontal stakeholders as a key factor in their successful planning work.

Subject matter experts who attended the DSDP walkthrough in May 2023 recommended that business lines dedicate employees to the DSDP development team to define and refine business requirements. This would involve an investment on the part of the operational area and in return they would receive a product that better meets internal user needs. Internal end users should be involved throughout the product development cycle and there should be a mechanism for feedback to be provided post-deployment.

Case studies also highlighted the need for end-user input, testing, and feedback as part of the development cycle. Resources should be set aside to support continuous feedback post-deployment from implicated areas.

Feedback

The evaluation team heard during inspector and case study interviews that suggested improvements to the platform made by users were not implemented. However, when questioned further, the vast majority did not know how to submit a ticket for a product improvement request (PIR) or bug. Only 13% of survey respondents said they had submitted a ticket.

The evaluation team leveraged the expertise of members of the working group to obtain further information on the process for submitting tickets for a PIR or bug. As a result, the evaluation team was invited to meetings of the Digital Services-Product Development Group (DS-PDG), interviewed employees and management from Platform and Applications-Development and Operations in IBSDB (for internal users) and Ask CFIA Team (for external users) examined SharePoint sites and excel spreadsheets tracking feedback, and learned about the newly created Intake Triage Committee led by IBSDB.

The process for how to submit a PIR was not clearly communicated to internal users and the process for how these requests would be prioritized also led to dissatisfaction with the user experience. IBSDB launched a new method for submitting PIRs in April 2023 but at that time it had not been communicated to staff.

Training

Text version - Satisfaction with duration and content of DSDP training

Satisfaction with duration of DSDP training

• Disagree (1-2): 41%
• Agree (4-5): 28%
• Neither agree nor disagree (3): 31%

Satisfaction with content of DSDP training

• Disagree (1-2): 41%
• Agree (4-5): 31%
• Neither agree nor disagree (3): 28%

Note: survey respondents refers to internal CFIA users of DSDP only.

Support

Survey respondents reported that the standard operating procedures (SOPs) for DSDP were the most common type of support used. 588 survey respondents said the SOPs were available to them, and 91% of those had used them.

However, only one third of respondents were satisfied with the SOPs. The evaluation team noted that many SOPs were updated following the survey period, in March 2023 which is a positive improvement. Other internal resources have not been updated since 2019.

The Evaluation Team observed that change management efforts to communicate where internal staff can go to access support, and promote consistency across geographic areas in Canada could be improved. The current approach has contributed to a lack of user confidence and satisfaction.

Text version - Access to DSDP support

The support services I accessed meet all of my needs

• Disagree (1-2): 30%
• Neither agree nor disagree (3): 33%
• Agree (4-5): 37%

The support services I accessed were timely

• Disagree (1-2): 26%
• Neither agree nor disagree (3): 34%
• Agree (4-5): 40%

I know where to go to get support

• Disagree (1-2): 23%
• Neither agree nor disagree (3): 25%
• Agree (4-5): 52%

I am able to get the support I need to use DSDP effectively

• Disagree (1-2): 27%
• Neither agree nor disagree (3): 24%
• Agree (4-5): 49%

Text version - Satisfaction with DSDP support

I am satisfied with the support available to DSDP

• Disagree (1-2): 32%
• Neither agree nor disagree (3): 33%
• Agree (4-5): 35%

Internal user satisfaction with the platform will not only be improved by making changes to the user interface but investments in training and support are required for users to feel more confident when using DSDP.

Text version - Confidence in ability to use DSDP

I feel confident in my ability to use DSDP

• Disagree (1-2): 40%
• Neither agree nor disagree (3): 19%
• Agree (4-5): 41%

Funding

To answer the question "How did the CFIA spend planned funds to achieve outcomes" the evaluation specifically examined two funding envelopes received by CFIA for digitization during the original evaluation period.

• $27.2 million over five years dedicated to digitization of export certificates (e-Cert) received in 2019
• $19.4 million over five years and $4.8 million in ongoing as part of Investing in CFIA (I-CFIA) received in 2020

The evaluation also examined $52.1 million in funding from 2014 for the ESDP which evolved into the DSDP.

e-Cert funding

The e-Cert and Digitization Project Vision created by International Affairs Branch (IAB) in fall 2020 had the close-out phase scheduled for January to April 2023. In FY 2020 to 2021, the CFIA lapsed $2.5 million of the $7.6 million received (33%), as it was intended to support the roll out of certificates for uptake by industry. As a result of delays to the digitization of certificates, activities supporting the "adoption roll out" of e-Cert originally scheduled for April 2021, could not take place as planned. Lapsed funds were re-allocated to later years. Progress on the automation of e-Cert was slowed by a number of unexpected obstacles:

• stakeholders identified challenges that had to be remedied before progress could commence on the digitization of export certificates (for example, critical functions that had to be in place in DSDP before e-Cert automation could be implemented, bugs that needed to be fixed)
• the COVID-19 pandemic caused a disruption to work in FY 2020 to 2021 and 2021 to 2022

Despite the initial lapse, by the end of FY 2022 to 2023, the CFIA spent 88% of funds allocated to e-Cert from FY 2019 to 2020 to FY 2022 to 2023 and 70% of the total e-Cert funds.

I-CFIA funding for digitization

I-CFIA included several funding initiatives, with only one, Digitization, being relevant to this evaluation. The evaluation was unable to examine the allocation of funding specifically for digitization under I-CFIA given that funding is not tracked by initiative in the CFIA's financial system. The decision to track funding and expenses for detailed initiatives is at the discretion of the initiative sponsor and lead. Considerations on whether to track at the detailed initiative level would depend on the complexity and materiality of each initiative as well as advice and guidance from the Corporate Management Branch.

The funds received for e-Cert were used to develop and deliver the core functionalities in DSDP, which can be leveraged to digitalize other services. IBSDB stakeholders believe that the level of effort needed to onboard new services will be significantly less than in the past due to the work completed on the platform over the past three years.

The outcomes and targets set for e-Cert were revised in August 2023 in light of the challenges experienced. e-Cert project owners believe that the revised outcomes and targets are achievable with additional funding (for a total cost of $42.4 million) and an extension in timeline to March 31, 2025.

Furthermore, while outside of the evaluation scope, it is important to note that through the COVID-19 Response funding, the agency received funding for "addressing Deferred Maintenance for Digital Services initiative". e-Cert received $8.061 million for FY 2022 to 2023 and 2024 to 2025 to address capacity required to build critical functionality, supporting digitized certificates, on DSDP.

Progress in achieving outcomes and targets identified

As of March 31, 2023, the CFIA had not met the planned targets set for indicators related to e-Cert and I-CFIA. For example, through the e-Cert funding, the CFIA had targeted to process at least 95% of export-related requests through DSDP by FY 2024 to 2025. As of the end of FY 2022, 6% of requests were being processed electronically.

The CFIA recognized that the original targets related to e-Cert were not well defined. e-Cert project owners acknowledged that as they matured their performance indicators and results work, they saw the need to reframe indicators to more meaningful ones. As a result, project artefacts for e-Cert were revised and internally approved on September 27, 2023.

The revised scope for e-Cert is now 74 European Union (EU) Traces certificates and 9 other certificates, for a total of 83, instead of the original 1,005. Program officials explained that most of the 1,005 certificates are either low export volume or for countries that may not accept electronic export certificates.

As of August 31, 2023, 21 services were currently online via My CFIA. There were 23 additional services listed as "to be released", 17 of which were export certificates, designed to support international trade and Canadian businesses by allowing the CFIA to communicate with other governments through more direct and reliable means.

While progress has been made to mature the platform, the evaluation findings relating to internal user experience do not suggest that the anticipated benefits and outcomes of digitization (based on the initial funding received to establish ESDP) have been met.

Acknowledgements

The evaluation team would like to acknowledge the contributions of all CFIA employees across Canada, and other government department representatives, who participated in the interviews and survey in support of this evaluation. Their perspectives and experiences were essential to this evaluation.

The evaluation team would also like to thank the members of the DSDP Evaluation Advisory Committee for their expertise and cooperation throughout this evaluation.

Annexes

Annex 1: Methodology

This evaluation focused on digitization of the CFIA's service delivery activities that took place from April 2018 to June 2023. This is the period to which the evaluation conclusion applies.

The evaluation team interviewed more than 85 CFIA employees located across Canada. The evaluation team spoke with employees from all activities that support service delivery using DSDP or My CFIA, including (but not limited to):

• Policy and Programs Branch (PPB)
• Innovation, Business, and Service Development Branch (IBSDB)
• Operations Branch
• International Affairs Branch (IAB)
• Corporate Management Branch (CMB)
• Human Resources Branch (HRB)
• Science Branch

From December 2022 to January 2023, approximately 3,000 CFIA employees trained to use DSDP were invited to answer a survey designed by the evaluation team in conjunction with working group members from implicated areas of the agency. 795 responses to the survey were received and analyzed by the evaluation team.

Using qualitative data analysis software and quantitative analysis using Excel, the evaluation team identified themes and trends within the survey responses.

The evaluation team also conducted an extensive review of hundreds of documents to inform the evaluation.

This evaluation was supported by an advisory committee that included members of branches involved in the evaluation scope, as nominated by their respective Vice Presidents. These members provided feedback on the evaluation terms of reference and preliminary findings, and confirmed the factual accuracy of the final report.

Interviews

The evaluation team conducted interviews with 33 CFIA inspectors in November 2022 and found that there were many issues commonly raised by internal DSDP users across Canada. The evaluation team sought to determine whether these issues were being experienced by the larger DSDP user community.

Survey

This voluntary survey was launched on December 16, 2022 and was designed to help answer one of our evaluation questions: Were DSDP users able to navigate the system to enter and/or access the data they needed to do their activities and make informed decisions?

The survey explored use of DSDP, impact of DSDP on work, user satisfaction, as well as training and support available to users.

The survey was targeted to all CFIA staff trained to use DSDP regardless of whether they are using it currently.

Not all questions were mandatory; 'respondents' refers to the number of people who answered a question unless otherwise stated.

The survey was live from December 18, 2022 to January 13, 2023.

The survey was distributed to 2969 CFIA employees and 795 responses were received, a response rate of 27%. The margin of error was +/- 2.97% at 95% confidence level.

Case studies

The evaluation team examined four case studies in order to learn about the experience of onboarding digitalized services from the perspective of business owners and other internal stakeholders. The case studies informed the findings and recommendations related to governance and user experience.

Certificate of Free Sale (CFS)

CFS is available for food products manufactured by licensed parties under the Safe Food for Canadians Act (SFCA) and Safe Food for Canadians Regulations (SFCR) when necessary to meet the requirements of importing countries. It's an export certificate service.

The CFS was chosen because it uses information already submitted for SFC licence to issue CFS automatically.

Domestic Movement Certification (DMC)

DMC is part of the effort to restrict the movement of commodities that have the potential to spread pests across Canada.

DMC was chosen to represent CFIA's Plant Health business line. DMC is the least developed of the case studies, with early adoption set to begin in 2024.

Live bovine (LB)

The CFIA uses electronic export certificates for live bovine to the United States using My CFIA. Participating exporters are from London, Ontario, Saint-Hyacinthe, Québec, and the provinces of British Columbia and Alberta.

LB was suggested by Animal Health business line and, with some components in early adoption, LB offers insight insights into the development process.

Safe Food for Canadians (SFC) licence

Safe Food for Canadians Regulations (SFCR) requires businesses that import or prepare food for export or to be sent across provincial or territorial boundaries to have licences.

The Safe Food for Canadians (SFC), the licence associated with SFCR, was chosen for the case study because it was the first licence to be digitized as part of the DSDP initiative, and has been fully onboarded since January 2019. SFC has already established functions on DSDP, and enrollment is required for external users. The SFC case study represents the CFIA's Food business line.

The evaluation team interviewed 38 CFIA employees (both former and current) as part of the case studies.

• CFS: 10
• DMC: 3
• LB: 8
• SFC licence: 17

The case studies were guided by the following evaluation questions:

• Are/were the governance model(s) for the implementation of the digitization of services effective?
• What challenges did the CFIA face to digitize services, what was the impact, and how did the CFIA address those challenges?
• Did the digitization of service delivery meet the needs of the internal users and external users?

The case studies were selected with input from the evaluation's advisory committee in consultation with members of the business lines.

Annex 2: Limitations and mitigations

The evaluation team used several strategies to mitigate the impact of limitations of the evaluation.

The CFIA has numerous activities related to digitization of service delivery. Evaluating all aspects of these activities was not feasible. Therefore, the evaluation team focused the evaluation on four services as case studies representing the three business lines, and that are in different stages of being onboarded to DSDP.

There are numerous internal and external stakeholders involved in digitization of service delivery. Conducting interviews with all stakeholders was not possible. The evaluation design used a purposeful sample for interviews, meaning that rather than randomly selecting interviewees, the selection was based on geographic area, commodity, and expertise. Furthermore, the evaluation team did not conduct any interviews or surveys directly with external users of My CFIA. There was limited data available on external user satisfaction via annual public opinion research.

The CFIA's digitization of service delivery was evolving during the time period of the evaluation: April 1, 2022, through June 1, 2023. This increased the complexity of the analysis. The evaluation team gathered information related to ongoing changes to the CFIA's digitization of service delivery activities and took this information into consideration when drafting findings, conclusions and recommendations.