Safety Isn't Optional: What GP Practices Must Do Before Deploying New Technology
Our general practices are becoming increasingly digital and they are now using AI scribes and online consultation tools to automated document processing, clinical decision support and prescribing systems.
Many of these developments offer opportunities to reduce administrative burdens, improve access and support clinicians to spend more time with patients.
But there is an important distinction between purchasing a system and being ready to use it safely.
A supplier can tell you that its product is safe. It cannot tell you that the way your practice intends to use it is safe.
That responsibility sits with the organisation deploying the technology.
What could actually go wrong?
When we talk about clinical safety or fairness, we often think about the obvious risks: an AI tool inventing information or a patient being directed to the wrong service.
But there are many moving parts and potential risks to consider.
AI scribes and dictation tools: What happens if important information is omitted, incorrectly transcribed or attributed to the wrong patient? Who checks the clinical record before it is saved?
Online consultations and AI triage: Could a patient describing urgent symptoms be directed into a routine pathway? Have local configurations, exclusion criteria and escalation arrangements been tested?
Automated document processing and coding: What happens if an abnormal result, referral recommendation or clinically significant letter is incorrectly classified or overlooked?
Clinical decision support and risk stratification: Are staff clear about the system's limitations? Could an incomplete record affect the advice or risk score presented to the clinician?
Prescribing and medicines management: How are recommendations checked, discrepancies managed and prescribing responsibilities understood?
Systems exchanging information with clinical records: How will the practice know that information has been transferred successfully, attached to the correct patient and made available to the right person?
There are information governance considerations too. Who can access the information? Where is it processed? What happens if the supplier experiences an incident? Have patients been given appropriate information about the processing?
These are foreseeable risks that should be considered as part of introducing or changing a system but many of these can be managed through relatively straightforward controls.
Your supplier says the system is safe. But have you read the small print?
Suppliers of relevant health IT systems have their own clinical safety responsibilities under DCB0129. This includes identifying hazards associated with their products, assessing the risks and documenting the controls needed to manage them.
But there is a second standard, DCB0160, which places separate clinical safety responsibilities on healthcare organisations deploying and using these systems.
And here is something our customers may not realise: a supplier's assurance that its system is safe may depend on the practice carrying out a considerable amount of local testing, monitoring and oversight.
These responsibilities are not always obvious from the sales demonstration or the supplier's headline assurances. They may be set out in the Clinical Safety Case Report, hazard log, deployment instructions, contractual terms or other supporting documentation.
For example:
AI triage systems: The supplier may have tested its algorithms and patient pathways but expect the practice to test its own appointment configurations, urgent escalation arrangements and handling of vulnerable patients.
AI scribes: The supplier may demonstrate high transcription accuracy but require clinicians to check every generated note, identify omissions or inaccuracies and correct them before saving the clinical record.
Automated document processing: The supplier may have assessed its classification and coding technology but rely on the practice to establish checks for incorrectly categorised documents, missed clinical actions and failures in the workflow.
Clinical decision-support systems: The supplier may validate its recommendations but expect the practice to ensure that staff understand the limitations, check the underlying patient information and exercise appropriate clinical judgement.
In other words, some of the controls that make a system safe are not built into the technology at all. They are things the practice is expected to do.
If those controls have not been implemented, the practice may be operating outside the conditions or assumptions on which the supplier's clinical safety assessment was based.
And this is where the legal and professional implications become important because, if a patient is harmed and the practice is subsequently asked to explain how the system was deployed, the practice may need to demonstrate that it understood the supplier's safety requirements, undertook appropriate local testing, implemented the necessary safeguards and maintained suitable oversight.
Simply producing the supplier's Clinical Safety Case Report is not going to be sufficient, particularly if that document identifies responsibilities that the practice was expected to fulfil.
The supplier is responsible for managing risks associated with its product. The practice is responsible for managing risks associated with its local deployment and use.
Not every digital system requires a formal DCB0160 assessment; applicability depends on the system and its intended use. However, proportionate data protection, information governance, security and operational assurance remain important even where formal clinical safety requirements do not apply.
Where LENS comes in
We developed LENS – Lawful, Explainable, Necessary and Safe – to make this process more manageable for practices.
Rather than expecting practice managers and governance leads to interpret multiple technical standards, assemble lengthy documents or work out which questions they should be asking suppliers, LENS takes users through a structured assessment.
Depending on the system, this includes considering data protection, security, transparency, local configuration, staff training, testing, human oversight and other deployment safeguards.
The questions are designed to be practical so, rather than simply asking whether the practice has implemented appropriate clinical safety controls, we might ask whether staff have tested the system using realistic patient scenarios, whether important information is checked before being saved to the record, or what happens when a system becomes unavailable.
The answers help establish what arrangements are in place, identify gaps and create an evidence trail.
LENS does not make the decisions for your practice but it helps you make informed decisions, take the necessary actions and document what you have done.
How does the Clinical Safety Officer fit into this?
The process works like this:
Step 1 – The practice completes its deployment activities
The practice manager, governance lead or nominated project sponsor works through the relevant tasks within the DPIA in myKafico, confirming local arrangements and completing any outstanding compliance or deployment tasks.
Step 2 – The assessment is submitted for clinical safety review
Once the required deployment work is complete, the practice notifies Kafico. We then trigger the draft LENS Clinical Safety Report using the information recorded via step 1.
Step 3 – The CSO reviews the assessment
The draft appears in the CSO's assessments section within LENS.
The CSO can review the identified hazards and existing controls, assess their likelihood and potential consequences, and determine whether the remaining risks are acceptable.
Step 4 – Further evidence or actions can be requested
Where necessary, the CSO can set specific tasks for the project sponsor, such as providing test results, confirming training arrangements or supplying evidence of a particular safeguard.
The project sponsor remains responsible for ensuring those actions are completed.
The CSO's role is to provide independent clinical safety scrutiny, not to undertake all the practice's deployment work.
Step 5 – The Clinical Safety Report is finalised
Once the CSO has completed the review and is satisfied that the assessment supports the clinical safety conclusions, the final report can be produced.
The practice's senior management remains responsible for authorising deployment and accepting any residual clinical risk on behalf of the organisation.
This gives the practice a documented record of the hazards considered, the controls in place, any outstanding responsibilities and the basis for its decision.
Ready to review your systems?
Log in to myKafico and complete the outstanding deployment assessments for your practice. If you are unsure which systems to prioritise or how to complete the steps use the Book DPO Meeting button within the platform and we can jump on a call!
Comments