Deepfakes: An Increasing Risk for our customers

The strongest protections are controls that prevent a single convincing communication from causing harm.

We have written on this topic before but it warrants revisitng. I think deepfakes should now be treated as a medium and increasing risk for sectors like healthcare, finance or IT.

The technology is becoming more accessible, convincing and much easier to use. These are not always sophisticated fake videos. In many cases, a cloned voice, manipulated image or impersonation can be enough to convince someone to make a payment or disclose information.

Our customers may be particularly exposed where they:

• publish large volumes of identifiable images or videos of children, patients or vulnerable people;

• have senior leaders who can authorise urgent payments;

• use telephone or video calls to verify identity;

• provide remote clinical services;

• have prominent spokespeople whose image could be used to promote false products or philosophies; or

• provide safeguarding, domestic abuse, sexual-health or mental-health services.

  
  

What could happen?

There are several realistic ways in which deepfakes could affect an organisation.

Payment and account fraud: Someone could use a cloned voice or fake video to pretend to be a senior member of staff and request an urgent payment, a change to bank details or access to confidential information.

False clinical or organisational communications: A fake video could make it look as though a professional is promoting a something, giving incorrect advice or making comments that damage the organisation’s reputation.

Abuse involving patients, staff or children: Photos shared online could be used to create sexualised, humiliating or abusive fake images, particularly affecting women, children and vulnerable people.

Identity and access fraud: A fake voice, face or message could be used to impersonate a patient or member of staff in order to reset an account, obtain personal information or persuade someone to disclose login details.

Fabricated evidence and complaints: An organisation could receive fake audio, video or images that appear to show misconduct, poor care or another serious incident. This means organisations should not automatically assume that digital evidence is genuine, but nor should they dismiss it without proper investigation.

The most effective protections aren't technical

Organisations should not rely on staff being able to identify a deepfake by looking for visual glitches or unusual speech patterns. As the technology improves, these signs may become harder to spot.

The strongest protections are controls that prevent a single convincing communication from causing harm.

Awareness

Staff should understand that a familiar face, voice or video is no longer reliable proof of identity, especially where a request is urgent, unusual or confidential.

Deepfake awareness should form part of wider fraud, phishing, safeguarding and information-security training rather than being treated as a completely separate issue.

Authority to challenge and delay

Staff must be explicitly empowered to pause, question and independently verify a request without fear of criticism.

This is particularly important where the request appears to come from a senior leader or trusted colleague. Urgency and seniority should not be allowed to override normal controls.

A member of staff who delays a payment or disclosure in order to verify it should be supported, not criticised.

Segregation of duties

No single person should be able to initiate and complete a high-risk action such as:

• changing bank details;

• making a significant payment;

• resetting access;

• releasing sensitive information; or

• overriding an established control.

Requiring a second person to review or approve the action reduces the likelihood that one convincing impersonation will succeed.

Independent verification through a trusted channel

High-risk requests should be confirmed using contact details or systems already held by the organisation.

Staff should not rely on replying through the same call, message or video channel used to make the request. For example, a payment request received by telephone should be verified by calling a previously recorded number or checking through an approved internal system.

Should policies be updated?

Deepfakes do not necessarily require an entirely new policy.

For many organisations, the best approach will be to update existing policies covering fraud, information security, identity verification, acceptable use, safeguarding and incident management.

A section dealing with masquerading or impersonation would be a particularly appropriate place to address the risk.

This could make clear that masquerading may involve:

• cloned voices;

• manipulated or synthetic video;

• altered images;

• impersonated email or messaging accounts; and

• AI-generated communications designed to appear as though they came from a trusted person.

  
  

The policy should also state that staff are authorised to pause and independently verify unusual or high-risk requests, and that established approval processes must not be bypassed on the basis of a voice call, video call or message alone.

It may also be appropriate to update the induction process to include the authority to challenge and to understand how urgent, high risk actions are taken in the organisation.

A proportionate response

Organisations do not need to assume that every video, image or telephone call is fake.

They do, however, need to recognise that seeing or hearing someone is no longer sufficient proof that the communication is genuine.

💡 Have a Data Protection Question?

Ask the experts through our low cost advice hub

🖱️ Interested in Healthcare myKafico

One month free trial of our DPO Compliance platform

Emma Cooper, AI and Data Protection Nerd