The statistics are alarming: 44% of FTSE100 companies talked about launching AI projects in their last annual report, and only seven linked these to cybersecurity. How can future disasters be prevented—pragmatically and concretely?
As the facts show, attacks on artificial intelligence systems are already happening. When the iPhoneX was released in 2017, Apple boasted of having created an extremely robust facial recognition system. A week later, and at a cost of less than $150, the Vietnamese cybersecurity company, Bkav, managed to create a mask that was capable of duping the application.
Artificial intelligence (AI) is revolutionising our daily lives: autonomous cars, behavioral biometry, predictive medicine, intelligent chatbots and content suggestion. New uses are appearing every day, but the cybersecurity risk management of using these new technologies is rarely discussed.
What are the specific risks associated with AI? What questions do you need to ask before tackling them? What security solutions can be applied in this innovative area? And how can you select one and put it in place?
44 % of companies mention AI or machine learning
7% only mention AI or ML in relation to cybersecurity
1 of those mentions AI or ML presenting a cyber risk – the others use it to help mitigate cyber risk (mostly through predictive analytics for financial crime prevention).
ARTIFICIAL INTELLIGENCE:LOOKING BEYOND THE BUZZWORDS — WHAT’S BEHIND THE CONCEPT?
Artificial intelligence can reproduce human intelligence. Its scope ranges from medical analysis programmes that detect tumors, to intelligence capable of driving your car completely autonomously.
In AI applications, we can distinguish between those whose rules are fixed in advance by experts and those with the ability to adapt their behavior to the situation. For this second case, we use the term machine learning. This second type of system relies on large amounts of data, manipulated using increasingly powerful computers and analysed to automatically recognise patterns, which then serve as the basis for decisions. There are various learning methods, including supervised learning, unsupervised learning, and reinforcement learning.
These systems, with their own learning mechanisms and the ability to modify decision thresholds in an adaptive way, offer a fundamentally different approach compared with historical systems. In addition, AI and machine learning are often conflated, including when considering cybersecurity. This Insight is no exception, and behind the broad title of Artificial Intelligence, it addresses, in particular, the management of cybersecurity risks related to the use of machine learning.
New Challenges for Cybersecurity Teams
Why attack AI?
Attackers mainly seek to:
/ Disrupt the AI application’s operation, by deliberately causing an erroneous decision using a chosen data set.
/ Sabotage the operation of the AI itself, preventing or disrupting the application’s operation.
/ Understand and «reverse engineer» the model by studying it behavior. Stealing, and then reselling, a model can be very lucrative, and can attract buyers who want to cut corners in the endless race for digital innovation.
/ Steal the data used by the application.
How is AI attacked?
Attacks that specifically affect machine-learning-based applications can be grouped into three categories.
This technique is the one least related to traditional methods because it specifically targets the automatic learning phase. With poisoning, an attacker seeks to modify the AI’s behavior in a chosen direction by influencing the data used for learning. Such techniques are particularly powerful when the data used for learning is poorly controlled: public or external data with a high-learning frequency.
With inference, an attacker experiments, successively testing different queries on the application, and studying the evolution of its behavior. The attacker looks either to collect the data used by the AI (in learning or in production) or steal the model (or some of its parameters).
With evasion, an attacker plays with the application’s input data to obtain a decision different than the one that the application would normally make. They seek to create the equivalent of an optical illusion for the algorithm, known as an adversarial example, by introducing a «sound» that is carefully calculated to remain discreet and undetectable.
Six key points to securely pursue an artificial intelligence project
Here are six essential areas to consider in mastering the risks related to AI business projects :
Protect data at every step of the project : Before even talking about protecting against data leaks, it’s essential to ensure that the desired use complies with current regulations, especially those related to data protection (the GDPR, medical data regulations, PCI DSS, etc.). This means defining, as clearly as possible, the project’s purpose and the associated data processing.
Protect the big data platform : In machine learning projects, this step takes on some special dimensions. There are large amounts of highly concentrated data, and therefore particularly exposed to the risk of the theft or modification of information.
Securing the learning process : The machine learning stage is both the key step in which the solution’s effectiveness and relevance is based, and the genuinely new part of the initiative in relation to existing systems.
Securing the application : Most machine learning-specific security measures focus on three areas: managing inputs, making processing reliable, and controlling outputs.
Defining your risk management and resilience strategy : In order to adopt approach at large scale, it’s important to develop security guidelines according to the type of AI project. For example, the guidelines can be structured by input data type, by learning frequency or by the solution’s level of exposure.
Think carefully before outsourcing : Opting for an external solution always carries risks, and, over and above the need to ensure publishers consider the following points : the intellectual property, the exposure to risks and the need for reversibility at the end of the contract.
The new threat to prepare for : Deepfake
Deepfake is the modification of images, audio, or video, using AI (and especially «deep learning») to present a fake version of reality.
A Deepfake file is created using two competing AI systems: one known as the generator, and the other the discriminator. The generatorcreates a false file (image, audio, video, etc.) and then asks the discriminator to determine if the file is real or false. Together, the generator and the discriminator form what is called a Generative Adversarial Network (GAN).
A prerequisite : engage your business functions to help protect your new artificial intelligence systems
Many of the AI initiatives being pursued by companies today are still in their research or experimental stages. Their initial objective is to demonstrate AI’s value to their various business lines. These proof of values (POVs) are often done on specific use cases, based on the company’s existing data. At the same time, identifying and categorising current and future corporate AI experiments and projects enables the process of defining the security measures to begin, which needs to be integrated by prioritising the company’s actual needs.
Related Insights
Chadi Hantouche
Gwenaëlle Boulet
Claire Delgove
Julien Floch
Ouala Barhoumi
Federico Chiarelli 
Simon Weber 
Federico Di Mauro