Attack surface of machine learning

Most conventional attack vectors (insider threat, denial of service, …) can be exploited against machine learning; and in fact, they possibly shine in a new light when examined through the machine learning lens. On that end, the attack surface might look slightly different but remains very much the same. However, it will expand through new vulnerabilities introduced by the design of machine learning. Potential adversarial interference includes: 

• attacks against the data used for training and decision-making, 
• attacks against the classifier in the training environment,
• as well as attacks against model in the deployment environment. 

Machine learning is a complex process, making the attack surface equally dynamic and multifaceted. To map the entire attack surface, it is therefore useful to go through  all steps of the machine learning process and explain the different threats and attack vectors step-by-step. The resulting surface will be generic, meaning that it will cover all machine learning models, but not the entire surface will be applicable to each machine learning model. Take for example an application using online machine learning therefore learning only when it is deployed. As no data is used in the training environment to train this model before deployment, attacking the data acquisition stage and poison training data would not be an applicable attack vector. That attack vector is however applicable to other forms of machine learning such as supervised machine learning. The subsequent analysis aims at breaking down the complex attack surface of machine learning into digestible pieces to provide clues for better securing it to harvest the full potential of machine learning.

The attack surface is divided into three larger sections: the training environment, the deployment environment and the outside world. Each section includes several stages of the machine learning process where attacks can occur. Some stages occur on the intersection between the outside world and the other environments, their analysis can be found in the sections covering the training environment (for the data acquisition stage) and the deployment environment (for the online learning and output stages) respectively.