Machine Learning and NCSC
The UK's top cybersecurity agency has released new guidance designed to assist developers and others identify and patch vulnerabilities in Machine Learning (ML) systems.
GCHQ's National Cyber Security Centre (NCSC) has laid out together its principles for the security of machine learning for any company that is looking to reduce potential adversarial machine learning (AML).
What is Adversarial Machine Learning (AML)?
AML attacks compromise the unique features of ML or AI systems to attain different goals. AML has become a serious issue as technology has found its way into a rising critical range of systems, finance, national security, underpinning healthcare, and more.
At its core, software security depends on understanding how a component or system works. This lets a system owner inspect and analyze vulnerabilities, these can be reduced or accepted later.
Sadly, it's difficult to deal with this ML. ML is precisely used for enabling a system that has self-learning, to take out information from data, with negligible assistance from a human developer.
ML behaviour and difficulty to interpret
Since a model's internal logic depends on data, its behaviour can be problematic to understand, and at times is next to impossible to fully comprehend why it is doing what it is doing.
This explains why ML components haven't undergone the same level of inspection as regular systems, and why some vulnerabilities can't be identified.
According to experts, the new ML principles will help any organization "involved in the development, deployment, or decommissioning of a system containing ML."
The experts have pointed out some key limitations in ML systems, these include:
- Dependence on data: modifying training data can cause unintended behaviour, and the threat actors can exploit this.
- Opaque model logic: developers sometimes can't understand or explain a model's logic, which can affect their ability to reduce risk.
- Challenges verifying models: it is almost impossible to cross-check if a model will behave as expected under the whole range of inputs to which it might be a subject, and we should note that there can be billions of these.
- Reverse engineering models and training data can be rebuilt by threat actors to help them in launching attacks.
- Need for retraining: Many ML systems use "continuous learning" to improve performance over time, however, it means that security must be reassessed every time a new model version is released. It can be several times a day.
In the NCSC, the team recognises the massive benefits that good data science and ML can bring to society, along with cybersecurity. The NCSC wants to make sure these benefits are recognised.
