{"id":26082,"date":"2023-06-16T13:03:58","date_gmt":"2023-06-16T07:33:58","guid":{"rendered":"https:\/\/www.getastra.com\/blog\/?p=26082"},"modified":"2025-10-28T12:05:57","modified_gmt":"2025-10-28T06:35:57","slug":"owasp-machine-learning-top-10","status":"publish","type":"post","link":"https:\/\/www.getastra.com\/blog\/security-audit\/owasp-machine-learning-top-10\/","title":{"rendered":"OWASP Machine Learning Top 10 Explained"},"content":{"rendered":"\n

With the rapid integration of machine learning technologies into various industries, the possibility of malicious attacks targeting them through vulnerabilities has grown. Machine Learning models are powerful yet prone to severe vulnerabilities due to data dependency and lack of standardized security measures.<\/p>\n\n\n\n

This rise in threats has prompted OWASP to release a list of vulnerabilities that would affect these models and the steps taken to mitigate them.<\/p>\n\n\n\n

The OWASP Top 10 for Machine Learning Includes:<\/strong><\/p>\n\n\n\n

    \n
  1. Input Manipulation Attack<\/a><\/li>\n\n\n\n
  2. Data Poisoning Attack<\/a><\/li>\n\n\n\n
  3. Model Inversion Attack<\/a><\/li>\n\n\n\n
  4. Membership Inference Attack<\/a><\/li>\n\n\n\n
  5. Model Theft<\/a><\/li>\n\n\n\n
  6. AI Supply Chain Attacks<\/a><\/li>\n\n\n\n
  7. Transfer Learning Attack<\/a><\/li>\n\n\n\n
  8. Model Skewing<\/a><\/li>\n\n\n\n
  9. Output Integrity Attack<\/a><\/li>\n\n\n\n
  10. Model Poisoning<\/a><\/li>\n<\/ol>\n\n\n\n

    <\/span>What is The OWASP Machine Learning Top 10? <\/span><\/h2>\n\n\n\n

    The OWASP Machine Learning Security Top 10<\/a> is a comprehensive guide developed by the OWASP foundation to address severe vulnerabilities in machine learning models and systems. Organizations can protect their ML systems against attacks that occur in the various stages of the ML lifecycle to ensure robust and reliable ML applications.<\/p>\n\n\n\n

    With the emergence of AI since OpenAIs GPT launch, OWASP accelerated its research on potential vulnerabilities in machine learning and LLMs. Ananda Krishna, CTO of Astra Security, has contributed to the machine learning top 10 list.<\/p>\n\n\n\n

    \"OWASP<\/figure>\n\n\n\n

    <\/span>OWASP Machine Learning Security Top 10 Explained with Examples<\/span><\/h2>\n\n\n\n

    ML01-2023: Input Manipulation Attack<\/h3>\n\n\n\n

    Input Manipulations are those in which attackers alter the input data to misdirect the machine learning model. These also include Adversarial attacks in which even slight and deliberate modification of the data can cause severe errors in the model predictions.<\/p>\n\n\n\n

    Example Attack Scenario for Input Manipulation Attack<\/h4>\n\n\n\n

    Input Manipulation in Image Classification Systems.<\/em><\/strong><\/p>\n\n\n\n

    Let\u2019s say a deep learning model is trained to classify images into various categories of dogs and cats. An attacker can alter or modify a few pixels of an image of a cat, appearing to be the original. Such small changes can easily cause the model to misclassify the image incorrectly as a dog. This can cause the manipulated image to bypass security measures or harm the system.<\/p>\n\n\n\n

    \u2139\ufe0f In 2023, researchers experimented with placing stickers on road signs to mislead the Tesla\u2019s autopilot system. It caused the cars to misinterpret stop signs as speed limit signs, leading to incorrect model behavior.<\/p>\n\n\n\n

    \"ML01-Input<\/figure>\n\n\n\n

    ML02-2023: Data Poisoning Attack<\/h3>\n\n\n\n

    Data Poisoning attacks occur when the attackers inject malicious data into the training data set, corrupting the learning phase of the model and leading to incorrect model behavior and predictions.<\/p>\n\n\n\n

    Example Attack Scenario for Data Poisoning Attack<\/h4>\n\n\n\n

    Training a Network Traffic Classification System<\/strong><\/em><\/p>\n\n\n\n

    The training data for the network traffic classification system in the machine learning model is poisoned through incorrect labeling of various types of traffic. As a result, the model misallocates network traffic to the wrong categories or network resources.<\/p>\n\n\n\n

    \u2139\ufe0f In 2022, a negative comment detector on Google’s Perspective API was targeted with a data poisoning attack. Attackers injected malicious data during the training phase, causing the model to misclassify toxic comments as non-toxic and vice versa.<\/p>\n\n\n\n

    ML03-2023: Model Inversion Attack<\/h3>\n\n\n\n

    Model Inversion Attacks occur when attackers reverse-engineer the model to reveal sensitive information from its output. Two possible ways such an attack can be executed are by stealing personal information and bypassing a bot detection model.<\/p>\n\n\n\n

    Example Attack Scenario for Model Inversion Attack<\/h4>\n\n\n\n

    Stealing Personal Information<\/em><\/strong><\/p>\n\n\n\n

    Attackers can train a facial recognition model and use it to invert the predictions of another face recognition model. This is done by exploiting vulnerabilities within the model implementation or API, with which the attacker can recover personal information used during the training phase.<\/p>\n\n\n\n

    \u2139\ufe0f In 2023, researchers performed a model inversion attack on a commercial facial recognition system to reconstruct images of individuals\u2019 faces by querying the model with various inputs. This attack exposed sensitive information in the training data.<\/p>\n\n\n\n

    ML04-2023: Membership Inference Attack<\/h3>\n\n\n\n

    Membership Inference Attacks occur when the attacker manipulates the training data used for the machine-learning model to expose sensitive data. This attack is carried out if the ML system does not have proper access controls, data encryption, or backup and recovery techniques. <\/p>\n\n\n\n

    Sensitive information about the dataset can be inferred, leading to privacy breaches.<\/p>\n\n\n\n

    Example Attack Scenario for Membership Inference Attack<\/h4>\n\n\n\n

    ML Model Used to Extract Financial Data<\/em><\/strong><\/p>\n\n\n\n

    Attackers use membership inference to query whether a particular individual\u2019s financial data was used to train a financial prediction model. Attackers can use this to extract sensitive private and financial information about the individuals.<\/p>\n\n\n\n

    \u2139\ufe0f In 2023, attackers used a machine-learning model trained on financial records to determine if specific people were part of the training data and extract their sensitive financial information.<\/p>\n\n\n\n

    \"ML04-Membership<\/figure>\n\n\n\n

    ML05-2023: Model Theft<\/h3>\n\n\n\n

    Model Theft or Model Extraction is an attack in which an attacker tries to replicate an ML model by repeatedly querying it, using the outputs to recreate the model, and gaining access to its parameters.<\/p>\n\n\n\n

    Example Attack Scenario for Model Theft<\/h4>\n\n\n\n

    Steal an ML Model From a Competitor<\/em><\/strong><\/p>\n\n\n\n

    Attackers execute this attack by either reverse engineering the model by decompiling the binary code or repeatedly querying it to gain access to its parameters. Once they have access to the model\u2019s parameters, attackers can start using it for themselves, causing financial and reputational damage to the competitor.<\/p>\n\n\n\n

    ML06-2023: AI Supply Chain Attack<\/h3>\n\n\n\n

    AI Supply Chain Attacks or Corrupted packages occur when an attacker modifies or replaces an ML library or model the system uses. <\/p>\n\n\n\n

    This usually occurs due to reliance on untrustworthy third-party code or heavy reliance on open-source packages that can be modified to affect the system upon downloading.<\/p>\n\n\n\n

    Example Attack Scenario for AI Supply Chain Attack<\/h4>\n\n\n\n

    Attack on an Organization\u2019s ML Models<\/em><\/strong><\/p>\n\n\n\n

    If an organization uses a public library in its applications, attackers can replace or modify the library’s code. When the target organization uses the compromised library, attackers can execute malicious activities within the organization\u2019s ML applications.<\/p>\n\n\n\n

    \u2139\ufe0f In 2022, an AI supply chain attack targeted an open-source machine learning library. Malicious code was injected into the library, which, when downloaded by users, allowed the execution of malicious actions.<\/p>\n\n\n\n

    \"AI<\/figure>\n\n\n\n

    ML07-2023: Transfer Learning Attack<\/h3>\n\n\n\n

    Transfer Learning attacks exploit vulnerabilities in pre-trained models, fine-tuning them and training them for another task, introducing weaknesses in the new model that lead to malicious model behavior.<\/p>\n\n\n\n

    Example Attack Scenario for Transfer Learning Attack<\/h4>\n\n\n\n

    Using Malicious Dataset to Train a Medical Diagnostics Model<\/em><\/strong><\/p>\n\n\n\n

    Attackers can train a dataset with manipulated images to target a Medical Diagnosis system. Once it starts using the tampered dataset, the system can make incorrect predictions, leading to incorrect diagnoses and harmful treatment suggestions.<\/p>\n\n\n\n

    \u2139\ufe0f In 2022, attackers exploited transfer learning by poisoning a pre-trained model, and when it was fine-tuned for malware classification, it misclassified malware samples as benign.<\/p>\n\n\n\n

    ML08-2023: Model Skewing<\/h3>\n\n\n\n

    Model Skewing attacks occur when an attacker manipulates the training data distribution to cause undesirable behavior in the models, affecting the accuracy of the predictions.<\/p>\n\n\n\n

    Example Attack Scenario for Model Skewing<\/h4>\n\n\n\n

    Targeted Product Suggestions Through Model Skewing<\/em><\/strong><\/p>\n\n\n\n

    Attackers skew the feedback data for a product suggestion model\u2019s predictions, leading to biased product suggestions for users.<\/p>\n\n\n\n

    \u2139\ufe0f In 2023, a model skewing attack was conducted on a credit scoring model used by a financial institution. Attackers manipulated the feedback data, causing the model to classify high-risk individuals as low-risk incorrectly.<\/p>\n\n\n\n

    ML09-2023: Output Integrity Attack<\/h3>\n\n\n\n

    Output Integrity Attacks are those in which the attackers modify or manipulate the output of an ML model, which leads to incorrect or altered results being presented to the users or the systems being used.<\/p>\n\n\n\n

    Example Attack Scenarios for Output Integrity Attack<\/h4>\n\n\n\n

    Modification of Fraud Detection System<\/em><\/strong><\/p>\n\n\n\n

    Output Integrity Attacks are those in which the attackers modify or manipulate the output of an ML model, which leads to incorrect or altered results being presented to the users or the systems being used.<\/p>\n\n\n\n

    \u2139\ufe0f In 2023, an output integrity attack targeted a healthcare diagnosis system. Attackers altered the model’s outputs, leading to incorrect medical diagnoses.<\/p>\n\n\n\n

    ML10-2023: Model Poisoning<\/h3>\n\n\n\n

    Model Poisoning attacks occur when the attacker manipulates the mode\u2019s parameters to degrade its behavior, affecting the performance or introducing specific biases.<\/p>\n\n\n\n

    Example Attack Scenarios for Model Poisoning<\/h4>\n\n\n\n

    Model Poisoning a Sentiment Analysis Model<\/em><\/strong><\/p>\n\n\n\n

    Attackers can input poisoned data during the training phase of the ML model, causing a sentiment analysis model to misinterpret positive sentiments as negative. They can lead to harmful automated responses for business emails.<\/p>\n\n\n\n

    <\/span>How To Prevent OWASP Machine Learning Top 10? <\/span><\/h2>\n\n\n\n
    \"How<\/figure>\n\n\n\n
      \n
    • Adversarial Training:<\/strong> Train models on adversarial examples to improve robustness against manipulation attacks and provide adversarial model training to reduce the chances of being successfully attacked. <\/li>\n\n\n\n
    • Data Validation and Verification:<\/strong> Data validation checks and multiple data labelers should be employed to ensure the accuracy of the labeled data. Data validation can be subcategorized into input validation and model validation.<\/li>\n\n\n\n
    • Access Control:<\/strong> Implement a strict role-based access control mechanism to limit who can interact with the model and avoid the unauthorized processing of the training data.<\/li>\n\n\n\n
    • Model Retraining:<\/strong> the machine learning model is regularly retrained to ensure that the system remains up to date, thus reducing the chances of information leakage from model inversion.<\/li>\n\n\n\n
    • Monitoring and Auditing: <\/strong>Monitoring and auditing the data regularly and at specific intervals can help detect anomalies and data tampering. Models should also be tested regularly for <\/strong>abnormalities to prevent attacks such as inference attacks.<\/li>\n\n\n\n
    • Package Verification:<\/strong> Verify the integrity of third-party libraries and tools used in the ML pipeline. <\/li>\n\n\n\n
    • Legal Protection:<\/strong> Ensure that the ML model is adequately protected in terms of legality with respect to patents and trade secrets. This will make it difficult for attackers to steal or use the model code and sensitive data and even provide a solid basis for legal action in case of theft.<\/li>\n<\/ul>\n\n\n