"November 27, 2023
Today, the California Privacy Protection Agency released draft automated decision-making technology (ADMT) regulations that define important new protections related to businesses’ use of these technologies. The proposed regulations would implement consumers’ right to opt out of, and access information about, businesses’ uses of ADMT, as provided for by the California Consumer Privacy Act (CCPA). The Agency Board will provide feedback on these proposed regulations at the December 8, 2023 board meeting, and the Agency expects to begin formal rulemaking next year. "
The CPPA released updates to this draft document in December 2023 and the most recent in March 2024. Before we deep dive into understanding the regulation and its key points, let's understand what automated decision-making technology is (henceforth referred to as ADMT.)
Link - https://cppa.ca.gov/meetings/materials/20240308_item4_draft_risk.pdf
What is ADMT?
Automated decision-making technology (ADMT) refers to software tools that use data, algorithms, and machine learning to automate choices previously made by humans.
ADMT translates data into decisions. It analyzes large amounts of information from various sources like databases, social media, and sensors to make automated choices.
ADMT relies on various technologies like:
- Machine learning algorithms that learn from data to make predictions
- Artificial intelligence techniques to mimic human-like decision making
So you would wonder, are ADMT and AI ( artificial intelligence) the same? No, ADMT (Automated Decision-Making Technology) and AI (Artificial Intelligence) are not identical, but AI can be a part of ADMT.
ADMT (Automated Decision-Making Technology): This is a broader term encompassing any technology that uses data and algorithms to automate choices. It can include various tools and techniques, and AI is one of them.
AI (Artificial Intelligence): This is a specific field of computer science focused on creating intelligent machines. AI offers a variety of techniques like machine learning that ADMT can leverage for decision-making.
Imagine ADMT as a workshop where decisions are made automatically. AI represents a set of advanced tools (machine learning algorithms, for instance) available in that workshop. ADMT might employ AI tools alongside other techniques to analyze data and reach conclusions.
Now that we have a clear understanding of ADMT and AI , let’s move forward with understanding the draft regulation by CPPA.
The core idea is to promote transparency, accountability, and consumer rights when it comes to high-risk data processing activities, especially those involving AI and automated decision-making.
Here are the important sections from this draft that a cybersecurity professional must take note of :
Risk Assessments: The Cornerstone of Responsible AI
One of the biggest requirements is for businesses to conduct comprehensive risk assessments before diving into certain processing activities that could significantly impact consumer privacy. We're talking about selling or sharing personal data, processing sensitive information like health records or biometrics, using automated systems for major decisions like employment or lending, and even training AI models on consumer data.
From a cybersecurity perspective, these risk assessments are a crucial first step in identifying and mitigating potential vulnerabilities associated with AI and automated decision-making systems. By thoroughly evaluating the purpose, data, and operational details of a proposed processing activity, businesses can better understand the potential attack vectors and tailor their safeguards accordingly.
For instance, if a business plans to use consumer data to train an AI model for loan approval decisions, the risk assessment would need to consider risks like data quality issues (e.g., incomplete or biased training data), model bias or discrimination, unauthorized access to sensitive financial information, and the potential for adversarial attacks on the AI model itself.
My recommendation for organizations would be to involve cybersecurity experts early in the risk assessment process. We can provide valuable insights into emerging threats, data security best practices, and technical safeguards that may not be immediately apparent to non-technical stakeholders.
These assessments aren't just a formality, either. Businesses have to get granular - identifying the specific purposes, data involved, operational details, and most importantly, weighing the benefits against the potential privacy risks to consumers. And we're not just talking about generic "service improvements" here; they need to spell out the tangible benefits, whether it's increased profits or something more altruistic.
If you analyze closely, this level of specificity is crucial. Too often, businesses fall into the trap of vague justifications for data processing, which can lead to scope creep and unintended consequences down the line. By clearly defining the purpose and benefits upfront, it becomes easier to evaluate whether the proposed activity is truly necessary and whether the potential risks are worthwhile.
On the risk side, they need to consider the gamut of potential harms - from unauthorized data access and discrimination to economic losses, physical dangers, and even psychological distress caused by things like deepfakes or invasive profiling. I can't stress enough the importance of taking a holistic view of risk. It's not just about data breaches or technical vulnerabilities; we need to consider the broader societal implications of AI and automated decision-making systems. Discrimination, privacy erosion, and the potential for physical or psychological harm are very real concerns that need to be addressed.
Organizations would have to involve a diverse range of stakeholders in the risk assessment process, including consumer advocacy groups, ethicists, and potentially impacted communities. This multifaceted perspective can help identify blind spots and ensure that all potential risks are accounted for.
But it's not just about listing risks; businesses also need to identify robust safeguards to mitigate those risks. We're talking technical controls like encryption and differential privacy, external audits, policies for human oversight, and more. The key is ensuring those safeguards are up to the task of protecting consumer privacy.
From a cybersecurity standpoint, I'd recommend a layered approach to safeguards, combining technical controls with robust policies, procedures, and ongoing monitoring. For instance, if a business plans to use AI for employment decisions, it might implement encryption and access controls to protect sensitive applicant data, conduct regular bias testing and model audits, and establish clear policies for human review and oversight of automated decisions. Access to the data provided to an AI is also crucial and needs a series of blog posts of its own.
And here's the kicker - if the risks outweigh the benefits, the business simply can't proceed with that processing activity. It's a hard stop, designed to protect consumers from potential privacy nightmares.
As a cybersecurity professional, I wholeheartedly support this risk-based approach. Too often, we've seen businesses prioritize short-term gains over long-term consequences, resulting in privacy breaches, discrimination scandals, and erosion of public trust. By mandating that benefits outweigh risks, these regulations help ensure that AI and automated decision-making systems are deployed responsibly and ethically.
Transparency and Consumer Control for Automated Decision-Making
Another major focus of this regulation is on transparency and consumer control when it comes to automated decision-making systems. Businesses have to provide detailed pre-use notices to consumers, explaining the purpose, logic, and intended outputs of these systems, as well as informing consumers of their rights.
From a cybersecurity perspective, transparency is crucial not just for consumer empowerment, but also for identifying potential vulnerabilities and attack vectors. By understanding the inner workings of an automated decision-making system, we can better assess its resilience to adversarial attacks, model drift, and other potential threats.
Consumers can opt out of having their personal data used for things like significant decisions (think employment, lending, etc.), extensive profiling, and behavioral advertising powered by automated systems. There are some exceptions, like fraud prevention and human appeals processes, but the general principle is giving consumers control over how their data gets used in these high-stakes scenarios.
The emphasis on consumer control and opt-out rights is extremely crucial from a privacy perspective. Not only does this empower individuals to make informed decisions about their personal data, but it also helps reduce the potential attack surface for malicious actors. The less personal data being processed by these automated systems, the lower the risk of unauthorized access or misuse. Organizations will have to develop ways to ensure that ADMT is safe and secure and the data processed does not violate any privacy laws.
Moreover, consumers have the right to access detailed information about how these automated systems were used in their specific case - the outputs, the logic applied, the key parameters that influenced the decision, and more. This level of transparency is crucial for holding businesses accountable and empowering consumers.
From a security standpoint, providing this level of access to consumers presents both challenges and opportunities. On one hand, organizations will need to implement robust access controls and data protection measures to ensure that sensitive information isn't inadvertently disclosed. On the other hand, this level of transparency can help identify potential biases or vulnerabilities in the automated decision-making process, allowing for timely remediation.
My advice to organizations would be to approach this access requirement with a "security by design" mindset. By integrating data protection and access controls from the outset, businesses can streamline the process of responding to consumer requests while minimizing potential risks.
High-Risk Scenarios: Raising the Bar
The regulations also call out specific high-risk scenarios that warrant extra scrutiny. Using automated systems for employment decisions, educational assessments, identity verification, and generating deepfakes - all of these trigger additional requirements around evaluating the systems for accuracy and preventing discrimination based on protected characteristics.
As a cybersecurity expert, I can't emphasize enough the importance of rigorous testing and validation for these high-risk use cases. Automated decision-making systems in areas like employment or education can have profound impacts on individuals' lives, and even seemingly minor inaccuracies or biases can lead to devastating consequences.
Organizations operating in these high-risk domains should partner with third-party security firms and ethical hacking teams to conduct penetration testing, adversarial attacks, and other forms of rigorous validation. It's not enough to rely solely on internal testing; these systems need to be stress-tested against real-world attack scenarios to identify potential vulnerabilities.
Businesses will need to have robust processes in place to ensure their AI models aren't perpetuating societal biases or making decisions that unfairly disadvantage certain groups. We're talking bias testing, human oversight policies, external audits - the works. From my perspective, addressing bias and discrimination in AI systems is not just an ethical imperative, but also a critical security concern. Biased or discriminatory models can inadvertently introduce vulnerabilities by making flawed decisions or reinforcing societal inequalities in ways that open the door to malicious exploitation.
To mitigate these risks, I'd recommend a multi-pronged approach that combines technical solutions like debiasing algorithms and fairness constraints with robust governance frameworks and human oversight. Additionally, organizations should consider engaging with impacted communities and advocacy groups to gain a deeper understanding of potential biases and ensure that their mitigation strategies are culturally sensitive and effective.
And if you're a company that's in the business of training AI models using consumer data and providing those models to other businesses, you have extra responsibilities too. You'll need to be transparent about any limitations or requirements associated with your models and provide all necessary information to enable those businesses to conduct their own risk assessments.
As a cybersecurity expert, I see this as a critical requirement for maintaining the integrity and security of AI supply chains. Just as we have rigorous standards and processes for vetting third-party software and hardware components, we need to apply the same level of scrutiny to AI models and training data.
My recommendation for AI model providers would be to implement robust documentation and testing processes to identify potential vulnerabilities, limitations, or biases in their models. This could include techniques like adversarial testing, stress testing with edge cases, and rigorous audits of training data and methodologies.
Additionally, it's crucial to establish clear lines of communication and responsibility with customer organizations. Providers should be upfront about any known issues or limitations, and work closely with customers to ensure that their models are being deployed and used securely and responsibly.
Compliance and Oversight: Keeping Everyone Honest
Of course, these regulations aren't just words on paper. There are real mechanisms in place to ensure compliance and oversight. Businesses will need to submit annual certifications and risk assessment information to the CPPA, with oversight from top executives. Note that these are just draft regulations and a lot may change when the final implementation takes place. It’s also important to understand the regulatory landscape around the world will change with AI at the forefront.
From a cybersecurity standpoint, this compliance and oversight process is essential for maintaining accountability and fostering a culture of responsible AI development and deployment. By requiring executive-level sign-off and regular reporting, these regulations help ensure that AI risk management isn't just a box-ticking exercise, but rather an integral part of an organization's overall security and privacy posture.
My advice to organizations would be to treat these compliance requirements with the same level of rigor and diligence as any other regulatory or industry standard. Establish clear ownership and accountability for risk assessment and compliance activities, and dedicate the necessary resources to ensure that these efforts are not just cursory, but truly robust and comprehensive.
This means establishing robust processes for documentation, certification, and submissions - no cutting corners allowed. Cybersecurity teams should be deeply involved in reviewing these materials for accuracy and completeness, as well as validating that businesses are truly following through on their risk assessment and safeguarding commitments.
From my perspective, this external oversight and validation is crucial. As much as we'd like to believe that organizations will self-regulate and prioritize consumer protection, history has shown that external pressure and accountability are often necessary to drive meaningful change. Every company wants to gain a competitive advantage today and that may in some cases mean skimping on the risks that may arise out of using ADMT.
By involving independent cybersecurity experts and regulatory bodies in the review process, we can help identify blind spots, challenge assumptions, and ensure that organizations are truly living up to their commitments. This external scrutiny can also help drive continuous improvement, as organizations strive to meet or exceed industry best practices.
It's a lot to take in, I know. With every technology, there are pros and cons. ADMT and AI are a game changer and will completely change the way we live and interact. Hence it’s prudent that while adopting these changes, we do it wisely and critically.
We're living in an era where these technologies can quite literally make or break lives - determining who gets hired, who gets a loan, who gets into college, and more. Without proper governance, the risks of perpetuating societal biases, eroding consumer privacy, and enabling discrimination are very real. And that's just the tip of the iceberg.
These regulations represent a crucial step towards mitigating those risks and harnessing the potential of AI responsibly and ethically. By mandating rigorous risk assessments, transparency, and consumer control, we're helping to build a future where these powerful technologies are deployed in service of the greater good, rather than narrow self-interests.
So, let's embrace this challenge head-on. As cybersecurity professionals, we must ensure these powerful technologies are developed and deployed responsibly, with consumer protection at the forefront. Let's roll up our sleeves, dive into these risk assessments, and help pave the way for a future where AI is a force for good, not harm.
Because at the end of the day, that's what matters – using our expertise to build a safer, more equitable digital world for all.
Here is a summary of what we have discussed.
Follow me on LinkedIn for updates.
Comments ()