Marcus Belke
CEO of 2B Advice GmbH, driving innovation in privacy compliance and risk management and leading the development of Ailance, the next-generation compliance platform.
Artificial Intelligence is quietly being incorporated into the software as part of regular version updates. For data protection and Compliance-For experts, this means that something fundamental is happening without the established approval processes kicking in. As a result, there’s a sense of unease in both medium-sized companies and large corporations: What new data flows and risks does this AI update bring with it? Do the original governance processes for software approval still apply? These questions highlight a blind spot in many organizations, as AI often sneaks in through the back door. What companies should look out for with AI updates.
Governance blindspot: AI update without approval
When introducing new software, organizations typically go through a rigorous review process: data protection impact assessments, IT security audits, and approvals by various committees. But what happens when software that has been in use for years suddenly gains new AI capabilities through an update? The original assessments do not account for these subsequent changes. The original purpose and data flows may change without anyone realizing that the approval granted at the time no longer aligns with the current scope of functionality. This creates a governance blind spot: The system is running in production and is considered „approved“ internally, while the new AI components lie completely outside the field of vision of those responsible.
This blind spot is dangerous. An AI function that was never originally approved suddenly changes the nature of the application. Risk profiles shift, but because there is no new project, there is no impetus for a new review. This means that changes in purpose can go unnoticed and with them potential violations of Data protection or Compliance, which were never consciously weighed up. Without Transparency and updating the Documentation AI remains literally invisible in running systems.
What Specifically Will Change With the New AI Features
New AI features in existing tools—that often sounds harmless. These might include an assistant mode, automatic analyses, or smart recommendations. In reality, however, they bring about very specific changes:
- Automated analyses instead of manual processes: Suddenly, the software begins interpreting data on its own. For example, a CRM system could automatically analyze customer sentiment or make „smart“ predictions where previously it only collected data. This leads to partially automated decision-making, with all its advantages and disadvantages.
- New data flows into the Cloud: AI functions often only work with the help of external computing power or models. As a result, data that previously remained internal now migrates to the Processing on Server of the provider or in the Cloud. A recent example is Microsoft's Outlook app, which, after an update, sends all emails to a central Microsoft server for AI analysis—Server transmits. Sensitive content slips out of one’s control unnoticed, only to be „enchanted“ by the AI. In many systems, such Cloud-Uploads in the background, without users or admins noticing right away. The "black box" effects are inevitable.
- Change in the Purpose of Data Processing: New AI features often use existing data for a purpose other than what was originally intended. For example, a tool that was primarily used for document storage could now also perform content analysis or sentiment analysis using AI. This changes the intended purpose of data processing, which is highly relevant under data protection law (keyword: change of purpose).
- Lack of Transparency for users: These features are often enabled by default or, at the very least, prominently promoted, without clearly communicating where data is sent or how the AI works. Users see the added value („finally, summaries of long emails“), but have little insight into what happens to their data behind the scenes. Complex opt-out settings overwhelm many users, and the default settings favor maximum data sharing.
New AI capabilities are changing the rules of the game. Local applications are evolving into distributed, cloud-based systems. Data that used to remain on-premises is suddenly leaving the secure fortress. And static tools are becoming learning systems that pose risks quite different from those of the original software.
Why traditional governance processes fail with AI updates
In companies, traditional governance and Compliance-Processes are usually project-oriented: They come into play when new software is procured or a major upgrade is set up as a project. However, from a formal standpoint, an update is not a new project. It is either provided via an auto-update from the manufacturer or installed as a routine patch by the IT department. For version 3.5.1, no project proposal is written, and no steering committee meets to discuss a new menu button.
As a result, AI changes via updates often fall through the cracks. The procurement department is not involved, as nothing new is purchased, and IT governance, such as change advisory boards, treats them as a technical patch with no strategic significance. Data protection and Compliance often only find out about it when something goes wrong. The existing approval processes are blind to incremental changes.
Another consideration is that even if the IT department reads the update notes, technicians do not always Compliance-Recognizing implications. A changelog announcing „AI-powered insights” might spark technical excitement in the IT department. But who translates that into data protection risks? Often, there is no clear designation of who should sound the alarm when new features are introduced. This creates a gap between technical maintenance and organizational governance.
On top of that, software providers are increasingly relying on SaaS models that involve continuous updates. Companies often receive new features automatically, whether they want them or not. If governance processes aren’t agile enough, they’ll lag behind these changes. Outdated review cycles, such as an annual review, fail to capture this dynamic. As the World Economic Forum recently demonstrated, the greatest operational risks posed by AI do not arise at the time of initial deployment, but rather at a later stage—when systems change or interact with others. Rigid governance cycles are barely able to capture these shifts.
Data protection and AI governance risks
Unnoticed AI updates harbor tangible risks in terms of data protection and Compliance:
- Change of Purpose and Lack of Legal Basis: When personal data are suddenly used for a new AI purpose, the question of the legal basis arises. The original Consent or contractual agreement may not cover the new use. For example, the service was purchased for X, but now also does Y with the data. This could be a change of purpose that is prohibited under Art. 6 para. 4 GDPR requires verification. Without new legitimization, the Processing on thin ice.
- Automated decisions: New AI functions can penetrate into areas covered by Art. 22 GDPR (Automated decisions). An update could, for example, introduce automated scoring that influences certain user decisions (such as automatic applicant pre-selection in HR software). However, such automated influences require special care, Transparency and, if necessary, options for objection. All this may not have been an issue when it was originally introduced, but it certainly is now.
- Data leakage and Cloud-Processing: AI features carry the risk of unintended data export. For example, personal data to third countries (keyword Cloud in the USA), which would require additional GDPR (e.g. Transfer Impact Assessments, Standard contractual clauses) is required. If no one knows about it, such requirements are, of course, not met. Data held by those entrusted with confidential information (trade secrets, confidential customer data) could end up on third-party servers—a nightmare for data protection officers.
- Bias and misanalysis: AI functions pose new risks related to content, such as bias or incorrect results. What if the automated analysis exhibits discriminatory tendencies or makes serious errors? At first, the software may have been „just a tool,“ but now it makes preliminary decisions. The result: The company bears responsibility for this. Without a reassessment of the risks, there is a danger of unknowingly violating principles of equal treatment or duties of care.
AI regulation and the consequences
The AI Regulation (EU AI Act) introduces new obligations. It classifies AI systems based on risk and requires strict measures for high-risk AI, such as risk management, Documentation or human supervision. Whether a system is classified as „high-risk“ depends on its intended use.
An existing tool could suddenly be reclassified into a higher risk category following an AI update. An example of this would be a human resources management tool that, after an AI update, uses AI to presort resumes. This would fall under the category of „use of AI in personnel decisions.“ This would actually mean Compliance-Measures are required under the AI Regulation, but no one has noticed this yet.
In addition, the AI Regulation imposes substantial penalties for violations: up to 35 million euros or seven percent of global revenues in cases of violations. These figures illustrate just how critical untested AI features can become.
Reading tip: Getting to grips with AI regulation with Ailance AI Governance
Shadow AI: unclear responsibility during operation
Why do such AI updates go unnoticed in the first place? A key problem is the lack of clarity regarding responsibility during ongoing operations. Once software has been implemented and approved, no one often feels explicitly responsible for conducting fundamental reassessments.
Although there is typically an application owner or process owner for the system, their focus is often on functionality and benefits in the specialist area and less on Compliance. The IT department keeps the system running from a technical standpoint and installs updates, but does not see itself as a data protection officer. Data protection officers and Compliance-Teams, on the other hand, usually focus on new projects and major changes, since their resources are limited. So it's no wonder that no one has a seemingly minor feature update on their radar.
As a result, there’s a sort of vacuum in responsibility: An AI assistant goes live, but who was supposed to evaluate it? The business unit? IT? Data protection? Everyone unconsciously assumed someone else was already on top of it. Formally, no one is tasked with assessing changes for risks after the rollout. There is no chain of responsibility for operations. Whereas with new acquisitions, clear Responsible persons While this is specified, there is often no similarly clear rule regarding who is responsible for raising the alarm for Feature X in Version Y.
Yet regulatory authorities such as BaFin have long required exactly that for the financial sector: responsibilities must be clearly assigned, and AI risks must be managed on an ongoing basis. However, many companies have not yet established internal roles such as an „AI owner per system.“ Without such a governance owner, AI in operations can quickly become unmanaged, and risks go unaddressed.
Necessary change of perspective: governance must include AI updates
In light of these developments, companies need to shift their perspective on governance. It is no longer enough to apply governance solely to large projects or acquisitions. The ongoing use and further development of tools that have already been implemented must also be a priority.
What does that mean, specifically? „Governance by Design“ must not end on the day a system goes live. Rather, there must be mechanisms in place to continuously—or at least regularly—verify that a system is still operating within acceptable parameters. Updates—whether they’re small patches or major version updates—should trigger a defined process. For example, a brief review should be conducted to determine whether new features have implications for data protection or security. If so, they must be integrated into the existing Compliance-processes are implemented (e.g., an addendum to the DSFA, an update to the Documentation, etc.).
Companies must proactively manage their tool landscape. This includes keeping an eye on software vendors’ roadmaps: Are AI features planned for the product roadmap? Are there beta programs that should be evaluated? This allows companies to assess early on what changes are on the horizon. Ideally, governance teams maintain an ongoing dialogue with vendors and obtain information about planned AI features well in advance.
Internally, too Awareness The question is: business departments and IT must be made aware that AI changes should not simply be dismissed as nice extra functions. Instead, it should be clear: Every major new function is an opportunity to pause for a moment and ask: „Are there any new risks or obligations here?“
This shift toward dynamic, continuous governance certainly requires new processes or tools. However, it is necessary to keep pace with rapid technological advancements. Agile methods aren’t limited to software development; we also need them in AI governance—moving away from one-off audits toward ongoing monitoring and adaptation. This is the only way to mitigate the greatest risks.
Specific Recommendations for AI Updates
How can the risks described above be effectively managed in practice? To conclude, here are some specific recommendations that every company—whether a small or medium-sized business or a large corporation—should implement:
- Define responsibilities per system: Appoint an owner for each important IT system who is not only responsible for the technology, but also explicitly for governance aspects. This person or committee is responsible for keeping an eye on changes, initiating risk assessments and acting as a link between the business department, IT and the IT department. Compliance to serve. Clear roles prevent a vacuum of responsibility.
- Implement a process for monitoring updates: Establish a process that regularly checks for updates, such as a quarterly review of the release notes for key software or a subscription to vendor news. It is crucial that you quickly assess each upcoming update by asking, „Is this relevant to data protection/governance—yes or no?“ Look out for suspicious keywords such as AI, machine learning, Cloud-Service, Analytics, etc., should automatically trigger a notification to the governance team.
- Define criteria for a revaluation: Define company-wide when an update requires a revaluation. For example: „Does the update process anew personal data? Are there data transfers to new recipients? Is there a new purpose for the Processing? Are automated decisions being introduced?“ As soon as one of these criteria is met, the data protection officer must be informed and, if necessary, a Data Protection Impact Assessment be updated. This list of criteria should be known and easy to apply.
- Feed new functions into the data protection, security and governance processes: Make sure that no feature goes live without first undergoing a Compliance-have undergone an audit. In practice, this might mean, for example, that the IT department notifies the data protection department of new feature toggles or modules before they are activated. Or the business unit may not use a new AI feature until the Compliance-teams have given their approval. This can be implemented through processes or technical means (such as disabling default settings). It is important that data protection and security are automatically taken into account when developing new features.
Automate the AI update process with Ailance AI Governance
Consider using a governance platform such as „Ailance AI Governance“ to effectively manage all of the measures mentioned above. Such tools offer, for example, model maps, automated workflows for risk assessments and approvals, and integration with existing data protection processes.
Reading tip: That's why model maps are so important for documentation
Ailance AI Governance makes it possible to manage every AI-Processing to register, Responsible persons to assign them and automatically trigger data protection checks as soon as personal data are in play. Workflows enforce that no approval is given without complete information, and reminders ensure that regular re-audits take place. Such a platform can Transparency and eliminate the blind spot by making updates, risks and evidence centrally visible and controllable.
Make sure you use AI without losing control. After all, the goal is for AI to create value for the company, not to pose an uncontrolled risk.
Marcus Belke is CEO of 2B Advice as well as a lawyer and IT expert for data protection and digital Compliance. He writes regularly about AI governance, GDPR-Compliance and risk management. You can learn more about him on his Author profile page.
German 
English 
Italian 
French