Key takeaways
- The Human Rights Commission is proposing a national approach to AI and digital technology risk assessment that would cover both public and private sector organisations.
- The report positions human rights due diligence and risk assessment as practical ways to turn broad AI governance principles into decisions about whether and how an AI system should be used.
- New Zealand professional services firms can start incorporating these questions into existing AI approval, procurement and governance processes now, without waiting for a formal national framework.
New Zealand organisations are becoming familiar with the language of responsible AI: privacy, security, transparency, human oversight and accountability. The latest report from Te Kāhui Tika Tangata Human Rights Commission adds an important new dimension by proposing a national approach to assessing and mitigating the human rights and Te Tiriti risks created by AI and digital technologies.
Importantly for businesses, the Commission says this approach should extend beyond government and include the private sector. That makes one part of the report particularly relevant to organisations adopting AI today: the idea of a structured human rights and Te Tiriti risk assessment before higher-impact technologies are deployed.
From AI principles to an actual decision process
One of the persistent problems with AI governance is that organisations can agree with responsible AI principles without knowing what those principles should change in practice. The Commission’s report attempts to bridge that gap by treating risk and impact assessment as a practical mechanism for applying human rights and Te Tiriti considerations throughout the design, development, deployment and governance of AI systems.
A useful assessment would consider what the technology does, who may be affected, which rights or interests could be engaged, the likelihood and seriousness of potential harm, the controls available to reduce those risks, and what happens if something still goes wrong. It should also address accountability, ongoing monitoring and the ability of affected people to challenge or seek remedy for harmful outcomes.
That is a more operational model than simply asking whether an organisation has an AI policy. It asks whether someone has systematically considered the consequences of a particular AI use before approving it.
Not every AI tool creates the same risk
A risk-based approach also recognises that AI uses are not equal. An employee using generative AI to improve the wording of an internal email is very different from an AI system that assesses employment candidates, analyses employee performance, recommends whether a customer receives a service, or produces decisions based on large datasets about people.
The Commission does not attempt to settle exactly where those thresholds should sit. Instead, it identifies issues that would need to be resolved in developing a national framework, including which technologies are covered, which rights should be assessed and which levels of risk should trigger more rigorous forms of assessment.
This distinction matters because the report is not creating a mandatory private-sector AI impact assessment today. It does, however, point towards a governance model in which organisations may increasingly be expected to demonstrate how AI risks were identified, considered and managed.
The private sector is clearly part of the picture
The Commission does not treat responsible AI as something government alone needs to address. Its report describes governance of digital infrastructure as a shared responsibility and says private-sector organisations should undertake human rights due diligence and risk mitigation so that the technologies they develop or deploy do not cause actual or potential human rights harm.
That is particularly relevant because most New Zealand professional services firms are not developing foundation models themselves. They are purchasing and deploying AI services supplied by Microsoft, Google, OpenAI and other technology providers.
Using someone else’s technology does not remove governance responsibility. The decision to introduce that technology into a workplace, client process or decision-making system remains an organisational decision, and the report is clear that responsibility should not simply be displaced onto the technology itself.
Human rights broadens the traditional AI risk conversation
Most professional services firms will already recognise some of the risks described in the report, particularly privacy breaches, misuse of personal information, cyber security concerns and inappropriate disclosure of confidential information.
The Commission’s framework is broader. It includes discrimination and bias, digital exclusion, lack of transparency and accountability, employment impacts, misuse of Māori data, risks to Māori rights and interests, and the environmental effects associated with digital infrastructure.
The report organises its human rights-based approach around five principles: Participation, Accountability and Transparency, Non-discrimination and Equality, Empowerment, and Legality, forming the acronym PANEL.
For an organisation evaluating an AI system, those principles translate into practical questions. Who is affected by this system? Do they know AI is involved where that matters? Could particular people or groups experience poorer outcomes? Can someone challenge a decision? Who inside the organisation is accountable? Does the person overseeing the system understand it well enough to intervene when necessary?
Those questions take AI risk assessment beyond technical performance and place greater emphasis on the consequences of using the system.
Te Tiriti and Māori data are part of the framework
A distinctive feature of the Commission’s proposal is that it does not separate human rights from Te Tiriti o Waitangi. The report argues that Māori should have meaningful influence over the governance and use of systems affecting Māori rights and interests, while Māori data sovereignty should be recognised as foundational to New Zealand’s wider approach to digital infrastructure.
The significance for businesses will depend heavily on the use case. Not every deployment of a generative AI tool will engage the same considerations, but an organisation using AI to analyse Māori data, make decisions materially affecting Māori, or process culturally significant information should not assume that a standard technology or privacy assessment answers every governance question.
This is one reason the Commission is arguing for a specifically New Zealand approach rather than simply importing an overseas AI governance model.
Human oversight needs to mean something
The report also reinforces a point that is becoming increasingly important across AI governance frameworks: human oversight should be meaningful rather than procedural.
It is not enough for an organisation to say that a person remains “in the loop” if that person lacks the knowledge, authority or information required to challenge the AI output. Effective oversight requires a clearly identified decision-maker who understands the context and has genuine authority to question, reject or override the system.
For professional services firms, this is particularly important where AI assists with legal, financial, employment, compliance, risk or other decisions carrying significant consequences. Human review should operate as an actual control, not merely as an approval step in a workflow.
What this means for your organisation
Professional services firms do not need to wait for a national framework before applying the underlying discipline. A practical starting point is to introduce a proportionate AI impact assessment into the organisation’s existing technology approval and procurement process.
For higher-risk systems, that assessment should consider:
- Purpose: What problem is the AI system intended to solve, and why is AI appropriate?
- People affected: Who may benefit, and who could experience harm or disadvantage?
- Information: What personal, confidential, Māori or other sensitive information will be used?
- Decision impact: Does the AI simply assist a person, or could its output materially affect an employee, client or other individual?
- Fairness: Could the system produce different or discriminatory outcomes for particular groups?
- Transparency: Will affected people know AI is involved where that knowledge is important?
- Human oversight: Who has the authority and capability to challenge or override the system?
- Mitigation and remedy: What controls reduce the identified risks, and what happens if the system causes an incorrect or harmful outcome?
- Accountability: Who owns the risk once the system is operating?
- Review: What changes, incidents or new uses would trigger reassessment?
The objective is not to create another lengthy compliance form. It is to make sure the important questions are answered before an organisation becomes dependent on the technology.
The bigger governance signal
Perhaps the most significant idea in the Commission’s report is its treatment of AI and digital technology as infrastructure, rather than simply a collection of software products.
Infrastructure becomes embedded. Organisations redesign processes around it, people become dependent on it, and decisions made when a system is first introduced can have consequences long after the original procurement decision.
The Commission’s response is to call for deliberate design, long-term planning, accountability and risk assessment rather than treating each new AI product as an isolated technology purchase.
For professional services firms, that is a useful way to think about the next stage of AI governance. The focus is beginning to move beyond rules about which AI tools staff may use and towards evidence that an organisation understood the consequences of introducing a particular system and made a considered decision about whether its use was appropriate.
A documented, proportionate risk assessment is one of the clearest ways to demonstrate that.
General information disclaimer
This article provides general information and commentary only. It is not legal, human rights, technology, governance or other professional advice. Organisations should obtain advice appropriate to their circumstances and intended use of AI and digital technologies.
Source
This article is based solely on the Commission’s report and our analysis of its implications for New Zealand organisations.
KiwiGen.AI 