Describe the change in the real world
Begin with the existing process. Who makes the decision today, what information do they use, and where does AI enter? An assessment of a customer-support assistant differs from one of a tool ranking job applicants. Document intended uses and reasonably foreseeable misuse. Make the boundaries explicit so the assessment has a clear subject.
Look beyond the organization’s own loss
A failed system can affect customers, employees, people represented in a dataset, or groups with limited ability to challenge an outcome. Consider inaccurate information, unequal treatment, loss of privacy, exclusion, and reliance on a result that should have been questioned. Include the people responsible for the affected process in the discussion.
Test assumptions against the actual users
A demonstration can work well while missing important languages, accessibility needs, document types, or unusual cases. Ask how representative the evaluation material is and which users are least well served. Where appropriate, consult people affected by the process. Record whose perspective is absent rather than pretending the assessment covers everyone.
Connect findings to design choices
An assessment should change something. You might restrict a use, require a reviewer, remove a sensitive field, improve retrieval permissions, or choose a less complex process. Record the decision, owner, evidence needed, and remaining uncertainty. If a risk cannot be managed within the intended context, a decision not to deploy is a valid result.
Keep different assessments connected
An AI impact assessment, a security review, and a privacy impact assessment may overlap without being interchangeable. Link them and assign owners for the gaps. Legal obligations depend on jurisdiction, organizational role, and use case; use the relevant official guidance when deciding whether a particular assessment is required.
PUT IT INTO PRACTICE
A useful next step
- Name three groups that could be affected.
- Write a plausible harm scenario for each.
- Choose a design change that reduces one harm.
- Identify evidence needed before deployment.
Keep the result: record the decision, owner, supporting evidence, and the next review trigger.
Go to the source
These links support further learning. The worked example and practical exercise are SyruSec’s own educational interpretation.
