Procurement Models
Applications
Plan
Source
Manage & Maintain
Decommission
Top 5 questions to ask
Scope: What is the model intended to be used for? Lifecycle: How will the model be integrated into a system, and what are the potential limitations that may arise (e.g. decommissioning)? Security and privacy: How will these models handle data privacy and security? Human and technical resource requirements: What capabilities, data, and infrastructure exist or will be needed to operationalise the AI model? Ethical Considerations: What are the key ethical considerations (e.g. potential for bias) related to the models, and how will these be evaluated or mitigated?
Reliability and testing: What is the expected reliability and performance of the model (and under what parameters)? How, or by whom, will testing and training be carried out? Vendor responsibility: What are the measures in place to continuously protect data and model integrity, including explainability? Licensing: What are the terms for licensing - who owns or is responsible for adapted versions of the model and any input or output data? Support: Is continued or long-term support required, for example for updates and maintenance of the AI models? Model use: How does the vendor ensure the appropriate and ethical use of their AI models?
Accuracy: What level of accuracy of the model is acceptable and what are the procedures for regular testing and validation of the models? Unexpected behaviour: What are the contingency plans for model failures or unexpected behavior? Compliance and documentation: How is compliance with privacy and ethical standards monitored? What documentation is recorded and provided for ongoing management? Updates: How are updates and maintenance managed (e.g. how do people keep up-to-date with any changes to the model, system, or standards)? Impact Assessment: What is the impact on current operations, and how will the impact of the output, or use, of the model be assessed?
Plan: What are the steps for securely decommissioning the AI models? Data: What are the needs to retain or transfer the data involved in the models? Are there guidelines in place for data disposal if necessary? Impacts: What are the impacts of decommissioning on current operations and stakeholders? Communication: How will the decommissioning process be documented and communicated? Compliance: Are there any legal or compliance considerations to address during decommissioning?
RAI considerations & Mitigation of inherent risk
Key principles Accountability - human oversight in systems need to be maintained Compliance - adhere to and monitor ethical and privacy obligations
Key Principles Fairness - ensure that the model is fit for purpose and understand its assumptions and limitations Security - ensure sensitive data, or intellectual property will be protected, especially is access is granted to third-parties
Key principles Transparency - ensure that relevant stakeholders know how, when and why AI is being used and potential implications for data privacy Explainability - understand how the AI is able to produce its outputs to ensure data accuracy and reliability
Key principles Reliability - ensure that core business functions are still able to be performed. Security - protect proprietary and sensitive data, including any intellectual property
Mitigation of inherent risk Complexity: Medium to High depending on the required specialisation of the model and necessary expertise of end users. Impact: High as decisions made here inform the remainder of the procurement process, and choice in model may have direct implications for business operations.
Mitigation of inherent risk Complexity: High, depending on the model and use, i.e. multiple vendors and systems may be involved. Impact: Medium, depending on specific clauses in contracts there is potential for vendor lock-in or potential mis-use.
Mitigation of inherent risk Complexity: High as it requires ongoing monitoring and tracking of data quality, model usage, compliance and stakeholder engagement. Impact: High as it involves the operationalisation of an AI model into organisational workflows which can directly impact core business processes.
Mitigation of inherent risk Complexity: Medium as it requires secure data disposal, and can involve a larger system Impact: Medium, depending on the scope of removal or replacement of the model.
Key mitigants Establish clear criteria for model evaluation that includes ethical considerations. Define the intended lifecycle of the model and potential/scope for iteration/tuning. Build internal capacity for model development, maintenance, and use.
Key mitigants Determine ongoing (lifecycle) costs, potential for damage, error, or liability. Define how model testing will be carried out - i.e. Accuracy, reliability and validity. Build awareness of potential compatibility issues, data quality, and model accuracy requirements.
Key mitigants Develop a system for auditing and oversight of use. Ensure regular and ongoing testing of the base model based on established testing procedures. Maintain proper documentation including input/outputs, architecture, limitations, requirements, performance (e.g. model cards).
Key mitigants Organisational champions that have technical familiarity with the model can be invaluable during the decommissioning process. Archive model documentation and decision making.