Prefer listening over reading? 🎧 Listen (AI-Generated Audio Version)

The model(s) used by an AI solution are a core component and should be managed accordingly, e.g. through versioning. However, there are some other components that integrate with the AI solution, contributing to its intellectual property (IP) and the overall value. These components must also be treated as essential elements of the solution.

Raw Data and Processing Pipelines

Most organisations recognise that their proprietary data is a critical component of their AI solutions. This data can be a real differentiator from competitors. However, in several cases, the pipelines required to prepare the data for analysis are not managed with the same level of importance and are treated as external tools rather than integral of the core AI solution. Changes in these pipelines can lead to issues within the core AI solution. In addition, any synthetic data created must be considered a core component of the AI solution. Version control and thorough integration testing are crucial to ensure consistency and resolve issues promptly.

Prompt Libraries

Prompts integrated within the solution should follow the same management processes as the solution’s codebase. For example, changes to prompts or model instructions should be part of the organisation’s peer-review process. Even minor changes to prompts can significantly impact results for end-users, making it crucial to track these .e.g user feedback. In addition, prompt changes that lead to poorer results should be easily reversible. Implement easy rollback options to revert to previous prompt versions if necessary.

Post-processing and Analytics Model

In several solutions, the output of the AI model undergoes further processing to align with organisational guidelines or to formal it appropriately. Similarly, analytics models are used to improve accuracy of end results or compensate for model inaccuracies. These components must be managed as part of the core AI solution. Due to their critical role in the overall output, these tools must be integrated into the core AI solution. Any changes to these tools must undergo end-to-end testing, as they can dramatically impact the end-user experience. Appropriate change management protocols are required in order to maintain output quality.

Fine-Tuning and Dashboards

The datasets used for fine-tuning, as well as method(s) employed, must be considered core component of the AI solution. Models should be version-controlled and linked to specific datasets and methods for fine-tuning. In addition, any dashboards or systems to track model accuracy, latency, and other KPIs, must be part of the solution. It is important for the organisation to maintain clear links between model versions and their corresponding fine-tuning datasets and methods.

Putting Everything Together

Depending on how organisations are structured and how they perceive their solutions, there may be other components to consider beyond those mentioned here. Organisations should adopt an end-to-end holistic view when developing or adopting AI solutions, ensuring that all components and their interconnections are fully understood. When they develop a strategy all these components must be considered and the relevant departments must be involved.