AI projects face security challenges that stem from the difficulties in ensuring model integrity and reliability. The Sleepy Pickle and HuggingFace models' silent backdoors are notable cases of such model security loopholes. They are evidence of the possibility of influencing an AI model's behavior directly or indirectly through malicious or authorized model modifications, manipulations, and adversarial attacks. These model breaches stem from the blind spots that exist during the development and post-development of AI projects. This lack of visibility leaves AI models and data vulnerable to these security compromises.

A recent survey by the Linux Foundation advocates adopting transparent and accountable development practices and strategies as a countermeasure to mitigating these security gaps and blindspots in AI systems.

This article highlights using software bills of materials (SBOMs) as a transparency and accountability strategy for attesting models and ensuring the model security of AI projects. Then, it explores using a tool that enables you to tailor these strategies for developing your AI models.

Understanding the AI security challenge

AI project components comprise data, models, and code. Changing the model component without a trace is often possible by modifying the data or code. A thorough documentation of the creation, operation, and lifecycle of your model’s dependencies, processes, metadata, artifacts, tools, libraries, etc, enables you to protect your AI system from common model breaches like data or model poisoning, model evasion, confidentiality attacks, and model flaws.

Having an inventory of all the dependencies, processes, metadata, artifacts, tools, and libraries for the different components of your AI project and having a way to verify them ensures complete transparency and accountability for your AI project. With a verifiable inventory of AI project components, you can identify when unauthorized or malicious changes are made to your model. Employing model attestation and SBOMs in your AI project can enable you to capture this information.

Why model attestation and SBOMs matter for your AI project

Model attestation, similar to a traditional software attestation security strategy, is a practice that enables you to establish a verifiable security supply chain for all system components. It is a strategy that verifies the integrity, authenticity, and lifecycle of your model's data, code, and artifacts and their relationship at the different stages of the development lifecycle.

By ensuring that malicious actors can’t introduce backdoors or manipulate the models, you can attest to a model's authenticity and integrity because you have verifiable evidence of its lineage and source. This transparency also strengthens you and your team's confidence in the model and fosters a culture of accountability in and out of your team.

Software Bill of Materials (SBOMs) or AI Bill of Materials (AIBOM), in the context of AI projects, is an inventory report you can use for model attestation. It provides an inventory of the components and the dependencies used in building a model. This inventory report should typically include data, model artifacts, dependencies, and their relationships.

AIBOMs enable you to identify and address cases of compromised models. In essence, AIBOMs empower you with the necessary information to proactively pinpoint the compromised models and swiftly take remedial action on incidents of compromised models or breaches. They ensure your project is resilient against attacks and manage the security risks associated with your AI projects. Although the use of AIBOMs is growing, some difficulties related to implementing and maintaining their inventories continue to limit their adoption, such as:

• A standard format for documenting AIBOM inventory to encourage interoperability and adoption.
• Automatic generation of the AIBOMs
• AIBOMs Verification engine
• Secure AIBOM Storage system ## Implementing model attestation and SBOMs in your AI project

There are various methods and standards for creating AI SBOMs for model attestation. These methods often require you to have some form of SBOM pipeline that extracts relevant information from your project and uses it to generate the SBOM. If you are using container-based technology, you can leverage information from the container images as your SBOM pipeline to create your AI SBOMs. You can directly generate your AI project's SBOMs from Docker container images using Syft.

Docker containers do not fit perfectly into ensuring the secure development of your AI project because they cannot track the model and data lifecycle. However, you can use other container-based technology tools like Buildpacks or KitOps to package your AI project and transform it into container images or open container initiative (OCI) artifacts.

A tool like KitOps is better suited for documenting the inventory of your AI project. KitOps can track the lineage of your model and model artifact's (i.e., datasets, code, configurations) development lifecycle with its container image, called ModelKit and Jozu Hub. For instance, it can capture the essential information an AI project SBOM needs to enable the verification of the origin and integrity of your models as OCI artifacts, such as:

• Data version
• Model training lineage
• Model artifacts
• Training metadata
• Dependency relationship
• The author of the model and its artifacts

Although KitOps doesn't have a tool like Syft that allows you to directly create SBOMs from ModelKits (i.e., container image), you can extract specific artifacts from your ModelKit and use them in your existing pipelines to generate your AI project's SBOMs.

How do ModelKits and Jozu enable model attestation and SBOMs for your AI projects?
ModelKit packages your AI projects and tracks your model's development lifecycle. Jozu hub enables you to securely store your ModelKits remotely. They both possess features that enforce a secure development process for your AI project, ensuring confidence when attesting to your model's integrity. These features also streamline the complexities of implementing and maintaining AIBOM inventories. Some of these features include:

OCI compliant format
Container-based technology is a widely adopted standard for developing projects. Since ModelKits primarily uses container technology—open container initiative-compliant (OCI) format—they are portable across the different environments and experts within your AI project. Using ModelKit as a standard format throughout development streamlines your AI project's inventory documentation, making it easy to generate your SBOMs without development blind spots.

Simple management with Kitfiles
ModelKits contain a Kitfile, a YAML-based configuration file where you specify the components of your AI projects (i.e., code, data, model, etc.) for simplified sharing and management of AI/ML components. This file facilitates the documentation of each component's dependencies, versions, and configurations, which is essential for creating comprehensive SBOMs. The Kitfiles enable you to maintain a transparent and detailed inventory of all components in your AI projects from the inception of your development, therefore enhancing security and accountability.

Tamper-proof packaging and ModelKit manifests
As an OCI artifact, Modelkit is immutable. Upon creating a ModelKit, a manifest is also generated, which signs each ModelKit's artifact and Kitfile with a cryptographic hash. Manifests have a unique identifier associated with each ModelKit. So, an update on any of the ModelKits in your project produces a new version of the file rather than altering the existing one, making it impossible to modify models or content (i.e., model artifacts) without a trace. This prevents the model and its artifacts from being vulnerable to tampering without evidence, providing assurance when auditing your AI project.

Jozu Hub’s secure storage:
Storing ModelKit in JozuHub allows you to view and compare ModelKit contents without pulling them, simplifying the discovery, sharing, usage, and tracking of models and their artifacts in your AI project. JozuHub displays the list of ModelKits and versions and the files within each ModelKit, making it easy to identify changes between versions.

Model attestation and SBOMs are essential for enhancing the security of your AI project. Despite their low adoption, they offer substantial transparency, accountability, and risk mitigation benefits. KitOps provides a robust solution for managing and securing AI models, making incorporating model attestation and SBOM practices from them into your AI project easier. Start using KitOps to adopt these best practices, safeguard your models against security threats, and ensure your AI project's integrity.