Additional scenarios

[sudo-bmitch]

Looking at #9, I'm trying to get a better understanding of how SBOMit delivers the following:

This specification proposes a means to generate metadata for an SBOM while the
software is being created. Furthermore, the means by which this information
is captured uses (in-toto)[https://in-toto.io] attestations and layouts. This
provides cryptograpic validation that this information is correct, handles
key distribution and management to indicate which parties should be trusted
for each step, and captures information about the environment in which the
steps are run.

As a result, using SBOMit provides a more accurate SBOM when parties are
honest. When malicious parties interfere in the process, SBOMit provides
a mix of traceability (knowing which party was malicious) and prevention
(blocking malicious software from being trusted), depending on how the
in-toto steps are configured.

For this I'm thinking of two scenarios. In both, the builds are not reproducible, so the hashes of the various outputs will be different for every build, and between any steps within the build.

In the first scenario, the build is run on a company's servers, and the end user only receives the generated binaries and an SBOM documenting the contents. For some reason the company doesn't want to document their usage of some 3rd party components, so they are taking deliberate steps to prevent that content from being included in the delivered SBOM. This company controls all of the signing keys and the servers where the steps are performed. How would SBOMit ensure that this is not possible and can be detected by the end user receiving the software?

In the second scenario, software is built on a public SaaS, e.g. GitHub Actions. The public action shows the SBOM generation steps being performed. and it is even signed using a project like sigstore to prove the build was run from the specific workflow and commit hash. The build itself could be verified with the GitHub SLSA provenance generator. In this scenario, what additional value would end users receive from verifying the SBOM from SBOMit vs verifying signature came from the workflow and commit running on the SaaS system?

[JustinCappos]

In the first scenario, the build is run on a company's servers, and the end user only receives the generated binaries and an SBOM documenting the contents. For some reason the company doesn't want to document their usage of some 3rd party components, so they are taking deliberate steps to prevent that content from being included in the delivered SBOM. This company controls all of the signing keys and the servers where the steps are performed. How would SBOMit ensure that this is not possible and can be detected by the end user receiving the software?

Since the company does not send information to the end user, then there would only be traceability in this case. (In general, if you don't share information, this is all you could accomplish). After some malicious behavior is detected, the company can rewind what happened and try to determine where in their pipeline things went wrong.

In the second scenario, software is built on a public SaaS, e.g. GitHub Actions. The public action shows the SBOM generation steps being performed. and it is even signed using a project like sigstore to prove the build was run from the specific workflow and commit hash. The build itself could be verified with the GitHub SLSA provenance generator. In this scenario, what additional value would end users receive from verifying the SBOM from SBOMit vs verifying signature came from the workflow and commit running on the SaaS system?

There are a lot of different value adds here. First of all, if there are more steps in the supply chain, those are also captured. So any test running, code review, etc. steps are also captured. Second, if GitHub is supposed be doing certain things or not doing specific things, those can also be captured in the layout. Finally, in practice, SLSA provenance may grow over time to include in-toto layouts, etc. as the way it does verification. If it does do this, and adds all the steps before and after the GitHub portions, and the SLSA provenance rules are the rules that you want to follow, and it generates an SBOM as part of the process, then it would be effectively equivalent to SBOMit in this case.

[sudo-bmitch]

Since the company does not send information to the end user, then there would only be traceability in this case. (In general, if you don't share information, this is all you could accomplish).

I'd be interested in seeing what that traceability looks like if the company is taking steps to hide the content, and using something like a malicious attestation tool.

There are a lot of different value adds here. First of all, if there are more steps in the supply chain, those are also captured. So any test running, code review, etc. steps are also captured. Second, if GitHub is supposed be doing certain things or not doing specific things, those can also be captured in the layout. Finally, in practice, SLSA provenance may grow over time to include in-toto layouts, etc. as the way it does verification. If it does do this, and adds all the steps before and after the GitHub portions, and the SLSA provenance rules are the rules that you want to follow, and it generates an SBOM as part of the process, then it would be effectively equivalent to SBOMit in this case.

Thanks. Does it make sense to find a common location to work on attestations for things like code review so that it gets included in SLSA and SBOMit?

[JustinCappos]

Since the company does not send information to the end user, then there would only be traceability in this case. (In general, if you don't share information, this is all you could accomplish).

I'd be interested in seeing what that traceability looks like if the company is taking steps to hide the content, and using something like a malicious attestation tool.

Right, so let's presume some scenario here where malicious functionality (like a cryptominer) gets included in a piece of software. There are a few cases here. The easier case is if the company can either tell what went wrong from the in-toto metadata (e.g., some cryptominer package was included and should not have been). If not, then the company would need to analyze the intermediate materials and products to see what went wrong. This can be done either by making sure they are reproducibly generated by an honest functionary or are stored over whatever period where one may want to go back and examine them forensically.

In any of these cases, it doesn't really matter which functionary or attestation component is malicious. There is in-toto metadata for all parts with signatures by the appropriate parties and you could go and figure out what happened.

Thanks. Does it make sense to find a common location to work on attestations for things like code review so that it gets included in SLSA and SBOMit?

Yes! That place would be in-toto, which is the common attestation framework, etc. for both! :)

[sudo-bmitch]

The easier case is if the company can either tell what went wrong from the in-toto metadata

If not, then the company would need to analyze the intermediate materials and products to see what went wrong.

This can be done either by making sure they are reproducibly generated...

All of those would be ruled out by the initial assumptions/scenario. To me this is telling that we're changing the problem to be one that in-toto can solve, and using that to claim it would solve all problems. As a user, my trust in a project is improved when there's clear guidance of what problems can and cannot be solved by the tool, rather than implying that all problems can be solved.

or are stored over whatever period where one may want to go back and examine them forensically.

I'm not sure those implementing these solutions are aware of the need to preserve a copy of the data after the hash is generated. We should make sure that's very clear. And they also wouldn't be available to end users verifying the builds of non-open source projects.

In any of these cases, it doesn't really matter which functionary or attestation component is malicious. There is in-toto metadata for all parts with signatures by the appropriate parties and you could go and figure out what happened.

Given the need to make exceptions to the scenario, I'm not seeing how an end user would be able to use this to detect a supplier that is intentionally concealing their dependencies.

[JustinCappos]

I'm not sure those implementing these solutions are aware of the need to preserve a copy of the data after the hash is generated. We should make sure that's very clear.

Sure. In the case that things are not reproducible and all information is hidden by the company then the company would need to preserve a copy of intermediate steps for forensics + traceability. We can make sure this is clear.

And they also wouldn't be available to end users verifying the builds of non-open source projects.
... I'm not seeing how an end user would be able to use this to detect a supplier that is intentionally concealing their dependencies.

Once again, if an organization hides all of their information about their supply chain, then an end user isn't going to be able to verify things directly. This is a tradeoff with the system that should be clear if it is not...