Skip to content

Conversation

@chait-slim
Copy link
Contributor

@chait-slim chait-slim commented Jan 7, 2026

Fixes three critical issues with Root.io data source integration:

  1. Version Comparison Crashes (TypeError)

    • Root versions mixed int and str types in comparison tuples
    • Example: '22.12.0-2.root.io.1' vs '22.12.0-2+deb12u1.root.io.5'
    • Solution: Reuse ecosystem-specific parsers (AlpineLinuxVersion,
      DebianVersion, packaging_legacy, MavenVersion)
    • Use suffix-based ecosystem detection (:Alpine:3.18 → Alpine)
    • Explicit error handling with ValueError for invalid versions
    • No fallback try/catch to let errors propagate properly
  2. PURL Collisions

    • All Root packages mapped to pkg:generic/root/ causing collisions
    • Example: Both Root:Alpine:3.18/curl and Root:Debian:12/curl
      generated identical pkg:generic/root/curl
    • Solution: Add sub-ecosystem-specific PURL mappings:
      • Root:Alpine → pkg:apk/root-alpine/
      • Root:Debian → pkg:deb/root-debian/
      • Root:Ubuntu → pkg:deb/root-ubuntu/
      • Root:PyPI → pkg:pypi/root/
      • Root:npm → pkg:npm/root/
      • Root:Maven → pkg:maven/root/
    • Parse hierarchical ecosystems (Root:Alpine:3.18 → Root:Alpine)
    • Add arch=source suffix for distro packages
  3. Maven PURL Encoding

    • Fixed encoding to preserve '/' in Maven group/artifact separation
    • Example: com.example:mylib → pkg:maven/root/com.example/mylib

Changes:

  • osv/ecosystems/root.py: Complete redesign with ecosystem-based
    version parser selection using match/case pattern
  • osv/purl_helpers.py: Added Root sub-ecosystem mappings and
    hierarchical ecosystem parsing
  • osv/purl_helpers_test.py: Added collision prevention tests
  • osv/ecosystems/root_test.py: Created 11 comprehensive unit tests
    covering Alpine, Debian, PyPI, Maven, and npm ecosystems

Testing:

  • All unit tests pass (11/11 in root_test.py)
  • Validated against real Root.io API with 8144 vulnerabilities
  • No PURL collisions detected across sub-ecosystems
  • Version comparison works correctly for all ecosystems

Fixes #4396

@chait-slim chait-slim mentioned this pull request Jan 7, 2026
6 tasks
@michaelkedar
Copy link
Member

  • Solution: Add sub-ecosystem-specific PURL mappings:

    • Root:Alpine → pkg:apk/root-alpine/
    • Root:Debian → pkg:deb/root-debian/
    • Root:Ubuntu → pkg:deb/root-ubuntu/
    • Root:PyPI → pkg:pypi/root/
    • Root:npm → pkg:npm/root/
    • Root:Maven → pkg:maven/root/

I'm not an authority on PURL, but adding root as a namespace to many of these PURL types would be considered invalid per the PURL Spec:

Inserting root into the namespace component like this technically creates malformed PURLs, which OSV should not be supplying to potential consumers.
Though, to be honest, I'm not exactly sure what the 'most correct' approach to PURLs for Root packages would be.

I've noticed that the OSV records from Root's REST API does not currently include the PURLs (were these silently removed recently? I thought I had seen them yesterday). OSV does not actually require PURLs.
These conversion functions are only really used for enriching the records with the computed PURL values, and for querying the OSV API using PURLs.
Since there's not really a clear and correct PURL representation of these, I am leaning towards removing these conversion functions for now.

@chait-slim
Copy link
Contributor Author

@michaelkedar

  1. I have removed the PURLs on our feed at let osv generate them
  2. The purls are now fixed according to spec. TBH, this is also something I have little knowledge about, so sorry for this hickup. This is the current change:
  • npm: @rootio/axiospkg:npm/%40rootio/axios (namespace extracted from scope)
  • PyPI: rootio-Pygmentspkg:pypi/rootio-Pygments (no namespace per spec)
  • Maven: io.root.example:mylibpkg:maven/io.root.example/mylib (namespace extracted from groupId)
  • Alpine/Debian: rootio-nghttp2pkg:apk/root-alpine/rootio-nghttp2?arch=source (custom distro namespace, following existing patterns)

@michaelkedar
Copy link
Member

  • npm: @rootio/axiospkg:npm/%40rootio/axios (namespace extracted from scope)
  • PyPI: rootio-Pygmentspkg:pypi/rootio-Pygments (no namespace per spec)
  • Maven: io.root.example:mylibpkg:maven/io.root.example/mylib (namespace extracted from groupId)

These PURLs would map to OSV's npm, PyPI, and Maven ecosystems, and refer to the packages on their respective public registries (i.e. https://www.npmjs.com/package/@rootio/axios or https://pypi.org/project/rootio-Pygments/ ) which Root does not publish to.

The best thing to do here would be to remove the PURL conversion code altogether. PURLs are not required in OSV.dev, and users will still be able to query the Root packages by their OSV ecosystem + names. We can always revisit adding PURLs later if the need arises and a clear mapping can be developed.

@chait-slim
Copy link
Contributor Author

@michaelkedar done.
Removed all Root PURL conversion

@chait-slim
Copy link
Contributor Author

Fixed lint. Other failures were related to urllib in unrelated files so I didnt change them

chait-slim and others added 5 commits January 13, 2026 21:29
These will be added in a separate PR after the code changes are merged.
    Fixes three critical issues with Root.io data source integration:

    1. **Version Comparison Crashes (TypeError)**
       - Root versions mixed int and str types in comparison tuples
       - Example: '22.12.0-2.root.io.1' vs '22.12.0-2+deb12u1.root.io.5'
       - Solution: Reuse ecosystem-specific parsers (AlpineLinuxVersion,
         DebianVersion, packaging_legacy, MavenVersion)
       - Use suffix-based ecosystem detection (:Alpine:3.18 → Alpine)
       - Explicit error handling with ValueError for invalid versions
       - No fallback try/catch to let errors propagate properly

    2. **PURL Collisions**
       - All Root packages mapped to pkg:generic/root/ causing collisions
       - Example: Both Root:Alpine:3.18/curl and Root:Debian:12/curl
         generated identical pkg:generic/root/curl
       - Solution: Add sub-ecosystem-specific PURL mappings:
         * Root:Alpine → pkg:apk/root-alpine/
         * Root:Debian → pkg:deb/root-debian/
         * Root:Ubuntu → pkg:deb/root-ubuntu/
         * Root:PyPI → pkg:pypi/root/
         * Root:npm → pkg:npm/root/
         * Root:Maven → pkg:maven/root/
       - Parse hierarchical ecosystems (Root:Alpine:3.18 → Root:Alpine)
       - Add arch=source suffix for distro packages

    3. **Maven PURL Encoding**
       - Fixed encoding to preserve '/' in Maven group/artifact separation
       - Example: com.example:mylib → pkg:maven/root/com.example/mylib

    Changes:
    - osv/ecosystems/root.py: Complete redesign with ecosystem-based
      version parser selection using match/case pattern
    - osv/purl_helpers.py: Added Root sub-ecosystem mappings and
      hierarchical ecosystem parsing
    - osv/purl_helpers_test.py: Added collision prevention tests
    - osv/ecosystems/root_test.py: Created 11 comprehensive unit tests
      covering Alpine, Debian, PyPI, Maven, and npm ecosystems

    Testing:
    - All unit tests pass (11/11 in root_test.py)
    - Validated against real Root.io API with 8144 vulnerabilities
    - No PURL collisions detected across sub-ecosystems
    - Version comparison works correctly for all ecosystems

    Fixes google#4396
@michaelkedar
Copy link
Member

/gcbrun

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Root.io Datasource

2 participants