100 Commits
Author SHA1 Message Date
Andrew Nesbitt 39b07ff349 Merge pull request #24321 from Homebrew/strings-c-locale
keg_relocate: strip high bytes before matched paths
2026-10-24 15:53:08 +00:00
Andrew Nesbitt f975977b22 Preserve UTF-8 paths in ELF loader strings
- Validate ELF loader strings as UTF-8 before checking paths, so
  valid non-ASCII references reach bottle relocation detection.
- Keep rejecting invalid byte sequences and cover loader paths
  and the binary relocation metadata produced by `brew bottle`.
2026-10-24 16:46:52 +01:00
Andrew Nesbitt 0235bb5551 keg_relocate: strip high bytes before matched paths
- Apple's `strings` under brew's UTF-8 locale can report a high byte
  just before an embedded path as part of the same run.
- `text_matches_in_file` then fails `File.exist?` on the joined string
  and drops the match, so `brew bottle` can mark a keg that embeds its
  Cellar path as `cellar: :any`.
- Strip those bytes rather than running `strings` with `LC_ALL=C`,
  which would split prefixes containing non-ASCII characters.
- Pass `-e S` to GNU `strings` on Linux, which otherwise splits
  those prefixes and misses them in text files.
2026-10-24 16:33:29 +01:00
Andrew Nesbitt e5d5d7d224 Merge pull request #24100 from Homebrew/semver-input-limit
Bound SemVer input length
2026-09-26 14:36:24 +00:00
Andrew Nesbitt bbc5b26a98 Bound SemVer input length 2026-09-26 14:41:57 +01:00
Andrew Nesbitt 740fe93932 Reject repeated SemVer prefixes 2026-09-26 13:39:08 +01:00
Andrew Nesbitt fd1873821d Merge pull request #23736 from Homebrew/advisory-match-overrides
vulns: correct upstream fix boundaries and add reviewed overrides
2026-09-02 10:59:34 +00:00
Andrew Nesbitt ecbeb2ef40 Merge pull request #23511 from kmarekspartz/vulns-fix-flags
vulns: include OSS-Fuzz patch fixes as GIT ecosystem, add additional "fixed" filtering (released vs. patched), and `--list-skipped`
2026-08-21 11:48:24 +00:00
Andrew Nesbitt e113b38fc3 Merge branch 'main' into vulns-fix-flags 2026-08-21 08:51:45 +01:00
Andrew Nesbitt ab33c27b01 Merge pull request #23516 from Homebrew/advisory-match-repology-index
advisory-match: add --repology to load a local index
2026-08-14 08:28:08 +00:00
Andrew Nesbitt e8b8e2c478 Merge pull request #23459 from kmarekspartz/sbom-purl-registry
sbom: add purl external references for sources using Vulns::Identify
2026-08-12 15:31:01 +00:00
Andrew Nesbitt 04cfe2a760 Merge branch 'main' into sbom-purl-registry 2026-08-12 15:54:24 +01:00
Andrew Nesbitt aaa15ff58d sbom: build pkg:brew purls via Vulns::Purl
Replace string-interpolated pkg:brew purls with Homebrew::Vulns::Purl so
formula names are percent-encoded per purl-spec (python@3.12 becomes
python%403.12, matching what vulns/osv_export already emits). Also fixes
pkg:brew//name output when the tap is nil and drops the @ segment when
there is no version.
2026-08-10 11:33:55 +01:00
Andrew Nesbitt fcec8a10f1 Merge pull request #23152 from kmarekspartz/vulns-fix-available-flag
vulns: add --fix-available and --no-fix-available flags
2026-08-05 07:52:19 +00:00
Andrew Nesbitt fd73448209 Merge branch 'main' into vulns-fix-available-flag 2026-08-05 08:33:53 +01:00
Andrew Nesbitt 55f2868df8 Merge pull request #23341 from andrew/vulns-api
generate-formula-api: attach vulnerabilities from advisory-database
2026-07-31 08:25:39 +00:00
Andrew Nesbitt d1e128b3c8 Merge branch 'main' into vulns-api 2026-07-31 09:07:12 +01:00
Andrew Nesbitt e7eec14b51 vulns/advisory_database: skip :not_applicable records in status_for
A record whose ECOSYSTEM range starts above pkg_version returns
state :not_applicable from range_status; that was falling through to
patched (if fix: patch) or fixed_count. Drop it instead so a version the
advisory never applied to is not counted either way. Also distinguish a
missing advisories key from a wrong-type value in the load error.
2026-07-28 17:26:03 +01:00
Andrew Nesbitt e36ace0ef4 generate-formula-api: attach vulnerabilities from advisory-database
Vulns::AdvisoryDatabase reads Homebrew/advisory-database's published
data/advisories.json (built by that repository's rake advisories:concat)
via the same CachedFeed mechanism as Vulns::Repology, and evaluates each
BREW-* record's ECOSYSTEM range against a given pkg_version using
Vulnerability#range_status.

generate-formula-api loads it once per build and attaches
hash["vulnerabilities"] = {open:, patched:, fixed_count:} to each
formula's public API JSON (_data/formula/<name>.json and
api/formula/<name>.json). open is records whose range still contains
pkg_version; patched is records with ecosystem_specific.fix: patch that
pkg_version is past; fixed_count is bump-fixed records that no longer
apply. The key is absent for formulae with no records so consumers can
tell checked-and-clean from not-covered. A feed load failure is
opoo'd and the field is omitted so an advisory-database outage does not
break the API build.

The internal API (FormulaStruct) is unchanged; brew info reading the
field from there is a follow-up.
2026-07-28 17:13:29 +01:00
Andrew Nesbitt 8d0b6afe7d Merge pull request #23329 from andrew/vulns-identify
vulns: add advisory matcher and dev-cmd/advisory-match
2026-07-28 15:46:43 +00:00
Andrew Nesbitt bd990a54dd Merge branch 'main' into vulns-identify 2026-07-28 16:20:14 +01:00
Andrew Nesbitt 966b021478 dev-cmd/advisory-match: explicit require "fileutils" 2026-07-28 16:19:00 +01:00
Andrew Nesbitt 119bb74bbd Merge branch 'main' into vulns-identify 2026-07-28 15:28:27 +01:00
Andrew Nesbitt 4f439a9272 vulns: lowercase github.com paths and accept sibling-formula Repology projects
Identify.repo_url lowercases the path for github.com URLs. OSV.dev's GIT
ecosystem indexes repository URLs case-sensitively but normalises
github.com to lowercase (GitHub itself is case-insensitive), so a
head "https://github.com/FFmpeg/FFmpeg.git" previously produced zero
hits where the lowercased form returns 231. GitLab and Codeberg are
case-sensitive so their paths are preserved.

Vulns::Repology.lookup no longer rejects a project whose Homebrew
entries include sibling formulae with a different base name (wget +
wget2, sqlite + sqlite-analyzer, ffmpeg + a third-party ffmpeg-full).
The sibling's distro srcnames flow through as extra low-confidence
distro queries whose upstream-CVE range check will not match this
formula's identity, so the cost is uncomparable candidates rather than
wrong :affected/:fixed claims. The published index still excludes these
(RepologyIndex in Homebrew/advisory-database applies the stricter rule),
so the recall gain applies to named-formula and PR-bot runs; --all needs
the same loosening in the index builder.
2026-07-28 15:00:55 +01:00
Andrew Nesbitt 2c1daef60a vulns/match: return :never_affected when Homebrew skipped the interval
first_fixed_version now distinguishes three outcomes for a currently-
:fixed hit: the pkg_version at the :fixed -> :affected boundary (or at
the last loadable revision, best-effort); :never_affected when the walk
reaches :not_applicable or the start of the formula's history without
seeing :affected (Homebrew jumped from below introduced straight past
fixed and never shipped an affected build); and nil when the current
aggregate is not :fixed. dev-cmd/advisory-match drops :never_affected
candidates instead of emitting {introduced: 0, fixed: <first-shipped>}.
2026-07-28 13:31:56 +01:00
Andrew Nesbitt 1ac19d222d vulns/match: keep versionless evidence uncheckable in the history walk
aggregate_state_at was substituting the historical formula version for
every evidence row, including distro evidence built with
subject_version: nil. That let a distro record's Debian-versioned range
be compared against our formula version and report :affected, which made
first_fixed_version stop at the current pkg_version for a hit whose only
comparable (registry) subject was already :fixed further back. Evidence
with a nil original subject_version is now skipped at every revision,
matching range_status.
2026-07-28 13:17:13 +01:00
Andrew Nesbitt 42064914e5 vulns/match: aggregate every subject per historical revision
first_fixed_version re-evaluates every evidence at each historical
revision using that revision's subject version (primary formula version
or the resource's pinned version there) and applies the same aggregate
rule as range_status. The walk stops at the first revision where any
subject drops back to :affected, so a primary that crossed its upstream
fix at formula 2.0 combined with a resource that crossed at 3.0 yields
3.0, not 2.0.
2026-07-28 13:07:15 +01:00
Andrew Nesbitt 9cb74dc0db vulns/match: keep per-evidence source records and aggregate subjects
Evidence gains :source_record, set at Hit construction to the record the
evidence was matched against. dedup_by_cve merges evidence from every
grouped hit, and each evidence keeps pointing at its own record; a GHSA
found via a PyPI query no longer loses its PyPI affected[] range when
deduped onto a CVE-id record found via GIT that only carries commit-SHA
ranges.

range_status evaluates every evidence against its own source record and
aggregates: :affected if any subject is affected (a fixed or
not-applicable primary cannot hide an affected resource), else :fixed if
any is fixed, else :not_applicable only when every comparable subject
says so. The chosen evidence is returned so to_brew_record attributes
resource/resource_purl to the subject that decided the state and
first_fixed_version re-runs that evidence per revision.

upstream_evidence in the emitted record excludes :source_record and
:advisory (both are internal handles, not serialisable metadata).
2026-07-28 12:54:59 +01:00
Andrew Nesbitt 35a92398f0 vulns/match: tighten range-state edge cases
dev-cmd/advisory-match drops :not_applicable hits before to_brew_record;
emitting them as {introduced: 0} with no fixed event reads to OSV
consumers as currently affected.

CPANSec.range_status decides :fixed by evaluating the full fixed_versions
constraint with satisfies? instead of a stripped >= bound, so
affected: ["<1.0"], fixed: [">1.0"] leaves 1.0 as :not_applicable
rather than :fixed.

cpansa_vulnerability takes the single id being handled so a multi-CVE
CPANSA advisory whose CVEs are absent from OSV yields one record per CVE
instead of collapsing under the lowest.

resolve_to_cves consults related only for ALSA-* records; the OSV schema
defines related as different vulnerabilities and only AlmaLinux is known
to use it for source CVEs.

Vulnerability#range_status marks an interval checked only after a
comparison succeeds, so a target that fails every comparison in the only
range returns nil (uncomparable) rather than :not_applicable.
2026-07-28 12:35:39 +01:00
Andrew Nesbitt a67b535365 dev-cmd/advisory-match: leave source: generated records untouched
A CVE that homebrew-core patches via a resolves annotation already has a
source: generated record with fix: patch from generate-vulns-advisories.
The matcher will typically also reach the same CVE via GIT/registry/
distro; overwriting drops the patch attribution for a derived
fix: null/bump. --output now skips existing files whose
database_specific.source is generated and reports the count.
2026-07-28 12:11:54 +01:00
Andrew Nesbitt 3f0f00fcf7 vulns/match: transitive upstream, CPANSA-only hits, threaded prefetch
resolve_upstream walks upstream transitively with a per-walk visited set
and hop budget so USN -> UBUNTU-CVE-* -> CVE-* resolves; related is
consulted for bare CVE ids only when upstream is empty (AlmaLinux ALSA
records). A record that reaches no CVE is kept as-is rather than dropped.

CPANSA advisories with no CVE alias (102 in the current feed) become hits
directly via a synthesised Vulnerability instead of being dropped when
their id 404s at OSV. CVE-bearing advisories keep the OSV record as
canonical, falling back to the synthesised one only if the fetch fails.

prefetch_vulnerabilities warms the record cache for a chunk's stub ids in
bounded-concurrency batches before per-formula processing, so
resolve_upstream reads mostly from cache instead of issuing serial GETs.

range_status returns [status, evidence] so first_fixed_version can re-run
the same evidence's range check against each historical subject version,
preserving last_affected and exclusive-bound semantics instead of
collapsing to a >= threshold. to_brew_record only sets fixed on
state == :fixed and records range_state in ecosystem_specific.

dev-cmd/advisory-match: --all now conflicts with --json (JsonEmitter
accumulates); the text report shows the three-state result.
2026-07-28 12:06:26 +01:00
Andrew Nesbitt 8aa628fb3d vulns: three-state range status over the union of matching entries
Vulnerability::RangeStatus.state is :affected, :fixed, or :not_applicable
(below every introduced boundary; distinct from :fixed so a version the
vulnerability never applied to is not recorded as a bump fix).
range_status now iterates every affected[] entry whose package matches
{ecosystem, name} rather than the first, so records that split one
package across several disjoint ranges (GHSA-jfh8-c2jp-5v3q's three
log4j-core entries) evaluate correctly. A last_affected boundary is not
reported as fixed_in for an affected version, and the not-affected side
of a last_affected interval requires target > boundary (not >=) to count
as :fixed.

CPANSec.range_status returns the same three states, using the highest
fixed lower-bound at or below the target for :fixed and reporting
:not_applicable when the version satisfies no affected constraint and
sits below every fixed bound.
2026-07-28 12:05:55 +01:00
Andrew Nesbitt 269b3ae4f0 vulns: raise let-free spec files to typed: strict
utils/repology_spec, identify_spec and purl_spec go to typed: strict
with sigs added on their helper methods. cpan_sec_spec, vulns/repology_spec,
match_spec and dev-cmd/advisory-match_spec stay at typed: true because
they use let, which generates a sig-less method that strict rejects (no
brew spec at typed: strict uses let).
2026-07-28 11:41:05 +01:00
Andrew Nesbitt ac858185a3 vulns/match: batch OSV queries across formulae and stream output
each_advisory_batch builds labelled queries for a chunk of formulae at
once, sends them through one OSV.query_batch (which slices at
BATCH_SIZE), and yields (formula, hits) in input order. advisories_for
becomes a single-element wrapper. The vulnerability cache still spans
chunks so a CVE fetched for one formula is reused for the next.

dev-cmd/advisory-match streams via an Emitter: --output writes each
record as it is produced and only accumulates counts; text mode counts;
--json still builds the array (single-formula / PR-bot use, so bounded).
This lets --all iterate the whole tap without holding every record in
memory or issuing one querybatch per formula.
2026-07-28 11:33:57 +01:00
Andrew Nesbitt f5b8b6a5b6 vulns/match: query versionlessly and evaluate ranges per evidence
All OSV queries are now versionless, so bump-fixed advisories are
returned. Evidence carries the {ecosystem, name} that was queried plus
the subject_version to check ranges against (formula version, pinned
resource version, or nil for distro), and the CPANSA Advisory for
:cpansa evidence.

resolve_upstream re-attributes each distro-ecosystem hit to the bare
CVE ids named in its upstream (Debian/Ubuntu/RH/openSUSE/...) or related
(AlmaLinux) fields, ignoring distro-prefixed intermediate ids so
USN -> [CVE-x, UBUNTU-CVE-x] does not produce a stray UBUNTU-CVE hit.
A multi-CVE advisory splits into one hit per CVE; a record naming no CVE
is kept as a low-confidence hit. Each resolved hit gains synthesised
own-identity evidence so range_status can check the CVE record's
affected[] against our version.

range_status(hit) walks each evidence in precision order and returns the
first Vulnerability::RangeStatus (or CPANSec.range_status result) a
comparable range yields. A GIT-SHA-only record, or a distro-resolved CVE
whose affected[] does not match our identity, returns nil.

to_brew_record derives fixed/fix: from range_status: not-affected sets
{fixed: pkg_version, fix: bump}; affected or uncomparable emits no fixed
event and fix: null, with confidence demoted for uncomparable.
first_fixed_version now uses the range_status fixed_in as the threshold
for the FormulaVersions walk.

bulk mode (Match.new(bulk: true), used by --all/--index) skips the live
Repology.lookup fallback so a full sweep never hits the rate-limited
per-project API for the ~3,200 formulae the published index does not
cover; single-formula runs still fall back for a formula the nightly
index has not seen.
2026-07-28 10:29:31 +01:00
Andrew Nesbitt d7e7161f24 vulns: add per-package range evaluation
Vulnerability#range_status(ecosystem, name, version) evaluates a version
against the single affected[] entry whose package matches, honouring
range type: SEMVER via Semver, ECOSYSTEM via Version, GIT skipped as
uncomparable (commit-SHA events cannot be ordered against a version
string). Returns {affected?, fixed_in} for the interval containing the
version, or the closing boundary of the highest interval below it, or
nil when no entry matches / no comparable range exists so callers can
tell not-affected from could-not-check.

CPANSec.range_status(advisory, version) evaluates the CPANSA
affected_versions/fixed_versions constraint grammar (comma-joined AND of
</<=/>/>=/==/=/bare terms; array entries OR) against a Version.

Vulnerability#identifiers reverts to id + aliases only. upstream is a
directed reference (a distro advisory naming one-or-more source CVEs)
and related links to different vulnerabilities; treating either as an
identity of this record over- or under-merges. Vulns::Match follows
upstream explicitly. severity_entries exposes the raw OSV severity
array for record emission.
2026-07-28 10:29:31 +01:00
Andrew Nesbitt 6525bc11ff vulns: raise spec files to typed: true
Sorbet rejects described_class::CONST at typed: true, so those references
are spelled out. The expect { loaded = ... } capture pattern gets an
explicit T.let so the block reassignment does not narrow to nil.
match_spec.rb stays typed: false pending its rewrite for per-evidence
range evaluation.
2026-07-28 10:29:31 +01:00
Andrew Nesbitt e2c5553eb5 Address style and Copilot review
- dev-cmd/advisory-match: extract text_mode? to satisfy
  Style/UnlessLogicalOperators.
- test/dev-cmd/advisory-match_spec: reorder let/before above the shared
  example, combine Repology stubs via receive_messages, use // regex.
- utils/repology: explicit require "erb" for ERB::Util.url_encode.
- test/vulns/identify_spec: cover multi-byte percent decoding
  (Integer#chr returns ASCII-8BIT for 128-255 so no encoding error;
  the Copilot suggestion to pass Encoding::ASCII_8BIT is unnecessary).
2026-07-28 10:29:31 +01:00
Andrew Nesbitt a03c8140c0 dev-cmd/advisory-match: wire Vulns::Match to a CLI
Three modes:

  brew advisory-match <formula>... [--json] [--output=DIR] [--no-history]
    Run every strategy against each named formula and emit candidate
    BREW-* records. Text mode prints a per-hit summary; --json prints the
    OSV hashes; --output writes one file per record via
    OsvExport.merge_existing so existing published/ranges are preserved.

  brew advisory-match --all [--json] [--output=DIR] [--no-history]
    As above for every formula in homebrew/core.

  brew advisory-match --index
    Emit the formula-identity index (name -> Identify keys) as JSON.

--no-history skips the FormulaVersions walk and uses the current
pkg_version as the fixed boundary. Hidden from the manpage; this is
authoring-time tooling for advisory-database CI and the homebrew-core PR
bot, not a user command.
2026-07-28 10:29:31 +01:00
Andrew Nesbitt 2e00163a78 vulns/vulnerability: include OSV upstream field in identifiers
Distro-ecosystem OSV records (DEBIAN-CVE-*, RHSA-*, OESA-*, etc.) carry
the underlying CVE in the schema's upstream field rather than aliases.
Including it in identifiers lets Match#dedup_by_cve collapse a distro
record onto the same CVE reached via GIT/registry, and lets
Scanner#partition_patched match a resolves annotation against a
distro-id result. related is read but excluded from identifiers since it
links to different vulnerabilities and would over-merge.
2026-07-28 10:29:31 +01:00
Andrew Nesbitt 7edb93f677 vulns/match: add to_brew_record and first_fixed_version
Match#to_brew_record emits a candidate BREW-* OSV hash for a (formula,
Hit) pair with database_specific: {source: matched, strategy, confidence,
upstream_evidence}. When the current formula (or resource, for a resource
hit) version is at or past the lowest comparable upstream fixed version,
the record carries {fixed: pkg_version} and ecosystem_specific.fix: bump;
otherwise no fixed event and fix: null. Resource hits record the resource
name and derived purl in ecosystem_specific. Distro-strategy fixed
versions (1:x.y-z, +dfsg-n) are not compared.

Match#first_fixed_version walks FormulaVersions history newest-first and
returns the pkg_version at the oldest revision where the subject version
was still at or past the threshold, caching the rev-list and per-revision
loads per formula. The dev-cmd passes this as first_fixed for new records;
OsvExport.merge_existing preserves hand-corrected ranges thereafter.
2026-07-28 10:29:31 +01:00
Andrew Nesbitt 6dff421708 vulns: add Match with identify and advisories_for
Match#identify composes Identify (git repo/tag, registry package for the
primary URL and each resource) with the Repology index (distro source
package names, live lookup fallback) into a single Identity per formula.

Match#advisories_for builds one OSV querybatch across GIT, language-
registry and versionless distro-ecosystem queries, routes CPAN packages
through CPANSec, fetches full records once per id (cached across calls),
and collapses hits sharing a CVE alias into one Hit whose strategy is the
highest-precision path that reached it, keeping every path as evidence.

Repology and CPANSec feeds are loaded once per Match instance and are
injectable for tests. This is authoring-time code for advisory-database
CI and the homebrew-core PR bot; it never runs on a user's machine.
2026-07-28 10:29:30 +01:00
Andrew Nesbitt e3a61648bb vulns/osv: generalise query_batch to arbitrary ecosystems
query_batch now takes {ecosystem:, name:, version:} with version nilable
(a nil version queries all known vulnerabilities for the package). Scanner
builds the GIT-ecosystem shape itself. This lets Vulns::Match issue PyPI/
npm/Debian/etc. queries through the same batching and pagination path.
2026-07-28 10:29:30 +01:00
Andrew Nesbitt 54412271a3 vulns/repology: delegate fetch_project to shared ::Repology client
utils/repology.rb gains an API_BASE constant, URL-encodes the project
name and pagination cursor, passes --fail, and raises with the curl
exit status on HTTP failure (still rescued to nil for brew bump, but
the detail now reaches the debug log instead of surfacing as a JSON
parse error on an HTML error body).

Vulns::Repology.fetch_project calls ::Repology.single_package_query and
raises Error on nil, dropping its duplicate curl invocation, API_BASE
and USER_AGENT constants. curl_args already sends HOMEBREW_USER_AGENT_CURL
so the custom UA was redundant.
2026-07-28 10:29:30 +01:00
Andrew Nesbitt 04209921f1 vulns: add Repology reader for advisory-database's distro index
Fetches data/repology.json published by Homebrew/advisory-database's
RepologyIndex (7-day TTL via CachedFeed) and exposes
#distro_packages_for(formula_name) returning
{osv_ecosystem => [srcname, ...]} with an @-versioned-name fallback.

.lookup(formula_name) is a live single-project API fallback for
formulae the published index doesn't yet cover: it fetches each
name_candidates variant, keeps projects whose Homebrew entries
include the formula (or its base) without grouping unrelated
formulae, partitions contributions into exact-name and base-name
pools, and resolves each by preferred Homebrew status the same way
RepologyIndex#resolve does — so the fallback is consistent with the
published index for the projects it can reach. .fetch_project raises
on transport/HTTP/JSON failures; only an empty (HTTP 200) response
means the project doesn't exist.

The OSV_DISTROS mapping and distil/homebrew_entries/PREFERRED_STATUSES
mirror RepologyIndex; a follow-up will consolidate the API client with
utils/repology.rb (which currently lacks a User-Agent).
2026-07-28 10:29:30 +01:00
Andrew Nesbitt 8eab7afb98 vulns: extract CachedFeed base class from CPANSec
Shared load/refresh/from_file for JSON feeds cached under
HOMEBREW_CACHE/vulns/: mtime-gated freshness check, download to a
per-process sibling Tempfile, validate, atomic rename, and stale-cache
fallback with a warning on failure. Subclasses implement .data_url,
.cache_filename and #initialize(data) and may override
.default_max_age. CPANSec now inherits it; behaviour unchanged apart
from the diagnostic wording naming the cache filename.
2026-07-28 10:29:30 +01:00
Andrew Nesbitt e650a474fa vulns: add CPANSec loader for cpan-security-advisory.json
Fetches the compiled cpan-security-advisory.json from
briandfoy/cpan-security-advisory into HOMEBREW_CACHE/vulns/ and exposes
advisories per CPAN distribution. Refresh downloads to a per-process
sibling Tempfile, validates the JSON, then atomically renames over the
cache; on network or validation failure it warns and falls back to the
stale copy. Range evaluation of affected_versions is left to the future
Vulns::Match consumer.
2026-07-28 10:29:30 +01:00
Andrew Nesbitt 093fd5ff5c vulns: extend Identify::FORGES with OSV-indexed GitLab hosts
Live querybatch probes against api.osv.dev confirm gitlab.gnome.org,
gitlab.freedesktop.org and invent.kde.org are indexed in the GIT
ecosystem (libxml2 56 hits, poppler 75, karchive 2); sr.ht,
salsa.debian.org and bitbucket.org returned zero and are not added.

FORGES becomes a {host => path_regex} hash. GitHub and Codeberg keep
the two-segment owner/repo capture; GitLab-family hosts (including
gitlab.com) use a lazy multi-segment capture bounded by .git, /-/,
/uploads/, /wikis/ or an optional trailing slash so nested subgroups
such as xorg/lib/libx11 resolve while host-level /-/ and /api/ routes
are rejected. repo_url now anchors on the host and strips a Wayback
Machine snapshot prefix so archived homepages still resolve.
2026-07-28 10:29:30 +01:00
Andrew Nesbitt 3dfee791d3 vulns: add Purl builder and Identify.registry_package
Homebrew::Vulns::Purl is a minimal purl-spec builder covering the ten
registry types we derive from formula source URLs, with per-type name
normalisation and RFC 3986 percent-encoding.

Homebrew::Vulns::Identify.registry_package(url) parses download URLs
from PyPI, npm, crates.io, RubyGems, Hackage, Hex, CPAN, Maven Central,
CRAN and NuGet into {ecosystem, name, version, purl}. Version
extraction is per-registry rather than a shared heuristic so names
containing hyphen-digit segments (es5-shim, base64-bytestring,
Perl6-Junction, iso-639) and RubyGems platform suffixes are handled
correctly. Nothing calls it yet; it is groundwork for the
advisory-match dev-cmd.
2026-07-28 10:29:29 +01:00
Andrew Nesbitt 7f40278258 vulns: extract Identify from Scanner
Move repo_url, tag, FORGES and TAG_PATTERNS into a new
Homebrew::Vulns::Identify module so the upcoming advisory-match
dev-cmd can share the URL-parsing helpers without pulling in Scanner.
Scanner requires the new module and calls it directly. No behaviour
change; specs moved to identify_spec.rb.
2026-07-28 10:29:29 +01:00
Andrew Nesbitt 4628b37681 Merge pull request #23314 from Homebrew/vulns-osv-export-dedup-references
vulns/osv_export: deduplicate references from OSV.dev
2026-07-26 18:41:06 +00:00
Andrew Nesbitt f6403c53ef vulns/osv_export: deduplicate references from OSV.dev
OSV.dev merges NVD and cve.org reference lists without normalising
percent-encoding, so a URL like the Fedora package-announce archive can
appear once as %40 and once as @ under the same reference type. Collapse
those on the way out while preserving the same URL under distinct types,
which the schema allows and which carries meaning.

Use `URI::RFC2396_PARSER.unescape` to match existing usage in
`AbstractFileDownloadStrategy`; it leaves literal + alone and passes
invalid %XX through unchanged rather than raising.
2026-07-26 19:19:49 +01:00
Andrew Nesbitt 77835e7501 Merge pull request #23223 from andrew/git-end-of-options
Use --end-of-options for Git arguments
2026-07-20 18:32:49 +00:00
Andrew Nesbitt c08eda3aff Force brewed Git on Catalina 2026-07-20 19:10:05 +01:00
Andrew Nesbitt 806fb27fe7 Use --end-of-options for Git arguments 2026-07-20 18:52:35 +01:00
Andrew Nesbitt e92f225437 Merge pull request #23140 from Homebrew/vulns-installed-scope
vulns: scan installed kegs by default and report untrusted-tap skips
2026-07-17 12:04:01 +00:00
Andrew Nesbitt 44e3ba4526 Merge branch 'main' into vulns-installed-scope 2026-07-17 04:49:49 -07:00
Andrew Nesbitt 0af3431b18 Merge branch 'main' into vulns-installed-scope 2026-07-16 12:22:00 -07:00
Andrew Nesbitt 6bc4e360cc cmd/vulns: use Utils.pluralize and prefer trusting the formula 2026-07-16 12:19:57 -07:00
Andrew Nesbitt be781f344e vulns: scan installed kegs by default and report untrusted-tap skips
No-arg `brew vulns` was branching on `tap_trust_configured?`, which is
always true now that `HOMEBREW_REQUIRE_TAP_TRUST` defaults on, so it hit
`Formula.all` and scanned every core formula instead of what is
installed. Enumerate installed racks directly, catching
`UntrustedTapError` so kegs from taps that lost trust in the 6.0
migration are reported and fail the run rather than silently vanishing
via `Formula.installed`'s bare rescue.

Also merge `--brewfile` entries with named arguments instead of
dropping the named ones, and validate `--severity` before enumerating
formulae.
2026-07-16 11:03:32 -07:00
Andrew Nesbitt 44e476a9e7 vulns: advisory database repo renamed to Homebrew/advisory-database
The homebrew- prefix was dropped so setup-homebrew stops treating it as
a tap in the regenerate workflow.
2026-07-14 16:17:27 -07:00
Andrew Nesbitt 6bd951d96e Merge pull request #23115 from Homebrew/vulns-osv-export-archaeology
generate-vulns-advisories: derive initial fixed via FormulaVersions
2026-07-14 22:39:17 +00:00
Andrew Nesbitt 4130be563c generate-vulns-advisories: derive initial fixed via FormulaVersions
For records with no existing file, walk homebrew-core git history
(newest-first) via FormulaVersions and use the pkg_version at the oldest
revision where the CVE still appears in resolved_ids as the `fixed`
boundary, instead of the current pkg_version. Existing records preserve
their on-disk ranges (per the merge behaviour in the previous commit),
so this only fires once per newly-annotated (formula, CVE) pair.

The FormulaVersions instance and its rev_list output are cached per
formula so subsequent CVEs for the same formula reuse the ~90s
path-filtered git rev-list and per-revision formula loads. OsvExport.run
now runs inside the SimulateSystem block so historical loads are
runner-OS-independent. Historical revisions are evaluated under the base
simulation only; a resolves inside on_linux/on_intel falls through to
the current pkg_version (documented; no such annotation exists in core
today).

Spot-checked against real homebrew-core: unzip CVE-2014-8139 -> 6.0_6
(current is 6.0_8), CVE-2022-0529 -> 6.0_8; libquicktime CVE-2016-2399
-> 1.2.4_5 (the resolves-added-later case; hand-correct to _4).
2026-07-14 14:09:30 -07:00
Andrew Nesbitt 945b48cb9f Merge pull request #23112 from Homebrew/vulns-osv-export-merge
vulns/osv_export: merge into existing records instead of overwriting
2026-07-14 20:49:07 +00:00
Andrew Nesbitt 70c621ff71 Merge branch 'main' into vulns-osv-export-merge 2026-07-14 12:38:48 -07:00
Andrew Nesbitt bea143bf74 vulns/osv_export: backfill published from modified; drop T.cast
When migrating a record written before `published` existed, fall back to
its `modified` timestamp rather than today's so first-rewrite doesn't
jump the publication date forward.

Narrow the upstream cache value with `is_a?(Hash)` at the call site
instead of `T.cast`, and collapse the failed-fetch guard to a single
`next if` now that reassignment isn't needed.
2026-07-14 12:37:18 -07:00
Andrew Nesbitt 8f9122b974 vulns/osv_export: merge into existing records instead of overwriting
Records now carry `published` and `database_specific: {source:
"generated"}`. `.run` reads any existing `<id>.json`, preserves its
`published` timestamp and `affected[].ranges` (so the `fixed` boundary
reflects when the annotation was first observed rather than drifting to
the current pkg_version), and only writes when the remaining content has
changed. Records for annotations no longer in core are simply not
visited, so they persist on the default branch.

A transient OSV.dev failure leaves an existing enriched record untouched
rather than stripping its summary/severity/references. The unchanged
comparison uses parsed hashes so key ordering in a hand-formatted
existing file does not cause spurious rewrites.
2026-07-14 12:23:19 -07:00
Andrew Nesbitt 714494585b vulns/osv_export: link OSV schema and tighten patch_ref return type
Add a pointer to ossf.github.io/osv-schema above SCHEMA_VERSION and note
where the Homebrew ecosystem/prefix were registered. patch_ref now
returns T::Hash[Symbol, T.any(String, T::Array[String])] via a PatchRef
alias; the input side stays T.untyped to match Formula#serialized_patches.
2026-07-14 10:20:00 -07:00
Andrew Nesbitt c0966f6ddb Merge pull request #23103 from andrew/vulns-non-forge-head
vulns/scanner: fall back to verbatim head URL for non-forge repos
2026-07-14 16:57:02 +00:00
Andrew Nesbitt a9f6cad1ad Merge branch 'main' into vulns-osv-export 2026-07-14 09:56:36 -07:00
Andrew Nesbitt d4470b95a2 vulns/osv_export: memoize upstream fetch per vuln id
When multiple formulae resolve the same CVE, fetch it from OSV.dev once
per run rather than once per (formula, id) pair. Uses Hash#fetch with a
block so a nil (fetch-failed) result is also cached.
2026-07-14 09:52:52 -07:00
Andrew Nesbitt c78d1a43fa Merge branch 'main' into vulns-non-forge-head 2026-07-14 09:38:07 -07:00
Andrew Nesbitt d09f70139c dev-cmd/generate-vulns-advisories: entry point for OsvExport
Replaces the tap's brew-vulns --osv-export DIR with a hidden dev-cmd
following the generate-formula-api pattern. Iterates homebrew/core under
with_no_api_env + SimulateSystem, unions each formula's serialized_patches
across all OS/arch variations from to_hash_with_variations so
platform-gated resolves annotations are not silently dropped, and hands
[formula, patches] pairs to Homebrew::Vulns::OsvExport.

--dry-run lists record ids without writing or querying OSV.dev.
Consumed by Homebrew/homebrew-advisory-database's regenerate workflow.
2026-07-14 09:35:34 -07:00
Andrew Nesbitt aa746060d6 vulns: add OsvExport for Homebrew-ecosystem OSV records
Ports the OSV record exporter from Homebrew/homebrew-brew-vulns. Emits
one BREW-<formula>-<CVE> record per security id declared in a formula's
patch resolves list, under the registered Homebrew ecosystem with
pkg:brew purls, marking the currently shipped pkg_version as the fixed
boundary. Upstream summary/severity/references are copied from OSV.dev
where available.

This is the generator for Homebrew/homebrew-advisory-database; the
dev-cmd entry point follows in the next commit.
2026-07-14 09:35:34 -07:00
Andrew Nesbitt 22d03a1650 vulns/scanner: hoist stable repo_url/tag derivation
Compute stable_repo_url and stable_tag once at the top of build_target
and reuse them in both the installed and non-installed branches, rather
than repeating the same target_repo_url/tag chain twice.
2026-07-14 09:17:11 -07:00
Andrew Nesbitt 19a23e1f3d Merge pull request #23104 from andrew/deps-brewfile
cmd/deps: accept --brewfile as an input source
2026-07-14 16:05:16 +00:00
Andrew Nesbitt 4b423089c6 cmd/deps: merge --brewfile inputs with named arguments
Previously --brewfile silently replaced any named arguments; combine
them instead so `brew deps foo --brewfile` includes foo alongside the
Brewfile entries, and dedupe the merged list.
2026-07-14 07:48:07 -07:00
Andrew Nesbitt 1d4cad75a4 Extract target_repo_url and cover the SBOM fallback path
Move the verbatim source/head URL fallback into a Scanner.target_repo_url
class method so build_target's three call sites share one implementation
written in the ||= style suggested in review, and add a spec for the
installed-keg SBOM branch where the SBOM downloadLocation host is not a
supported forge.
2026-07-14 07:30:54 -07:00
Andrew Nesbitt 4c904dc1d5 cmd/deps: accept --brewfile as an input source
Read brew and cask entries from a Brewfile and treat them as if they
had been passed as named arguments, so --brewfile composes with --tree,
--graph, --for-each, --union and the default flat listing. --formula and
--cask filter which entry types are loaded. A bare --brewfile reads
./Brewfile and --brewfile=<path> reads a specific file, matching
brew vulns.
2026-07-14 07:20:49 -07:00
Andrew Nesbitt 719e9562f3 vulns/scanner: fall back to verbatim head URL for non-forge repos
repo_url only normalises GitHub, GitLab and Codeberg, so formulae whose
stable, head and homepage URLs are all hosted elsewhere (bash, gmp,
gnu-tar, go, make, openconnect, zsh) were skipped outright. Restore the
tap's last-resort fallback: query OSV with the source URL verbatim when
its path yielded a tag, otherwise the head URL verbatim. OSV's GIT
ecosystem returns nothing for these hosts today, so this is parity with
the tap and future-proofing rather than a detection change.

Follow-up to #23080.
2026-07-14 06:56:57 -07:00
Andrew Nesbitt 570b1f07f9 Merge remote-tracking branch 'fork/vulns-cmd' into vulns-cmd 2026-07-13 17:06:51 -07:00
Andrew Nesbitt 20563c04be Restore vulns scanner target fallbacks from the tap
Widen repo_url to also consider the formula homepage, fall back to
stable.specs[:tag] then the stable version string when no tag can be
extracted from the source URL, and read the SBOM versionInfo alongside
downloadLocation. This recovers coverage for formulae like curl, git,
node and libssh2 whose stable URL is not a supported forge archive.
2026-07-13 17:05:06 -07:00
Andrew Nesbitt 5d56078a1e Accept bare --brewfile in brew vulns
Restores the optional-argument form from the tap so `brew vulns --brewfile`
defaults to ./Brewfile (or HOMEBREW_BUNDLE_FILE). `--brewfile=<path>` still
selects a specific file; the bare/empty value is normalised to nil for
Bundle::Brewfile.read.
2026-07-13 17:04:57 -07:00
Andrew Nesbitt e1e3bcfb21 Merge branch 'main' into vulns-cmd 2026-07-13 14:48:03 -07:00
Andrew Nesbitt d51148c873 Use tap trust for brew vulns 2026-07-13 14:34:04 -07:00
Andrew Nesbitt 4a81eacf1f Inline minimum vulnerability severity 2026-07-13 14:31:43 -07:00
Andrew Nesbitt 6d474d7e42 Merge branch 'Homebrew:main' into vulns-cmd 2026-07-13 05:48:53 -07:00
Andrew Nesbitt 7abeaefa61 Regenerate man page and completions for brew vulns 2026-07-12 22:53:16 +01:00
Andrew Nesbitt d7e60e7c14 Add brew vulns command 2026-07-12 22:47:09 +01:00
Andrew Nesbitt 87cf9fed34 Scan installed keg's SBOM source URL, not the current formula
For installed formulae, read the source download URL from the keg's
sbom.spdx.json (SPDXRef-Archive-*-src) so the OSV query and reported
version reflect what is actually on disk. Patch suppression only applies
when the scanned keg matches the current recipe's PkgVersion. When an
outdated keg has no SBOM the formula is recorded in
Results#outdated_without_sbom for the caller to surface.
2026-07-12 22:47:09 +01:00
Andrew Nesbitt 6d53ee38ba Add Homebrew::Vulns::Output text and JSON renderers 2026-07-12 21:14:56 +01:00
Andrew Nesbitt b390ec387a Add Homebrew::Vulns::Scanner#scan orchestration and Results 2026-07-12 20:58:12 +01:00
Andrew Nesbitt faf548e838 Add Homebrew::Vulns::Scanner URL/tag/resolves extraction helpers 2026-07-12 20:45:25 +01:00
Andrew Nesbitt 0df30eb6c9 Validate OSV querybatch result count matches queries 2026-07-12 20:38:34 +01:00
Andrew Nesbitt df6e7a6387 Follow per-result next_page_token in OSV querybatch 2026-07-12 20:15:16 +01:00
Andrew Nesbitt 9963b6bb9b Add Homebrew::Vulns::OSV client on Utils::Curl 2026-07-12 20:09:55 +01:00
Andrew Nesbitt a9a0392667 Add Homebrew::Vulns::Vulnerability OSV record wrapper 2026-07-12 19:18:53 +01:00
Andrew Nesbitt 876abd4213 Add Homebrew::Vulns::CVSS v3.x base score and severity 2026-07-12 19:02:06 +01:00
Andrew Nesbitt d10df41000 Trim Semver module comments to match brew density 2026-07-12 19:02:06 +01:00
Andrew Nesbitt 692fe388fd Reject leading zeroes and empty identifiers in Semver parser 2026-07-12 18:45:13 +01:00