Pius Architecture - Three-Phase Discovery Pipeline

Pius uses a three-phase concurrent pipeline to discover organizational assets. This page explains how plugins are orchestrated, how data flows between phases, and the key design decisions.

How does the Pius pipeline work?

The pipeline has three phases plus an enrichment step:

1. Phase 0 (Independent) - Domain and CIDR plugins with no dependencies run concurrently
2. Phase 1 (Handle Discovery) - whois and edgar discover RIR organization handles
3. Handle Enrichment - Handles are grouped by registry and injected into pipeline input
4. Phase 2 (Handle Resolution) - RDAP/RPSL plugins resolve handles to CIDR blocks
5. Output Filtering - Internal handle findings are stripped; only domains and CIDRs reach the user

Pipeline diagram

pius run --org "Acme Corp" --domain acme.com
              |
              v
       Plugin Registry
    (init() auto-registration)
              |
   +----------+-------------------------------------+
   | Phase 0 (concurrent, independent)              |
   |  crt-sh   apollo   github-org   gleif          |
   |  passive-dns   reverse-whois                   |
   |  dns-brute*  dns-zone-transfer*                |
   |  doh-enum*  favicon-hash*                      |
   |  asn-bgp                                       |
   +----------+-------------------------------------+
              | Emits domains + CIDRs directly
   +----------+-------------------------------------+
   | Phase 1 (concurrent)                           |
   |  whois   edgar                                 |
   +----------+-------------------------------------+
              | Emits RIR org handles
              v
       enrichWithHandles()
   Input.Meta["arin_handles"] = "ACME-1"
   Input.Meta["ripe_handles"] = "ORG-ACME-RIPE"
              |
   +----------+-------------------------------------+
   | Phase 2 (concurrent)                           |
   |  arin   ripe   apnic   afrinic  lacnic         |
   +----------+-------------------------------------+
              | Emits CIDR blocks
              v
       filterOutput()
   (strips internal handle findings)
              |
              v
      Domains + CIDRs

* active mode only

Project structure

cmd/pius/               CLI entrypoint (Cobra-based)
pkg/
  cache/                Two-tier caching: API responses (JSON) + RPSL databases (gzip)
  cidr/                 IP range to CIDR conversion and /24 subnet splitting
  client/               Shared HTTP client with retries and 10 MB response limit
  plugins/              Plugin interface, registry, confidence scoring
    all/                Blank imports to trigger all plugin init() registrations
    cidrs/              CIDR discovery plugin implementations
    domains/            Domain discovery plugin implementations
  runner/               Pipeline orchestration, mode filtering, output formatting

Key design decisions

Why a three-phase pipeline?

The three-phase design separates RIR handle discovery (Phase 1) from CIDR resolution (Phase 2). This is necessary because you must first find an organization's RIR handles before you can look up their IP allocations. Phase 0 runs concurrently with everything else for plugins that don't need handles.

How does plugin registration work?

Plugins self-register using Go init() functions. Each plugin calls plugins.Register() with a factory function. Adding a new plugin requires zero changes to the runner -- just implement the interface and add a blank import to pkg/plugins/all/all.go.

How does confidence scoring work?

Name-matching plugins (github-org, reverse-whois, apollo) score ambiguous matches on a 0-1 scale. Findings above 0.65 are emitted normally. Findings between 0.35-0.65 are flagged needs-review. Below 0.35 are dropped as noise.

How does caching work?

Pius uses two cache tiers stored in ~/.pius/cache/:

Tier	Used By	TTL	Format
API response cache	apollo, github-org	24 hours	JSON per key
RPSL registry database	apnic, afrinic	24 hours	Decompressed gzip

How does graceful degradation work?

Plugin errors are logged but never fail the pipeline. If a plugin times out or returns an error, partial results from other plugins are still returned. Missing API keys cause plugins to silently skip via the Accepts() check.

Finding types

Type	Description	Visible in output?
FindingDomain	Discovered domain name	Yes
FindingCIDR	Discovered IP range	Yes
FindingCIDRHandle	RIR organization handle	No (internal, filtered)

Related pages

• Plugins - All 23 plugins with data sources and auth requirements
• Configuration - Cache management and CLI reference
• Contributing - How to add new plugins