Skip to content

ADR-0008 — Six Eyes Framework as Content Quality Gate

Status: Accepted
Date: 2026-05-25
Author: João Luís Brazão
Deciders: João Luís Brazão
Vikunja: #838

Context

As the Down a Rabbit Hole brain ingests content from multiple sources (Brave Search discovery, YouTube transcripts, email submissions, manual uploads), a consistent quality gate is needed to:

  1. Prevent low-quality, duplicative, or off-topic content from polluting the knowledge graph
  2. Give the operator (João) a transparent, auditable reason for every accept/reject decision
  3. Allow the system to auto-ingest high-confidence content without human review, while routing uncertain items for manual triage

The system processes hundreds of candidates per week across 13+ domains; manual review of every item is not scalable.

Decision

Adopt the Six Eyes Framework (from João's book Awake: The Practice of Critical Thinking in an Age of Soft Lies) as the structured scoring model for all content entering the brain.

The Six Eyes

Eye Dimension What it evaluates
Attention Factual content Is the information factually grounded?
Emotion Tone Is the tone neutral / credible or sensationalist?
Framing Problem framing Is the right question being addressed?
Structure Logic Is the reasoning coherent and well-structured?
Contrast Perspective Does it offer genuine alternative viewpoints?
Integrity Truthfulness Is the source trustworthy and verifiable?

Each eye is scored 0–10 by the brain.six-eyes-scorer skill (LLM-assisted). The composite score is the arithmetic mean of all six dimensions.

Routing thresholds

Composite score Verdict Action
≥ 6.0 pass Auto-ingest to brain
4.0 – 5.9 quarantine Route to /admin/curation for manual review
< 4.0 rejected Discard with logged reason

Thresholds are configurable per domain via discovery_config.scoring_rubric.

Implementation

  • brain.six-eyes-scorer skill: scores each brain_staging candidate
  • brain.six-eyes-triage skill: handles bulk triage of quarantine queue, including idle_nudge mode for items sitting >48h without review
  • Six Eyes Browser Extension v1.1: scores Brave Search results in-browser; auto-submits pass/quarantine/reject directly to the staging pipeline
  • brain_staging.six_eyes_* columns: store individual dimension scores, composite, verdict, and scored_at timestamp for full auditability

Iteration model

The Six Eyes model iterates: every human approve/reject decision in /admin/domain-explorer or /admin/curation is a labelled training signal. The source_reputation table aggregates per-source signal (accept rate, average score) to adjust future scoring calibration.

Alternatives Considered

Option Reason Rejected
Binary accept/reject No explainability; hard to tune thresholds
Single credibility score Collapses distinct failure modes (factual errors vs framing bias vs tone)
Human review only Not scalable at 100+ candidates/week
Pure ML classifier Requires labelled training data we don't have yet; LLM-assisted scoring works now
Six Eyes (chosen) Grounded in operator's own framework; explainable; tunable per dimension

Consequences

Positive: - Every routing decision is auditable (six dimension scores stored) - Per-domain calibration: cars domain can weight Attention (factual accuracy) higher; geopolitics can weight Integrity (source trust) higher - Human feedback loop: approve/reject decisions improve source_reputation scores over time without retraining any model - Operator familiarity: João wrote the framework; he understands the scores intuitively

Negative / Risks: - LLM scoring has inherent variance: same article may score ±0.5 across runs - Six dimensions are not always independent (a structurally poor argument is also often emotionally manipulative) - Threshold miscalibration can cause good content to land in quarantine; requires periodic review via /admin/curation/pipeline coverage stats

  • ADR-004: LiteLLM as only LLM path → Six Eyes scorer routes through LiteLLM
  • ADR-005: Skills Gateway → scorer and triage are gateway skills