Project Aquarius // Protocol v1.0
Return to Index

Aquarius Governance Standard

2025-12-30
System

Introduction

Project Aquarius operates on a decentralized model where GitHub serves as the single source of truth for all research and engineering activities. The official website (aquarius.abanhasan.net) serves as a discovery and monitoring layer, automatically ingesting data from various repositories within the Project-Aquarius-White organization and beyond.

To maintain consistency and enable automated tracking, every repository associated with Project Aquarius must adhere to the following two requirements:

  1. Repository Topic: The repository must be tagged with the project-aquarius topic.
  2. Project Manifest: The repository root must contain an aquarius.project.json file.

This governance standard ensures that all reproductions are discoverable, measurable, and standardized across the organization.

The Six Stages of Reproduction

Project Aquarius utilizes a rigid six-stage milestone model to track the lifecycle of a paper reproduction. This progression ensures that a project moves from theoretical understanding to verified implementation and finally to public release.

Stage 1 - SPEC: Paper Analysis

The initial phase focuses on analytical de-noising. The primary deliverables include a comprehensive analysis of the target paper, definition of the core innovation to be reproduced, and explicit acceptance criteria. This stage is complete when the technical specifications are formalized.

Stage 2 - PROTO: Notebook Reproduction

This stage involves quick validation of the core logic using interactive environments such as Jupyter notebooks. The goal is to achieve logical parity with the paper's mathematical foundations without the overhead of a full software architecture.

Stage 3 - LIB: Clean Implementation

Once the logic is validated, the project moves to a clean-room implementation. This involves refactoring the prototype into a modular, testable library. High test coverage and adherence to software engineering best practices are required at this stage.

Stage 4 - BENCH: Experiment Harness

With a stable implementation, the focus shifts to creating a reproducible experiment harness. This includes configuration management, command-line interfaces (CLI) for training/evaluation, and ensuring that all experiments can be run in a containerized or deterministic environment.

Stage 5 - MATCH: Verification

The verification stage is the most critical. Results generated by the implementation are compared directly against the claims and benchmarks reported in the original paper. A project remains in this stage until the reproduction results match the paper within a statistically significant margin.

Stage 6 - SHIP: Publication

The final stage involves the formal publication of the reproduction. This includes a detailed technical writeup (retrospective), a formal release of the codebase, and the finalization of documentation for public consumption.

Metadata Standard

The aquarius.project.json file is the primary machine-readable manifest for each project. It must contain the following fields described below.

Versioning and Core Identity

  • schema_version: The version of the Aquarius metadata standard used (e.g., "1.0.0").
  • project.title: The formal name of the reproduction project.
  • project.type: The category of the project (e.g., "paper-reproduction", "tooling", "infrastructure").
  • project.reproduction_stage: The current stage of the project, ranging from "Stage 1" to "Stage 6".
  • project.status: The operational status of the project (e.g., "active", "completed", "archived", "on-hold").

Paper Reference

  • paper.title: The original title of the research paper being reproduced.
  • paper.authors: A list of the original authors.
  • paper.year: The year of publication.
  • paper.venue: The conference or journal where the paper was published (e.g., "NeurIPS", "ICLR", "arXiv").
  • paper.pdf_url: A direct link to the paper's PDF.

Resource Links

  • links.repo: The URL of the GitHub repository.
  • links.website: The project-specific landing page (if applicable).
  • links.writeup: A link to the technical retrospective or publication.

GitHub Conventions

Consistency across GitHub repositories is maintained through standardized labels and milestones.

Labels Taxonomy

Repositories must use the following label categories to categorize work:

  • stage:*: Corresponds to the six stages (e.g., stage:1-spec, stage:3-lib).
  • status:*: Indicates the priority or state of an issue (e.g., status:blocker, status:in-progress).
  • backend:*: Identifies the technical stack involved (e.g., backend:tinygrad, backend:pytorch).
  • experiment:*: Categorizes experiment-related tasks (e.g., experiment:hyperparameter-tuning).

Milestones and Sprints

  • Milestones: Every repository should have milestones named after the six stages (S1-SPEC through S6-SHIP) to track macro-progress.
  • Sprint Labels: Active development is tracked using weekly sprint labels in the format sprint:YYYY-W## (e.g., sprint:2025-W52).

Issue Templates

Standardized issue templates must be used for bug reports, feature requests, and stage-specific tasks to ensure that all necessary technical context is captured at the point of creation.

Getting Started

To initialize a new reproduction repository within the Project Aquarius organization:

  1. Create a new repository and apply the project-aquarius topic.
  2. Initialize the repository with the standard Aquarius directory structure.
  3. Create the aquarius.project.json file at the root using the fields defined in the Metadata Standard section.
  4. Set up the six milestones (S1-S6) in the GitHub repository settings.
  5. Apply the standard label set using the organization's repository template or automation tools.
END OF TRANSMISSION
ARCHIVE