farmops: routine operations.

What it is

farmops is meant to run routine operations, such as security patching on a group of hosts in a weekly window, on top of vpscfgfarm (the door) and badfarm (discovery). A catalog entry describes one routine task: which hosts, when, how fast, and what to check before and after.

What exists today

The repository is a skeleton. It holds:

• The catalog schema (farmops/schema.py): version 1 of the entry format, closed to unknown keys, with validate() returning a normalised copy.
• The catalog loader (farmops/catalog.py): load_entry(path) and load_catalog(directory), with a strict YAML reader.
• Dependency pointers (farmops/deps.py): how a farmops checkout names the vpscfgfarm and badfarm checkouts it uses, and records which commit each is at.
• farmops's own journal (farmops/journal.py): append-only, hash-chained JSONL, journal-or-refuse.
• Tests and fakes (tests/): hermetic tests with fake door, badfarm and signer modules. They use temporary directories and never contact a host.

Requirements

• Python 3.9 or later.
• PyYAML, pinned with sha256 hashes in requirements.lock.
• pytest to run the tests (the test extra in pyproject.toml).

Try it

Clone the repository, install the pinned dependency and run the tests:

git clone https://github.com/afterdarksys/farmops.git
cd farmops
python3 -m pip install --no-deps --require-hashes -r requirements.lock
python3 -m pytest -q

Then load a catalog. Save the example entry from the catalog format below as debian-security-web.yaml in a private directory (the loader refuses a directory that group or others can write), and read it from the checkout:

mkdir -m 700 ~/farmops-catalog
# save the example entry as ~/farmops-catalog/debian-security-web.yaml, then:
python3 -c '
import os, sys
from farmops import catalog
entries = catalog.load_catalog(os.path.expanduser(sys.argv[1]))
for name, entry in entries.items():
    print(name, entry["template"], entry["schedule"]["window_s"])
' ~/farmops-catalog

For the example this prints debian-security-web debian/apt-security-upgrade@1 7200: the entry name, its template, and the two-hour window in seconds. Change max_hosts to 0 and the load fails with hosts.max_hosts: 0 is outside 1-64. One invalid entry refuses the whole catalog.

The catalog format

A catalog is one directory, read non-recursively, with one entry per file named <entry>.yaml, where the name matches [a-z][a-z0-9-]{0,62} and equals the file's entry: key. Hidden files are skipped; any other file name refuses the whole catalog. A catalog holds at most 256 entries, each file at most 64 KiB.

version: 1                    # required; exactly 1
entry: debian-security-web    # required; equals the file name
template: debian/apt-security-upgrade@1   # required; <family>/<name>@<version>
description: "Debian security patching on web hosts"
hosts:
  select: 'host.os.id == "debian" and host.os.version_id == "12"'
  require_groups: [web]
  include: []
  exclude: [web3]
  max_hosts: 3
schedule:
  tz: America/New_York
  days: [sun]
  start: "03:00"
  end: "05:00"
after: []
rollout:
  hosts_per_window: 1
  canary: 0
runtime:
  max_runtime_s: 2700
  verify_grace_s: 900
  postcheck_grace_s: 1800
params:
  reboot: if_required
prechecks:
  - index_fresh: 12h
  - badfarm_clear: [health.failed-units, health.disk-fault]
  - badfarm_present: [software.pending-security-updates]
postchecks:
  - badfarm_rescan: {resolves: [software.pending-security-updates], no_new: [health.failed-units]}
rollback: none
notify: [halt, finish, missed, skipped, blocked]
ttl: 7d

Fields

Strict YAML

Entry files must be printable ASCII plus newlines, hold exactly one YAML document, and nest at most 16 levels. On top of PyYAML's safe loader the reader refuses duplicate keys at any depth, anchors, aliases and merge keys, YAML 1.1 boolean words such as yes and on, floats, timestamps and nulls, explicit tags, and any whole number that is not plain decimal (so 0644 cannot silently become 420). Times must be quoted, because an unquoted 3:00 reads as a number. Values are never coerced.

The farmops journal

farmops records what it did and saw in its own journal, separate from vpscfgfarm's authoritative journal. Each record is one line of canonical JSON chained to the previous line by sha256. The whole file is verified before every append; an edited, reordered, truncated or spliced journal refuses the append, and journal-or-refuse means the step that wanted the record does not run. The journal lives under ~/.farmops/journal in the home directory from the password database (never $HOME), with every path component checked for symlinks, ownership and permissions. A secret masker is required, and field names that look like secrets are refused.

The library entry points are Journal(stream, mask=...), with append(), verify(), records(), tail(n) and run_journaled(event, step). In production the masker is meant to be vpscfgfarm's secret masker; only tests pass the explicit NO_MASK.

Planned

The public README lists the rest as later work. None of it exists yet:

• A compiler that turns a catalog entry into a routine policy that vpscfgfarm enforces.
• Templates (the template: key names one, but no templates ship).
• A scheduler that runs due entries with no human present, under a standing grant that a human approved once at a terminal, with a signature. The door is to check that grant on every call.
• A runner, reports and a command-line interface.

How it fits with the other tools

The design puts farmops last in the chain: badfarm observes, vpscfgfarm enforces, and farmops would schedule routine work between them. Today only the data side of that link exists. Catalog prechecks and postchecks name badfarm check IDs, such as health.failed-units and software.pending-security-updates, and deps.py records the vpscfgfarm and badfarm commits in use. Nothing in farmops calls either tool yet.

Links

• README on GitHub
• The catalog format (docs/CATALOG.md)
• Releases: none yet; the package version is 0.0.1.
• MIT license
• Issue tracker
• Security policy

Built from the public farmops repository (README.md, docs/CATALOG.md and the module docstrings in farmops/) at commit 6de9abd.