Bro Technical Specification Document

B.R.O. = Build Repeat Orchestrator

1. Overall Architecture

Bro uses a layered architecture:

┌─────────────────────────────────────┐
│           CLI Layer                 │
│    run/list/clean/watch/version     │
├─────────────────────────────────────┤
│        Configuration Layer          │
│  YAML/TOML parse, validate, merge   │
├─────────────────────────────────────┤
│         Scheduler Layer             │
│  DAG build, topo sort, executor     │
├─────────────────────────────────────┤
│           Cache Layer               │
│  CAS, fingerprint, remote cache     │
├─────────────────────────────────────┤
│         Runtime Layer               │
│  embedded shell (mvdan/sh), env     │
└─────────────────────────────────────┘

Design principles:

• Stateless core: scheduling and caching logic does not depend on a daemon.
• Pluggable cache: local cache and remote cache implement the same interface.
• Transparent failures: any step failure provides a clear task chain and error context.

2. Core Concepts

2.1 Task

A task is the smallest execution unit and contains:

• name: task identifier
• cmd: command to execute
• deps: list of prerequisite tasks
• inputs: list of input-file globs
• outputs: list of output files/directories
• env: environment variables
• cache: cache switch with three-state semantics — default auto (enable cache only when inputs or input_cmds are declared; disable when neither is declared); explicit cache: true/false overrides the default
• input_cmds: list of commands run before fingerprint computation; their stdout is included in the fingerprint to capture implicit inputs such as toolchain versions (e.g., go version). A failing command aborts the run — it is never silently ignored
• dir: task working directory (relative to project root); defaults to project root. Applies to command execution, input glob expansion, and output paths
• shell: shell to use (default is the embedded shell interpreter; see Section 11)

2.2 Fingerprint

The unique identifier of a task; determines whether the cache hits. Computed as:

fingerprint = hash(
  task.name,
  task.cmd,
  sorted(task.env),
  hash_of_inputs,
  hash_of_input_cmds,
  hash_of_deps_outputs
)

Here hash_of_inputs is the hash of all input-file contents after sorting; hash_of_input_cmds is the combined hash of each input_cmds command's stdout in declared order, used to capture implicit inputs such as toolchain versions; hash_of_deps_outputs is the combined hash of all dependency task output contents.

input_cmds run through the same shell mechanism as the task command (embedded mvdan/sh by default) on every fingerprint computation, so they should be fast commands. A failing input_cmds command aborts the run with an error naming the task and the command — implicit inputs are never silently dropped. Tasks with substituted CLI arguments (Section 13) are fingerprinted with the substituted cmd, so different arguments never share a cache entry.

2.3 Cache Entry

A cache entry contains:

• fingerprint: task fingerprint
• outputs: archive of output files/directories (tar.gz or zip)
• metadata: execution time, exit code, stdout/stderr (replayed on cache hit so CI and similar scenarios see logs consistent with a real execution)

3. Configuration File Specification

Default configuration file: bro.yaml (or bro.toml).

version: "1"

# Global settings
global:
  shell: "sh"           # Optional: fallback to system shell; default uses embedded mvdan/sh interpreter
  jobs: 4               # Maximum parallelism
  cache-dir: ".bro/cache"
  max-cache-size: "5GB" # Local cache capacity limit; evicted by LRU when exceeded

# Environment variables inherited by all tasks
env:
  NODE_ENV: "development"

# Task definitions
tasks:
  lint:
    cmd: "golangci-lint run"
    inputs: ["**/*.go", ".golangci.yml"]
    env:
      GOLANGCI_LINT_CACHE: ".cache/golangci-lint"

  test:
    deps: ["lint"]
    cmd: "go test -coverprofile=coverage.out ./..."
    inputs: ["**/*_test.go", "**/*.go", "go.mod", "go.sum"]
    outputs: ["coverage.out"]

  build:
    deps: ["test"]
    cmd: "go build -o bin/app ./cmd/app"
    inputs: ["cmd/**/*.go", "internal/**/*.go", "go.mod"]
    outputs: ["bin/app"]

  deploy:
    deps: ["build"]
    cmd: "./scripts/deploy.sh"
    cache: false        # Pure command task, no inputs; explicitly disable caching

3.1 Field Details

Field	Type	Required	Description
version	string	Yes	Configuration file version, used for future compatibility
global.shell	string	No	System shell to fall back to; default is the embedded mvdan/sh interpreter (see Section 11)
global.jobs	int	No	Maximum number of parallel tasks; defaults to CPU core count
global.cache-dir	string	No	Local cache directory; defaults to .bro/cache
global.max-cache-size	string	No	Local cache capacity limit; defaults to 5GB; evicted by LRU when exceeded
remote-cache	mapping	No	Remote cache backend (spec Section 9.3); type: s3 (requires endpoint/bucket, optional prefix/region) or type: http (requires endpoint, optional prefix); credentials come from the environment
env	map	No	Global environment variables
tasks.<name>.cmd	string	Yes	Task command; may reference CLI arguments via {{args}} / {{argN}} templates (spec Section 13)
tasks.<name>.deps	[]string	No	Dependency task names
tasks.<name>.inputs	[]string	No	Input-file globs; support **, *, ?; resolved relative to the task's dir
tasks.<name>.outputs	[]string	No	Output files/directories; resolved relative to the task's dir
tasks.<name>.env	map	No	Task-level environment variables; override globals
tasks.<name>.cache	bool	No	Cache switch; default auto: enable cache only when inputs or input_cmds are declared; disable when neither is declared; explicit true/false overrides the default
tasks.<name>.input_cmds	[]string	No	Commands run before every fingerprint computation; stdout included in the fingerprint (e.g., go version); a failing command aborts the run. Keep them fast
tasks.<name>.dir	string	No	Task working directory (relative to project root; must exist); defaults to project root. Applies to command execution, input globs, output paths, and input_cmds
tasks.<name>.shell	string	No	Task-level shell (fallback to system shell)

Long-term direction: multi-package workspace configuration discovery (automatic discovery of task configs in subdirectories) is out of scope for MVP and Phase 2; to be evaluated now that the dir field has landed.

4. Task Execution Model

4.1 Execution Flow

1. Load configuration
2. Build DAG (nodes = tasks, edges = dependencies)
3. Detect circular dependencies
4. Topologically sort the subgraph of the target task
5. Schedule tasks in topological order
6. For each task:
   a. Compute fingerprint (running `input_cmds` first when declared — a failing command aborts the run)
   b. Query cache (local → remote)
   c. On hit: restore outputs and replay cached stdout/stderr (CI scenarios need logs consistent with real execution), mark complete
   d. On miss: execute cmd (in the task's `dir`, defaulting to the project root)
   e. On success: archive outputs into cache
7. Summarize results

4.2 Parallel Scheduling

Uses a worker-pool model:

• Initially all tasks with in-degree 0 enter the ready queue.
• jobs workers take tasks from the queue and execute them.
• When a task finishes, decrement the in-degree of its downstream tasks; enqueue them when in-degree reaches 0.
• If any task fails, cancel all downstream tasks that depend on it (fail-fast).

5. DAG Scheduling Algorithm

5.1 Graph Construction

type TaskNode struct {
    Name     string
    InDegree int
    Deps     []string
    ReverseDeps []string
}

5.2 Cycle Detection

Use Kahn's algorithm or DFS coloring. If nodes remain unprocessed after topological sorting, a cycle exists and the cycle path is output.

5.3 Subgraph Pruning

When executing bro run build, only the transitive closure of build needs to be built; nodes outside the target task graph do not need to be loaded.

6. Content-Addressable Cache (CAS)

6.1 Cache Key

The cache key is the fingerprint, using SHA-256.

6.2 Local Cache Layout

.bro/cache/
├── entries/
│   └── ab/cd/abcdef123456.../
│       ├── metadata.json
│       └── outputs.tar.zst
├── hash-cache.json
└── index/
    └── task-name-to-fingerprints.json

Two-level directory storage avoids too many files in a single directory.

hash-cache.json is the (path, mtime, size) → content-hash cache described in Section 7.5; it is a pure performance optimization and can be deleted safely. index/task-name-to-fingerprints.json is used only for statistics and bro list/historical display; the fingerprint itself is computed and does not depend on this index. If the index is corrupted, it can be safely rebuilt.

6.3 Output Archiving

• Use tar + zstd compression.
• Preserve file permissions, modification time, and symbolic links.
• On Windows, permission information may be ignored or use an ACL summary.

6.4 Cache Invalidation

A cache hit requires:

1. Exact fingerprint match
2. Output archive is complete and readable
3. Output paths have not been externally modified (optional secondary check)

6.5 Concurrency Safety

Multiple bro processes may share the same cache directory (e.g., multiple local terminals, concurrent CI jobs):

• Write: write to a temporary file first, then atomically rename to the final path after flushing to disk, so other processes never read a half-written file.
• Read: verify archive integrity (length + checksum) when reading; incomplete archives are treated as misses and re-executed.
• Restore: extract the archive into a staging directory inside the cache directory (which is excluded from input expansion), then swap each declared output into place via rename-with-replace. Rename is atomic, so a concurrent process observes either the old or the new content, never a half-written file. On Windows a rename-with-replace can still fail with ERROR_ACCESS_DENIED while the destination has open handles (antivirus, indexer, or another bro process reading the output), so the rename is retried with a short backoff before falling back to clearing the destination first; a non-empty directory output always requires clearing before rename. That fallback opens a brief window in which the output path does not exist, so correctness never relies on atomicity alone (see below). When a custom cache-dir lives on a different filesystem, restore degrades to direct extraction.
• Output verification tolerance: after executing or restoring a task, the declared-output check and output hashing are retried with a short backoff, because a concurrent process's restore fallback (above) can transiently remove a path. If the outputs are still inconsistent after a successful execution, the task command is re-executed once before failure is declared; after a restore, the task degrades to re-execution. Concurrency anomalies therefore never fail a task that actually succeeded and never produce wrong outputs — worst case, work is recomputed.

6.6 Cache Eviction

• Local cache has a capacity limit configured by global.max-cache-size (default 5GB; suffixes KB/MB/GB/TB are binary multiples, validated at config-load time).
• Only the entries/ tree counts toward the limit; small side files (hash-cache.json, staging dirs) are excluded.
• Last-access time is tracked as the entry directory's mtime: set at write time and refreshed on every cache hit. When the limit is exceeded, the least recently accessed entries are evicted until the directory is within the limit.
• Enforcement runs after every cache write; the entry just written is never evicted by its own enforcement pass.
• bro clean still empties the entire cache.

7. Input Fingerprint Computation

7.1 File Fingerprint

For each input file, compute the content hash first, then write it into the task fingerprint hasher with a length prefix:

content_hash = sha256(file_content)
write: len(path) + path + content_hash

Path and content are not concatenated directly — "ab"+"c" and "a"+"bc" would collide. A unified length-prefixed framing scheme is used (write path length first, then path, then fixed-length content_hash) to eliminate ambiguity. Including the path in the fingerprint also avoids path conflicts that would arise from using only content hashes. Pseudocode in 17.1.

7.2 Glob Expansion

Supported glob patterns:

• *: match any file in the current directory
• **/*.go: recursively match all .go files
• ! prefix: exclude (Phase 2 support)

Expanded paths are sorted by relative path to ensure stable fingerprints. Paths under the cache directory are always excluded from expansion, so broad patterns such as **/* never fold cache content into fingerprints.

7.3 Directory Inputs

If inputs contains a directory, recursively traverse all files inside, sort by relative path, and compute the hash.

7.4 Environment-Variable Fingerprint

By default, only the env variables explicitly declared for the task are included in the fingerprint — that is, the task-level env merged over the global env, since every task inherits the global values. System environment variables are excluded. Optional env-include / env-exclude configuration controls whether system environment variables are included.

7.5 Hash Cache

To avoid re-reading and re-hashing all input files on every run, cache file content hashes keyed by (path, mtime, size):

• When file metadata is unchanged, reuse the previous content_hash, skipping disk reads and hash computation.
• Re-read and recompute only when any metadata field changes.
• This cache is stored under .bro/cache/ as a pure performance optimization; it can be safely deleted, after which the next run will recompute everything.
• This is the key technique for achieving the < 50 ms cache-hit target in large repositories.

8. Output Capture and Verification

8.1 Output Declaration

Before a task executes, paths in outputs should not exist or will be overwritten. After successful execution, Bro archives outputs into the cache. Declared outputs are slash-separated paths relative to the task's dir; trailing slashes and ./ prefixes are normalized at config load (dist/ is the same output as dist), and absolute or root-escaping paths are rejected.

8.2 Output Verification

• If a task declares outputs but they are not produced after execution, it is treated as a failure. A concurrent process's cache restore can transiently remove a declared output, so the check tolerates that window (retry, then one re-execution) before failure is declared (Section 6.5).
• When restoring cache outputs, extract into a staging directory and swap each output into place via rename-with-replace; only when rename is not possible (non-empty directory destination, or a cache-dir on a different filesystem) is the target path cleared first and the archive extracted directly (Section 6.5).
• When restoring outputs from cache, refresh file mtimes to the current time to prevent downstream tools based on mtime (such as Make) from mistaking restored files as stale.

8.3 Output Conflicts

Two tasks' outputs paths must not overlap. Conflicts are detected at config-load time with an error indicating the task names.

9. Remote Cache

9.1 Interface Design

// Cache is the context-aware cache-backend interface. It operates on entry
// content: the metadata JSON document and the output archive, keyed by
// fingerprint. Local and remote entries share the same format and layout
// (entries/ab/cd/<fp>/{metadata.json,outputs.tar.zst}), so they are
// interchangeable.
type Cache interface {
    // Get returns (nil, nil, nil) on a miss; a corrupt entry is an error.
    Get(ctx context.Context, fingerprint string) (meta []byte, archive io.ReadCloser, err error)
    Put(ctx context.Context, fingerprint string, meta []byte, archive io.Reader, archiveSize int64) error
    Exists(ctx context.Context, fingerprint string) (bool, error)
}

The scheduler never talks to remote backends directly: the Store front-end composes the local CAS with an optional remote Cache. Lookups go local → remote; a remote hit is verified and atomically committed into the local CAS before use, so subsequent runs are local-fast. Writes commit locally first, then upload best-effort (upload failure = warning, never a build failure).

9.2 Backend Implementations

Backend	Protocol	Description
local	Filesystem	Default; used for local development
s3	S3 API	Compatible with MinIO, AWS S3, Cloudflare R2
http	REST API	Generic interface for self-hosted cache servers: GET /{prefix}{key} (404 = miss), PUT /{prefix}{key}, HEAD /{prefix}{key} for existence; two objects per entry mirroring the CAS layout (metadata.json + outputs.tar.zst under entries/ab/cd/<fp>/)

9.3 Configuration Example

remote-cache:
  type: s3
  endpoint: "s3.amazonaws.com"
  bucket: "bro-cache"
  prefix: "myproject/"
  region: "us-east-1"

remote-cache:
  type: http
  endpoint: "http://cache.local:8080"
  prefix: "myproject/"

Implementation notes (M4):

• The endpoint may carry an http:// or https:// scheme selecting the transport; a bare host[:port] implies https (use http://localhost:9000 for a local MinIO).
• Credentials come from the environment, never from the config file. Lookup order: AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY (with optional AWS_SESSION_TOKEN), then MINIO_ROOT_USER/MINIO_ROOT_PASSWORD, then the legacy MINIO_ACCESS_KEY/MINIO_SECRET_KEY.
• The bucket is created automatically when missing.
• Remote caching is entirely optional: with no remote-cache section behavior is purely local. If the section exists but the backend cannot be initialized (no credentials, unreachable endpoint), bro warns once and continues with the local cache only.

Implementation notes for the http backend (M5):

• The endpoint is the base URL of the cache server; objects are addressed as {endpoint}/{prefix}entries/ab/cd/<fp>/{metadata.json,outputs.tar.zst}.
• Authentication is an optional Bearer token from the BRO_REMOTE_HTTP_TOKEN environment variable, sent on every request when set.
• Like for S3, uploads write the archive object before the metadata object, downloads are verified (metadata fingerprint match + full archive decode) before entering the local CAS, and any backend error degrades to local behavior with a warning.

9.4 Consistency

• Before uploading, the just-written local entry is re-verified: the metadata fingerprint must match the cache key and the archive must fully decode.
• On download, the entry is staged into a temporary directory inside the cache, verified (metadata fingerprint match + full archive decode), and only then atomically renamed into the local CAS — a tampered remote object never becomes visible locally; the run warns and degrades to re-execution.
• The archive object is uploaded before the metadata object, so a metadata object always implies a complete archive.

9.5 Trust Model

In the MVP phase, the remote cache has only checksums, which can defend against transmission corruption but cannot defend against malicious poisoning — anyone with write permission can upload tampered entries. Recommendations:

• Use read-only credentials in CI for downloading; use separate write credentials for uploading, narrowing the poisoning surface.
• Entry signature verification (e.g., cosign) is a long-term direction.

10. Concurrency Control

10.1 Worker Pool

type Scheduler struct {
    jobs    int
    graph   *TaskGraph
    cache   Cache
    workers chan struct{}
}

10.2 Synchronization Mechanisms

• Use sync.WaitGroup to wait for all tasks to complete.
• Use errgroup.Group to propagate the first error and cancel the remaining tasks.
• Task status is stored in a concurrency-safe map.

10.3 Resource Contention

• Output-path conflicts are detected at the configuration stage.
• The same task is never scheduled twice.

11. Cross-Platform Command Execution

11.1 Shell Choice

By default, use the embedded POSIX shell interpreter mvdan/sh (pure Go implementation, same approach as go-task):

• All platforms execute the same shell semantics, independent of the system shell; consistent behavior on Windows, macOS, and Linux.
• No extra tools need to be installed on Windows to run POSIX-style commands.

The shell configuration item (global or task-level) is retained, allowing users to explicitly specify a system shell (e.g., bash, pwsh) as a fallback for tasks that depend on specific shell features.

11.2 Command Execution

Default path: the embedded interpreter parses and executes commands in-process:

r, _ := interp.New(
    interp.Dir(projectRoot),
    interp.Env(expand.ListEnviron(mergeEnv(os.Environ(), task.Env)...)),
)
r.Run(ctx, parsedScript)

When the user specifies a system shell, fall back to os/exec:

cmd := exec.Command(shell, "-c", task.Cmd)
cmd.Dir = projectRoot
cmd.Env = mergeEnv(os.Environ(), task.Env)

The working directory (interp.Dir / cmd.Dir) is the task's dir resolved against the project root — projectRoot itself when the task does not set dir. The same directory is used for input glob expansion, output paths, and input_cmds.

11.3 Path Handling

• Use / uniformly as the path separator in the configuration file.
• Convert to the local path separator before execution based on the runtime platform.
• Display relative paths in output logs to keep cross-platform consistency.

12. Watch Mode

12.1 Implementation

bro watch <task> (implemented in internal/watch on top of fsnotify) runs the target's transitive closure once at startup, then monitors its inputs:

1. Watch the directories of every input pattern of every task in the closure: the pattern's static prefix (the deepest leading path without glob magic) plus all of its subdirectories. fsnotify watches are not recursive (and Windows has no native recursive mode), so subdirectories are subscribed individually, and newly created directories are subscribed as they appear. A literal-file pattern contributes its parent directory; a not-yet-existing prefix climbs to its nearest existing ancestor. The cache directory is excluded so cache writes never retrigger runs.
2. Events only arm the debounce timer. When it fires, every task's inputs are re-expanded (so newly created/deleted files are seen) and content-hashed through a session hash cache; the per-task diff against the previous snapshot is the dirty marking — the affected tasks and their downstream. Editor temp files, task outputs, and cache writes never trigger spurious re-runs.
3. The re-run schedules the full closure through the normal scheduler. Because a task's fingerprint covers its dependencies' output hashes and the scheduler records those on cache hits too, running the full closure keeps fingerprints identical to a plain bro run; only tasks whose inputs actually changed (plus downstream) re-execute — everything else hits the cache in milliseconds. Declared outputs that are also inputs are re-baselined after each run so the run's own writes do not echo as another trigger.

The snapshot is deliberately content-based, not mtime-based: a cache hit restores outputs with a refreshed mtime (Section 6.4), so an mtime diff would retrigger on every run forever. A side benefit is that a bare touch does not cause a re-run.

A failing run does not stop the session: the error is printed and watching continues, so the next save picks up the fix. Ctrl-C (SIGINT/SIGTERM) cancels the session context and exits with status 0.

12.2 Debouncing

Aggregate change events and trigger rerun only after no new events occur within 100 ms, avoiding frequent builds caused by rapid saves.

13. CLI Design

bro init              # Create a bro.yaml template
bro list              # List all tasks
bro run <task>        # Run a task (and its dependencies)
bro run <task> [args...] # Pass arguments to a task (see below)
bro run build --force # Force execution, skipping cache
bro clean             # Clear local cache
bro clean --outputs   # Clear all declared outputs
bro watch <task>      # Watch inputs and rerun automatically
bro version           # Show version

Implementation notes (M5, authoritative — see internal/cli/):

• bro init is not implemented: a project is created by writing a bro.yaml by hand (see docs/getting-started.md).
• bro run --force is not implemented: force re-execution by changing an input or running bro clean first.
• The implemented command set is exactly run, list, clean [--outputs], watch, version (plus cobra's built-in help and completion), with global flags --quiet/-q, --verbose/-v, and --debug.

13.0 Task Arguments

bro run <task> [args...] passes the arguments after the task name to the task via template substitution in cmd:

• {{args}} — all arguments joined by spaces.
• {{arg1}}, {{arg2}}, ... — positional arguments (1-based).

tasks:
  build:
    cmd: "go build -tags {{arg1}} -o bin/app ./cmd/app"

bro run build -- release runs go build -tags release -o bin/app ./cmd/app. (The -- separator is optional but recommended when an argument starts with a dash, so the CLI parser does not mistake it for a flag.)

Rules:

• Substitution happens before fingerprinting, so arguments are part of the cache key: bro run build -- release and bro run build -- debug never share a cache entry.
• Arguments are inserted literally (no shell quoting).
• {{argN}} beyond the number of supplied arguments is an error naming the task and the placeholder — required arguments have no default.
• Passing arguments to a task whose cmd has no placeholder is an error — the arguments would otherwise be silently ignored.
• Placeholders are exact ({{args}}, {{arg1}}); unrecognized {{...}} forms are left as-is.
• bro watch passes no arguments; parameterized tasks are not supported in watch mode.

13.1 Output Format

[lint]  ⏳  computing fingerprint...
[lint]  ✅  cached (12ms)
[test]  ⏳  running go test ./...
[test]  ✅  passed (3.4s)
[build] ⏳  running go build -o bin/app ./cmd/app
[build] ✅  done (1.2s)

3 tasks, 1 cached, 2 executed, 4.6s total

Implementation notes (M3):

• The emoji indicators are used when stdout is a TTY. On piped output (CI logs) they degrade to plain ASCII: [test] $ go test ./... for a running task, [test] done (3.4s), [lint] cached (12ms), and [test] failed (0.3s): <err> on the error stream.
• With parallel workers, each task's captured stdout/stderr is printed atomically when the task finishes, so output blocks never interleave.
• On failure, skipped tasks are reported as [name] skipped (upstream task "x" failed) and the summary line gains failure/skip counts: 4 tasks, 0 cached, 1 executed, 1 failed, 2 skipped, 0.7s total.
• Durations render as milliseconds below one second (12ms) and with one decimal above (3.4s).

14. Error Handling and Logging

14.1 Error Classification

Error Type	Description	Handling
Config error	YAML parsing failure, task does not exist	Exit immediately; output detailed location
Circular dependency	Cycle exists in DAG	Output cycle path
Task failure	Command returns non-zero	Output failed task and its dependency chain
Cache error	Cache read/write failure	Degrade to re-execution; warn user

14.2 Log Levels

• --quiet: output only errors
• --verbose: output debug information such as fingerprint computation and cache queries
• --debug: output full commands, environment variables, and stack traces

15. Dependency Library Selection

Purpose	Candidates	Choice
CLI framework	cobra / urfave/cli	cobra
Config parsing	yaml.v3 / toml	yaml.v3 (YAML first)
File watching	fsnotify	fsnotify
Compression	zstd / gzip	zstd (higher compression ratio)
Hashing	crypto/sha256	standard library
Glob	doublestar	doublestar (supports **)
Shell interpreter	mvdan/sh	mvdan/sh (cross-platform consistency)
Concurrency	errgroup / sync	standard library + golang.org/x/sync
S3 client	minio-go / aws-sdk-go-v2	minio-go
Colored output	fatih/color	fatih/color

16. Project Structure

bro/
├── cmd/
│   └── bro/
│       └── main.go
├── internal/
│   ├── config/        # Configuration parsing and validation
│   ├── dag/           # DAG construction and topological sorting
│   ├── exec/          # Worker-pool scheduler and command execution (embedded mvdan/sh + system-shell fallback)
│   ├── cache/         # Cache interface and implementations
│   ├── fingerprint/   # Fingerprint computation
│   ├── watch/         # Watch mode (fsnotify, debounce, dirty-marking)
│   ├── cli/           # CLI command implementations
│   └── testutil/      # Test helpers
├── docs/
│   ├── getting-started.md
│   ├── benchmarks.md
│   ├── benchmark-comparison.md
│   └── technical-specification.md
├── internal-docs/        # contributor-facing docs (not published)
│   ├── requirements-analysis.md
│   ├── development-plan.md
│   └── benchmark-plan.md
├── e2e/               # End-to-end CLI tests
├── examples/          # Example project configurations
├── bro.yaml           # Bro dogfoods itself: the repo's own task pipeline
├── Makefile
├── go.mod
└── README.md

All implementation packages live under internal/ — they are not a public API and must not be importable by other modules.

17. Key Algorithm Pseudocode

17.1 Fingerprint Computation

func computeFingerprint(task Task, inputs map[string][]byte, depOutputs []string) string {
    h := sha256.New()

    // Length-prefixed framing eliminates concatenation ambiguity between "ab"+"c" and "a"+"bc"
    writeFrame := func(data []byte) {
        binary.Write(h, binary.BigEndian, uint64(len(data)))
        h.Write(data)
    }

    writeFrame([]byte(task.Name))
    writeFrame([]byte(task.Cmd))

    for k, v := range sorted(task.Env) {
        writeFrame([]byte(k))
        writeFrame([]byte(v))
    }

    for path, content := range sortedByKey(inputs) {
        writeFrame([]byte(path))
        contentHash := sha256.Sum256(content) // compute content hash first
        writeFrame(contentHash[:])
    }

    for _, out := range runInputCmds(task.InputCmds) {
        writeFrame([]byte(out)) // command stdout included in fingerprint
    }

    for _, depOutput := range depOutputs {
        writeFrame([]byte(hashFile(depOutput)))
    }

    return hex.EncodeToString(h.Sum(nil))
}

17.2 Topological Sort

func topoSort(graph *TaskGraph, target string) ([]string, error) {
    // 1. Compute transitive closure from target and prune to subgraph (see Section 5.3)
    sub := transitiveClosure(graph, target)

    // 2. Copy subgraph in-degrees and run Kahn's topological sort on the copy
    inDegree := copyInDegree(sub)
    queue := []string{}
    result := []string{}

    for name, node := range sub.Nodes {
        if node.InDegree == 0 {
            queue = append(queue, name)
        }
    }

    for len(queue) > 0 {
        name := queue[0]
        queue = queue[1:]
        result = append(result, name)

        for _, dep := range sub.Nodes[name].ReverseDeps {
            inDegree[dep]--
            if inDegree[dep] == 0 {
                queue = append(queue, dep)
            }
        }
    }

    // 3. Validate against subgraph node count to detect cycles
    if len(result) != len(sub.Nodes) {
        return nil, errors.New("cycle detected")
    }

    return result, nil
}

18. Testing Strategy

18.1 Unit Tests

• Configuration parsing (valid/invalid YAML)
• DAG construction and cycle detection
• Fingerprint computation stability
• Cache read/write

18.2 Integration Tests

• End-to-end execution of simple task graphs
• Cache hit and invalidation scenarios
• Cross-platform command execution

18.3 Performance Tests

• Scheduling time for 100 tasks with all cache hits
• Fingerprint computation time for large-file inputs
• Remote cache upload/download time

19. Milestones

Phase	Time	Deliverable
M1	Weeks 1-2	Configuration parsing, DAG construction, basic CLI
M2	Weeks 3-4	Local CAS cache, task execution, output capture
M3	Weeks 5-6	Parallel scheduling, embedded shell and cross-platform execution, error handling
M4	Weeks 7-9	Remote cache, watch mode
M5	Weeks 10-12	Windows adaptation verification, test coverage, documentation, first release

Note: The original 8-10 week schedule was overly optimistic — Windows adaptation and remote cache effort are typically underestimated, so M4 is split and the overall schedule is relaxed to 10-12 weeks.

---

文档版本：0.2
日期：2026-08-02