Spec-driven development with approval loops
Failure modes
| Symptom | Cause | Fix | Severity |
|---|---|---|---|
ValueError: plan_path must be a non-empty string |
Empty string or None passed to read_spec / PipelineOrchestrator |
Resolve the path before passing it | high |
FileNotFoundError: Plan file not found |
The plan file does not exist at the given path | Verify the path; use os.path.abspath to be sure |
high |
PipelineResult.success is False |
One or more tasks failed to execute or failed a gate | Iterate result.tasks; check each TaskResult.error, quality_gate_passed, tests_passed |
high |
| A task never appears in output | The XML task block is malformed or absent | Call read_spec directly and inspect the returned list |
medium |
| Execution resumes from the wrong task | SpecState.completed / current is stale |
load_state to inspect, then clear_state to reset |
medium |
get_pending_tasks returns [] unexpectedly |
SpecState.completed already holds all task IDs |
State is stale or the plan finished; clear_state to start fresh |
medium |
| Quality gate always passes | skip_gates=True left in from development |
Remove the flag to re-enable gates | medium |
| State comment vanished after editing the plan | An editor/formatter/VCS step stripped HTML comments | Ensure tooling preserves HTML comments in .md; confirm the comment is present after save_state |
medium |
Risk areas
- Resuming re-runs tasks when state drifts.
get_pending_tasksmatchestask_idvalues fromSpecState.completedagainst the task list fromread_spec. If task IDs are renumbered or reordered between sessions, completed tasks can look pending and run twice. Treat plan files as append-only once execution starts; if you must edit mid-run,clear_statefirst. - Skip flags silently lower quality guarantees.
skip_gates,skip_tests, andskip_simplifyset the correspondingTaskResultfields toNone/Falserather than raising.PipelineResult.successstill returnsTrueif all tasks executed, even with gates skipped. After any skip-flag run, inspectquality_gate_passed,tests_passed, andgate_scoreexplicitly. read_specdoes not warn on empty task lists. A valid file with no parseable XML task blocks returns[]silently, and downstream orchestration completes with nothing to do. Check for a non-empty list before orchestrating.on_task_completeerrors abort the pipeline. An unhandled exception in the callback stops the run at that task. Run viaexecute_with_approval(thespeclayer) and state is saved with that task markedcurrentbefore the callback fires, so resuming re-runs it; barerun_alldoes no state-saving of its own. Wrap callback logic intry/exceptand checkTaskResult.errorbefore acting.
Diagnosis order
- Reproduce with a minimal
read_spec(plan_path)call — if it raises, the problem is the path or the plan file. - Inspect persisted state:
load_state(plan_path); checkcompleted,current,schema_version. - Clear stale state and retry:
clear_state(plan_path). - Re-run with
skip_gates=Trueto isolate gate failures from task logic. Ifsuccessflips toTrue, the gate thresholds or scores are the cause — inspectgate_details. - Iterate
result.tasksand print each failingTaskResult(error,gate_score,gate_details,tests_passed). - Run the related tests:
pytest -k "spec" -v.