What is Decision Forge?
Decision Forge is an open-source Python tool for turning an organization’s policies and workflow knowledge into verified training data for decision models. It takes material such as handbooks, standard operating procedures, approval rules, and workflow definitions, then supports a process for reviewing rules, generating decision scenarios, checking labels against policy, and exporting the resulting data. It is aimed at organizations and teams building models that need to reflect their own rules for how work is assigned, information is shared, actions are approved, and outcomes are assessed. The project is in alpha.
Decision Forge Features:
Document Ingestion: Ingests text, Markdown, JSON and YAML, office documents, PDFs, HTML, CSV and TSV, with local OCR available.
Rule Compilation: Drafts rule specifications from organizational documents through a model endpoint selected by the user, with quotations tied to source material.
Business Review: Allows proposed rules to be reviewed and accepted, while authored specifications can enter the pipeline without document compilation.
Scenario Generation: Builds decision scenarios around workflow points, organizing them into families and distributions across four decision surfaces.
Policy Label Replay: Recomputes scenario labels with a deterministic policy interpreter so each example reflects the declared rules.
Training Data Exports: Exports verified decision examples in four formats, including a candidate-readout decision contract.
Family-Based Splits: Keeps related scenario variants together when assigning data splits, and reports and drops duplicates found across splits.
Decision Forge Benefits:
Auditable Decisions: Source quotations and provenance make it easier to inspect which organizational rule supports a training example.
Consistent Labels: Replaying approved rules provides a reproducible basis for labels instead of relying only on manual judgment.
Organization-Specific Data: Teams can build supervision around their own policies and thresholds rather than assuming every organization makes the same decisions.
Local Data Pipeline: After rules are reviewed or authored, the core scenario, labeling, and export workflow can run without API keys or model downloads.
Cleaner Evaluation: Keeping related variants in the same split reduces the risk that near-duplicate examples appear in both training and held-out data.
Use Cases:
Task Allocation: Create examples for a model that must decide which person or agent should take ownership of a workflow task.
Information Sharing: Train decision models on rules for which recipients may receive particular information, including restrictions on what can be shared.
Action Approval: Build examples that teach an agent when to proceed with a workflow action, adjust it, or request authorization.
Outcome Review: Prepare training data for assessing whether a completed deliverable meets an organization's stated acceptance criteria.
Policy-Based Agents: Turn handbooks, approval rules, and standard operating procedures into reviewed decision examples for agent training.

