Packaging and verifying runs

A package is a folder that holds a model's files, the run configuration it was run with, the inputs files that configuration names, the results and, optionally, the compiled IR, with a manifest, cfdl.package.json. The manifest records:

the engine's name and version, and each pack's name and version;
the model file compilation starts from;
every file with its role (model, run, inputs, ir, results) and the SHA-256 checksum of its bytes;
model_hash, and every ledger hash: the base run's, each scenario's and the Monte Carlo trials'.

The manifest's shape is the package schema.

Making a package

Run the model, write its results into the folder, then package the folder:

cfdl compile deal/ --out deal/ir.json
cfdl run deal/ir.json --config deal/run.json --out deal/results.json
cfdl package deal/ --root model.cfdl --run run.json --results results.json --ir ir.json

Every .cfdl file under the folder is a model file, and every inputs file the run configuration names is included. cfdl package refuses a results document whose ledger hashes are not its own: one changed after its run.

Verifying a package

cfdl verify deal/

cfdl verify runs nothing. It:

checks every file's checksum;
recompiles the model files and checks that they give the manifest's model_hash, and the package's IR when it carries one;
checks that the results' run section, the run configuration as the run used it, is the package's run.json with the inputs files it names;
recomputes every ledger hash from the results' series, journal, transitions and trace, and checks them against the manifest.

On a different engine version than the manifest's, it checks everything but the recompile, and says it did not recompile.

Reproducing a package

cfdl reproduce deal/

cfdl reproduce runs the model under run.json and requires every ledger hash to come out as the manifest records it. It runs only on the manifest's engine version, since another engine may produce another ledger.

The report

Both commands print a report: every check with what it found, and a note naming the checks that failed. They exit 1 when a check fails. A changed byte in any file fails its checksum, which the report names.

From Python and the browser

import cfdl_sdk

manifest = cfdl_sdk.package("deal/", root="model.cfdl", run="run.json",
                            results="results.json", ir="ir.json")
assert cfdl_sdk.verify("deal/")["ok"]
assert cfdl_sdk.reproduce("deal/")["ok"]