TokMesh Lite · Session 02 · Field workbook

A contract meets real rows.

We turned an abstract Rust validator into a runnable synthetic Nigeria lithium learning product, broke its inputs deliberately, and learned to read every failure as evidence.

22 August 2026 · checkpoint 2e00acd
12contract columns
11tests passing
4deliberate defects
2prior checkpoints
01 / ORIENT

What this session accomplished

TokMesh Lite already knew how to parse a YAML manifest and validate CSV data. This session gave that engine a concrete, runnable learning product. All records are explicitly synthetic: they are test data, not claims about real mines, operators, reserves, grades, or production.

Data-product outcome

A portable contract

One manifest describes ownership, a 12-column schema, sovereignty metadata, and executable quality expectations. Two small CSV files demonstrate success and failure without infrastructure or containers.

Learning outcome

Failures became instruments

Missing files, malformed YAML, broken CSV records, and semantically invalid values showed us different failure boundaries and exit codes. We repaired one layer at a time.

Accuracy boundary. Jurisdiction names and lithium vocabulary provide a Nigeria-focused learning context. The individual site identifiers, names, coordinates, grades, dates, and activities are synthetic.
02 / PRODUCT

The three-file learning product

examples/nigeria-lithium/
├── product.yaml   # contract: schema + quality + ownership
├── valid.csv      # eight rows expected to pass
└── invalid.csv    # eight rows with controlled defects
Contract areaDeclared behaviorWhy it matters
Identityid, name, semantic version, descriptionLets people and automation identify the product and its release.
OwnershipNigeria · TokMesh Learning LaboratoryA data product has an accountable owner, not merely a file location.
Schema12 required columnsDefines the expected shape and primitive value types.
QualityUnique IDs, accepted categories, numeric rangesTurns expectations into executable checks.
Sovereigntynational, allowed country NigeriaIntroduces policy metadata without pretending the CLI enforces a full governance platform.
LicenseCC0-1.0Makes the synthetic example easy to reuse.
The 12 columns

site_id, site_name, jurisdiction, district_code, zone, deposit_type, primary_mineral, li2o_grade_percent, activity, latitude, longitude, and sample_date.

03 / LAB

Failure laboratory: what broke and why

1. The fixture did not exist

The CLI process ran, but it could not read product.yaml. That is an execution/input problem, so the command returned exit code 2, not validation exit code 1.

error: could not read .../examples/nigeria-lithium/product.yaml:
No such file or directory (os error 2)

2. YAML indentation was invalid

After the file existed, parsing reached line 2 and stopped. YAML uses indentation as structure, so a misplaced mapping key changes—or destroys—the document.

mapping values are not allowed in this context at line 2 column 7

3. CSV records were physically split

The header had 9 fields in the early draft while subsequent physical lines had 4, 6, or 7. A CSV reader expects a consistent record width unless multiline fields are correctly quoted.

awk -F',' '{print "line", NR, "has", NF, "fields"}' \
  examples/nigeria-lithium/valid.csv

line 1 has 9 fields
line 2 has 4 fields
line 3 has 6 fields

4. Repair restored 12 fields per line

A backup was made first. Wrapped fragments and leading indentation were repaired, then awk confirmed that all nine physical lines—one header plus eight records—contained exactly 12 fields.

line 1 has 12 fields
...
line 9 has 12 fields

5. “Invalid” data was still valid

Copying valid.csv created a structurally useful starting point, but it contained no broken promise. The validator correctly returned 0. We then introduced controlled defects one at a time.

Core lesson: a file can exist but fail to parse; parse successfully but have broken record structure; or be structurally valid while violating the product contract. Each layer needs a precise test and a distinct explanation.
04 / TDD

Four controlled defects, observable through public behavior

We used two already agreed public seams: the CLI process boundary for exit behavior and validate_data for detailed issue reports. We did not test private helper functions.

CSV locationDeliberate valueExpected issueStatus
row[3].site_idNG-LI-000001 repeatedquality.unique.duplicateGREEN
row[4].jurisdictionUnknownquality.accepted_values.rejectedGREEN
row[5].li2o_grade_percent12.50quality.range.outsideGREEN
row[6].site_nameempty fielddata.value.requiredGREEN
row[7].sample_datestill valid: 2026-01-20data.value.typeNEXT RED

Report-level test pattern

#[test]
fn nigeria_lithium_invalid_example_rejects_missing_site_name() {
    let root = std::path::Path::new(env!("CARGO_MANIFEST_DIR"));
    let manifest_path = root.join("examples/nigeria-lithium/product.yaml");
    let data_path = root.join("examples/nigeria-lithium/invalid.csv");

    let manifest = load_manifest(&manifest_path).expect("manifest should parse");
    let report = validate_data(&manifest, &data_path)
        .expect("CSV should be readable");

    assert!(report.issues.iter().any(|issue| {
        issue.code == "data.value.required"
            && issue.location == "row[6].site_name"
    }));
}

CLI process test pattern

let status = Command::new(env!("CARGO_BIN_EXE_tokmesh-lite"))
    .args([
        "data",
        "validate",
        &format!("{root}/examples/nigeria-lithium/product.yaml"),
        &format!("{root}/examples/nigeria-lithium/invalid.csv"),
    ])
    .status()
    .expect("CLI should run");

assert_eq!(status.code(), Some(1));
Why these tests are worth keeping. They describe caller-visible promises: which command succeeds, which exit code represents invalid data, and which stable issue code identifies each broken contract expectation.
05 / RUST

Rust constructs encountered

Compile-time environment

env!(...)

The macro reads a Cargo-provided value while compiling. CARGO_MANIFEST_DIR gives tests a stable repository root instead of relying on the terminal’s current directory.

Paths

Path::new and .join

They build filesystem paths using platform-aware path handling. Joining path components is clearer and safer than hand-building separators.

Result handling

.expect(...)

Unwraps a successful Result. If setup fails, the test stops with the supplied explanation, making fixture problems readable.

Iterator predicate

.iter().any(...)

Borrows each issue and returns true as soon as one issue satisfies the closure. It does not consume the report.

Closure

|issue| { ... }

An inline function receiving one issue. The body defines exactly what matching evidence means.

Assertions

assert! and assert_eq!

assert! expects a Boolean condition. assert_eq! compares two values and prints both sides when they differ.

Logical conjunction

&&

Both the stable issue code and exact location must match. This prevents a test from passing because an unrelated problem happened elsewhere.

Process integration

Command::new

Starts the compiled CLI as a user would, providing confidence in arguments, printed behavior, and operating-system exit status.

06 / COMMANDS

Command atlas: what every part means

Command or tokenMeaning
cargo run -- product validate FILEBuilds if necessary, runs the binary, and passes everything after -- to TokMesh Lite.
cargo fmtRewrites Rust source into standard Rust formatting.
cargo fmt --checkChecks formatting without changing files; useful before a commit or in CI.
cargo testRuns unit tests, integration tests, and documentation tests.
cargo test --test validationSelects the integration-test binary built from tests/validation.rs.
NAME -- --exactSelects a test name; the separator passes --exact to Rust’s test harness so partial names do not match.
cargo clippyRuns Rust’s linter for likely mistakes and non-idiomatic code.
--all-targetsChecks library, binary, tests, examples, and other Cargo targets.
--all-featuresEnables every declared Cargo feature while checking.
-- -D warningsPasses a compiler flag after the separator; -D means deny, so warnings fail the command.
\ at line endThe shell continues the same command on the next physical line. It improves readability without creating another command.
|A pipe sends standard output from the command on its left to standard input of the command on its right.
sed -i '...'Runs the external Unix stream editor and modifies a file in place. Zsh or Bash parses the command; sed performs the edit.
sed -n '6p'Suppresses automatic printing and prints only physical line 6.
cut -d',' -f1-3Uses comma as delimiter and prints fields 1 through 3.
awk -F',' 'NR == 6 { ... }'Uses comma as field separator, selects line 6, and can print NF, the number of fields.
cp SOURCE DESTINATIONCopies a file. We used it to preserve the pre-repair CSV in /tmp.

Why the focused test command has two separators

cargo test --test validation TEST_NAME -- --exact
│     │    │                 │         └─ argument for Rust test harness
│     │    │                 └────────── separator: Cargo stops parsing here
│     │    └──────────────────────────── integration-test target
│     └───────────────────────────────── run tests
└─────────────────────────────────────── Rust package manager/build tool
07 / GIT

Checkpointing without mystery

git status
git add tests/cli.rs tests/validation.rs \
  examples/nigeria-lithium/product.yaml \
  examples/nigeria-lithium/valid.csv \
  examples/nigeria-lithium/invalid.csv
git diff --cached --check
git commit -m "feat: add Nigeria lithium validation example"
git push origin main
CommandRepository effect
git statusRead-only inventory of tracked, modified, staged, and untracked files.
git add FILE...Copies the selected current file contents into the staging area for the next commit.
git diff --cached --statSummarizes what is staged. --cached means the index/staging area.
git diff --cached --checkChecks staged changes for whitespace errors. Silence means none were found.
git commit -m "..."Creates a local, named snapshot from the staging area.
git push origin mainExplicitly sends local main commits to origin/main.
git push -u origin mainPushes and sets the upstream tracking relationship. -u is short for --set-upstream.
git pushAfter upstream exists, pushes the current branch to its configured upstream.
Published checkpoint: d3883f9 contains the Nigeria example and tests. 2e00acd adds prominent rendered-library links to the README. Both reached origin/main, and the working tree was clean.
08 / PAGES

Why docs/index.html exists

GitHub’s repository viewer is a source-code browser. Clicking an HTML file under github.com/.../blob/... shows its source. GitHub Pages is the static web host that renders the same committed HTML.

main branch
└── docs/                      # configured Pages source
    ├── .nojekyll              # publish static files directly
    ├── index.html             # default learning-library home
    └── sessions/
        ├── 2026-08-20-session-01.html
        └── 2026-08-22-session-02.html

Open the rendered TokMesh Lite learning library

Opening a separate tab: GitHub READMEs cannot reliably force target="_blank". Use Ctrl+click on Linux/Windows, Command+click on macOS, middle-click, or the browser’s “Open link in new tab” menu.
09 / QUESTIONS

Your questions, preserved with their answers

?
Shouldn’t we add and commit at every stage?

Commit at coherent GREEN checkpoints—not while a test is intentionally RED. Several closely related fixture behaviors can form one focused commit after the complete quality gates pass.

?
What is TDD?

Test-driven development is a RED→GREEN learning loop: state one public behavior in a failing test, implement only enough to make it pass, then verify. Refactoring comes during review.

?
Is sed -i ... a Bash command?

sed is an external Unix program. Bash or Zsh is the shell that parses quoting, line continuations, and pipes before launching it.

?
What does cargo fmt do?

It applies the standard Rust formatter. cargo fmt --check only checks and is ideal as a quality gate.

?
What does the long Clippy command mean?

It lints every target with every Cargo feature and turns warnings into failures, keeping the project warning-free.

?
Why did the missing fixture produce exit code 2 instead of 1?

Code 1 means the validator successfully evaluated data and found contract violations. Code 2 means it could not perform validation because input was missing, unreadable, or malformed.

?
Why did the invalid CSV initially pass?

It was copied from the valid fixture and had not yet been mutated. A filename does not make data invalid; an observable broken contract promise does.

?
Why count CSV fields with awk?

It exposed physical record breaks before the Rust validator could reach semantic checks. Every row now has the contract’s 12 fields.

?
What does -u change when pushing?

It records an upstream relationship while pushing. Once local main tracks origin/main, plain git push and git pull know the default destination/source.

?
What was index.html for?

It is the default entrance to the GitHub Pages learning library, linking session workbooks and durable guides in a readable website.

?
Can GitHub display the HTML directly?

The normal repository view shows source. The Pages URL renders the page. The README now links directly to that rendered address.

?
Can the README force the library into a new browser tab?

No reliable repository-controlled method exists because GitHub sanitizes such HTML behavior. Keyboard modifiers, middle-click, or the browser menu remain the dependable choices.

?
What is the “agreed public seam” we paused before?

It is the caller-visible boundary where behavior is observed. For this feature, that means the public validate_data report and the CLI process exit code—not private parsing helpers.

10 / EVIDENCE

The final verification run

cargo fmt --check
# passed with no output

cargo clippy --all-targets --all-features -- -D warnings
Finished dev profile ...

cargo test
tests/cli.rs:        4 passed; 0 failed
tests/validation.rs: 7 passed; 0 failed
doc tests:           0 failed

INVALID — 4 issue(s)
- [quality.unique.duplicate] row[3].site_id
- [quality.accepted_values.rejected] row[4].jurisdiction
- [quality.range.outside] row[5].li2o_grade_percent
- [data.value.required] row[6].site_name

The suite contains 11 executable integration tests in total. The invalid CLI output demonstrates four simultaneous findings while the focused report tests prove three of them by exact code and location. Duplicate-ID behavior is also confirmed by the CLI test and output.

11 / RESUME

Resume from one precise boundary

First: understand and reconfirm the seam.

The next test observes validate_data, because consumers care that an invalid date becomes a stable report issue at an exact row and column. They do not care which private helper discovers it.

Then add one test expecting data.value.type at row[7].sample_date. Confirm RED while row 7 still contains valid 2026-01-20. Change only physical CSV line 7 to an unambiguously invalid month such as 2026-13-20, then confirm GREEN.

Do not conflate this fixture slice with the future date-engine feature. The current checker handles shape and broad month/day ranges. Calendar-valid parsing remains a separate roadmap item.