1. Delivery package
Each delivery is one directory:
mpg-import-v0.2/
├── classification.csv
├── images.csv
├── manifest.csv
└── images/
└── <relative paths named by images.csv FilePath>
manifest.csv is for transfer verification, not wiki ingestion. It contains
FilePath,Bytes,SHA256 for every delivered image.
The pilot directory is named mpg-import-v0.2-pilot. It also includes
report.json, report.md, selection.json and provenance.json for review
and verification; only the classification and image CSVs are importer inputs.
The image directory contains the canonical images selected for this delivery. Site graphics, thumbnail copies and obsolete size variants are excluded. The pilot contains four published images; it is not the full image migration.
2. CSV rules
- UTF-8 without a BOM; comma-separated; RFC 4180 quoting and doubled embedded double quotes; one header row using the exact names below.
- Record IDs and numbering-system values are strings. Do not coerce Hodges or phylogenetic numbers to floating point or remove meaningful zero padding.
- Empty/unknown values are empty strings. Never emit the text
NULL,null,N/A, or a FileMaker?error marker. - Boolean fields use
1and0.BinomialChangeusesYesandNobecause that is what the current wiki template tests. - A full delivery is an idempotent upsert snapshot. An absent row does not mean delete. Deletes require a future explicit action file.
- The adapter must emit the same bytes for the same source state. Validation failure stops packaging and produces a rejection report; it never silently drops a bad row.
3. classification.csv
One row per FileMaker MPG New_Taxa record that belongs in the wiki taxonomy.
The file deliberately flattens the fields that currently feed both Cargo
Classification and the species-content portion of the entity page. Bryan
called this the “Classification table” delivery; it is an importer shape, not a
claim that every column lives in Cargo Classification.
| Column | Required | FileMaker source / transform | Contract meaning |
|---|---|---|---|
Species |
yes | MPG New_Taxa::Original Name |
Taxon name at every rank despite the column's species-oriented name. |
Level |
yes | Canonical text label derived from MPG New_Taxa::Rank |
Rank such as Order, Family, Genus, Species, or species synonym; labels come from FileMaker's Rank type calculation without the numeric prefix. |
LevelNumber |
no | The wiki's ClassificationLevels.Idx for Level, looked up by label from a pinned snapshot of the wiki page "Classification levels" (adapter/wiki-levels.json) |
Sort/scan key the entity module joins to ClassificationLevels. It is not FileMaker's rank number: the wiki numbers the band above Superfamily differently (Order is 0.1 on the wiki, rank 0 in FileMaker). Empty, with a validation warning, for a level the wiki has no row for (Tribe synonym, subspecies group). |
ID |
yes | Taxa RecID |
FileMaker record ID and existing Classification.ID; unique primary key. |
Parent |
except root | Parent ID No |
Immediate parent FileMaker ID; empty only for the root. |
SynonymOf |
no | Sr Taxa RecID calc for synonym ranks |
Comma-separated accepted-name FileMaker IDs, as Bryan requested. This version normally emits zero or one ID. |
PhylogeneticRevision |
no | Parsed from preferred P No 2023, falling back to P No |
Lowercase revision letter inserted after the first two digits of a P number; for 16a0075.1, emit a. Empty when absent. |
PhylogeneticNumber |
no | Preferred P No 2023, falling back to P No, with the revision letter split out |
P number used for plate order. For 16a0075.1, emit 160075.1. |
HodgesNumber |
no | MONA MPG Calc |
Hodges/MONA number as text, zero-padded to four digits by FileMaker's own calculation. The CSV retains this formatting unchanged. |
BinomialChange |
yes | Yes when brackets left is (; otherwise No |
Whether author/year are parenthesized because the current genus differs from the original combination. |
CommonName |
no | Common Name |
Published common name. |
SpeciesAuthor |
no | Author |
Describing author. |
AuthorYear |
no | Author Year |
Trimmed free text. A four-digit year is the common form; bracketed nomenclatural forms such as [1875] and [no date] are valid. |
Notes |
no | species_note, falling back to C_Notes |
General published species notes. This fallback is an explicit PSC assumption and is easy to correct after Steve reviews a real row. |
TaxonomicNotes |
no | Taxonomic Note |
Published taxonomic note. Do not export editor-only notes. |
Distribution |
no | Distribution |
Published range/locality narrative shown under “Distribution.” |
Description |
no | Description_Field Marks |
Published “Description/Field Marks” content. |
Genitalia |
no | Genitalia |
Species-level genitalia text/reference. Genitalia photographs are separate image rows with IsGenitalia=1. |
ReferenceList |
no | Related References Registry::Reference Link rows, joined on Taxa RecID, ordered by RecID |
Sanitized reference entries joined by the exact delimiter ;;;. The delimiter is forbidden inside an entry, and does not occur in any source value. |
Source field names may change in FileMaker; the delivered column names and meanings above remain stable. Prince Street handles source-field mapping changes.
Rich text
Notes, TaxonomicNotes, Distribution, Description, Genitalia, and each
ReferenceList entry are deterministically sanitized at the adapter boundary,
then delivered as wikitext-safe text that can be placed in a template parameter
unchanged. The source allowlist is:
- elements:
a,b,strong,i,em,br,p,ul,ol,li; - attributes:
hrefandtitleonaonly; - URL schemes:
https,http, andmailto.
Before allowlist validation, the adapter performs three narrow deterministic
normalizations: remove target attributes, unwrap font and span elements
while preserving their contents, and substitute ♂ / ♀ with
♂ / ♀. Scripts, styles, event handlers, images, iframes, and every other
unknown element or attribute are rejected. Plain FileMaker paragraph breaks
become <br>. There is no LLM cleanup step.
MediaWiki does not render <a> in wikitext, so the delivered text contains no
<a> elements (version 0.1 delivered them, and they displayed literally on the
page). Instead:
- Each link is a wikitext external link,
[url text]. Formatting inside the link text is kept, and a]in the link text is written]. A link with empty text shows its URL. A link with an emptyhrefkeeps only its text. - In URLs, space,
",<,>,[,],{,}and|are percent-encoded, and&is written&(MediaWiki decodes it). A relativehrefresolves against the legacy site,https://mothphotographersgroup.msstate.edu/; almost all of them point at its Books pages. Atitleattribute is dropped. - Plain-text sequences that wikitext would read as markup become character
references that render identically:
|as|, the first character of[[,]],{{,}}and of an apostrophe run, and the first of three tildes. - A link nested inside a link is a source defect and rejects.
The remaining elements (b, strong, i, em, br, p, ul, ol, li) are
in MediaWiki's HTML whitelist and are delivered as tags. The validator rejects any
delivered value that is not unchanged by both sanitizing and the wikitext step, so
a raw <a> or an unescaped | cannot reach the importer.
4. images.csv
One row per canonical image to upload or backfill. FilePath points into the
delivery's images/ directory. The extra columns beyond Bryan's list are fields
already required by the live Image Upload template; Prince Street supplies them
so the delivery can populate visible, usable pages in one pass.
| Column | Required | Source / transform | Contract meaning |
|---|---|---|---|
FilePath |
yes | Relative archive/package path | POSIX relative path under images/; no leading slash, .., or URL. Its basename becomes the wiki file name. Basenames must be unique case-insensitively. |
NameType |
yes | Derived | Hodges Number when HodgesNumber is present, otherwise Phylogenetic Number. |
PhylogeneticNumber |
no | Related taxon P number | Secondary join key retained because existing Cargo data uses both numbering systems. |
HodgesNumber |
no | Image/taxon MONA value | Primary image-to-taxon join in this version, matching Bryan's stated preference. At least one of the two number fields is required. Delivered in FileMaker's zero-padded form; the two sources format it differently and are matched on a canonical numeric key before export. |
Date |
no | Collection Date or source-table equivalent |
Trimmed source text. Cargo currently stores this as a string and the edit form permits month names, so this version does not invent day precision. Empty in this pilot. |
Credit |
no | Published webdb credit/caption, or Copyright full where a FileMaker row matches |
Published photographer credit, normalized to plain text; retain the copyright symbol when present. |
LiveCollection |
yes | Derived from published source table | Exact enum Live or Collection, not a boolean. Living and larva source tables are Live; the primary large and genitalia source tables are Collection. These flags follow the source table, including primary photos that depict a living moth. |
IsGenitalia |
yes | Published genitalia source table | Boolean. |
IsLarva |
yes | Published larva source table | Boolean. |
Address |
no | Locality or source-table equivalent |
Human-readable locality. Empty in this pilot. |
Coordinates |
no | GPS or source-table equivalent |
Decimal latitude and longitude as lat, lon; validator enforces valid ranges. Empty in this pilot. |
IsPrimary |
yes | Legacy large selection |
Main species-page image. At most one primary image per taxon; all others 0. |
Reviewed |
yes | Derived | 1 for a row reproduced from the published production site and passing validation. |
Approved |
yes | Derived | 1 for a row reproduced from the published production site and passing validation, so it is visible in current wiki queries. |
The adapter assembles image rows from the relevant FileMaker/image sources rather
than assuming MPG Images to Add is a complete store of record. Known sources
include MPG Images to Add, MPG Genitalia, MPG Larvae, the legacy webdb
image-bearing tables, and the archive paths they reference.
5. Validation and acceptance
Before a package can be delivered:
- CSVs parse under the declared UTF-8/RFC 4180 rules and have exactly one header row with no duplicate columns.
classification.IDvalues are unique; every non-emptyParentandSynonymOfID exists in the same delivery.- The parent graph has one root, no cycles, and no orphaned rows. A source row with no resolvable parent is rejected and reported; the adapter never invents or substitutes a parent.
- Every image has an existing file whose byte count and SHA-256 match
manifest.csv; image basenames are unique after MediaWiki normalization. - Every image has at least one numbering-system join that resolves to a classification row; conflicting Hodges/P-number joins are rejected.
- Enums, booleans, coordinates, years, rich text, and the
;;;reference delimiter conform to this contract. - The report includes source/output row counts, warnings, rejected rows, and duplicate/orphan counts. A non-zero rejection count blocks the delivery.
The first acceptance run is a small, representative pilot: one higher taxon, one ordinary species, one synonym, one P-number revision, one primary collection image, one live image, one larva, one genitalia image, and one record containing rich text and multiple references. After the pilot imports idempotently twice, run the full snapshot and reconcile counts.