Methodology

How CSVFixLab decides what is safe to repair

A transparent boundary between what can be proven from CSV structure and what requires knowledge of the source or destination system.

Core principle

CSVFixLab automates representation changes only when the parsed structure is unambiguous. It does not treat “the file parses after an edit” as proof that the edit preserved the intended business data.

1. Prove structure

Validate encoding, parse delimiters and quotes, identify the header, and compare row widths.

2. Separate ambiguity

Distinguish formatting that can be normalized from missing values, duplicate meanings, or unclear field boundaries.

3. Block unsafe output

When a correction requires guessing, report the evidence and keep the repair download disabled.

What the browser worker checks

UTF-8Auto mode uses strict UTF-8 decoding. A non-UTF-8 file is not assigned a guessed legacy charset for repair.
CSV parsingThe parser identifies fields, delimiters, quoted values, and parse errors.
HeadersEmpty headers, duplicates, and surrounding whitespace are reported.
RowsEach non-empty data row is compared with the header's field count.
Safe outputWhen allowed, normalized output uses comma separation, CRLF line endings, trimmed headers, removed fully empty rows, and UTF-8 text.

Errors that block automatic repair

  • Unclosed or malformed quoted fields.
  • Inconsistent field counts.
  • Duplicate header names.
  • Empty header names.
  • Unknown legacy source encoding in Auto mode.

Why source encoding is never guessed for repair

Browsers can decode several legacy encodings when the label is known, but they do not provide a universal reliable charset detector for arbitrary uploaded files. CSVFixLab therefore uses Auto only to confirm valid UTF-8. If strict UTF-8 fails, the user must select the known source encoding before conversion can be enabled.

Platform rules are a separate layer

Generic CSV syntax cannot prove that a Shopify handle exists, that a CRM identifier is valid, or that an ERP accepts a particular status value. Platform-specific pages only add rules that can be checked offline from the file itself, and they state when live-account validation is outside scope.

Reproducible fixtures

The lab case library publishes small known inputs for delimiter normalization, column mismatch, broken quotes, non-UTF-8 confirmation, duplicate headers, and a basic Shopify required-header scenario. These fixtures let users verify both successful normalization and intentional refusal to repair.

Current limitations

  • This is not a schema inference engine.
  • It does not restore bytes that were already corrupted by another program.
  • It does not connect to destination accounts for live validation.
  • It does not guarantee that a structurally clean file will be accepted by a destination platform.
Reviewable behavior is the goal. A blocked repair with a clear reason is preferable to a downloadable file whose correctness depends on an unspoken guess.

Methodology reflects the current browser-based CSVFixLab MVP implementation and its published lab fixtures.