All Posts

This fourth post in the entity resolution series closes on sustainability. A deterministic PPRL pipeline using NFKC normalization, HMAC-SHA256, and Jaccard similarity over 1,000-bit Bloom filters measures at 0.000147 gCO₂e per 1,000 records matched—1,400× less carbon than a single LLM query. Profiled with Criterion.rs and Intel RAPL. Rare earth mining, e-waste, and Bornean orangutans make this more than an engineering argument. Write code as if paying for it in joules.

Encoding fragmentation breaks entity resolution pipelines. A sixty-year-old Unicode anomaly fractures entity graphs and drops database joins. We examine the root cause of these byte-level mismatches. We detail the mechanics of internationalization (i18n) engineering, the evolution of the ICU library family, and how Rust's ICU4X datagen architecture handles multilingual text with zero-copy deserialization and compiler-enforced memory safety.

Privacy-Preserving Record Linkage (PPRL) lets two organizations determine which records refer to the same person — without either party seeing the other's data. Healthcare networks, judicial agencies, and government registries link records using cryptographic Bloom filters and HMAC-keyed hashing. But if strings are not Unicode-normalized before encoding, the hash diverges and matches fail silently. This post shows the full pipeline: q-grams, Bloom filters, CLKs, and normaliza

Entity resolution runs on joins, comparisons, and blocking keys that assume “same-looking text” means “same bytes.” That assumption is wrong once you move outside ASCII. To solve this encoding chaos, data architectures must standardize text at the lowest possible layer: the Unicode byte structure. A normalization strategy that's too weak misses matches; one that's too aggressive destroys signal and creates false positives.