Deduplicating 401,000 Equipment Auction Records with LLM Calibration
Equipment auction data is messy. The same Caterpillar D6 bulldozer sells three times across different states, and each auction house records the model slightly differently. "580 Super L" in one rec...

Source: DEV Community
Equipment auction data is messy. The same Caterpillar D6 bulldozer sells three times across different states, and each auction house records the model slightly differently. "580 Super L" in one record becomes "580SL" in another. "D6H LGP" appears alongside "D6H" — same machine, different level of detail. We took the Kaggle Bulldozer Blue Book dataset — 401,125 auction records across 53 columns — and threw it at GoldenMatch to see what happens. The result: 27,937 duplicate clusters, 384,650 records matched, and the LLM learned the perfect threshold from just 200 pairs. Total LLM cost: $0.01. The Dataset The Bulldozer dataset comes from a Kaggle competition on predicting auction sale prices. Each row is a single auction transaction: 401,125 rows, 53 columns Key fields: fiModelDesc (model description), fiBaseModel (base model code), YearMade, state, SalePrice, ProductGroup, fiProductClassDesc The same machine appears multiple times across different auctions, states, and years Model descri