Just compared rolling my own AI model vs using ChatGPT for data cleanup and it was no contest after 18 hours of fiddling

I spent a full weekend setting up a local LLM to scrub a client's messy CSV files, then ran the same task through ChatGPT in 5 minutes and got way cleaner results, so am I just wasting time trying to do it myself or is there a real advantage to going local for privacy?

3 comments

3 Comments

owens.laura16d agoTop Commenter

Have you tried a smaller model like Llama 3.2 for something quick and local? I had almost the same problem last month cleaning up addresses, and the local model kept messing up zip codes. I gave up and used Claude for that batch, it took maybe 10 minutes. If you have any personal info in the CSV files, going local might still be worth it for privacy alone, but for normal business data the online models are just way faster and more reliable in my experience.

paulnguyen16d ago

Did you ever figure out if it was a formatting thing with the zip codes? Honestly a buddy of mine was trying to clean up a mailing list with Llama 3.2 and it kept swapping digits around on 5-digit zips, like turning 90210 into 92010. He spent like two hours debugging it before switching to the Claude API and finishing everything in 15 minutes flat.

ben_fisher16d ago

Yeah I had the same issue with Llama messing up zip codes, switched to GPT-4 for address cleanup and it was way more accurate.