Originally Posted by
sgtdetritus
Have you tried exporting the data to Excel, examining it then re-importing in smaller segments?
It sounds a little dumb but we have a DB that takes a big file dump from SAP and brings it in. One of the problems we came across is that certain special characters that come in from international customers interface created something strange in the delimiting that threw off everything after that special character. Since our table in question is mostly text fields it was hard to spot until it tried to import text into a date or number field.
What we ended up doing is dumping everything out in 1000 record chunks. Import a chunk at a time to a backup edition of the DB (not the production version). rinse, repeat, until you get an error. That at least gave us an idea where the strange crap was. It was tedious, but it worked. we were able to identify and excise the bad record and everything else went back into alignment.
After that we worked out how to add additional controls to the import process. including importing first to a temporary table to keep pollution out of the production data and letting import errors happen there before appending things to the production tables. This may not be the most elegant method, but it may work for you.