@RW9 wrote:
Why do you not just reject the file? Surely according to your data import agreement and contract with the data provider you will know what you are expecting from the file. Me, if someone sent me a file called .csv (Comma Separated Variable file) which wasn't in that format, or in fact a proper usable format, I would reject with the reponse of poor data received.
This isn't always possible.
I've seen this due to either restrictions on what the survey tool exports or an application system. This is one way they get you with 'free' survey options, if you want a decent data file you need to pay to get a SPSS or SAS version. Another is, I've worked for social services agencies and doctors. There focus is on patients/clients and data is secondary, that's why they hired me in the first place. Am I really going to ask a social worker to skip an appointment with a client or a doctor to skip time with a patient to fix data? No. I'm going to try and see if I can fix it first before 'rejecting' the file. And if I had a staff member who did reject a file, they'd likely get reprimanded at minimum. 99% of the time data cleaning is part of the job, and if someone doesn't want to do that part they're not doing their job.
... View more