BookmarkSubscribeRSS Feed
Vergoulas
Calcite | Level 5

I would like to investigate if methods of advanced statistical process control (such as control charts) could be applied to pinpoint potentially erroneous data in a large dataset. 

Are there any available data sets that I could use? 

I am an MSc student and I could use any available data sets for my dissertation. 

 

Thank you,

Vaggelis Vergoulas

3 REPLIES 3
PaigeMiller
Diamond | Level 26

@Vergoulas wrote:

I would like to investigate if methods of advanced statistical process control (such as control charts) could be applied to pinpoint potentially erroneous data in a large dataset. 



I don't think YOU specifically need to do an investigation of this type. I think the answer is already known.

 

The answer is that these types of control charts were not designed to pinpoint erroneous data, but they certainly have been used to detect problems in the data. Of course, once a "problem" in data happens, the issue then becomes: is it real data, or an erroneous data? I don't think any statistical method can answer this question, the only way a decision that the decision can be made that the data is erroneous is to apply subject matter knowledge.

--
Paige Miller
Vergoulas
Calcite | Level 5

Dear Paige, thank you very much for your quick and very informed answer.

 

My aim is to investigate if these charts could be used to detect these “problems” or better “potential problems” as they happen and request attention to the process. Theoretically, they are made for that. I wanted to focus on the points of data capture (or creation if you will). I do not know if such methods are in use for data. Your input on this subject could be very helpful.

 

Thank you very much.

PaigeMiller
Diamond | Level 26

I guess I don't know what you mean by "focus on the points of data capture", and how that differs from just analyzing the data.

 

Certainly control charts can be used on any data, and properly used under the right conditions control charts will detect problems and can request immediate attention to the process.

 

It sounds as if you are starting from scratch in this research, even though control charts are among the most widely used statistical tools ever invented, and furthermore there is a huge amount of published literature on their use.

--
Paige Miller

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

How to connect to databases in SAS Viya

Need to connect to databases in SAS Viya? SAS’ David Ghan shows you two methods – via SAS/ACCESS LIBNAME and SAS Data Connector SASLIBS – in this video.

Find more tutorials on the SAS Users YouTube channel.

Discussion stats
  • 3 replies
  • 694 views
  • 1 like
  • 2 in conversation