topic How can we retrieve huge amount of data from flat files or from database quickly?few techniques? in SAS Programming

How can we retrieve huge amount of data from flat files or from database quickly?few techniques?

MSK4 — Fri, 28 Aug 2020 04:03:23 GMT

(huge data 100 millions of records)Detail explanation required please,highly appreciated?

Re: How can we retrieve huge amount of data from flat files or from database quickly?few techniques?

SASKiwi — Fri, 28 Aug 2020 04:24:55 GMT

@MSK4 - Yes SAS can handle reading a lot of data from large flat files and databases, but if you ask a broad question without providing any details of your requirements then you are not going to get detailed or useful answers.

Re: How can we retrieve huge amount of data from flat files or from database quickly?few techniques?

Kurt_Bremser — Fri, 28 Aug 2020 05:08:29 GMT

You read flat files with a data step; to write it, you follow the documentation of the flat file. The target library needs to have enough free space to accommodate the resulting dataset file; depending on the variable types and layouts, using the COMPRESS= option may be advised.

Re: How can we retrieve huge amount of data from flat files or from database quickly?few techniques?

Astounding — Sat, 29 Aug 2020 12:49:47 GMT

Here are some basic details you need to supply if you want any kind of an answer.

Which is it, a flat file or a database that you want to read from. Wouldn't you expect the details for your request to be quite different depending on where the data is coming from?

Do you want all the fields? If not, how am I supposed to know which fields you want?

Do you want all the rows? If not, which rows do you want?

Is the result supposed to be a SAS data set?

If you are reading from a database, supply a program capable of reading from that database.

That would be a starting point to enable any sort of discussion.