topic SAS GRAPH FROM A BIG DATA in SAS Programming

SAS GRAPH FROM A BIG DATA

LineMoon — Wed, 09 Mar 2016 21:22:49 GMT

Hello,

If I have a big data( more than 3000 000 observations ) for every year, and I have more than 20 years.

I want to make a graph from a many big tables without making :

data Tab;

set T1 T2 T3 T4 ... Tn;

run;

Re: SAS GRAPH FROM A BIG DATA

LinusH — Wed, 09 Mar 2016 21:26:09 GMT

Because you don't want to duplicate the data?
Storing time series of data in one table is often convenient since reduces the steps to out it together each time.
But, do what you are suggesting but as a view instead.

Re: SAS GRAPH FROM A BIG DATA

LineMoon — Wed, 09 Mar 2016 21:46:58 GMT

Thank you.

Yes, I do not want to duplicate data likes this :

I do not want to do this :

data tab;

set T1 T2 ....Tn;

run;

"Storing time series of data in one table" it can be more expenssive, I will have a big data.

I want to make my graph without making any concatanation..

Re: SAS GRAPH FROM A BIG DATA

ballardw — Wed, 09 Mar 2016 22:19:34 GMT

Without specifying what graph and options you need it is difficult to provide exact help. However, If your final graph is to have Year as a component it may be that what you should do is summarize each year of data into a separate summary data set, make sure there is a year variable in the summary, then combine those summarized sets before graphing.

There really isn't going to be much hope of getting a graph from multiple sets as none of the SAS graphing approaches I am aware of support more than a single input data set.

I usually summarize data before graphing anyway so that I know exactly what is being calculated as any of the graph procudure's summaries usually fall afoul of a data gotcha somewhere with my data.

Re: SAS GRAPH FROM A BIG DATA

Reeza — Wed, 09 Mar 2016 22:24:33 GMT

If you want to use all your data, you need to process all your data.

This can be as indicated, either stack your data together and process. Depending on your set up, 60 million rows may not be much. I regularly worked with a dataset that was 30 million per year - after removing multiple entries.

A second method is to process each data set individually and stack the results - the loopy way - usually accomplished via a macro.

A third method is to create a view such that the data is only combined for the analysis and then not stored permanently.

http://support.sas.com/documentation/cdl/en/lrcon/62955/HTML/default/viewer.htm#a001278887.htm

Re: SAS GRAPH FROM A BIG DATA

LinusH — Thu, 10 Mar 2016 07:04:59 GMT

Concatenation must take place if you want a single graph output.
If you don't want to store the data concatenated the only option I see is a view. Should be sufficient.