DATA Step, Macro, Functions and more

SAS GRAPH FROM A BIG DATA

Reply
Super Contributor
Posts: 371

SAS GRAPH FROM A BIG DATA

Hello,

 

If I have a big data( more than 3000 000 observations ) for every year, and I have more than 20 years.

I want to make a graph from a many big tables without making :

 

data Tab;

  set T1 T2 T3 T4  ... Tn;

run;

 

 

 

Super User
Posts: 5,424

Re: SAS GRAPH FROM A BIG DATA

Because you don't want to duplicate the data?
Storing time series of data in one table is often convenient since reduces the steps to out it together each time.
But, do what you are suggesting but as a view instead.
Data never sleeps
Super Contributor
Posts: 371

Re: SAS GRAPH FROM A BIG DATA

Thank you.

Yes, I do not want to duplicate data likes this :

 

I do not want to do this :

 

data tab;

  set T1 T2 ....Tn;

run;

 

"Storing time series of data in one table" it can be more expenssive, I will have a big data.

I want to make my graph without making any concatanation..

 

 

Super User
Posts: 11,343

Re: SAS GRAPH FROM A BIG DATA

Without specifying what graph and options you need it is difficult to provide exact help. However, If your final graph is to have Year as a component it may be that what you should do is summarize each year of data into a separate summary data set, make sure there is a year variable in the summary, then combine those summarized sets before graphing.

 

There really isn't going to be much hope of getting a graph from multiple sets as none of the SAS graphing approaches I am aware of support more than a single input data set.

 

I usually summarize data before graphing anyway so that I know exactly what is being calculated as any of the graph procudure's summaries usually fall afoul of a data gotcha somewhere with my data.

Super User
Posts: 19,768

Re: SAS GRAPH FROM A BIG DATA

If you want to use all your data, you need to process all your data.

This can be as indicated, either stack your data together and process. Depending on your set up, 60 million rows may not be much. I regularly worked with a dataset that was 30 million per year - after removing multiple entries.

 

A second method is to process each data set individually and stack the results - the loopy way - usually accomplished via a macro.

 

A third method is to create a view such that the data is only combined for the analysis and then not stored permanently. 

http://support.sas.com/documentation/cdl/en/lrcon/62955/HTML/default/viewer.htm#a001278887.htm

 

 

 

Super User
Posts: 5,424

Re: SAS GRAPH FROM A BIG DATA

Concatenation must take place if you want a single graph output.
If you don't want to store the data concatenated the only option I see is a view. Should be sufficient.
Data never sleeps
Ask a Question
Discussion stats
  • 5 replies
  • 263 views
  • 3 likes
  • 4 in conversation