topic Re: Exchanging data between SAS and ROOT. in New SAS User

Exchanging data between SAS and ROOT.

marty1885 — Tue, 03 Sep 2019 05:34:33 GMT

Hi all.

I'm new to SAS but have been using other analysis tools for a while.

I have been analysing data using ROOT and all of my data is stored in ROOT's TTree format. Recently I'm tasked to export some of my data for statisticians whom use SAS primarily. Are there better way to exchange data between the two frameworks? CSV and JSON are not a viable option as they are huge and clunky. Also, what would be a good way to get data from SAS back into ROOT?

Re: Exchanging data between SAS and ROOT.

Kurt_Bremser — Tue, 03 Sep 2019 09:15:25 GMT

I can't think of anything more compact than a gzip'ed csv file for data transfer. Supply a documentation of the structure along with the file, so that the recipients can exactly recreate the data.

SAS can read a gzip'ed csv file in one action like this:

filename ziptest zip 'class.csv.zip'; /* insert the path where you stored the file */

data class;
infile ziptest dlm=',' dsd;
input
  name :$8.
  sex :$1.
  age
  height
  weight
;
run;

The file was created with

data _null_;
file ziptest dlm=',' dsd;
set sashelp.class;
put name sex age height weight;
run;

and the end result of the first piece of code will be an exact replica of the SAS demo dataset SASHELP.CLASS.

PS I used zip as a file format as the forum software does not allow uploading a .gz file.

The zip compression results in 348 bytes, the gzip gives you 248. Change the filename statement to

filename ziptest zip '$HOME/sascommunity/class.csv.gz' gzip;

to use gzip compression.

Re: Exchanging data between SAS and ROOT.

Ksharp — Tue, 03 Sep 2019 12:57:45 GMT

Access data ?

Re: Exchanging data between SAS and ROOT.

marty1885 — Tue, 03 Sep 2019 14:18:03 GMT

Maybe some context can help. Now, I have a ~100GB compressed ROOT archive which contains a lot of observation (event in root terms) and 50 variables (branch in root terms) per observation from an physics experiment. And from experience and rough math, I'm expecting a 1TB JSON or a 400GB CSV file if I de-compress my root archive. And my college, the statistician's job is to go through these few hundred million observations and find the outliers, then append statistics to each observation so I can find malfunctions in the equipments.

So, here's the problem. As he primarily uses SAS, he needs a way to import all my data in ROOT into SAS. But JSON is obviously nonviable and CSV doesn't scale (I need to generate the CSV then compress it). The opposite problem of sending data from SAS to ROOT is also a big one.

Re: Exchanging data between SAS and ROOT.

Tom — Tue, 03 Sep 2019 15:40:29 GMT

A quick google search found at least one example of a program to convert Root TTree file into CSV file.

https://root-forum.cern.ch/t/root-tree-to-csv-file-format/27633/3

If you can get that to run then in SAS you can just use the PIPE engine to run it on the fly.

data want;
  infile 'root2csv -f myrootfile' pipe dsd truncover ;
  ...
run;

Re: Exchanging data between SAS and ROOT.

Ksharp — Wed, 04 Sep 2019 11:22:38 GMT

100G is too big for ACCESS file. Using CSV as Tom suggested.

Re: Exchanging data between SAS and ROOT.

marty1885 — Fri, 06 Sep 2019 03:49:50 GMT

Thank you! This looks like the solution I need.

Is communication in the opposite direction possible? Can SAS output it's data into stdout?

Re: Exchanging data between SAS and ROOT.

marty1885 — Fri, 06 Sep 2019 03:52:10 GMT

I think the document is referring to Accessing Data not Microsoft Access

Data saved into one or several ROOT files can be accessed from your PC, from the web and from large-scale file delivery systems used, for example in the GRID.