topic Re: Big DATA Imports in SAS Programming

Big DATA Imports

fatemeh — Wed, 12 Jul 2017 20:46:08 GMT

Big DATA Imports[ New ]

data alaki;
   do trt = -1 to 1 by 2;
     do gene = -1 to 1 by 1;
       do time=1 to 3;
          input responder@;
          output;
       end;
   end;
   end;
datalines;
4.22027067   4.37189508   4.42832633
12.342489   12.3535905   12.2709339
6.10929307   6.34392243   6.43155656
4.43362232   4.54115629   4.2886178
12.6404601   12.715388   12.7065314
6.07044046   6.06052994   6.11873577
;
run;
proc print;run;

hello.my question is how i can read my data as a seperate sas dataset in the above program,and then use set statement to read from this new dataset instead of using datalines ,because my data is big data and i can not place it in datalies .i used proc import but it gives me header ,and considers it as columns which do loop doesn't realize it .if anybody knows the answer we appreciate to share it.

thanks

Big DATA Imports

fatemeh — Wed, 12 Jul 2017 20:34:48 GMT

hello.my question is how i can read my data i aseperate sas dataset .and then use set statement to read from this new dataset instead of using datalines ,because my data is big data and i can not place it in datalies .i used proc import but it gives me header .

thanks

Re: Big DATA Imports

ballardw — Wed, 12 Jul 2017 21:21:47 GMT

You might post a few lines of data as text and show what the result should be. If your data is sensitive then something with the same properties. Use a codebox opened with the forum {i} menu icon so the text does not get reformatted by posting in the main message window.

Does your actual data not have values for the trt, gene and time and you have to assign them based on the row in the data???

Re: Big DATA Imports

Reeza — Wed, 12 Jul 2017 22:33:53 GMT

What's your definition of big?

What's the source?

You can read from a file using an INFILE staement to point at a text file, or a straight PROC IMPORT will often work as well. Without further detailed information we can't help you.

Re: Big DATA Imports

ChrisNZ — Wed, 12 Jul 2017 22:34:21 GMT

Read the documentation.

Proc import has options DATAROW and GETNAMES to deal with headers.

Re: Big DATA Imports

fatemeh — Wed, 12 Jul 2017 22:49:10 GMT

hi. my data has hundred thousands rows ,i use small data to understand . when i use proc import the sas output gives me the data set with headers that looks like thistable.

i am trying to use this below program but it does not give me the numbers in the table as responder.

data alaki;
infile'C:\Users\FATEMEH\Desktop\research project\alak.xlsx' ;
   do trt = -1 to 1 by 2;
     do gene = -1 to 1 by 1;
       do time=1 to 3;
          input responder;
          output;
       end;
   end;
   end;

run;
proc print data=alaki;run;

Re: Big DATA Imports

Reeza — Wed, 12 Jul 2017 23:00:27 GMT

You need to first import your data (PROC IMPORT)

THen you need to add the multiple options, since your code is replicating your data or assuming a structure. Either way, move that to a different data step. If your code works on your small sample it will work on hte big data (thanks SAS) so working with small samples to start is a very smart idea.

Here's some video tutorials on SAS and the UCLA link above has many many tutorials on everything from reading sas to how to interpret your regression analysis output.

http://video.sas.com/#category/videos/how-to-tutorials

Re: Big DATA Imports

andreas_lds — Thu, 13 Jul 2017 07:50:35 GMT

Interesting strategy: repeating a question while ignoring answers already given.

A data-step can import almost any text-file. Multiple datasets are created by using the output statement.

Re: Big DATA Imports

Oligolas — Thu, 13 Jul 2017 09:07:37 GMT

Probably something like that should work:

PROC IMPORT OUT= WORK.tmp 
            DATAFILE= "C:\Users\FATEMEH\Desktop\research project\alak.xlsx" 
            DBMS=EXCEL REPLACE;
     SHEET="Please complete"; 
     GETNAMES=YES;
     MIXED=YES;
     SCANTEXT=NO;
     USEDATE=NO;
     SCANTIME=NO;
RUN;

Reading the documentation about how to import and handle data might help too.

Cheers