topic Re: Append into single dataset in New SAS User

Append into single dataset

animesh123 — Wed, 14 Dec 2022 16:18:38 GMT

If we have 100 Dataset with datset name as day1 , day2 till day100.
How to append this into single dataset?

Re: Append into single dataset

animesh123 — Wed, 14 Dec 2022 16:21:27 GMT

Does the below code will work?
Data day;
Set day1- day100;
Run;

Re: Append into single dataset

ballardw — Wed, 14 Dec 2022 17:24:52 GMT

@animesh123 wrote:
Does the below code will work?
Data day;
Set day1- day100;
Run;

It should if three conditions are met:

1) all the sets are in the work library. If they were in a different library you should be able to use the library name such as: lib.day1 - lib.day100

2) that sort of list will require continuous numbers without any gaps. If you don't actually have a Day25 for example then it will result in an error but could be gotten around by using actually continuous lists: set day1-day24 day26-day100; for example.

3) and perhaps the most critical: all variables that are in common in the datasets must be of the same type. If any variable is of a different type in some sets then it will generate an error and additional work will be needed. Caution: if variables have different lengths in the data sets you will get a warning about possible truncation of data. Pay attention because that may result in lost data or unexpected values.

Typically problems of different types or lengths occur when using Proc Import or other widget based tool to bring data into SAS. Depending on the source the tools make guesses for type and length based on content for each file and may result in inconsistent characteristics.

Note that variables that do not occur in other data sets will result in missing values for records from other sets.

Re: Append into single dataset

animesh123 — Wed, 14 Dec 2022 17:58:28 GMT

Ok ,So what is the efficient way to append in this situtation

Re: Append into single dataset

Quentin — Wed, 14 Dec 2022 18:37:04 GMT

Can you describe the situation?

The code should work, if it meets @ballardw 's conditions / warnings.

Did you try the code? If so, did you get an errors? Unexpected results?

Re: Append into single dataset

animesh123 — Thu, 15 Dec 2022 13:40:09 GMT

Hey, I tried the below code
data want;
set 😧 ;
run;
As i was getting an unexpected results.

Re: Append into single dataset

animesh123 — Thu, 15 Dec 2022 13:43:21 GMT

Hi @ballardw.
Thanks for the information
Lets say if variables have different lengths in the all of the given data sets.How will i append that data.
Just to understand how the process works?

Re: Append into single dataset

PaigeMiller — Thu, 15 Dec 2022 13:44:21 GMT

A request was made for you to explain. Just saying code doesn't work explains nothing. Saying you get unexpected results explains nothing. So repeating the request: Explain what the problem is that you are trying to solve. Explain the desired outcome. Explain the undesired results.

Re: Append into single dataset

ballardw — Thu, 15 Dec 2022 15:56:13 GMT

@animesh123 wrote:
Hi @ballardw.
Thanks for the information
Lets say if variables have different lengths in the all of the given data sets.How will i append that data.
Just to understand how the process works?

That would require a LENGTH statement before the SET statement to set the length of those variables to the longest value to avoid any truncation.

Dummy code:

data want; 
    length thisvar $ 25  thatvar $ 10;
   set set1-set100;
run;

If you don't know, or just want to make sure what the longest value for length is this code can point you to one way to get a report on such. Again creates some small sets to demonstrate;

data set1;
   x='abc';
run;
data set2;
   x='a;sdlfkjasfj';
run;
data set3;
   x='much longer text than other sets';
run;

proc tabulate data=sashelp.vcolumn;
   where libname='WORK' and memname=:'SET';
   class  name;
   var length;
   table name,
         length*max
         ;
run;

The data view SASHELP.COLUMN contains descriptions of all the variables in all the data sets in the current SAS session.

To select the data set information you want to specify the LIBNAME and the MEMNAME(s) of interest. In this case the =: is asking for the data set names whose names start with the letters SET. The Libname and Memname information are stored in the view in uppercase. Name, the name of variables, is not. So you may show similar text with different spellings like Var1 and var1 for the identical variable. Pick the largest length value as the data step doesn't which spelling is used but treats all as the same variable.

The SASHELP.VCOLUMN view may take a bit of time to run depending on how many sets and libraries you have so if you don't get a result for the Proc Tabulate code in just a few seconds that may be the reason.

Re: Append into single dataset

Tom — Thu, 15 Dec 2022 16:23:00 GMT

@animesh123 wrote:
Hi @ballardw.
Thanks for the information
Lets say if variables have different lengths in the all of the given data sets.How will i append that data.
Just to understand how the process works?

It probably makes more sense to explain the process that creates the datasets and fix that so that they are always created with the same structure.

If you really had to do something with a bunch of datasets with inconsistently defined variables you probably will want to analyze the variable metadata first.

proc sql;
create table variables as 
  select min(varnum) as varnum
       , upcase(name) as name
       , count(*) as nds
       , max(type) as type
       , max(length) as length
  from dictionary.columns
  where libname='SASHELP' and memname like 'CLASS%'
   group by calculated name
   order by 1,2
;
quit;

Once you have that the list of variables with their intended type and length you can use that to generate a LENGTH statement to define the variables.

proc sql noprint;
select varnum
     , catx(' ',nliteral(name),cats(case when (type='char') then '$' else ' ' end,length))
  into :varlist, :varlist separated by ' '
  from variables
  order by varnum
;
quit;

data want;
  length &varlist ;
  set sashelp.class: ;
run;

Re: Append into single dataset

animesh123 — Fri, 16 Dec 2022 13:22:19 GMT

I have tried using this
data want;
set D : ;
run;