BookmarkSubscribeRSS Feed
🔒 This topic is solved and locked. Need further help from the community? Please sign in and ask a new question.
Sangho
Obsidian | Level 7

I have a question regarding adding rows in between another rows. (maybe using do loops?)

I have a data for each quater for each firm. quarters without observations are removed. How can I create rows with zero for those quarters without observations?

For simplicity, let's say there are three firms, for year 2000 and 2001. And I have a data like this:

data table1;
input firm_id year quarter value;
cards;
0001 2000 1 10
0001 2000 3 20
0001 2001 2 45
0001 2001 3 14
0001 2001 4 45
0012 2000 1 34
0012 2000 2 10
0047 2000 1 34
0047 2001 3 34
;
run;

And eventually I want a data looks like

Capture3.PNG

Thanks!

1 ACCEPTED SOLUTION

Accepted Solutions
mkeintz
PROC Star

MIssing years as well as missing quarters?  If you know you want a fixed range of years for every ID, then it's relatively straightforward:

  1. Instead of processing data one year at a time, process data one id at a time.
  2. Add an outer loop over years containing the inner loop over quarters.
  3. And instead of a one-dimensional array saving flags for a single year, have a two-dimensional array saving flags for quarters within multiple years:

 

data want (drop=qtr_found:);

  array qtr_found {2000:2001,4} ;

  /* Read one ID to establish qtr_found flags */
  do until (last.id);
    set table1 (keep=id year quarter);
    by id ;
    qtr_found{year,quarter}=1;
  end;

  /* Loop over the flags to control re-reading and output */
  do year=lbound1(qtr_found) to hbound1(qtr_found);
    do quarter=1 to 4;
      if qtr_found{year,quarter} then set table1;
      else value=0;
      output;
    end;
  end;
run;

 

Notes:

  1. Arrays can have multiple dimensions, and those dimensions don't have to have lower bound of 1.  Note the first dimension of qtr_found is specified as 2000:2001 which means lower bound of 2000 and upper bound of 2001.  But the second dimension is 4, which defaults to lower bound of 1 ... and upper bound of 4.

  2. The LBOUND1 and HBOUND1 functions provide the lower and upper bounds of dimension 1, so it's easy to directly index the array on year as well as quarter.

 

--------------------------
The hash OUTPUT method will overwrite a SAS data set, but not append. That can be costly. Consider voting for Add a HASH object method which would append a hash object to an existing SAS data set

Would enabling PROC SORT to simultaneously output multiple datasets be useful? Then vote for
Allow PROC SORT to output multiple datasets

--------------------------

View solution in original post

5 REPLIES 5
mkeintz
PROC Star

This can be done by reading each year of data twice.  The first read establishes 4 flags, 1 for each quarter, indicating whether that quarter is present in the given year.   The second read goes through the 4 flags.  If the flag is 1, then read a single record, if not then set value to 0.  Then output the record.

 

data want (drop=qtr_found:);

  array qtr_found {4} ;
  do until (last.year);
    set table1 (keep=id year quarter);
    by id year;
    qtr_found{quarter}=1;
  end;

  do quarter=1 to 4;
    if qtr_found{quarter} then set table1;
    else value=0;
    output;
  end;
run;


--------------------------
The hash OUTPUT method will overwrite a SAS data set, but not append. That can be costly. Consider voting for Add a HASH object method which would append a hash object to an existing SAS data set

Would enabling PROC SORT to simultaneously output multiple datasets be useful? Then vote for
Allow PROC SORT to output multiple datasets

--------------------------
Sangho
Obsidian | Level 7

Below picture is the result based on your code.

But for firm_id=12, it has results only for year 2000. I guess it is because in the original data, there was no observation at all for firm 12 in year 2001. But I need 4 rows all with zero for firm 12 for year 2001. How can I fix this problem?

 Capture4.PNG

Thanks again

art297
Opal | Level 21

I like @mkeintz's approach, but it didn't account for the missing year. I think the following will:

 

data table1;
  set table1;
  by firm_id year;
  if last.firm_id and year eq 2000 then do;
    output;
    year=2001;
    quarter=1;
    value=0;
    output;
  end;
  else output;
run;

data want (drop=qtr_found:);
  array qtr_found {4} ;
  do until (last.year);
    set table1 (keep=firm_id year quarter);
    by firm_id year;
    qtr_found{quarter}=1;
  end;

  do quarter=1 to 4;
    if qtr_found{quarter} then set table1;
    else value=0;
    output;
  end;
run;

Art, CEO, AnalystFinder.com

mkeintz
PROC Star

MIssing years as well as missing quarters?  If you know you want a fixed range of years for every ID, then it's relatively straightforward:

  1. Instead of processing data one year at a time, process data one id at a time.
  2. Add an outer loop over years containing the inner loop over quarters.
  3. And instead of a one-dimensional array saving flags for a single year, have a two-dimensional array saving flags for quarters within multiple years:

 

data want (drop=qtr_found:);

  array qtr_found {2000:2001,4} ;

  /* Read one ID to establish qtr_found flags */
  do until (last.id);
    set table1 (keep=id year quarter);
    by id ;
    qtr_found{year,quarter}=1;
  end;

  /* Loop over the flags to control re-reading and output */
  do year=lbound1(qtr_found) to hbound1(qtr_found);
    do quarter=1 to 4;
      if qtr_found{year,quarter} then set table1;
      else value=0;
      output;
    end;
  end;
run;

 

Notes:

  1. Arrays can have multiple dimensions, and those dimensions don't have to have lower bound of 1.  Note the first dimension of qtr_found is specified as 2000:2001 which means lower bound of 2000 and upper bound of 2001.  But the second dimension is 4, which defaults to lower bound of 1 ... and upper bound of 4.

  2. The LBOUND1 and HBOUND1 functions provide the lower and upper bounds of dimension 1, so it's easy to directly index the array on year as well as quarter.

 

--------------------------
The hash OUTPUT method will overwrite a SAS data set, but not append. That can be costly. Consider voting for Add a HASH object method which would append a hash object to an existing SAS data set

Would enabling PROC SORT to simultaneously output multiple datasets be useful? Then vote for
Allow PROC SORT to output multiple datasets

--------------------------
Ksharp
Super User
data table1;
input firm_id year quarter value;
cards;
0001 2000 1 10
0001 2000 3 20
0001 2001 2 45
0001 2001 3 14
0001 2001 4 45
0012 2000 1 34
0012 2000 2 10
0047 2000 1 34
0047 2001 3 34
;
run;

proc sql;
create table want as
select a.*,coalesce(b.value,0) as value
 from 
 (select * from (select distinct firm_id from table1),
   (select distinct year from table1),
   (select distinct quarter from table1) ) as a
 left join table1 as b 
 on a.firm_id=b.firm_id and a.year=b.year and a.quarter=b.quarter;
quit;

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

How to Concatenate Values

Learn how use the CAT functions in SAS to join values from multiple variables into a single value.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 5 replies
  • 2480 views
  • 5 likes
  • 4 in conversation