Converting an array to a data set

[ArtemisFowl]

Hello,

I have created a 9801 by 201 array in a data step, but apparently this array is too large to be saved in SAS, i.e. there are too many variables for SAS to save all of them in a data set. The error message of SAS is the following:

ERROR: Too many variables defined for file WORK.BLABLA.  This file may not have more than 32767 variables.

So I would like instead to convert this array into 201 variables each with 9801 records, but without having to leave the data step in which I define the array - the reason being that I cannot save the contents of my array. If the array is saved instead in 201 variables, it is possible for me to include the data in other data sets or sort it etc.

Here's where I define my array (what my code does is not really important):

data _null_;


array a[9801,201];


do i = 2 to 100;
    do j = 2 to 100;
        n = 99*(i-2)+(j-1);
        link fillArray;
    end;
end;


return;


fillArray:
a[n,201] = i;
do k = 2 to j;
    do x = 201 to 1 by -1;
        if a[n,x] = . then leave;
        a[n,x] = a[n,x]*i;
    end;
    do y = 201 to 1 by -1;
        if a[n,y] = . then leave;
        if a[n,y] ge 10 then do;
            a[n,y-1] = sum(a[n,y-1],floor(a[n,y]/10));
            a[n,y] = a[n,y] - floor(a[n,y]/10)*10;
        end;
    end;
end;
return;


run;

[SAPPER]

I am pretty sure , you would have thought of this....the case looks to me as a good fit for a table (201 columns and 9000 rows). If it is possible in the context of your code to use a table instead of an array, try creating a dataset using proc SQL or other data step elements, it is much more efficient and easy to maintain than an array.

Thanks

SAPPER

[PGStats]

data save(keep=col:);

array a[9801,201];
array col{201};

do i = 2 to 100;
    do j = 2 to 100;
        n = 99*(i-2)+(j-1);
        link fillArray;
    end;
end;

do j = 1 to dim(a,1);
do i = 1 to dim(a,2);
  col{i} = a{j,i};
  end;
output;
end;
return;

fillArray:
a[n,201] = i;
do k = 2 to j;
    do x = 201 to 1 by -1;
        if a[n,x] = . then leave;
        a[n,x] = a[n,x]*i;
    end;
    do y = 201 to 1 by -1;
        if a[n,y] = . then leave;
        if a[n,y] ge 10 then do;
            a[n,y-1] = sum(a[n,y-1],floor(a[n,y]/10));
            a[n,y] = a[n,y] - floor(a[n,y]/10)*10;
        end;
    end;
end;
return;

run;

PG

[chang_y_chung_hotmail_com]

here is one way. hth.

  data one;
    array a[9801,201] _temporary_;
    link doFill;
 
    array var[1:201] var001-var201;
    link doOutput;
 
    return;
 
    doFill:
      ;/* do something here */
    return;
 
    doOutput:
      do i = 1 to dim(a,1);
        do j = 1 to dim(a,2);
          var[j] = a[i,j];
        end;
        output;
        keep var001-var201;
      end;
    return;
  run;
  /* on log
     NOTE: The data set WORK.ONE has 9801 observations and 201 variables.
  */

: Sorry, I did not know that you have already given basically the same answer... Although it is quite assuring that we came up with the same thing!

[Tom]

Why have you conceptualized the problem as an array instead of a dataset?

What do the 9801 rows of the array represent?  Are they samples?

What do the 201 columns of the array represent?  Replicates?

Is this some sort of time series? 

What is "A"?  Is it some variable that you are simulating?  For example: Height, TreeTrunkDiameter, StockPrice?

Why does filling the array involve looping and FLOOR() function calls?

[ArtemisFowl]

Thank you all. It works the way I want it to now.

@SAPPER: I don't know proc sql, doesn't it require sql knowledge?

@PGStats and @chang_y_chung: Thanks, I see the idea behind what you did.

@Tom:

Tom: Why have you conceptualized the problem as an array instead of a dataset?

Me: It was easier for me programming-wise to read the numbers into an array. To save time, I would have prefered to read them directly into a data set, and maybe this is possible by changing the code slightly. Actually, I think I'll do that. I'm learning how to use SAS, but not in a very linear way.

Tom: What do the 9801 rows of the array represent?  Are they samples?

What do the 201 columns of the array represent?  Replicates?

Is this some sort of time series? 

What is "A"?  Is it some variable that you are simulating?  For example: Height, TreeTrunkDiameter, StockPrice?

Why does filling the array involve looping and FLOOR() function calls?

Me: I'm doing this "puzzle", where I determine the amount of distinct numbers of the form a^b, where 2 <= a, b <= 100 (for instance 2^4 = 4^2). It turned out that SAS could not handle numbers that large, so I read the digits of the numbers into an array. The length of the largest number (100^100) is 201, so I made room for 201 digits. There's (100-1)*(100-1) = 9801 different combinations of a and b. The rows of the array are as follows:

2^2

2^3

2^4

...

2^100

3^2

3^3

...etc

The fillArray part is a multiplication algorithm, where I make sure each digit of a^b gets its own cell in the array.

[PGStats]

Hi,

If you are interested/allowed, it might be a good idea to submit your puzzle in its mathematical form as a new discussion on this forum. The diversity of proposed approaches is always fascinating.

PG