turn on suggestions

Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

Find a Community

- Home
- /
- SAS Programming
- /
- Base SAS Programming
- /
- Converting an array to a data set

Topic Options

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

05-31-2012 02:53 PM

Hello,

I have created a 9801 by 201 array in a data step, but apparently this array is too large to be saved in SAS, i.e. there are too many variables for SAS to save all of them in a data set. The error message of SAS is the following:

ERROR: Too many variables defined for file WORK.BLABLA. This file may not have more than 32767 variables.

So I would like instead to convert this array into 201 variables each with 9801 records, but without having to leave the data step in which I define the array - the reason being that I cannot save the contents of my array. If the array is saved instead in 201 variables, it is possible for me to include the data in other data sets or sort it etc.

Here's where I define my array (what my code does is not really important):

data _null_;

array a[9801,201];

do i = 2 to 100;

do j = 2 to 100;

n = 99*(i-2)+(j-1);

link fillArray;

end;

end;

return;

fillArray:

a[n,201] = i;

do k = 2 to j;

do x = 201 to 1 by -1;

if a[n,x] = . then leave;

a[n,x] = a[n,x]*i;

end;

do y = 201 to 1 by -1;

if a[n,y] = . then leave;

if a[n,y] ge 10 then do;

a[n,y-1] = sum(a[n,y-1],floor(a[n,y]/10));

a[n,y] = a[n,y] - floor(a[n,y]/10)*10;

end;

end;

end;

return;

run;

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to ArtemisFowl

05-31-2012 03:28 PM

I am pretty sure , you would have thought of this....the case looks to me as a good fit for a table (201 columns and 9000 rows). If it is possible in the context of your code to use a table instead of an array, try creating a dataset using proc SQL or other data step elements, it is much more efficient and easy to maintain than an array.

Thanks

SAPPER

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to ArtemisFowl

05-31-2012 03:32 PM

data save(keep=col;

array a[9801,201];

array col{201};

do i = 2 to 100;

do j = 2 to 100;

n = 99*(i-2)+(j-1);

link fillArray;

end;

end;

do j = 1 to dim(a,1);

do i = 1 to dim(a,2);

col{i} = a{j,i};

end;

output;

end;

return;

fillArray:

a[n,201] = i;

do k = 2 to j;

do x = 201 to 1 by -1;

if a[n,x] = . then leave;

a[n,x] = a[n,x]*i;

end;

do y = 201 to 1 by -1;

if a[n,y] = . then leave;

if a[n,y] ge 10 then do;

a[n,y-1] = sum(a[n,y-1],floor(a[n,y]/10));

a[n,y] = a[n,y] - floor(a[n,y]/10)*10;

end;

end;

end;

return;

run;

PG

PG

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to ArtemisFowl

05-31-2012 03:47 PM

here is one way. hth.

**data** one;

array a[**9801**,**201**] _temporary_;

link doFill;

array var[**1**:**201**] var001-var201;

link doOutput;

return;

doFill:

;/* do something here */

return;

doOutput:

do i = **1** to dim(a,**1**);

do j = **1** to dim(a,**2**);

var[j] = a[i,j];

end;

output;

keep var001-var201;

end;

return;

**run**;

/* on log

NOTE: The data set WORK.ONE has 9801 observations and 201 variables.

*/

: Sorry, I did not know that you have already given basically the same answer... Although it is quite assuring that we came up with the same thing!

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to ArtemisFowl

05-31-2012 07:38 PM

Why have you conceptualized the problem as an array instead of a dataset?

What do the 9801 rows of the array represent? Are they samples?

What do the 201 columns of the array represent? Replicates?

Is this some sort of time series?

What is "A"? Is it some variable that you are simulating? For example: Height, TreeTrunkDiameter, StockPrice?

Why does filling the array involve looping and FLOOR() function calls?

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to ArtemisFowl

06-03-2012 12:03 PM

Thank you all. It works the way I want it to now.

@SAPPER: I don't know proc sql, doesn't it require sql knowledge?

@PGStats and @chang_y_chung: Thanks, I see the idea behind what you did.

@Tom:

*Tom: Why have you conceptualized the problem as an array instead of a dataset?*

Me: It was easier for me programming-wise to read the numbers into an array. To save time, I would have prefered to read them directly into a data set, and maybe this is possible by changing the code slightly. Actually, I think I'll do that. I'm learning how to use SAS, but not in a very linear way.

*Tom: What do the 9801 rows of the array represent? Are they samples? *

*What do the 201 columns of the array represent? Replicates?*

*Is this some sort of time series? *

*What is "A"? Is it some variable that you are simulating? For example: Height, TreeTrunkDiameter, StockPrice?*

*Why does filling the array involve looping and FLOOR() function calls?*

Me: I'm doing this "puzzle", where I determine the amount of distinct numbers of the form a^b, where 2 <= a, b <= 100 (for instance 2^4 = 4^2). It turned out that SAS could not handle numbers that large, so I read the digits of the numbers into an array. The length of the largest number (100^100) is 201, so I made room for 201 digits. There's (100-1)*(100-1) = 9801 different combinations of a and b. The rows of the array are as follows:

2^2

2^3

2^4

...

2^100

3^2

3^3

...etc

The fillArray part is a multiplication algorithm, where I make sure each digit of a^b gets its own cell in the array.

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Posted in reply to ArtemisFowl

06-03-2012 03:23 PM

Hi,

If you are interested/allowed, it might be a good idea to submit your puzzle in its mathematical form as a new discussion on this forum. The diversity of proposed approaches is always fascinating.

PG

PG