Hash Memory used the varible length kept are the data having in that variables

[sas_]

Hash

Hi in my code i have created hash objects the data was huge as 15 crores and i need to create the 4 tables ,but i am getting memory issue.where the hash actually stores the files
2.I told that i have created 4 hash tables based on 4 varibale and the variable length
is 100 i have give and the data in this variables will use 12 onlye
ex: attrib x1 length 100$

the data that was used max was up to 12 only Ex:x1=hihru1234545

In this case the hash will creat the table for 100 lenght or for the length the variable that exactly used(12).

If i reduce the length of variable used in the hash tables the memory will increase

[FriedEgg]

150 million rows of data will require a decent about of memory to store the hash object.

1. Make sure your system has enough memory available total to even store the object.

2. Make sure you setup the sessions memory settings appropirately so that SAS will have that memory available to it.

As far as I know the extranious length of a assigned variable does not affect the amount of space required to store it in memory for a hash object.  See the following as proof:

16         data foo;

17          length bar $100;

18          set sashelp.class end=done;

19          orig_age=age;

20          do i=1 to 500000;

21           bar=cats(name,put(i,6.));

22           age=orig_age+rand('table',.25,.25,.25,.25);

23           output;

24          end;

25          keep bar age;

26         run;

NOTE: There were 19 observations read from the data set SASHELP.CLASS.

NOTE: The data set WORK.FOO has 9500000 observations and 2 variables.

NOTE: DATA statement used (Total process time):

      real time           4.83 seconds

      user cpu time       4.06 seconds

      system cpu time     0.75 seconds

      Memory                            2493978k

      OS Memory                         2499944k

27        

28         data age10(where=(10<=age<15)) age15(where=(15<=age<20));

29          if 0 then set foo;

30          declare hash ha(dataset:'foo');

31          declare hiter iter('ha');

32           ha.definekey('bar');

33           ha.definedata(all:'y');

34           ha.definedone();

35          iter.first();

36          do while(iter.next()=0);

37           output;

38          end;

39          stop;

40         run;

NOTE: There were 9500000 observations read from the data set WORK.FOO.

NOTE: The data set WORK.AGE10 has 2374832 observations and 2 variables.

NOTE: The data set WORK.AGE15 has 7000595 observations and 2 variables.

NOTE: DATA statement used (Total process time):

      real time           13.27 seconds

      user cpu time       11.93 seconds

      system cpu time     1.32 seconds

      Memory                            2493978k

      OS Memory                         2499944k

16         data foo;

17          length bar $13;

18          set sashelp.class;

19          orig_age=age;

20          do i=1 to 500000;

21           bar=cats(name,put(i,6.));

22           age=orig_age+rand('table',.25,.25,.25,.25);

23           output;

24          end;

25          keep bar age;

26         run;

NOTE: There were 19 observations read from the data set SASHELP.CLASS.

NOTE: The data set WORK.FOO has 9500000 observations and 2 variables.

NOTE: DATA statement used (Total process time):

      real time           5.71 seconds

      user cpu time       4.08 seconds

      system cpu time     1.56 seconds

      Memory                            2493978k

      OS Memory                         2499944k

27        

28         data age10(where=(10<=age<15)) age15(where=(15<=age<20));

29          if 0 then set foo;

30          declare hash ha(dataset:'foo');

31          declare hiter iter('ha');

32           ha.definekey('bar');

33           ha.definedata(all:'y');

34           ha.definedone();

35          iter.first();

36          do while(iter.next()=0);

37           output;

38          end;

39          stop;

40         run;

NOTE: There were 9500000 observations read from the data set WORK.FOO.

NOTE: The data set WORK.AGE10 has 2373910 observations and 2 variables.

NOTE: The data set WORK.AGE15 has 7000782 observations and 2 variables.

NOTE: DATA statement used (Total process time):

      real time           13.78 seconds

      user cpu time       11.93 seconds

      system cpu time     1.82 seconds

      Memory                            2493978k

      OS Memory                         2499944k