topic Re: scan function with only spaces as delimiter in SAS Programming

scan function with only spaces as delimiter

mathias — Tue, 21 Jun 2016 09:45:02 GMT

Hello,

I'm trying to use the scan function with only spaces as delimiter

because when I don't specify delimiters, a dash is also interpreted as a delimiter.

For example,

scan('w_gamma_2 1-w_alpha_1',1) --> w_gamma_2
scan('w_gamma_2 1-w_alpha_1',2) --> 1
scan('w_gamma_2 1-w_alpha_1',3) --> w_alpha_1

But I need to get `w_gamma_2` and `1-w_alpha_1`, not split by the dash

I tried these 3 things:

scan('w_gamma_2 1-w_alpha_1',i,' ')
scan('w_gamma_2 1-w_alpha_1',i,%str(' '))
scan('w_gamma_2 1-w_alpha_1',i,'',s)

but all 3 make all my enterprise guide session completely crash

> An error occurred executing the workspace job "xxxxxx".
> The server is disconnected

Deleting all my tables in the WORK library.

Any ideas on what is wrong here?

I'm in a data step
I'm using Enterprise Guide 7.1 and SAS 9.3

Thanks!

Re: scan function with only spaces as delimiter

RW9 — Tue, 21 Jun 2016 10:03:55 GMT

What is the full code your running, can't see from this snippet. However in your second code:

scan('w_gamma_2 1-w_alpha_1',i,' ')

You mention a variable i which isn't in the first set of code. Normally:

data want;
  result=scan('w_gamma_2 1-w_alpha1',1,' ');
run;

Should work fine.

Re: scan function with only spaces as delimiter

mathias — Tue, 21 Jun 2016 10:14:45 GMT

i = 1, 2, 3... from a do iteration.

If you think the scan(,,' ') should be fine

then the crash may be caused by the rest of the code

This is the whole data step



data PROPORTIONED;

	length
		Germ $10
		Year 3
		AgeGroupDALY $10 
		AgeGroupSPMA $10 
		Gender $1
		State $99
		Category1 $50 
		Category2 $50 
		value 8 
		;

	*** make link to hash table ;
	if _n_=1 then do;

		***modelvalues ----------------;
		declare hash h1(dataset:'modelData');
		h1.definekey ('Germ', 'Category1', 'Category2', 'AgeGroupDALY', 'Gender') ;
		h1.definedata('Value');
		h1.definedone();
		call missing(Germ, Value, Category1, Category2);
		* e.g. rc=h1.find(KEY:Germ, KEY:"State", KEY:"property", KEY:AgeGroupDALY, KEY:Gender);

	end;

	set EDGED;
	put "°°°°° _n_=" _n_;


	put "Germ=" Germ;
	edgesList=tranwrd(edgesList,'||',' ');
	put "edgesList=" edgesList;

	Probability=1;
	*ProbabilitiesList = '++++';

	do i=1 to countw(edgesList);
		put "- i=" i;
		*Edge = scan(edgesList,i,' '); * --> crash ;
		*Edge = scan(edgesList,i,%str(' ')); * --> crash ;
		*Edge = scan(edgesList,i,'',s); * --> crash ;
		Edge = scan(edgesList,i); * --> no crash ;
		put "edge=" Edge;


		* deal with trans probabilities;
		*if find(edge,'1-','i') ge 1 then do;
		if Edge =: '1-' then do;
			put 'i trans probability';
			edge=tranwrd(edge,'-1','');
			marker=1;
			put "edge=" Edge;
		end;

		*lookup;
		rc=h1.find(KEY:Germ, KEY:Edge, KEY:"Probability", KEY:AgeGroupDALY, KEY:Gender);

		if marker=1 then do;
			value = 1 - value;
		end;
		
		*IF lookup fail;
		if rc ^= 0 then do;
			*rc=h1.find(KEY:Germ, KEY:Edge, KEY:"Probability", KEY:AgeGroupDALY, KEY:Gender);
			value = 777777;
		end;
		if value = . then do;
			*rc=h1.find(KEY:Germ, KEY:Edge, KEY:"Probability", KEY:AgeGroupDALY, KEY:Gender);
			value = 666666;
		end;

		Probability = Probability * value;
		put "value=" value;
		put "Probability=" Probability;
		*ProbabilitiesList = tranwrd(ProbabilitiesList,'+','--');
	end;


	Proportion = Probability;

	DALY = Cases * Duration * Severity * Proportion;
	DALYperCase = Duration * Severity * Proportion;

	drop Probability rc i Edge Category1 Category2 Value;

run;

Re: scan function with only spaces as delimiter

RW9 — Tue, 21 Jun 2016 10:22:59 GMT

Hi,

Ok, you are using Hash Tables, I have never found a need to use it so can't help further on that. Could be any number of things from system resources etc. All I can say is the scan statement is correct:

data want;
  result=scan('w_gamma_2 1-w_alpha1',1,' ');
run;

How that interacts with Hash Tables I can't say. Try doing the code using Base SAS.

Edit: Not IML, Hash Tables. Again, never found a need for them, so not used.

Re: scan function with only spaces as delimiter

mathias — Tue, 21 Jun 2016 10:24:39 GMT

May I ask where I'm using IML?

I don't think I have SAS/IML and I'm not aware I never used it.

Re: scan function with only spaces as delimiter

RW9 — Tue, 21 Jun 2016 10:26:45 GMT

See edit, I meant Has Tables.

Re: scan function with only spaces as delimiter

Kurt_Bremser — Tue, 21 Jun 2016 10:52:05 GMT

If you use countw() to determine the number of "words" in your string, you must supply the same list of delimiters that you use in the scan() function, or it will give you a higher number of "words" than are present in the string.

Depending on your settings, this might produce enough log messages so that you run out of memory (which is already under stress due to the hash object) or something else that crashes your server session.

Re: scan function with only spaces as delimiter

mathias — Tue, 21 Jun 2016 11:04:39 GMT

I'm solving the scan problem by using '_1_' instead of '1-'.

With this, no need to change the countw and the scan anymore.

And I agree with the memory issue

I got the error again and after commenting all the put statements it runs better.

Thanks!

Re: scan function with only spaces as delimiter

mathias — Wed, 22 Jun 2016 12:37:56 GMT

Does someone know how can I solve this memory problem ?

Because my table is getting 10k times bigger and it already is crashing.

I did look at my computers RAM and CPU -> no spikes before the crash and much available

I also looked and the BI server's RAM and CPU --> no spikes before the crash and much available

How can I diagnose the crash ?

Is there a parameter in SAS or EG to increase allocated memmory?

Or maybe there is huge file being created on the disks somwhere?

Thanks!

Re: scan function with only spaces as delimiter

Kurt_Bremser — Wed, 22 Jun 2016 19:45:38 GMT

I'd try to specify logging into a file with logparms write=immediate. You will have to play around with the server's config files for this.

You can also locate your current work directory in the filesystem and look for files that grow excessively.

Are you sure you have removed all causes for unnecessary log entries?

And if the base file for the hash object grows, you will run out of memory sooner or later. Some things need to be solved with datasets only, as there only disk storage is the limit.