DATA Step, Macro, Functions and more

scan function with only spaces as delimiter

Accepted Solution Solved
Reply
Frequent Contributor
Posts: 124
Accepted Solution

scan function with only spaces as delimiter

Hello,

 

I'm trying to use the scan function with only spaces as delimiter

because when I don't specify delimiters, a dash is also interpreted as a delimiter.

For example,

 

scan('w_gamma_2 1-w_alpha_1',1) --> w_gamma_2
scan('w_gamma_2 1-w_alpha_1',2) --> 1
scan('w_gamma_2 1-w_alpha_1',3) --> w_alpha_1

 

But I need to get `w_gamma_2` and `1-w_alpha_1`, not split by the dash

I tried these 3 things:

 

scan('w_gamma_2 1-w_alpha_1',i,' ')
scan('w_gamma_2 1-w_alpha_1',i,%str(' '))
scan('w_gamma_2 1-w_alpha_1',i,'',s)

but all 3 make all my enterprise guide session completely crash

 

Image 8.png

 

> An error occurred executing the workspace job "xxxxxx".
> The server is disconnected

 

Deleting all my tables in the WORK library.

 

Any ideas on what is wrong here?

 

I'm in a data step
I'm using Enterprise Guide 7.1 and SAS 9.3

 

Thanks!


Accepted Solutions
Solution
‎06-21-2016 07:05 AM
Super User
Posts: 6,946

Re: scan function with only spaces as delimiter

If you use countw() to determine the number of "words" in your string, you must supply the same list of delimiters that you use in the scan() function, or it will give you a higher number of "words" than are present in the string.

Depending on your settings, this might produce enough log messages so that you run out of memory (which is already under stress due to the hash object) or something else that crashes your server session.

---------------------------------------------------------------------------------------------
Maxims of Maximally Efficient SAS Programmers

View solution in original post


All Replies
Super User
Super User
Posts: 7,407

Re: scan function with only spaces as delimiter

What is the full code your running, can't see from this snippet.  However in your second code:

scan('w_gamma_2 1-w_alpha_1',i,' ')

You mention a variable i which isn't in the first set of code.  Normally:

data want;
  result=scan('w_gamma_2 1-w_alpha1',1,' ');
run;

Should work fine.

 

Frequent Contributor
Posts: 124

Re: scan function with only spaces as delimiter

[ Edited ]

i = 1, 2, 3... from a do iteration.

 

If you think the scan(,,' ') should be fine

then the crash may be caused by the rest of the code 

 

 

This is the whole data step

 



data PROPORTIONED;

	length
		Germ $10
		Year 3
		AgeGroupDALY $10 
		AgeGroupSPMA $10 
		Gender $1
		State $99
		Category1 $50 
		Category2 $50 
		value 8 
		;

	*** make link to hash table ;
	if _n_=1 then do;

		***modelvalues ----------------;
		declare hash h1(dataset:'modelData');
		h1.definekey ('Germ', 'Category1', 'Category2', 'AgeGroupDALY', 'Gender') ;
		h1.definedata('Value');
		h1.definedone();
		call missing(Germ, Value, Category1, Category2);
		* e.g. rc=h1.find(KEY:Germ, KEY:"State", KEY:"property", KEY:AgeGroupDALY, KEY:Gender);

	end;

	set EDGED;
	put "°°°°° _n_=" _n_;


	put "Germ=" Germ;
	edgesList=tranwrd(edgesList,'||',' ');
	put "edgesList=" edgesList;

	Probability=1;
	*ProbabilitiesList = '++++';

	do i=1 to countw(edgesList);
		put "- i=" i;
		*Edge = scan(edgesList,i,' '); * --> crash ;
		*Edge = scan(edgesList,i,%str(' ')); * --> crash ;
		*Edge = scan(edgesList,i,'',s); * --> crash ;
		Edge = scan(edgesList,i); * --> no crash ;
		put "edge=" Edge;


		* deal with trans probabilities;
		*if find(edge,'1-','i') ge 1 then do;
		if Edge =: '1-' then do;
			put 'i trans probability';
			edge=tranwrd(edge,'-1','');
			marker=1;
			put "edge=" Edge;
		end;

		*lookup;
		rc=h1.find(KEY:Germ, KEY:Edge, KEY:"Probability", KEY:AgeGroupDALY, KEY:Gender);

		if marker=1 then do;
			value = 1 - value;
		end;
		
		*IF lookup fail;
		if rc ^= 0 then do;
			*rc=h1.find(KEY:Germ, KEY:Edge, KEY:"Probability", KEY:AgeGroupDALY, KEY:Gender);
			value = 777777;
		end;
		if value = . then do;
			*rc=h1.find(KEY:Germ, KEY:Edge, KEY:"Probability", KEY:AgeGroupDALY, KEY:Gender);
			value = 666666;
		end;

		Probability = Probability * value;
		put "value=" value;
		put "Probability=" Probability;
		*ProbabilitiesList = tranwrd(ProbabilitiesList,'+','--');
	end;


	Proportion = Probability;

	DALY = Cases * Duration * Severity * Proportion;
	DALYperCase = Duration * Severity * Proportion;

	drop Probability rc i Edge Category1 Category2 Value;

run;

 

Super User
Super User
Posts: 7,407

Re: scan function with only spaces as delimiter

[ Edited ]

Hi,

 

Ok, you are using Hash Tables, I have never found a need to use it so can't help further on that.  Could be any number of things from system resources etc.  All I can say is the scan statement is correct:

data want;
  result=scan('w_gamma_2 1-w_alpha1',1,' ');
run;

How that interacts with Hash Tables I can't say.  Try doing the code using Base SAS.

 

Edit: Not IML, Hash Tables.  Again, never found a need for them, so not used.

 

 

Frequent Contributor
Posts: 124

Re: scan function with only spaces as delimiter

May I ask where I'm using IML?

 

I don't think I have SAS/IML and I'm not aware I never used it.

 

Super User
Super User
Posts: 7,407

Re: scan function with only spaces as delimiter

See edit, I meant Has Tables.

Solution
‎06-21-2016 07:05 AM
Super User
Posts: 6,946

Re: scan function with only spaces as delimiter

If you use countw() to determine the number of "words" in your string, you must supply the same list of delimiters that you use in the scan() function, or it will give you a higher number of "words" than are present in the string.

Depending on your settings, this might produce enough log messages so that you run out of memory (which is already under stress due to the hash object) or something else that crashes your server session.

---------------------------------------------------------------------------------------------
Maxims of Maximally Efficient SAS Programmers
Frequent Contributor
Posts: 124

Re: scan function with only spaces as delimiter

I'm solving the scan problem by using '_1_' instead of '1-'.

With this, no need to change the countw and the scan anymore.

 

And I agree with the memory issue

I got the error again and after commenting all the put statements it runs better.

 

Thanks!

Frequent Contributor
Posts: 124

Re: scan function with only spaces as delimiter

[ Edited ]

Does someone know how can I  solve this memory problem ?

 

Because my table is getting 10k times bigger and it already is crashing.

 

I did look at my computers RAM and CPU -> no spikes before the crash and much available

I also looked and the BI server's RAM and CPU --> no spikes before the crash and much available

 

How can I diagnose the crash ?

Is there a parameter in SAS or EG to increase allocated memmory?

 

Or maybe there is huge file being created on the disks somwhere?

 

 

Thanks!

Super User
Posts: 6,946

Re: scan function with only spaces as delimiter

I'd try to specify logging into a file with logparms write=immediate. You will have to play around with the server's config files for this.

You can also locate your current work directory in the filesystem and look for files that grow excessively.

Are you sure you have removed all causes for unnecessary log entries?

And if the base file for the hash object grows, you will run out of memory sooner or later. Some things need to be solved with datasets only, as there only disk storage is the limit.

 

---------------------------------------------------------------------------------------------
Maxims of Maximally Efficient SAS Programmers
☑ This topic is SOLVED.

Need further help from the community? Please ask a new question.

Discussion stats
  • 9 replies
  • 576 views
  • 3 likes
  • 3 in conversation