J111 Tracker

Re: Data size limitations for running code

J111 — Thu, 14 Aug 2025 12:28:23 GMT

Hello We solved the issue by changing the data structure Thanks everyone The topic can be closed

Re: Data size limitations for running code

J111 — Wed, 13 Aug 2025 12:33:55 GMT

Just updating that we plan to change the data structure and this might also solve this issue. I can check this in the next few days.

Re: Data size limitations for running code

J111 — Tue, 12 Aug 2025 09:04:02 GMT

Thanks for the proc printo idea, yes we are using EGuide

8

Re: Data size limitations for running code

J111 — Mon, 11 Aug 2025 10:46:15 GMT

ok Thanks for the advice.

Re: Data size limitations for running code

J111 — Mon, 11 Aug 2025 10:07:35 GMT

Hello,

The administrator checked and the WORK space was fine during the run.

Seems like not a code problem either.

any idea ?

Re: Data size limitations for running code

J111 — Mon, 11 Aug 2025 08:50:57 GMT

Hello,

I could not find code issues with obs=10

the only note is

NOTE: Mathematical operations could not be performed at the following places. The results of the operations have been set to missing values

-----

We have at least 150 GIGA space for work on the server.

----

Matching table has less than 2500 rows.

Why by limiting it to (obs=2500) the trns_table could be larger than (obs=177) ? and even (obs=300)

---

there is no consistency - next run with combination of 2500 & 300 ended with connection error.

Thanks

Re: Data size limitations for running code

J111 — Mon, 11 Aug 2025 08:22:46 GMT

Hello,

I am running several combinations of obs=

for the combination below, the program runs fine.

once increase obs from 300 to 350 the connection to the server is lost.

I shall check all suggestions posted and update regarding the results.

------------------------

filename sas_code temp ;

data _null_ ;

Set MATCHING_TABLE_FUNC(obs=2500) end = last ;

file sas_code ;

if _N_ = 1 then put "options ls = 256 errors = max nospool ; Data Formula_ ; set TRNSP_SERIES_LIST(obs=300) ;" ;

put Target '=' Formula ";" ;

if last then put "Run ;" ;

run ;

data _null_ ;

infile sas_code ;

input ;

put _infile_ ;

run ;

%include sas_code ;

Re: Data size limitations for running code

J111 — Mon, 11 Aug 2025 07:20:50 GMT

Hello,

The following log error

Error: The connection to the server has been lost.

appears for (obs=177)

for obs with lower values such as 170, 175, 176 the program runs fine.

------------------------------------------------------------------------------------

filename sas_code temp ;

data _null_ ;

Set MATCHING_TABLE_FUNC end = last ;

file sas_code ;

if _N_ = 1 then put "options ls = 256 errors = max nospool ; Data Formula_ ; set TRNSP_SERIES_LIST(obs=177) ;" ;

put Target '=' Formula ";" ;

if last then put "Run ;" ;

run ;

data _null_ ;

infile sas_code ;

input ;

put _infile_ ;

run ;

%include sas_code ;

Re: Data size limitations for running code

J111 — Mon, 11 Aug 2025 07:01:34 GMT

Target and formula are character variables in the matching table.

Re: Data size limitations for running code

J111 — Mon, 11 Aug 2025 06:59:37 GMT

Hello,

The TRNP_SERIES_LIST columns looks like this

DATE x1 x2 x3 until x4623

The x(i) are variables from TRNS_SERIES_LIST that appear in the formula in the MATCHING_TABLE.

The matching table includes target name of a new variable to calculate and the formula of variables.

for example new_var = x1 + x2 /x3 + x4

Re: Data size limitations for running code

J111 — Mon, 11 Aug 2025 06:50:17 GMT

Hello,

I will try to answer your questions:-

The code is valid it runs very good for 175 rows in the TRNS_SERIES_LIST table.

The matching table includes 2454 rows each row with one target formula.

Each formula includes up to 20 variables.

The TRNS_SERIES_LIST includes one column of date and thousands columns of variables.

These variables appear in the formula.

Thanks

Data size limitations for running code

J111 — Mon, 11 Aug 2025 05:58:48 GMT

Hello,

Are there data size limitations for the following code,

if so is there a solution rather than a smaller data input:

----------------------------------------------------

filename sas_code temp ;

data _null_ ;

Set MATCHING_TABLE_FUNC end = last ;

file sas_code ;

if _N_ = 1 then put "options ls = 256 errors = max nospool ; Data Formula_ ; set TRNSP_SERIES_LIST ;" ;

put Target '=' Formula ";" ;

if last then put "Run ;" ;

run ;

data _null_ ;

infile sas_code ;

input ;

put _infile_ ;

run ;

%include sas_code ;

----------------------

The log has an error of connection to the server was lost.

The matching table includes thousands of formula such as

target= x1 + x2

The TRNSP_SERIES_LIST SAS file includes 487 rows and 4624 columns.

If the TRNSP_SERIES_LIST includes only 175 rows there is no log error of connection and the program runs smoothly.

We work with Linux, SAS EG 8.2

-------------------------

I can invent data if it is required.

Thanks in advance.

Re: How to extract file information on a web server

J111 — Mon, 17 Mar 2025 13:16:06 GMT

Hello, We get the same error if the csv is local or if it is located on the web and not at the Linux server.

Thanks your reply is helpful.

How to extract file information on a web server

J111 — Mon, 17 Mar 2025 09:56:21 GMT

Hello,

we are able to extract the last modified date of a file on Linux using the following code,

and we would like to extract similar information from a web server/ local.

* Running on a Linux folder ;

filename myfile '/opt/sas/spf/xyz/abc.csv' ;

data last_updated ;

opt=FOPTNAME(FOPEN('myfile'),5) ;

optval=FINFO(FOPEN('myfile'),upcase(opt)) ;

run ;

Here we get the required information

Last Modified

31Jan2023:15:29:01

However a log ERROR appears for running on a web server/windows :

filename fileref 'C:\Temp\abc.csv' ;

data a(drop=fid);

infile fileref truncover obs=1;

fid=fopen('fileref');

moddate=finfo(fid,'Last Modified');

run;

ERROR: Physical file does not exist, /opt/sas/config/Lev1/SASApp/C:\Temp\abc.csv.

Do we need to install Samba in order to solve this issue ? another solution you might have ?

What would the code be ?

Thanks a lot

Re: String Parsing - how to extract words between comma and dot

J111 — Wed, 22 Jan 2025 09:04:22 GMT

Hello,

Seems like this can work for TYPE B,

Data next ;

set have ;

xx = scan(data,-1,"/") ;

Array x_ [40] $55 x1-x40 ;

do i = 1 to 40 ;

x_[i] = scan(scan(xx,-i,','),1,'.') ; ;

end ;

drop i xx data ;

Run ;

String Parsing - how to extract words between comma and dot

J111 — Wed, 22 Jan 2025 09:13:43 GMT

Hello,

Please find below data Have and data Want.

Data Have ;
input data $100. ;
cards ;
data/dataflow/BPS.STQR/EZR/1.0/RER_QWE_MED.D.AUD.MED, RER_QWA_MED.D.CAD.WOW.WOW.OF00, QWZ_PLO_WOW.D.POP
;
Run ;

Data Want ;
input x1 $60. ;
cards ;
RER_QWE_MED , RER_QWA_MED , QWZ_PLO_WOW
;
Run ;

We would like to extract X1 from the data according to the following rules for Type A and Type B:

Type A

1. it appears after the last back slash in the sentence

2. and also before the first dot following this word (if there is a dot)

3. and this word contains only capital letters or underscore in it (without small letters)

4. if such a word is not found write 'NOWORD'

Type B

here we have to extract more than one word and the logic is as follows:

1. it appears after the last back slash in the sentence

2. continue in the same row until we find a comma and then take the word which is between the comma and the next first dot.

recursively look for all the words between comma and next dot (as long as we are after the last backslash in the row).

the purpose is to clean relevant words from garbage, for example '.CAD.' is garbage.

for type A we have the solution

x1 = scan(scan(data,-1,"/"),1,'.') ; *after last back slash and before first period ;

if count(x1,lowcase(x1)) = 1 then x1 = 'NOWORD' ;

We need a solution for type B

best solution would handle both type A and type B in the same row.

------------------------------------------------------

BTW Next step we split the want data by delims.

Data Audit_Split ;

set Audit_short ;

delims = '+,' ;

Array s_ [40] $55 s1-s40 ;

do i = 1 to 40 ;

s_[i] = left(scan(string, i, delims)) ;

end ;

drop delims i ;

Run ;

----------------------------------------------------------------------------------------------------------

Thanks in advance.

Re: Data Parsing - How to extract specific words from a sentence by rules

J111 — Wed, 08 Jan 2025 13:57:27 GMT

A little clarification

The purpose is to clean the words from the garbage

as long as they have upercase or underscores.

Thanks

Re: Data Parsing - How to extract specific words from a sentence by rules

J111 — Wed, 08 Jan 2025 11:11:41 GMT

Hellow,

Seems I found a solution for calculating X1 - view data test:-

Would appreciate your help regarding X2..

Data test ;
set have ;
X1 = scan(scan(data,-1,"/"),1,'.') ;
if count(x1,lowcase(X1)) = 1 then X1 = 'NOWORD' ;
Run ;

Data Parsing - How to extract specific words from a sentence by rules

J111 — Wed, 08 Jan 2025 09:53:44 GMT

Hello,

Please find below data Have and data Want.

We would like to extract X1 from the data according to the following rules:

1. it appears after the last back slash in the sentence

2. and also before the first dot following this word (if there is a dot)

3. and this word contains only capital letters or under score in it (without small letters)

4. and there is no garbage after this word (such as '..')

5. if it can not find a word that follows all theses rules than write NOWORD

We would like to extract X2 from the data according to the following rules:

1. it appears always after this word BPS.STQR/ or this word BPS.STQR,

in other words it appears after BPS.STQR and is surrounded with '/' or with ','

Thanks in advance

----------------------------------------------------------------------------------------------

Data have ;
input data $60. ;
cards ;
data/dataflow/BPS.STQR/WAB/1.0/NER_PGTABC
data/dataflow/wow/BPS.STQR,WAB,1.0/NER_QZW
data/dataflow/wow/BPS.STQR,WAB,1.0/NER_QZW ..
data/dataflow/wow/BPS.STQR,WAB,1.0/NER_QZW.ABCDEFG
/availability/dataflow/*/*/*/*/-
/availability

;
Run ;

Data Want ;
input x1 $10. x2 $7. ;
cards ;
NER_PGTABC WAB
NER_QZW WAB
NER_QZW WAB
NER_QZW WAB

NOWORD

;
Run ;

Re: Bufsize for long concatination

J111 — Mon, 09 Dec 2024 06:10:47 GMT

This works !

data want;
set matrix ;
length want $32767;
want =catx(", ", of col:) ;
run;