topic Re: Scan Function for data input in SAS Studio

Scan Function for data input

jacksonan123 — Tue, 01 Aug 2017 14:20:49 GMT

I have the following code to scan the input from my files test1.txt and test2.txt using SAS University edition. I have also placed it in the question using the SAS icon.

%macro datm1;
%do i=1% to 2;
data NEW&i;
libname OLD "/folders/myfolders/bootprocess/";
/*Input D*/
datafile="/folders/myfolders/bootprocess/TEST&i..DAT/";
INPUT @ 1 CHECK  $CHAR200.@;
/*DATA NAME IS CHECK WHICH IS INDEXED W/'THETA   =' THIS MUST BE EXACT*/
S=INDEX(CHECK,'THETA     =');
/*DO LOOP TO PICK OUT CORRECT LINE OF CODE LINE W/STRING WILL BE NUMBERED WITH 1 OTHERS=0*/
/*SCAN FUNCTION ALLOWS VARIABLES TO BE OUTPUT FROM THAT POSITION WITH THE DESIRED NAME*/
IF S>0 THEN DO;
 THETA_=SUBSTR(CHECK,S+0,110);
 K13=SCAN(THETA_,1,' ');
 K23=SCAN(THETA_,2,' ');
 DURATION=SCAN(THETA_,3,' ');
 LAG=SCAN(THETA_,4,' ');
 LOGIT=SCAN(THETA_,5,' ');
 K30  =SCAN(THETA_,6,' ');
 K34= SCAN(THETA_,7,'   ');
 VOLUME=SCAN(THETA_,8,'  ');
  OUTPUT;
 END;
PUT  K13  K23 DURATION LAG LOGIT K30 K34 VOLUME; 
KEEP  K13  K23 DURATION LAG LOGIT K30 K34 VOLUME;
 *PROC PRINT;
RUN;

dATA PEW;
SET NEW;
PROC PRINT;
RUN;

When I run the code I get no output but an error 49 which states " The meaning of an identifier after a quoted string might change in a future SAS release. Inserting white space between a quoted string and the succeeding identifier is recommended." I really habe no idea what this means. Can anyone tell me what is the problem and how I can solve it? The data sets test1 and test 2 are the same.

Re: Scan Function for data input

art297 — Tue, 01 Aug 2017 14:48:32 GMT

You need to show us more! You create a macro, but don't show how it ends (i.e., a %mend; statement), you start a %do loop (the % to may or may not work .. should be %to), but I didn't see a %end statement, you create new1 and new2, but then try to print new, and you use a datafile statement (which I've never seen before) where you probably want to use an infile statement, your libname and datafile statements end with / which might be ok for the libname (but not needed), but I don't think will work with the infile statement, and, finally, you never show us an example dataset or show us the statement that calls the macro.

Art, CEO, AnalystFinder.com

Re: Scan Function for data input

jacksonan123 — Tue, 01 Aug 2017 14:52:39 GMT

I apologize but I did forget to post the entire program. It is now
attached.

%macro datm1;
%do i=1% to 2;
data NEW&i;
libname OLD "/folders/myfolders/bootprocess/";
/*Input D*/
datafile="/folders/myfolders/bootprocess/TEST&i..smr/";
INPUT @ 1 CHECK $CHAR200.@;
/*DATA NAME IS CHECK WHICH IS INDEXED W/'THETA =' THIS MUST BE EXACT*/
S=INDEX(CHECK,'THETA =');
/*DO LOOP TO PICK OUT CORRECT LINE OF CODE LINE W/STRING WILL BE NUMBERED
WITH 1 OTHERS=0*/
/*SCAN FUNCTION ALLOWS VARIABLES TO BE OUTPUT FROM THAT POSITION WITH THE
DESIRED NAME*/
IF S>0 THEN DO;
THETA_=SUBSTR(CHECK,S+0,110);
K13=SCAN(THETA_,1,' ');
K23=SCAN(THETA_,2,' ');
DURATION=SCAN(THETA_,3,' ');
LAG=SCAN(THETA_,4,' ');
LOGIT=SCAN(THETA_,5,' ');
K30 =SCAN(THETA_,6,' ');
K34= SCAN(THETA_,7,' ');
VOLUME=SCAN(THETA_,8,' ');
OUTPUT;
END;
PUT K13 K23 DURATION LAG LOGIT K30 K34 VOLUME;
KEEP K13 K23 DURATION LAG LOGIT K30 K34 VOLUME;
*PROC PRINT;
RUN;

dATA PEW;
SET NEW;
PROC PRINT;
RUN;

TITLE ' DISTRIBUTION DATA';
data distrR;
set %do i=1% to 2;
new&i %end ;;
proc append base=master&i data=new&i;
*PROC PRINT;
RUN;
%end;
%mend datm;
%datm;
run;

Re: Scan Function for data input

art297 — Tue, 01 Aug 2017 16:39:29 GMT

The following corrects for the errors I mentioned in my first post, as well as a couple of additional problems. NEVER use the * type comment in a macro!

I'm not sure what you are trying to do, but try and then examine the results of the following code:

%macro datm1;
/*  %do i=1% to 2;*/
  %do i=1 %to 2;
    data NEW&i;
      libname OLD "/folders/myfolders/bootprocess/";
      /*Input D*/

/*      datafile="/folders/myfolders/bootprocess/TEST&i..smr/";*/
      infile "/folders/myfolders/bootprocess/TEST&i..smr" truncover;
/*      INPUT @ 1 CHECK $CHAR200.@;*/
      INPUT @ 1 CHECK $CHAR200.;
      /*DATA NAME IS CHECK WHICH IS INDEXED W/'THETA =' THIS MUST BE EXACT*/
      S=INDEX(CHECK,'THETA     =');
      /*DO LOOP TO PICK OUT CORRECT LINE OF CODE LINE W/STRING WILL BE NUMBERED
      WITH 1 OTHERS=0*/
      /*SCAN FUNCTION ALLOWS VARIABLES TO BE OUTPUT FROM THAT POSITION WITH THE
      DESIRED NAME*/
      IF S>0 THEN DO;
        THETA_=SUBSTR(CHECK,S+0,110);
        K13=SCAN(THETA_,1,' ');
        K23=SCAN(THETA_,2,' ');
        DURATION=SCAN(THETA_,3,' ');
        LAG=SCAN(THETA_,4,' ');
        LOGIT=SCAN(THETA_,5,' ');
        K30 =SCAN(THETA_,6,' ');
        K34= SCAN(THETA_,7,' ');
        VOLUME=SCAN(THETA_,8,' ');
        OUTPUT;
      END;
      PUT K13 K23 DURATION LAG LOGIT K30 K34 VOLUME;
      KEEP K13 K23 DURATION LAG LOGIT K30 K34 VOLUME;
/*      *PROC PRINT;*/
    RUN;

/*    dATA PEW;*/
/*      SET NEW;*/
    PROC PRINT;
    RUN;

    TITLE ' DISTRIBUTION DATA';
/*    data distrR;*/
/*      set %do i=1% to 2;new&i %end ;;*/
    proc append base=master data=new&i;
    /**PROC PRINT;*/
    RUN;
  %end;
/*%mend datm;*/
%mend datm1;
%datm1;
/*run;*/

Art, CEO, AnalystFinder.com

Re: Scan Function for data input

jacksonan123 — Tue, 01 Aug 2017 17:37:39 GMT

Changing the input to (infile=) from (datafile=) and adding truncover
helped it to run ok.

However a macro issue surfaced. I noted that the following code was
commented out in your reply:

/* data distrR;*/
/* set %do i=1% to 2;new&i %end ;;*/
proc append base=master data=new&i;
/**PROC PRINT;*/

I actually need that code because I need the distrR data set to be created
so that it can be further processed.

When I include the code you commented out I get an error that Work.new3 does
not exist. Can you tell me why I am getting this error? My macro states 1
to 2:

data distrR;
set %do i=1 %to 2;
new&i %end ;;
proc append base=master&i data=new&i;

Thanks for your help.

Re: Scan Function for data input

art297 — Tue, 01 Aug 2017 18:05:10 GMT

I commented those lines out because: (1) you were setting the macro variable i in two overlapping loops and it didn't make any sense (to me) to be using an append statement to create two master files that each would only include the files you already created.

You would have to post the code you actually ran but, before doing so, change the second loop to something other than I (e.g., J)

And, if you want to solve it yourself, first run:

options mprint mlogic symbolgen;

That way you will be able to see what is going on in the background throughout the code.

Art, CEO, AnalystFinder.com

Re: Scan Function for data input

jacksonan123 — Tue, 01 Aug 2017 18:39:39 GMT

I put in the code to have the notes and mprint printed out and whether I put
in J or left in the I for the append it seems to have the same issue. It
looks for a work.new3 which doesn't exist. It went through 1 and 2 with
no issue but instead of stopping it resolves :

With the following code for the distr data step:

data distrR;
set %do i=1 %to 2;
new&i %end ;;
proc append base=master&i data=new&i;
*PROC PRINT;
RUN;
%end;
%mend datm1;
%datm1;
run;

Symbolgen: Macro variable I resolves to 3

Mprint(datm1): proc append base =master3 data=new3

Error: File work,new3 does not exist.

If I put in j:

Symbolgen: Macro variable j resolves to 3

Mprint(datm1): proc append base =master3 data=new3

Error: File work,new3 does not exist.

Can you tell me why the macro does not stop at 2?

Re: Scan Function for data input

art297 — Tue, 01 Aug 2017 18:58:47 GMT

Makes perfect sense .. but the logic of your code doesn't!.

Think about it! You have a major loop where you are creating NEW1 and NEW2. However, in the middle of that loop you are trying to create three copies of the same thing, one called distrR and the other called master1 and master2.

Since the I loop doesn't %end until the end of the program, the first time through it can't create distrR the way you want because only one of the files had been created at that point. The loop doesn't end until I gt 2 (i.e., 3), thus that is its value by the time it gets to the proc append statement.

You'll have to tell us what you really want to create in those two segments (i.e., distrR and master&i) in order for anyone to give you good advice.

Art, CEO, AnalystFinder.com

Re: Scan Function for data input

Tom — Tue, 01 Aug 2017 19:23:19 GMT

I would suggest taking a step back and getting the data step working for a single file first. Later you can back the macro %DO loop if you need it. It is much easier to debug without having to use macro code.

Also post an example input file and we can help you simplify the data step. If the data is really space delimited then why not just use the INPUT statement to read it rather than reading it into a character varaible and then having to parse it. Plust that way it will be easier to read the numeric fields into numeric variables.

Something like this might work.

data new1 ;
  infile "/folders/myfolders/bootprocess/TEST1.DAT";
  input @;
  if index(_infile_ , 'THETA     =') then do;
    input @'THETA     =' +110 THETA K13 K23 DURATION LAG LOGIT K30 K34 VOLUME;
    output;
  end;
run;
data _null_;
  set new1 ;
  put (_all_) (=/);
run;

Re: Scan Function for data input

jacksonan123 — Tue, 01 Aug 2017 19:28:39 GMT

I got a response that allowed me to use my original code by using an infile
statement with truncover instead of datafile.

The changed code is listed:

%macro datm1;
%do i=1 %to 2;

data NEW&i;
libname OLD "/folders/myfolders/bootprocess/";
/*Input D*/
/*datafile="/folders/myfolders/bootprocess/TEST&i..smr/";*/
infile "/folders/myfolders/bootprocess/TEST&i..smr/" truncover;

Thanks for your response and I will try your code later.

Re: Scan Function for data input

Tom — Tue, 01 Aug 2017 19:30:46 GMT

Why did you still post the start of data step with a LIBNAME statement embedded into the middle of it?

The LIBNAME statement will have NOTHING to do with the data step. It will be compiled before the data step starts. Plus you code isn't even referencing the libref it creates.

Re: Scan Function for data input

jacksonan123 — Tue, 01 Aug 2017 19:47:39 GMT

The current macro that I have runs just fine with the libname and when it is
removed I get an error.

The libname is being used by SASuniversity for the infile statement.

Re: Scan Function for data input

Tom — Tue, 01 Aug 2017 20:06:51 GMT

@jacksonan123 wrote:
The current macro that I have runs just fine with the libname and when it is
removed I get an error.

The libname is being used by SASuniversity for the infile statement.

Unless the INFILE statement looks like this

infile "%sysfunc(pathname(OLD))/TEST&i..smr" truncover;

then the INFILE statement is in no way using the libname statement for anything.

Now your code or your SAS/Studio session might be using the libref OLD for something. But since its definition does NOT depend on the macro variables used in the macro there is no reason why it cannot be defined outside of the macro.

libname OLD "/folders/myfolders/bootprocess/";
%datm1;

Or if it is defined in the macro then define it outside of the %DO loop instead of defining it twice.

%macro datm1;
libname OLD "/folders/myfolders/bootprocess/";
%do i=1% to 2;
...
%end;
%mend datm1;

Re: Scan Function for data input

art297 — Tue, 01 Aug 2017 21:21:32 GMT

Looking at your raw data, I will guess that you really want to read some of the other fields in your data. However, the following matches most of the calculations described in the data. This time, I chose to read them directly rather than scanning a text field:

%macro datm1;
  %do i=1 %to 2;
    data NEW&i;
      /*Input D*/
      infile "/folders/myfolders/bootprocess/TEST&i..smr";
      INPUT @ 'THETA     =' K13 K23 CL V3 D1 TLAG2 LOGITT;
      ALAG2=sum(D1,TLAG2);
      TVF1=1/(1+EXP(-LOGITT));
      F1=TVF1;
      S3=V3/1000;
      F2=1-F1;
      K30=CL/V3;
    run;
  %end;

  TITLE ' DISTRIBUTION DATA';
   data distrR;
     set %do i=1 %to 2;new&i %end ;;
   run;
%mend datm1;
%datm1;

Art, CEO, AnalystFinder.com

Re: Scan Function for data input

ballardw — Tue, 01 Aug 2017 22:34:21 GMT

@jacksonan123 wrote:

When I run the code I get no output but an error 49 which states " The meaning of an identifier after a quoted string might change in a future SAS release. Inserting white space between a quoted string and the succeeding identifier is recommended." I really habe no idea what this means. Can anyone tell me what is the problem and how I can solve it? The data sets test1 and test 2 are the same.

If you read carefully that is a WARNING. There aer several code structures, date, datetime, time and name literals that use a quoted value followed immediately by a special character to indicate the quoted value is to be used in a specific way.

x = '01JAN2017'd; for example says that the variable x is assigned a date value (the trailing d). So if you use code with any quoted string such as: 'some string followed by'anything the 'a triggers the generic warning because the character was not one of the currently used d,dt, t, n (and I think there's another couple I don't remember of the top of my head).

Note that if you had used one of the data indicators you may have generated an error instead:

446  data _null_;
447     x='23Fre2017'd;
          ------------
          77
ERROR: Invalid date/time/datetime constant '23Fre2017'd.
ERROR 77-185: Invalid number conversion on '23Fre2017'd.

448  run;

because the value in the quotes does not match the type indicated by the trailing character.

Re: Scan Function for data input

jacksonan123 — Wed, 02 Aug 2017 00:12:13 GMT

Code works well.

Thanks

Re: Scan Function for data input

jacksonan123 — Wed, 02 Aug 2017 00:24:39 GMT

The code works well. I had one additional question. In my data output I
would like to capture the string MINIMIZATION SUCCESSFUL or in some other
cased MINIMIZATION TERMINATED as part of my output?

( K13 K23 CL V3 D1 TLAG2 LOGITT MINIMIZATION SUCCESSFUL) or ( k13 k23
..MINIMIZATION TERMINATED)

Re: Scan Function for data input

ccaulkins9 — Mon, 02 Apr 2018 15:54:13 GMT

I know this is old and resolved but just wanted to chip in a bit of "clarity" as to the error @jacksonan123? is seeing: if you were to provide the part of the error log where the message is printed there's a good chance theres' some coordinates to point out where those unbalanced quotes maybe are lurking - anybody give this a shot?

Re: Scan Function for data input

ccaulkins9 — Mon, 02 Apr 2018 15:54:51 GMT

in fact, this error actually isn't barely related to SAS Studio, but more like Base SAS related