reading multiple textfiles

[R_A_G_]

Hello,

I have 150 .dat files which are space delimited. I need to read the last column.

please help

Thanks

[art297]

Didn't you just ask this question last week?  I believe the answer was:

filename allcsv "C:\CDM1\5K\5K_10000obs\5K_10000obs_RIT\OUT\respondents_*.dat";

data out.class;

  infile allcsv lrecl=10000 filename=filename;

  input;

  lastfield=input(scan(_infile_,1,,'bs'),best12.);

run;

Didn't that end up working for you?

[R_A_G_]

yes it did but it doesn't work for this one. I am tring to change it a bit and see if it can read this as well.

Thanks

[art297]

You didn't include any data, code or anything in your post.  It would help if you provided your code and an idea of what the new problem is that you are confronting.

[R_A_G_]

My new problem is that each one of these files have a corresponding file from another directory that I need to read and after ward do do some calculation.

My question is that how do I read another file from a different location into the file had created to then do some math calculations?

this is what I had with your help, which I put it in a Macro:

%macro class;

OPTIONS MACROGEN

%let num_reps=150;

%do seed=1 %to &num_reps;

%LET commandfile=respondents_&SEED;

LIBNAME Out "C:\CDM1\5K\OUT";

RUN;

filename allcsv "C:\CDM1\5K\5K_5000obs\5K_5000obs_RIT\OUT\commandfile..dat";

data out.class;

infile allcsv lrecl=10000 filename=filename;

input;

file_1=input(scan(_infile_,1,,'bs'),best12.);

run;

data out.class1_&seed;

set class;

obs=_n_;

output;

run;

%END;

%MEND;

I need the 2nd column of the file the following file, I can attach it if that would be more helpful

This is what the file looks like:

1 5 0 0 0 0 1 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0

2 32 1 0 1 1 1 1 1 0 1 1 1 1 0 1 1 1 1 1 1 0 1 1 1 1 0 0 1 1 1 1 1

3 13 1 0 0 1 1 0 1 1 1 0 0 1 0 0 1 0 0 1 0 1 1 0 0 1 0 1 0 0 0 0 0

4 8 0 1 1 1 1 1 1 0 0 0 1 1 1 0 1 0 0 0 1 0 0 0 1 0 0 0 0 0 1 0 1

5 31 0 0 1 1 1 1 1 0 0 0 1 1 0 1 1 0 0 1 0 1 0 1 1 1 0 1 1 1 0 1 1

6 32 0 1 1 1 1 1 0 1 0 0 0 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

7 18 0 0 0 0 0 1 0 1 0 1 0 0 1 0 0 1 1 1 1 0 0 1 1 1 1 1 0 0 0 0 0

Thank you

[art297]

This is starting to sound too much like a homework problem and the code you provided can't work as you refer to at least one macro variable without preceding it with an ampersand.

You can probably read the second column by simply changing the scan statement to go for the second rather than the first column.

I'd suggest trying to solve it yourself and then, if you still need help. post the code you wrote.

[Tom]

Problem would probably be easier for you to understand and for others to help you with if you could explain more about what you are actually doing.  What do these files represent. Why do you only want the last value for each observation?    Normally if you want to combine data from two different files there should be some key variables that you can use to match (merge) the records.  In your example the first value of every line seems to be the same as the line number.  Is that an ID variable? Didn't you receive any specifications on what the content of these files was?

[R_A_G_]

you are right about the first line being observation number.

One of these files is the estimated value and the other is the original data. I have simulatd about 150 of these files and now I am trying to grab the estimated variable from one file and the coresponding original variable form the other and calculate the bias, standard error, and RMSE of these variables 150 of them.

Thanks

This is my code, the only problem is that I cannot make the second file to read the 2nd column,

%macro RRRUN;

%let num_reps=1;

%do seed=1 %to &num_reps;

%LET commandfile=respondents_&SEED;

%LET dat=dat&SEED;

LIBNAME class "C:\CDM1\5K\OUT\class";

RUN;

filename allcsv "C:\CDM1\5K\5K_5000obs\5K_5000obs_RIT\OUT\&commandfile..dat";

data class1_&seed;

infile allcsv lrecl=10000 filename=filename;

input;

file_&seed=input(scan(_infile_,1,,'bs'),best12.);

run;

data class1_&seed;

set class1_&seed;

obs=_n_;

output;

run;

/******************************/

filename allcsv "C:\CDM1\5K\5K_5000obs\5K_5000obs_RIT\IN\&dat..dat";

Data class2_&seed;

infile allcsv lrecl=10000 filename=filename;

input;

file2_&seed=input(scan(_infile_,1,,'bs'),best12.);

run;

data class2_&seed;

set class2_&seed;

obs=_n_;

output;

run;

data class;

merge class1_&seed class2_&seed; by obs;

Run;

set

%END;

%MEND;

%RRRUN;

[art297]

Okay, it is beginning to look and sound less like a homework problem.  Thus, a couple of points:

1. you don't need to create the variable obs in each of the files.

2. you don't need and output statement in either datastep.  All of the records will be output, with or without the statement.

3.  If you need to scan for the second column, don't scan for the last column.  i.e., instead of using:

file2_&seed=input(scan(_infile_,1,,'bs'),best12.);

use

file2_&seed=input(scan(_infile_,2,,'s'),best12.);

4.Don't use:

data class;

merge class1_&seed class2_&seed; by obs;

Run;

You only want to merge the each line of the one file with its corresponding line from the other file.  Just use:

data class;

  set class1_&seed;

  set class2_&seed;

Run;