Desktop productivity for business analysts and programmers

How to import all observations from an one record infile (@@ is not enougth)

Accepted Solution Solved
Reply
New Contributor
Posts: 3
Accepted Solution

How to import all observations from an one record infile (@@ is not enougth)

Hi,

I have a txt file with all my observation in just one record, similiar to this:

[{"Var1":"a1234","Var2":"a567","var3":0,"var4":0},{"Var1":"b1234","Var2":"b567","var3":1,"var4":0},{"Var1":"c1234","Var2":"c567","var3":0,"var4":0}, ..............................................]

I'm trying to import its content to a dataset. But even using double trailing I just end up with 2 observations (the txt file has something about 20,000 observations). This is the code:

data dataset1;

  infile '/local/arquivo_1.txt' dlm='{[,:"} ;';

  input var1 $ Var1_content : $9. var2 : $12. Var2_content : $12. var3 : $16.  Var3_content $ var4 : $12. Var4_content $ @@;

run;


Accepted Solutions
Solution
‎03-17-2015 04:59 PM
Super User
Super User
Posts: 6,845

Re: How to import all observations from an one record infile (@@ is not enougth)

Add a LRECL option to the INFILE statement.  You should be able to set it to a billion (1 with 9 zeros).

If that doesn't work then read the file using STREAM input and convert all of the '}' to '0A'x and then the line lengths will be shorter.

View solution in original post


All Replies
Solution
‎03-17-2015 04:59 PM
Super User
Super User
Posts: 6,845

Re: How to import all observations from an one record infile (@@ is not enougth)

Add a LRECL option to the INFILE statement.  You should be able to set it to a billion (1 with 9 zeros).

If that doesn't work then read the file using STREAM input and convert all of the '}' to '0A'x and then the line lengths will be shorter.

New Contributor
Posts: 3

Re: How to import all observations from an one record infile (@@ is not enougth)

Thank you very much Tom! It worked perfectly

Super User
Posts: 11,134

Re: How to import all observations from an one record infile (@@ is not enougth)

If this was a one-time project I would be very tempted to use and editor to replace the string },

with }<new line> where new line is the appropriate one for your OS.

Valued Guide
Posts: 2,177

Re: How to import all observations from an one record infile (@@ is not enougth)

data dataset1;

  infile '/local/arquivo_1.txt' dlm='{[,:"} ;';

length varn varn_conts $20 ;

Do case = 1 by 1 ;

  input varn Varn_conts @ ;

  If varn = " " then input ;

  Else output ;

End ;

stop ;

run;

Super User
Posts: 19,157

Re: How to import all observations from an one record infile (@@ is not enougth)

Is the original file JSON?

Super User
Posts: 9,867

Re: How to import all observations from an one record infile (@@ is not enougth)

It is JSON file which is a stream file , You need to tell SAS it is a stream file by defining recfm=n ;

data dataset1;

  infile '/local/arquivo_1.txt' dlm='{[,:"} ;'    recfm=n   ;

  input var1 $ Var1_content : $9. var2 : $12. Var2_content : $12. var3 : $16.  Var3_content $ var4 : $12. Var4_content $ @@;

run;

Xia Keshan

🔒 This topic is solved and locked.

Need further help from the community? Please ask a new question.

Discussion stats
  • 6 replies
  • 435 views
  • 5 likes
  • 6 in conversation