DATA Step, Macro, Functions and more

Import raw data

Reply
Regular Contributor
Posts: 212

Import raw data

[ Edited ]

Hi,

 

Need help on importing raw file to SAS.

The example below shows how is the file:

 

 

011      30000092 10498100037500000000003000006274966080000018000  Condo                   180,00 Ha   05/11/2015 13:16  Ha
031      34300000 39900000000000000000000000000023800000000000000  Fat Porto               794,02 Ha   05/11/2015 14:05  Ha
031      34000003 39994000000000000000000000000024100000000000000  car porto
2.568,42 Ha   05/11/2015 14:45  Ha    010      34000000 00190000090142509602911765526170266030000074031  carro                      740,31 Ha   05/11/2015 07:46  Ha

 

If you see, the first five values of the fourth line, should be at the third line.

And the fourth line should begin in the 010.

 

The next textbox is how it should be:

 

031      34340033 39994000000000000000000000000024100000000000000  cartão porto             2.568,42 Ha   05/11/2015 14:45  Ha
010      34000000 00190000090142509602911765526170266030000074031  carro                      740,31 Ha   05/11/2015 07:46  Ha

 

How can i solve this when i import the file ?

 

Thanks in advance

Super User
Super User
Posts: 7,404

Re: Import raw data

TBH, I would go back to source and ask them to provide a decent file.  Should be part of the Import Agreement which you would have in place, that describing what data will be sent?  

 

Your examples don't seem to match either, I cant find 34349340 in the first, and several of the data items seem to be different between the two.  One option you could try is just to read the whole thing as $2000. then post process that line.  I note the very long number, 3rd element, wouldn't be easy to use also.  Myabe provide a datastep with some test data and required output.

Regular Contributor
Posts: 212

Re: Import raw data

I import every part of this file as strings.

The third part is a doc number and it is a string column.

It does make sence because this is how it is in my raw file.

This file is extracted with a robot.

 

The generated file is as shown before.

I need to import every part of this as strings, after all of this, i can change the formats, but what i need for now is a way to import when cases like this happens. And because some politicies they can't change the file to be generated in its right form.

 

Thanks

 

Respected Advisor
Posts: 3,894

Re: Import raw data

[ Edited ]

As it looks like: Your file structure gets messed up and a record gets distributed over more than one line. Or is this just how you're "looking" at your record?

If you're opening this text file in a decent editor like Notepad++: Is the "end of line" indicator at the end of the record or does this "vary"?

 

If you don't have a clean "end of line" indicator: What would identify a new record?

 

SAS is very flexible when it comes to reading text files; but we still need to determine the logic to be used to figure out when a new record starts.

 

Ideally: You have an interface contract where the record structure provided to you has been agreed upon. If that's not the case then you're in a lot of pain. You can probably code for the data "at hand" but then in the next data delivery something new happens and your code won't work. So here it's really about responsibility. Don't accecpt everything. Get an agreement from the data provider (=Interface contract).

Ask a Question
Discussion stats
  • 3 replies
  • 185 views
  • 0 likes
  • 3 in conversation