topic Re: Dataflux - Reading Strings as Integer fields in SAS Data Management

Dataflux - Reading Strings as Integer fields

LuisLeiteBR — Fri, 25 Nov 2016 13:00:18 GMT

Hello Everybody!

I´m using Dataflux DMStudio 2.6 to read those values below:

"asString";"asInteger"
"A001127209";"A001127209"
"000379043B";"000379043B"
"00085351C6";"00085351C6"
"0014479443";"0014479443"

Case A. Reading from a txt file containing that lines:

I read the first column (Field Name asString) as an STRING(10) field type, and the second column as an INTEGER(10) field type.

In that case, when I preview the data the results are:

1 - Generates a "0" result;

2 - Preserve only the left significative numeric characters (379043),

3 - Preserve only the left significative numeric characters (85351),

4 - Preserve only the left significative numeric characters, in that case, the real value.

Case B. Reading as a Job Specific Data:

In that case, when I preview the data the results are:

1 - Generates a "null" result;

2 - Preserve only the left significative numeric characters (379043),

3 - Preserve only the left significative numeric characters (85351),

4 - Preserve only the left significative numeric characters, in that case, the real value.

So, I´d like to understand:

a) Why Dataflux accepts an alphanumeric value (even part of it) as a input of an Integer field?
b) Why not treat all alphanumeric strings as "null" results?
c) Why Dataflux treats the first string in different ways when reading a file and for Job Specific Data ( "0" x "null" )?
d) Where can I find documentation for content transformation between different field types?

I welcome any information that helps to clarify this situation.

Thank you all.

Luís Leite

Re: Dataflux - Reading Strings as Integer fields

andrew_s — Tue, 29 Nov 2016 14:27:39 GMT

Luis,what you have identified is unfortunately some disparate behaviour between these two components. It would be good if they behaved the same, but unfortunately they do not. The text file node will read until the first non-numeric character, and the job specific data will issue null if it finds a non-numeric value. It's possible that this will be fixed in a future version.

Re: Dataflux - Reading Strings as Integer fields

LuisLeiteBR — Tue, 29 Nov 2016 19:27:02 GMT

Ok Andrew!!

Would you know where can I find any kind of documentation about this flield type transformation?
i.e. between Integer, Strings, Real, Boolean, etc...

Thank´s a lot!

Re: Dataflux - Reading Strings as Integer fields

andrew_s — Tue, 29 Nov 2016 20:04:46 GMT

Luis, the documentation does not appear to be too detailed in this regard. This is some information from the code which generally applies to these types of transformations (and as you have noted, there seems to be an exception in the Job Specific Data node):
String -> integer - Read the first characters until a non-numeric value is found. The integer is the digits read.
String-> Real - Same as integer, but allows a decimal point (Which must be a '.' Character)
String -> Date - String is parsed, and the components are analyzed. If there is ambiguity between month and day, it will either use what you specify (ie in text input or job specific data node) or it will use your locale setting.
String -> Boolean - Will look for values true, t, yes, y, on to represent true and false, f, no, n, off to represent false

Re: Dataflux - Reading Strings as Integer fields

LuisLeiteBR — Tue, 06 Dec 2016 19:41:48 GMT

Thank´s Andrew!!!
It´s never too late ...