SAS Data Integration Studio, DataFlux Data Management Studio, SAS/ACCESS, SAS Data Loader for Hadoop and others

Dataflux - Reading Strings as Integer fields

Accepted Solution Solved
Reply
Occasional Contributor
Posts: 17
Accepted Solution

Dataflux - Reading Strings as Integer fields

[ Edited ]

Hello Everybody!

I´m using Dataflux DMStudio 2.6 to read those values below:

 

"asString";"asInteger"
"A001127209";"A001127209"
"000379043B";"000379043B"
"00085351C6";"00085351C6"
"0014479443";"0014479443"

 

Case A. Reading from a txt file containing that lines:

 

I read the first column (Field Name asString) as an STRING(10) field type, and the second column as an INTEGER(10) field type.

000.PNG 

 

In that case, when I preview the data the results are:

001.PNG

1 - Generates a "0" result;

2 - Preserve only the left significative numeric characters (379043),

3 - Preserve only the left significative numeric characters (85351),

4 - Preserve only the left significative numeric characters, in that case, the real value.

 

Case B. Reading as a Job Specific Data:

 

 

002.PNG

 

In that case, when I preview the data the results are:

 

004.PNG

 

1 - Generates a "null" result;

2 - Preserve only the left significative numeric characters (379043),

3 - Preserve only the left significative numeric characters (85351),

4 - Preserve only the left significative numeric characters, in that case, the real value.

 

So, I´d like to understand:

a) Why Dataflux accepts an alphanumeric value (even part of it) as a input of an Integer field? 
b) Why not treat all alphanumeric strings as "null" results?
c) Why Dataflux treats the first string in different ways when reading a file and for Job Specific Data ( "0" x "null" )?
d) Where can I find documentation for content transformation between different field types?

 

I welcome any information that helps to clarify this situation.

 

Thank you all.

Luís Leite


 


Accepted Solutions
Solution
‎12-06-2016 02:40 PM
SAS Employee
Posts: 5

Re: Dataflux - Reading Strings as Integer fields

Posted in reply to LuisLeiteBR
Luis, the documentation does not appear to be too detailed in this regard. This is some information from the code which generally applies to these types of transformations (and as you have noted, there seems to be an exception in the Job Specific Data node):
String -> integer - Read the first characters until a non-numeric value is found. The integer is the digits read.
String-> Real - Same as integer, but allows a decimal point (Which must be a '.' Character)
String -> Date - String is parsed, and the components are analyzed. If there is ambiguity between month and day, it will either use what you specify (ie in text input or job specific data node) or it will use your locale setting.
String -> Boolean - Will look for values true, t, yes, y, on to represent true and false, f, no, n, off to represent false

View solution in original post


All Replies
SAS Employee
Posts: 5

Re: Dataflux - Reading Strings as Integer fields

Posted in reply to LuisLeiteBR

Luis,what you have identified is unfortunately some disparate behaviour between these two components. It would be good if they behaved the same, but unfortunately they do not. The text file node will read until the first non-numeric character, and the job specific data will issue null if it finds a non-numeric value. It's possible that this will be fixed in a future version.

Occasional Contributor
Posts: 17

Re: Dataflux - Reading Strings as Integer fields

Ok Andrew!!

Would you know where can I find any kind of documentation about this flield type transformation?
i.e. between Integer, Strings, Real, Boolean, etc...

Thank´s a lot!

Solution
‎12-06-2016 02:40 PM
SAS Employee
Posts: 5

Re: Dataflux - Reading Strings as Integer fields

Posted in reply to LuisLeiteBR
Luis, the documentation does not appear to be too detailed in this regard. This is some information from the code which generally applies to these types of transformations (and as you have noted, there seems to be an exception in the Job Specific Data node):
String -> integer - Read the first characters until a non-numeric value is found. The integer is the digits read.
String-> Real - Same as integer, but allows a decimal point (Which must be a '.' Character)
String -> Date - String is parsed, and the components are analyzed. If there is ambiguity between month and day, it will either use what you specify (ie in text input or job specific data node) or it will use your locale setting.
String -> Boolean - Will look for values true, t, yes, y, on to represent true and false, f, no, n, off to represent false

Occasional Contributor
Posts: 17

Re: Dataflux - Reading Strings as Integer fields

Thank´s Andrew!!!
It´s never too late ...
☑ This topic is solved.

Need further help from the community? Please ask a new question.

Discussion stats
  • 4 replies
  • 251 views
  • 0 likes
  • 2 in conversation