topic Re: SAS dataset variable values cleansing in SAS Programming

SAS dataset variable values cleansing

jayakumarmm — Fri, 23 Sep 2016 19:59:16 GMT

Hi,

I am trying to import data from excel into SAS dataset, I am facing issues with data contains double quotes and ampersand.

Code used:

libname xlsFile XLSX "/path/monthly.xlsm";
options validvarname=v7;
options SYMBOLGEN MPRINT;


PROC SQL;
    create table  work.raw_data  as 
	(select * from xlsFile.datal);
quit;

Excel data:

1. "Online system" and "Mobile data"

2, Online system & Mobile data

SAS dataset:

1. "Online system" and "Mobile data"

2, Online system & Mobile data

Expected data:

1. Online system and Mobile data

2, Online system & Mobile data

Re: SAS dataset variable values cleansing

Reeza — Fri, 23 Sep 2016 19:58:52 GMT

How are you importing your data?

Re: SAS dataset variable values cleansing

jayakumarmm — Fri, 23 Sep 2016 20:00:22 GMT

Hi,

Given below is the code

Code used:

libname xlsFile XLSX "/path/monthly.xlsm";
options validvarname=v7;
options SYMBOLGEN MPRINT;


PROC SQL;
    create table  work.raw_data  as 
	(select * from xlsFile.datal);
quit;

Re: SAS dataset variable values cleansing

Reeza — Fri, 23 Sep 2016 20:10:40 GMT

You can use dequote() to strip quotes or compress() to remove them from the text.

If the text has quotes in Excel it will in SAS and that seems the correct behaviour to me. I also don't get the & converted to HTML (amp) so I think there's something else behind the data in Excel? Or the forum changed the value?

I'm using SAS 9.4 TS1M3 and Excel 2010

I get the following, which is exactly what I'd expect.

"Online system" and "Mobile data"

Online system & Mobile data

Re: SAS dataset variable values cleansing

jayakumarmm — Fri, 23 Sep 2016 20:41:13 GMT

Forum is displaying the same & values which I have updated.

Re: SAS dataset variable values cleansing

Reeza — Fri, 23 Sep 2016 20:58:21 GMT

To remove the quotes use compress() on the field.

Re: SAS dataset variable values cleansing

Ksharp — Sat, 24 Sep 2016 03:02:42 GMT


1) try to use PROC IMPORT .

2)
data have;
a=' "Online system" and "Mobile data" '; b=htmldecode(a);output;
a='  Online system & Mobile data  '; b=htmldecode(a);output;
run;

Re: SAS dataset variable values cleansing

Tom — Sat, 24 Sep 2016 13:27:00 GMT

I cannot recreate your problem. When I create an Excel spreadsheet with those values SAS reads them in the same as they are in Excel. What version of SAS and Excel are you using? Why are you using an XLSM file instead of an XLSX file?

Now if you want to remove the quotes from the middle of you string your best option is to just strip them out using the COMPRESS() function.

VAR1 = compress(VAR1,'"');

And if you really do want to translate HTML codes like &AMP; back into the characters they represent then use HTMLDECODE() function.

VAR1 = htmldecode(VAR1);