Whenever I convert SAS to csv, I noticed that it would output extra quotation marks if a value contains quotation marks.
For example:
Original value:
hello "M" world
Output to CSV value:
"hello""M""world"
Here is my code to convert from SAS to CSV. I expect it has something to do with my put statement?
%Let outfile = '\path\to\save\csv\table.csv' DLM = '|' DSD;
DATA _NULL_;
SET library.table;
FILE &outfile;
put (_ALL_) (:);
RUN;
Any help would be great, Thanks!
@jim_toby wrote:
@Tom I want to convert it from SAS to csv in order to load it to netezza. If it adds extra quotations, it will appear with extra quotations in Netezza. So I would like for it to not manipulate any of the values. I don't have plans to convert it back to the source data since I'm not actually manipulating the SAS table but rather creating a new csv file.
So you need to ask Netezza what rules it uses for parsing text files. A number of databases I have used behave as if they were C code or Unix shell commands and expect to see backslashes in the data to "escape" special characters.
But most of them also have options so that they can handle normal CSV files (as defined by the standard that @data_null__ provided the link for). For example if you are using the Redshift COPY command you can add the CSV option to have it handle quoted strings properly.
@jim_toby wrote:
Whenever I convert SAS to csv, I noticed that it would output extra quotation marks if a value contains quotation marks.
For example:
Original value:
hello "M" world
Output to CSV value:
"hello""M""world"
Here is my code to convert from SAS to CSV. I expect it has something to do with my put statement?
%Let outfile = '\path\to\save\csv\table.csv' DLM = '|' DSD; DATA _NULL_; SET library.table; FILE &outfile; put (_ALL_) (:); RUN;
Any help would be great, Thanks!
First thing CSV = Comma Separated Values. If you use DLM='|' or anything except a comma the file is not csv. This does have serious potential for causing problems with programs that expect a CSV file to have commas.
I get different results:
Hello,"""M""",world
Since your post of the example output does not include any delimiter than I have to assume the post is from a program that modified the result and so the quotes around the words hello and world were placed by that other program.
Note that when checking the output of delimited file do not open them with anything except a plain text editor such as NotePad or even the SAS editor. Spreadsheets are almost always going to change something, especially if you then save from the spreadsheet.
One might ask why you thought you needed quotes around a variable to begin with?
Very simple to remove quotes with the COMPRESS function.
@ballardw my example was just one value surrounded by pipes.
So it would look like: |hello "M" world|....|.......
And I did not construct this table so I don't want to change any of the data inside of it. Therefore I would like to leave the quotes
https://en.wikipedia.org/wiki/Comma-separated_values
"1997","Ford","E350"
1997,Ford,E350,"Super, luxurious truck"
1997,Ford,E350,"Super, ""luxurious"" truck"
See this issue: https://communities.sas.com/t5/SAS-Procedures/Proc-Export-adding-extra-quotation-marks/td-p/90021
He was able to solve it by doing this:
DATA _null_;
SET SASHelp.table;
FILE &outfile;
put (_ALL_) ('|');
RUN;
So this would indeed get rid of the extra quotations but it also adds the pipe delimeter at the start of every row. Main point is, is that it doesn't add the extra quotations.
I don't see any extra quotes. Just the required quotes needed to make sure that the resulting file can actually be converted back to the source data.
@Tom I want to convert it from SAS to csv in order to load it to netezza. If it adds extra quotations, it will appear with extra quotations in Netezza. So I would like for it to not manipulate any of the values. I don't have plans to convert it back to the source data since I'm not actually manipulating the SAS table but rather creating a new csv file.
@jim_toby wrote:
@Tom I want to convert it from SAS to csv in order to load it to netezza. If it adds extra quotations, it will appear with extra quotations in Netezza. So I would like for it to not manipulate any of the values. I don't have plans to convert it back to the source data since I'm not actually manipulating the SAS table but rather creating a new csv file.
So you need to ask Netezza what rules it uses for parsing text files. A number of databases I have used behave as if they were C code or Unix shell commands and expect to see backslashes in the data to "escape" special characters.
But most of them also have options so that they can handle normal CSV files (as defined by the standard that @data_null__ provided the link for). For example if you are using the Redshift COPY command you can add the CSV option to have it handle quoted strings properly.
@Tom Sounds good, I'll do that. Thanks!
@Reeza I want to convert it from SAS to csv in order to load it to netezza. If it adds extra quotations, it will appear with extra quotations in Netezza. So I would like for it to not manipulate any of the values.
Except for the CSV standard (which you're using pipes for some reason) requires double quotes. Most apps reading would expect it as well. Is Netezza adding the extra quotes?
For example, I tried putting your text into Excel and then saved it as a CSV, it looks like this:
"hello ""M"" world"
And then when I read it back in, it's fine. I'd be really really surprised if Netezza didn't do the same thing. Any particular reason you're not uploading straight to Netezza? Assuming no connection for some reason.
@jim_toby wrote:
@Reeza I want to convert it from SAS to csv in order to load it to netezza. If it adds extra quotations, it will appear with extra quotations in Netezza. So I would like for it to not manipulate any of the values.
@Reeza I'm not loading the file straight from SAS because the table contains over 500 million records and I get a communication link error when I do a bulkload due to the inactivity timeout limit
One thing to try is to not use the DSD option on the FILE statement.
This will mean that you are not protected from embedded delimiters or unbalanced quotes. So you need to check your data before writing it.
It also will means that missing (aka NULL) values will be represented as a single character instead of by adjacent delimiters. So you need check to how Netezza will lines like this:
value1| |value3
instead of like this:
value1||value3
A lot of databases will allow you to include the work "null" (or perhaps some other character string) to indicate missing values. If so you might be able to use user defined formats to fix that.
data have;
length yr 8 make model $20 description $100 ;
yr=1997;
make='Ford';
model='E350';
do description = ' ',"Super, luxurious truck","Super, ""luxurious"" truck" ;
output;
end;
call missing(yr);
output;
run;
proc format ;
value null low-high=[best32.] other='null';
value $null ' '='null' other=[$200.];
run;
filename csv temp;
data _null_;
file csv dlm='|' ;
set have;
format _numeric_ null. _character_ $null. ;
put (_all_) (:) ;
run;
data _null_;
infile csv ;
input;
list;
run;
RULE: ----+----1----+----2----+----3----+----4----+----5----+----6----+----7 1 1997|Ford|E350|null 19 2 1997|Ford|E350|Super, luxurious truck 37 3 1997|Ford|E350|Super, "luxurious" truck 39 4 null|Ford|E350|Super, "luxurious" truck 39
Did you try PROC EXPORT or macro %DS2CSV() ?
Join us for SAS Innovate 2025, our biggest and most exciting global event of the year, in Orlando, FL, from May 6-9. Sign up by March 14 for just $795.
Need to connect to databases in SAS Viya? SAS’ David Ghan shows you two methods – via SAS/ACCESS LIBNAME and SAS Data Connector SASLIBS – in this video.
Find more tutorials on the SAS Users YouTube channel.