About Autotelic

Autotelic · ‎11-06-2017

Great! This worked and it gives us a clue.. The code below output not two, but three lines. The header's row first, the problematic data line last and inbetween it also output the line just before the problematic line. Data _null_; Infile "file_path_name" LRECL=32767 ENCODING="UTF-16" TERMSTR=CRLF; File "other_file_path_name" LRECL=32767 ENCODING="UTF-16" TERMSTR=CRLF; Input; If _n_ = 1 | _n_ = 10524 then put _infile_; Run; The importation of "other_file_path_name" worked fine too.

Autotelic · ‎11-06-2017

That code doesn't seem to write anything. I experimented with different values of _n_ ans always got zero results. Any idea of what's going on here? NOTE: 16809 records were read from the infile "C:\Users\10173847.GRUPOECI\Documents\My SAS Files\Performance\Inputs MicroStrategy\Performance_RT_20171102.csv". The minimum record length was 260. The maximum record length was 572. NOTE: 0 records were written to the file "C:\Users\10173847.GRUPOECI\Documents\My SAS Files\Performance\test.csv".

Autotelic · ‎11-04-2017

This thread is getting long. Apologies for this unrelated post, but I want to thank everyone that has been trying to help: thank you very much! I'm going to keep working on this issue.

Autotelic · ‎11-04-2017

Thanks, I will only be able to try this in about 40 hours. Actually I expect 16809 records, that's because the file ends with a blank line, so there are as many CRLFs as there are datalines. The character that can't be transcoded is unrelated to this, IMO. I actually asked about this here: https://communities.sas.com/t5/SAS-Enterprise-Guide/Importing-text-file-with-Unicode-U-0099/m-p/409930#M26273

Autotelic · ‎11-04-2017

No idea. I'm way out of my element here, trying to convey as much useful information as I can. When I import the file manually, SAS is able to do it correctly, but it seems to preprocess the file first as the file in the infile statement is a .txt and it uses the enconding WLATIN1. If I try to import using this encoding I get an error and it says that 'A byte-order mark indicates that the data is encoded in "utf-16le"'. How would I go about finding that 2-byte character that might be in the file? I can't find anything out of the ordinary when I open the file in Notepad++, nor on Notepad for that matter. Plus, when I import the file in pandas (with the encoding UTF-16), it imports it fine as well. I'm at a loss.

Autotelic · ‎11-04-2017

I'm not sure, but I don't think so, I checked the "extra" lines specifically and they all end with LF and continue in the next line. I also checked and whenever it ends with LF, the data for that line isn't over yet, it carries onto to the next line and when it reaches the end of the data, I find a CRLF. Everything seems to be working fine with these lines. What I'm asking about, as far as I can tell, is unrelated to this since every line the vicinity of the problematic line actually ends with CRLF.

Autotelic · ‎11-04-2017

Notepad++ sees as many lines as SAS reports after I run the first of the snippets above: NOTE: 16813 records were read from the infile However this number doesn't correspond to the number of CRLFs in the file. I expect to have 16809 observations (as many CRLFs as there are in the file).

Autotelic · ‎11-04-2017

But at the point a line ends, it's supposed to be a "CRLF" and that's what I see in Notepad++. Am I misinterpreting something here?

Autotelic · ‎11-04-2017

How do you identify the end of line mark? I can't find it. But, judging by the log and my knowledge of the data, it should be the two consecutive full stops in the line above "ZONE 4555454422222222222222466646667677222333323323322232323333222323328222323328222322232002333333333333". No, I'm not sure about the encoding. I can't remember at the moment (and also can't check), but I think I copied the encoding from what SAS EG chose when I manually imported the file. Also, Notepad++ seems to support this. Is there a way to get an "official" answer to what the encoding is?

Autotelic · ‎11-03-2017

I added the code. The vertical purple line isn't a character, it's the spot where I placed the cursor. Thanks!

Autotelic · ‎11-03-2017

Thanks, David. I understood everything. Is there a way to make it so that this specific error, specifically for this character, doesn't yield a warning?

Autotelic · ‎11-03-2017

I'm on SAS Enterprise Guide 7.11, SAS 9.4. When trying to import a csv file, there's a line that disappears. It is the one mentioned in the error below. (See block quote of this at the bottom of this question). I've opened the file in Notepad++, but can't find anything wrong with the missing line. Here's a screen shot placed at the missing line at column 283 (where SAS claims the error begins): The code I'm using is Data Want; Length 'Nº de Operación de Venta C&C'n $ 29 "Estado Operación"n $ 46 "Código de Pedido"n 8 "Estado del Pedido"n $ 46 "Fecha de Pedido"n 8 "Hora de Pedido"n 8 "Referencia ECI"n $ 18 "Descripción Referencia ECI"n $ 40 "División"n $ 5 "Descripción División"n $ 20 "Subdivisión"n $ 7 "Descripción Subdivisión"n $ 20 "Método de Envío"n $ 16 "Fecha de Compromiso Original"n 8 "Fecha de Compromiso"n 8 "Importe Original"n $ 10 Importe $ 10 Unidades 8 "Unidades Pedidas"n 8; Format 'Nº de Operación de Venta C&C'n $CHAR29. "Estado Operación"n $CHAR46. "Código de Pedido"n BEST13. "Estado del Pedido"n $CHAR46. "Fecha de Pedido"n Date9. "Hora de Pedido"n TIME8. "Referencia ECI"n $CHAR18. "Descripción Referencia ECI"n $CHAR40. "División"n $CHAR5. "Descripción División"n $CHAR20. "Subdivisión"n $CHAR7. "Descripción Subdivisión"n $CHAR20. "Método de Envío"n $CHAR16. "Fecha de Compromiso Original"n DATE9. "Fecha de Compromiso"n DATE9. "Importe Original"n $CHAR10. Importe $CHAR10. Unidades Comma12. "Unidades Pedidas"n Comma12.; Informat 'Nº de Operación de Venta C&C'n $CHAR29. "Estado Operación"n $CHAR46. "Código de Pedido"n BEST13. "Estado del Pedido"n $CHAR46. "Fecha de Pedido"n Date9. "Hora de Pedido"n TIME8. "Referencia ECI"n $CHAR18. "Descripción Referencia ECI"n $CHAR40. "División"n $CHAR5. "Descripción División"n $CHAR20. "Subdivisión"n $CHAR7. "Descripción Subdivisión"n $CHAR20. "Método de Envío"n $CHAR16. "Fecha de Compromiso Original"n DATE9. "Fecha de Compromiso"n DATE9. "Importe Original"n $CHAR10. Importe $CHAR10. Unidades Comma12. "Unidades Pedidas"n Comma12.; Infile "file_path_name" Firstobs=2 LRECL=600 ENCODING="UTF-16" TERMSTR=CRLF DLM="," MISSOVER DSD; Input 'Nº de Operación de Venta C&C'n : $CHAR29. "Estado Operación"n : $CHAR46. "Código de Pedido"n : BEST13. "Estado del Pedido"n : $CHAR46. "Fecha de Pedido"n : MMDDYY10. "Hora de Pedido"n : TIME5. "Referencia ECI"n : $CHAR18. "Descripción Referencia ECI"n : $CHAR40. "División"n : $CHAR5. "Descripción División"n : $CHAR20. "Subdivisión"n : $CHAR7. "Descripción Subdivisión"n : $CHAR20. "Método de Envío"n : $CHAR16. "Fecha de Compromiso Original"n : YYMMDD10. "Fecha de Compromiso"n : MMDDYY10. "Importe Original"n : $CHAR10. Importe : $CHAR10. Unidades : Comma12. "Unidades Pedidas"n : Comma12.; Run; NOTE: Invalid data for 'Unidades Pedidas'n in line 10524 283-318. REGLA: ----+----1----+----2----+----3----+----4----+----5----+----6----+----7----+----8----+----9----+----0 10524 "00401430761020520170823225656","Cancelled","1723549001754","Cancelled","8/23/2017","22:57 ","001002 101 535973537 ","SET PULSERAS 4 ","00102","COMPLEMENTOS ","0010258","B 201 ISUTERIA ","HomeDelivery","2017-09-07","9/7/2017","4.45 €","0.00 €","1","0".."004014307610 ZONE 4555454422222222222222466646667677222333323323322232323333222323328222323328222322232002333333333333 NUMR 93545291000000000002C28FD545C965292C22017D09D072C29F7F20172C24E45002C20E00002C212C202DA2004014307610 301 20520170823225656","Cancelled","1723549001754","Cancelled","8/23/2017","22:57 ","001016687233862042" 401 ,"CMST E/RE N/C COMB. LUREX ","00109","JOVEN ELLA ","0010919","MARCAS PRO.JOV 501 EN E ","HomeDelivery","2017-09-07","9/7/2017","2.99 €","0.00 €","1","0" 572 'Nº de Operación de Venta C&C'n=00401430761020520170823225656 'Estado Operación'n=Cancelled 'Código de Pedido'n=1723549001754 'Estado del Pedido'n=Cancelled 'Fecha de Pedido'n=23AUG2017 'Hora de Pedido'n=22:57:00 'Referencia ECI'n=001002535973537 'Descripción Referencia ECI'n=SET PULSERAS 4 'División'n=00102 'Descripción División'n=COMPLEMENTOS 'Subdivisión'n=0010258 'Descripción Subdivisión'n=BISUTERIA 'Método de Envío'n=HomeDelivery 'Fecha de Compromiso Original'n=07SEP2017 'Fecha de Compromiso'n=07SEP2017 'Importe Original'n=4.45 € Importe=0.00 € Unidades=1 'Unidades Pedidas'n=. _ERROR_=1 _N_=10523

Autotelic · ‎11-03-2017

Hi, David. It's wlatin1.

Autotelic · ‎11-02-2017

I'm on a local server. Alas, I do not have access to an administrator.

Autotelic · ‎11-02-2017

I'm on SAS Enterprise Guide 7.11 HF5 (7.100.1.2856) (64-bit), SAS 9.4. When importing a csv file, I get the warning "WARNING: A character that could not be transcoded has been replaced in record 1726." in some rows (the number at the end changes according to the row it is reading). I was able to determine that the character causing the problem is the Unicode Character U+0099. According to Notepad++, the fiile is enconded in "UCS-2 LE BOM" and I'mporting the file in SAS using the encoding "UTF-16". Can't SAS read this character? Am I not usin the correct encoding option in SAS? Am I doing something wrong? If I'm not, how can I get SAS to ignore this warning?

Online Status	Offline
Date Last Visited	‎09-07-2018 07:37 PM

Re: Connecting SAS to Jupyter kernell.

Connecting SAS to Jupyter kernell.

Re: How to chain format and type changes in Proc SQL?

How to chain format and type changes in Proc SQL?

Re: Autofit column width in proc print

Autofit column width in proc print

Re: Inserting datalines into file

Inserting datalines into file

Re: Error importing line in CSV (Invalid data for variable in line *...

Re: Error importing line in CSV (Invalid data for variable in line *...

Re: How to chain format and type changes in Proc SQL?

Re: How to chain format and type changes in Proc SQL?

Re: Error importing line in CSV (Invalid data for variable in line *...

Re: Error importing line in CSV (Invalid data for variable in line *...

Re: Error importing line in CSV (Invalid data for variable in line *...

Re: Error importing line in CSV (Invalid data for variable in line *...

Re: Error importing line in CSV (Invalid data for variable in line *...

Re: Error importing line in CSV (Invalid data for variable in line *...

Re: Error importing line in CSV (Invalid data for variable in line *...

Re: Error importing line in CSV (Invalid data for variable in line *...

Re: Error importing line in CSV (Invalid data for variable in line *...

Re: Error importing line in CSV (Invalid data for variable in line *...

Re: Error importing line in CSV (Invalid data for variable in line *...

Re: Error importing line in CSV (Invalid data for variable in line *...

Re: Error importing line in CSV (Invalid data for variable in line *...

Re: Importing text file with Unicode U+0099

Error importing line in CSV (Invalid data for variable in line *line...

Re: Importing text file with Unicode U+0099

Re: Importing text file with Unicode U+0099

Importing text file with Unicode U+0099