Thank you so much! ... View more

Could you provide more details on how to do this? I deeply appreciate your kind help if you could offer a sample code. Thank you! ... View more

I have an additional question on this issue. Since my question is the same for everyone in the chat, I will only post it in a response to @Tom. My question is: is it possible to import the CSV files in a parallel instead of a sequential manner? My experience in importing such large files is that the importation process of even a single file takes a lot of time. Therefore, "parallel importation", a word I coined from "parallel computing", is preferred. I wonder if SAS can realize this. ... View more

Thank you so much for your very informative and helpful reply! So my question can now be resolved by a macro. ... View more

Is it possible, then, to specify the starting and ending row of the .csv.gz file and let SAS read in the designated subset of data only? ... View more

Thank you for your patient illustration! Could you please tell me where to specify the COMPRESS=YES option? ... View more

Because loading in the entire dataset is too large. I understand that the importation process might not need all of the file to be loaded into memory in SAS, but the question is the resultant imported dataset is too large to be stored in memory as well. ... View more

Thank you! I consulted Deepseek on resolving this issue in R and it provided a "flowing decompression" method of dealing with this problem. In short, batches of observations are decompressed, imported, selected and stored. When one cycle finishes, a second batch is decompressed while the first batch of decompressed file is deleted, and so on. The stored observations, which is what we finally want yet is distributed as multiple small datasets for the time being, is stacked to form a large one. Can SAS do something like this? ... View more

So the selection process still takes place after the entire importation is done, right? ... View more

@Tom wrote: @Season wrote: @Tom wrote: @Season wrote: It is possible to read it in R. The compressed file is about 30G, with some 40 variables. What guesses did R make for how to define the 40 variables?  You could use those to help you write the code. A function is applied before importation of the whole compressed file to read its first row, which stores the names of variables. No such functions seem available in SAS, are there? Note the NAME is the only metadata that a CSV file does have.  What it does not have is any information about what types of variables those names represent.  Are they character strings? If so how long. Are the numbers? Perhaps dates?   That is where the guessing needs to happen. Thank you for your reminder! ... View more

Thank you so much! That saves the time of manually typing the variable names in the DATA step. ... View more

@Tom wrote: @Season wrote: It is possible to read it in R. The compressed file is about 30G, with some 40 variables. What guesses did R make for how to define the 40 variables?  You could use those to help you write the code. A function is applied before importation of the whole compressed file to read its first row, which stores the names of variables. No such functions seem available in SAS, are there? ... View more

It is possible to read it in R. The compressed file is about 30G, with some 40 variables. ... View more

Thank you so much for your solution! My questions are the same for both of you, so I will reply to one of the duo: (1) Does it mean that in importing .csv.gz files, we have to specify the features of all variables manually? (2) Is it possible to filter the observations in the course of importation instead of doing so post hoc, which is the usual practice? Can columns and rows both undergo filtering in the importation process? ... View more