topic Re: Converting very large sd2 datasets to sas7bdat in SAS Data Management

Converting very large sd2 datasets to sas7bdat

williamty — Thu, 06 Jul 2023 14:52:02 GMT

I have a very large sd2 dataset file which is about 42GB. I can only load it into SAS by a script which qualifies certain variables. I cannot read the dataset structure nor open it in SAS universal viewer. I have tried all the methods I can find on the Internet and forums, including but not limited to using old v6. For example, all of the following scripts don't work:

libname in v6 'path';
libname out v9 'path';
proc copy inlib=in outlib=out;
run;

libname new 'path';
libname old v6 'path';
data new.test;
set old.test; 
run;

libname out v9 'path';                                                                                                        
libname in v6 'path';                                                                                                               
                                                                                                                                        
data in.test;                                                                                                                       
  infile 'path';                                                                                                       
  input;                                                                                                                                
run;

I think the main error is that: Write access to member 'IN.TEST.DATA' is denied. I don't know why. How can i convert the sd2 file to sas7bdat?

Re: Converting very large sd2 datasets to sas7bdat

Kurt_Bremser — Thu, 06 Jul 2023 16:45:08 GMT

Avoid having two LIBNAMEs pointing to the same path. You can't have test.sd2 and test.sas7bdat in the same directory, SAS does not allow this.

Re: Converting very large sd2 datasets to sas7bdat

Tom — Thu, 06 Jul 2023 16:55:38 GMT

SAS does not have any trouble with a directory having mixed version 6 and version 7/8/9 format files in the same directory.

But you do need to tell it which engine to use for which libref. Otherwise it will pick one for you (and I am not sure what logic it uses to pick).

libname in v6 'path';
libname out v8 'path';

@Kurt_Bremser wrote:

Avoid having two LIBNAMEs pointing to the same path. You can't have test.sd2 and test.sas7bdat in the same directory, SAS does not allow this.

Re: Converting very large sd2 datasets to sas7bdat

Tom — Thu, 06 Jul 2023 17:03:15 GMT

Your first step looks right. You are copying the V6 dataset to a V9 dataset.

Your second step also looks right, but the libref NEW is WRONG (or at least potentially wrong or confusing). If you don't tell SAS what engine to use it will pick one based on the files in the directory. If the only SAS datasets that are there are V6 then it will pick V6.

Use BASE as the engine and SAS will use the default engine for whatever version of SAS you are running.

libname new base 'path';

The last one is flat out wrong. You cannot write a V6 dataset unless you are running SAS 8 or earlier. INFILE is for reading TEXT files (or reading binary files as text). You cannot use it to read a SAS dataset or a directory.

Re: Converting very large sd2 datasets to sas7bdat

Al_CUA — Fri, 17 Nov 2023 06:03:18 GMT

Did you try this?
SAS Help Center: Overview: PROC MIGRATE

Re: Converting very large sd2 datasets to sas7bdat

yabwon — Fri, 17 Nov 2023 07:09:04 GMT

Hi @williamty

1) can you share log from the 1st and 2nd steps.

2) and you are working on Windows, right? (e.g. Linux does not have v6 engine)

Bart