Data extraction from a particular library on server.

[Visal_Chourasia]

I am trying extract a very large dataset (more than a billion records) from a particular library to my local library. but it is giving me memory and timeout error. I tried to extract it by creating small chunks also but still it is giving me the same error. 
I would appreciate if someone provides me any solution to load this data to my local library efficiently. 

Thanks

[AlanC]

Since SAS datasets are also ODBC compliant, try pulling them vs pushing them. I am not sure whether SAS would still have an issue but a simple Python, C#, PowerShell apparatus may work. You could also just try a binary copy using robocopy (assuming Windows) and set up timeout parameters. PowerShell is also highly efficient.

At the heart of it, the sas7bdat is just a simple file. Don't try to read and export, but just copy. If you need it in a different form, consider the ODBC route. You don't indicate what your final form should be so hard to tell the needs. 

[Kurt_Bremser]

Define "local". Does it mean you want to download the dataset to your desktop PC, or do you want to copy it to a library of yours on the SAS server?

Also post the log from your failed code; include the LIBNAME for your "local" library.

[LinusH]

The fact "more than a billions of records" should trigger you to rethink your use case.

Try to use the data where it resides, and only extract subsets/aggregates.

This can be achieved using SQL pass through (explicit and/or implicit).

Or if the dataset is a SAS dataset, use SAS/CONNECT or other methods to use it directly.