topic Batch Download from Oracle in ODS and Base Reporting

Batch Download from Oracle

ImPrasad — Sat, 25 Jul 2015 05:41:41 GMT

Hello Everyone,

Being Data Scientist,One of the prominent challenge is to extract data from server to local machine.Is there any way out to extract data from oracle server more speedy than traditional Data;Set;Run; Process.

Majority data extraction for us are blanket extraction i.e. without applying any filter.Is there any bulk download process which wont read every line of the data and download all the files in batch mode.

Ragards,

Prasad Devasthali

HDFC Bank

Re: Batch Download from Oracle

Ksharp — Sat, 25 Jul 2015 11:25:40 GMT

Add option readbuff=100000 into your libname statement will fast your speed .

Re: Batch Download from Oracle

ImPrasad — Fri, 07 Aug 2015 11:49:17 GMT

Thanks Xia,this option really helped me a lot to reduce TAT.However I am unable to increase value more than 32000 on my local system against ReadBuff value.My sas version is 9.3 and I am operating it through 32 Bit processor

Re: Batch Download from Oracle

Ksharp — Fri, 07 Aug 2015 12:14:34 GMT

Sorry. Maybe you should talk to SAS Support .

OR post your Q at Administrator Forum, there are more experts about DB in there .

Re: Batch Download from Oracle

jakarman — Fri, 07 Aug 2015 12:37:10 GMT

Optimizing run time is decreasing the technical overhead. Read buffers is mostly sufficient There should be no logging and no locking when reading (writing is different).

You local system could be a bottleneck (SSD?) as is the network transfer rate. Optimizing buffering bufsize aligniofiles could help. SAS/ACCESS(R) 9.2 for Relational Databases: Reference, Fourth Edition Xia give you that one already
Mostly it is adiviced to work "federated" leave data as much as possible at his source, avoiding copies. With mining that is not really possible.
As of Views and other technical stuff in Oracle it could be Oracle is not faster and being the limitation. Ask your oracle DBA for that area.