BookmarkSubscribeRSS Feed
deleted_user
Not applicable
hi
my question is ,now to pull huge data using sas programming.
4 REPLIES 4
Cynthia_sas
SAS Super FREQ
Hi:
This is not a simple question. You need to know how your data will be accessed for INPUT, what kind of processing will be performed, what kind of transformations (if any) have to take place and what kinds of OUTPUT (either output data or output report) need to be produced.

For example, is your data in "flat files" or ASCII text files? Is your data in Oracle or DB2 tables? Is your data in SQL server files? Does the data live on the same platform/operating system as SAS? Are you using a DATA step? Are you using PROC SQL? Is SAS/Access to Oracle or SAS/Access to DB2 involved? Is your data in Teradata or SAP or SPD Server? What is "huge"? A million rows? Or is it only 200,000 rows, but very wide?? Or is the data both very wide (lots of variables) and very long (lots of rows)?

There's no simple answer to this question, because the answer is VERY specific to your configuration, where your INPUT data lives, the limitations of the operating system, the limitations of the network, the kind of processing you need to do, the kind of output you need to produce, etc, etc. You will always have to live within constraints
-- operating system constraints: like CPU cycles and I/O operations;
-- space constraints: physical storage constraints and work area constraints;
-- time constraints: time to load, time to process, time to transmit across a network; and
-- manpower constraints: time to maintain and or recode programs.

If you optimize for one constraint, you could end up having a negative impact on another constraint.

If you go to Google and enter:
SAS accessing large data
or
SAS efficient access data
in the search box, you will find some hits for SUGI papers that talk about efficiencies for accessing large data files. This would be a good place to start your investigations.

cynthia
deleted_user
Not applicable
How huge is your SAS datasets in clinical trials?

Generally there's no huge data in clinical trials! The data structure is complex or messy, but definitely not huge...

A standard Phase III study, 100 patient, 300 pages each patient, 20 data points on every page...
deleted_user
Not applicable
I have a similar question. I am supposed to extract data from eDataFax using SAS. Any suggestions? I never use eDataFax before. Thanks a bunch!
Cynthia_sas
SAS Super FREQ
Hi,
Apparently, this is a topic that DataFax discusses in their documentation and other users appear to have written papers about -- a quick Google search revealed these links:
SAS-L Posting: http://listserv.uga.edu/cgi-bin/wa?A2=ind0403a&L=sas-l&F=&S=&P=8582 message recommends reading "DataFax Programmer Guide, Chapter 10 DFsas: DataFax to SAS. The chapter is about 30 pages long".

http://www.datafax.com/UserGroup/2002/presentations/Harris.pdf
http://www.datafax.com/UserGroup/2006/presentations/richardson.pdf
http://www.datafax.com/UserGroup/2007/presentations/ZO.pdf

http://www.basas.com/pages/baysas.htm presentation "Single Step Processing, DataFax to SAS Datasets in a Windows Environment (Simplifying Your End User Interface)", by Brad Harris


cynthia

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

New Learning Events in April

 

Join us for two new fee-based courses: Administrative Healthcare Data and SAS via Live Web Monday-Thursday, April 24-27 from 1:00 to 4:30 PM ET each day. And Administrative Healthcare Data and SAS: Hands-On Programming Workshop via Live Web on Friday, April 28 from 9:00 AM to 5:00 PM ET.

LEARN MORE

Discussion stats
  • 4 replies
  • 1089 views
  • 0 likes
  • 2 in conversation