Yes Reeza, I see all the file names. In all from all these folders I have 1,102 .txt files to import. For the most part all the columns SHOULD be the same. I wished they would just get this into a database already. Last time I had to split it up. One person did our Medicare the other Medicaid. Different populations although there are duals. Took 3 months for them to complete the project. Now, Medicare still has the .txts but thankfully they built a datalab to hold the files and I did all this teradata coding and now the code runs in 1 hour. So from 3 months to 1 hour. Not too shabby. But Medicaid is HUGE. Our population is gigantic and they have yet to build a lab. So I have to use these .txt files and told them I sincerely hope nothing changes like column headers or datatypes because I am going to create SAS code to import all this stuff, then work it because it does have to be cleaned and if the next time we run (once a quarter is the ask) if they change one little thing the SAS automation portion that cleans and does all the manipulating will be thrown out of whack. It is the same with the lab. If they change something. Then I get oh, so it isn't truly automated??? I want to scream clueless people. The only way to automate and account for changes of incoming stuff is to create a brain that can say oh if the datatype is now this then do that. But if datatype stays the same do this. They don't get they are talking about 1,102 separate .txt files (as of now), and if it were as easy a balancing a checkbook then everyone would and could do it. thanks. I will look at the links.
... View more
I have SAS EG and everything is directed through our SAS Server.
I already found out that I cannot point my code to a shared drive within our organization that contains multiple zipped folders with multiple various .txt files and get all of what I need to import.
So, the way around it is to move all the zipped to my local drive. That took 5 hours. Then I unzipped everything and that took about 3 hours. Now I have 10 folders on my local hard drive and just starting with 1 of the folders. There are 288 .txt files in this particular folder. Some of them have an extension of flaggedevents.txt and the others details.txt
I want to import all of the .txt files that have the extension details.txt. Is there a way I can do this with SAS code??? The reason I want to use the details.txt is because the file names vary. Like adol_well_care_16_ch_details.txt and the next is adult_bmi_assmt_details.txt
... View more
Thanks. RW9, what if I write a libname statement to point to the Sharedrive so I can see the zipped files in the tree and then try extracting that way. Do you think that would work? I have all these people in my org saying just write a libname statement to point to the shared drive and I have tried and it does not find the path I am putting in.
... View more
I have SAS EG 7.1 on my machine. I have been given read access only to a shared drive. In this shared drive there are multiple folders. Within these multiple folders and multiple zipped files. I need these zipped files imported into my SAS EG which runs through a SAS Server that is Unix based. I have tried all sorts of unzipping or import codes and everything comes back stating I insufficient rights. I can move the files to my local machine and right now it is saying it will take 4 hours to move them. I have already tested the exact same code but on my local machine with one zipped and it works. I do not get the error. I believe the problem is I have read access only to this shared drive. Can anyone confirm if this is the case? If you must have write access in order to import zipped files to your SAS EG through a SAS Server.
... View more
I have had some great help on this board and Chris helped me with finding the right path to connect to a shared drive and finding the zipped files. I have 12 folders that have multiple zipped files and I was coding each zipped file as I will show in my code and each .txt file I need from the zipped file. But there are over 200 .txt files in this one zipped folder and I only need the files that have the extension detail.txt. The other files I can disregard. They are flagged.txt. Here is an example of what I have as code now and just want to find an easier way than going back and forth to the shared drive and typing in all these lengthy names:
[code]
filename fromzip zip '\\agpcorp\files\va1\Private\managedcaresvcs\health-qualitymgmt\hedis rates_datasets_benchmarks\hedis 2017\datasets\2016_03_mar hedis 2017_my2016\me1\hedis2017_mar2016_md medicaidchip.zip';
data _null_;
length memname $256;
input memname $;
infile fromzip memvar=memname end=eof;
eof=0;
do while(not eof);
input;
put memname= _infile_;
end;
datalines;
%_rye_mbr_benefit_16_rider16_detail.txt;
adh_med_schizo_16_saa16_detail.txt;
adhere_antipsy_16_am_saa16am_detail.txt;
adol_well_care_16_ch_awc16ch_detail.txt;
adolesc_well_care_16_awc16_detail.txt;
adult_bmi_16_am_aba16am_detail.txt;
adult_bmi_assmt_16_aba16_detail.txt;
adult_access_16_aap16_detail.txt;
ambulatory_care_16_amb16_detail.txt;
ambulatory_care_16_ch_amb16ch_detail.txt;
annual_dental_16_adv16_detail.txt;
annual_mon_rx_16_am_mpm16am_detail.txt;
annual_monitor_rx_16_mpm16_detail.txt;
antemmedmgmntasthma16_mma16wl_detail.txt;
antenatal_stds_16_am_ans16am_detail.txt;
antibiotic_util_16_abx16_detail.txt;
antidepr_meds_16_am_amm16am_detail.txt;
antidepress_meds_16_amm16_detail.txt;
anitpsy_psychosoc_16_app16_detail.txt;
app_test_pharyng_16_cwp16_detail.txt;
app_treatment_uri_16_uri16_detail.txt;
asthma_admit_16_am_aar16am_detail.txt;
asthma_med_ratio_16_amr16_detail.txt;
avoid_abx_bronch_16_aab16_detail.txt;
behavior_risk_16_ch_bra16ch_detail.txt;
breast_cancer_16_am_bcs16am_detail.txt;
cahps_adult_survey_16_cpa16_detail.txt;
cahps_child_16_ch_cpc16ch_detail.txt;
cahps_child_survey_16_cpc16_detail.txt;
breast_cancer_16_bcs16_detail.txt;
cardio_schizo_16_smc_16_detail.txt;
care_trans_16_am_ctr16am_detail.txt;
cervical_c_scr_16_am_ccs16am_detail.txt;
chf_admit_16_am_chf16am_detail.txt;
child_access_16_ch_cap16ch_detail.txt;
child_imm_w_lead_16_cis16q_detail.txt;
childhood_imm_16_ch_cis16ch_detail.txt;
childhood_imm_16_cis16_detail.txt;
childrens_access_16_cap16_detail.txt;
chlamyd_screen_16_ch_chl16ch_detail.txt;
chlamydia_screen_16_chl16_detail.txt;
chlamydia_scrn_16_am_chl16am_detail.txt;
colorectal_cancer_16_col16_detail.txt;
comp_diabetes_16_cdc16_detail.txt;
control_hbp_16_am_cbp16am_detail.txt;
control_high_bp_16_cbp16_detail.txt;
copd_admit_16_am_cpr16am_detail.txt;
csr_for_ntsv_16_ch_csr16ch_detail.txt;
dental_sealant_16_ch_sea16ch_detail.txt;
dev_scrn_1_3_16_ch_dsv16ch_detail.txt;
dev_scrn_age_1_16_ch_dvs16cha_detail.txt;
dev_scrn_age_2_16_ch_dvs16chb_detail.txt;
dev_scrn_age_3_16_ch_dvs16chc_detail.txt;
diab_monit_schizo_16_smd16_detail.txt;
diab_screen_schiz_16_ssd16_detail.txt;
diabetes_a1c_16_am_cdc16am_detail.txt;
diabetes_admit_16_am_dsr16am_detail.txt;
diabetes_poor_16_am_hpc16am_detail.txt;
dmard_rheum_arthr_16_art16_detail.txt;
ed_utilization_16_edu16_detail.txt;
elec_delivery_16_am_edr16am_detail.txt;
fllwup_hosp_mh_16_ch_fuh16ch_detail.txt;
cervical_cancer_16_ccs16_detail.txt;
follow_up_hosp_mh_16_fuh16_detail.txt;
;
[\code]
... View more
Yes, it is UNIX so I have to dig around and find the actual path. I wonder if I just open UNIX and type in \\agpcorp or I found the server name of \\va10fsstd003.agp.corp.ads if that will tell me. Because doing the E: or the \\server does not come back in the file exists. I just realized it comes back 0 and not 1 and I know we have to update our UNIX password every 45 days so it is definately UNIX.
... View more
Yep. Thanks. When I put the exact path in fileexist, it works just fine. It can see the .zip file. When I run my libname and zip file entry alone without the data step, I get no errors. When I add that data step to actually open the .txt file is when the error occurs.
... View more
Yes. The \\ appears before all of our mapped drives. I can try taking that out and see if that makes a difference. It did not make a difference. I still get an error opening the file.
As far as bandwidth, the 45 hours is simply copying all those files from the shared drive and moving them to my local machine. That has nothing to do with SAS. I was trying to point my SAS session to look at this shared drive and use SAS to unzip and get the files into our library in our SAS EG grid folder. At this time I don't know what the time involved is for that because I cannot get my SAS to even point to the folder correctly.
As for Net Use, I don't know what that is so I will have to Google to figure it out. I just thought maybe someone had run into zipped files on a Shared Drive in the past and might know he correct path to put in or way to put it in. So far everyone in my organization has no idea. I am sure there is a way. I cannot imagine SAS would create a software product like SAS Base of PC SAS or SAS EG that does not have the ability to unzip files in a share drive since so many organizations for security purposs store data on shared drives. I have been Googling all day yesterday to find an answer and so far all the information has been how to use SAS to zip your files. That is why I finally posted in the forum hoping to get an answer and stop spending all this time researching since I am running out of time to find an answer.
... View more
I have a shared drive with a very long name and have mapped to that drive and it happens to be e: There are 12 folders and within the 12 folders there are several zipped files. Using the first folder, I have 3 zipped files. Within the zipped files there are 32 .txt files so in all that would be 96 .txt files. If I take these 12 folders and just try and move them to my local machine, I get a response that it will take 45 hours to move them. On top of that I have to unzip them all and import all the .txt files into SAS. There are a total of 612 .txt files in all. I have to merge all of these into one and then do my coding to manipulate. I tried the following code in SAS to just zipfile them all but I get an error that the location I am pointing to cannot be found. Here is my code:
<code>
libname MYTD '/global/users/ad36742/';
filename zipfile zip
'e:\\agpcorp\files\Private\ManagedCareSvcs\Health-QualityMgmt\Hedis Rates_Datasets_Benchmarks\Hedis 2016\Datasets\2016_02_Feb Heids 2016_MY2015\ME1\hedis2016_feb2016_md ssi.zip';
DATA newdata;
INFILE ZIPFILE(MDVBPAmbAdult_exMH_AMBAD16MDb_Detail.txt) dsd DLM = '|';
RUN;
</code>
Any advice?
... View more
I am not sure I am doing this correctly, but I have a very large dataset, 10 MILLION ROWS that I am trying to obtain the largest amount in the group. Here is an example of my data have and data need and the code, which I get an error: DATA HAVE ID SERVICE NETPDAMT 111 9000 $10.00 111 9001 $5.00 111 1010 $15.00 111 1049 $50.00 222 4000 $2.00 222 5000 $5.00 222 7000 $15.00 222 8001 $10.00 DATA NEED ID SERVICE NETPDAMT 111 1049 $50.00 222 7000 $15.00 DATA NEED; DATA HAVE; large1=largest(1,of netpdamt); by id service; run;
... View more
I am not sure I understand, however if you mean does the column header with those descriptions contain distinct values in each, then yes. _51 will always be multiple surgical procedures _21 will always be prolonged evaluation and management There are a total of 47 different ones. This proc_cd just happen to have only these. I am not sure what a retain statement is. I will try and find that somewhere online.
... View more
I have a table with 17598 rows. In this table there are numerous procedure codes that have various columns of data contained within. I am trying to group all these various columns of data by the procedure code. I have tried a proc sql group by the proc but that did not work. Then tried by the proc and all these columns of data and that did not work. DATA HAVE PROC_CD _80 RT AS _51 _59 LT _79 11981 11981 DISTINCT PROCEDURAL SERVICE 11981 DISTINCT PROCEDURAL SERVICE LEFT SIDE PROCEDURE UNRELATED PROCEDURE BY SAME PHYSICIAN POST OPERATIVE 11981 RIGHT SIDE PROCEDURE DISTINCT PROCEDURAL SERVICE UNRELATED PROCEDURE BY SAME PHYSICIAN POST OPERATIVE 11981 RIGHT SIDE PROCEDURE MULTIPLE SURGICAL PROCEDURES DISTINCT PROCEDURAL SERVICE 11981 RIGHT SIDE PROCEDURE MULTIPLE SURGICAL PROCEDURES DISTINCT PROCEDURAL SERVICE LEFT SIDE PROCEDURE DATA NEED PROC_CD _80 RT AS _51 _59 LT _79 11981 RIGHT SIDE PROCEDURE MULTIPLE SURGICAL PROCEDURES DISTINCT PROCEDURAL SERVICE LEFT SIDE PROCEDURE UNRELATED PROCEDURE BY SAME PHYSICIAN POST OPERATIVE
... View more
This works awesome. Thanks. I noticed that my data type was different in the data have for the prov_mpin. So, I had to do a let statement at the very beginning for that so I could select the MPIN to pull data on. Then, I did remove the total just because my manager and those reviewing the output do not want to see that. Thank you so much for your help. *restudy breakout mpin; proc report nowd data=cardiac.mpinrestudyfinal (where =(prov_mpin= &prov_MPIN2)); column('Cardiac Metric Component' study) ('Calculation by Place of Service and Procedure' location, procedure, count); define study /group ' ' order=formatted; define location / across ' ' order=data style(column)=[width=30%]; define procedure / across ' ' order=data style(column)=[width=30%]; define count / sum ' '; run; ODS RTF CLOSE;
... View more
Yes, the prior post was when the data was formatted that way. But after QAing the output, I found the end results were not correct and had to change the code that gets me to my data have. I ended up having to recode about 13 pages that feed into my final data have information. I see your attached does work if I just had those 2 MPINS, but I have 27k mpins in my data have and have to figure out how to get that where statement right so the output will be correct. The reports are not an output of all the MPINS but only ones being asked for. So, someone might say I need only MPIN 1, 2 and 3. So, I have a let statement at the top for MPIN and state. This is because there is a ton of other data above this particular section. I only gave the 2 examples because 27k would have been far too much. I will read thru some output documentation and figure out how to output only a certain MPIN. I had the where(strata = &prov_mpin in there, but I think it needs to be defined in the ods and just noprint. Will try that and see what i get back. Thanks
... View more
I am trying to get my proc report output to output the correct counts but it seems it is a bit off. Here is my code: proc report nowd data=cardiac.mpinrestudyfinal (where =(prov_mpin = &prov_mpin2)); column ('Cardiac Metric Component' study) ('Calculation by Place of Service and Procedure' location ) ,procedure count; define study / group ' ' order=formatted; define location / across ' ' order=formatted; define procedure / across ' ' style(column)={just=c}; define count/ computed noprint; compute count; count=1; endcomp; quit; I do have code the creates an rtf report prior. This is just the meat of my proc report. This is the data I have: DATA HAVE location procedure study PROV_MPIN count OUTPATIENT CATH 3-MONTH MAJOR RESTUDY RATE 104534 2 OUTPATIENT PCI 1-30 DAY PCI REDO RATE 104534 1 OUTPATIENT PCI 1-182 DAY PCI REDO RATE 104534 1 INPATIENT CATH 3-MONTH MAJOR RESTUDY RATE 623256 1 INPATIENT TTE 3-MONTH MINOR RESTUDY RATE 623256 1 OFFICE/CLINIC TTE 3-MONTH MINOR RESTUDY RATE 623256 1 OFFICE/CLINIC TTE 3-MONTH MINOR RESTUDY RATE 623256 1 OUTPATIENT CATH 3-MONTH MAJOR RESTUDY RATE 623256 1 OUTPATIENT PCI 1-30 DAY PCI REDO RATE 623256 1 OUTPATIENT PCI 1-182 DAY PCI REDO RATE 623256 1 OUTPATIENT PCI 1-182 DAY PCI REDO RATE 623256 2 OUTPATIENT CATH 3-MONTH MAJOR RESTUDY RATE 623256 1 When I run this report, it outputs the attached information.
... View more
