08-04-2020
pensarchem
Obsidian | Level 7
Member since
05-09-2019
- 39 Posts
- 3 Likes Given
- 0 Solutions
- 6 Likes Received
-
Latest posts by pensarchem
Subject Views Posted 3660 08-03-2020 02:04 AM 3662 08-03-2020 02:02 AM 3666 08-03-2020 01:58 AM 3789 08-02-2020 11:47 AM 3884 08-02-2020 03:48 AM 1935 08-01-2020 02:50 PM 2072 07-28-2020 08:03 PM 2089 07-28-2020 06:40 PM 2147 07-28-2020 05:34 PM 2160 07-28-2020 05:25 PM -
Activity Feed for pensarchem
- Got a Like for ODS tagsets.excelxp complex formating question. 09-16-2022 12:47 PM
- Posted Re: improve proc sql join efficiency on SAS Programming. 08-03-2020 02:04 AM
- Posted Re: improve proc sql join efficiency on SAS Programming. 08-03-2020 02:02 AM
- Posted Re: improve proc sql join efficiency on SAS Programming. 08-03-2020 01:58 AM
- Posted Re: improve proc sql join efficiency on SAS Programming. 08-02-2020 11:47 AM
- Posted improve proc sql join efficiency on SAS Programming. 08-02-2020 03:48 AM
- Posted Re: macro Do loop for a set of programs on SAS Programming. 08-01-2020 02:50 PM
- Liked Re: macro Do loop for a set of programs for mkeintz. 08-01-2020 02:44 PM
- Liked Re: macro Do loop for a set of programs for ballardw. 08-01-2020 02:44 PM
- Posted Re: macro Do loop for a set of programs on SAS Programming. 07-28-2020 08:03 PM
- Posted Re: macro Do loop for a set of programs on SAS Programming. 07-28-2020 06:40 PM
- Posted Re: macro Do loop for a set of programs on SAS Programming. 07-28-2020 05:34 PM
- Posted macro Do loop for a set of programs on SAS Programming. 07-28-2020 05:25 PM
- Posted Re: unzip sas problems: input and unzip files by batch on SAS Programming. 07-17-2020 02:41 PM
- Liked Re: unzip sas problems: input and unzip files by batch for Tom. 07-17-2020 02:34 PM
- Posted Re: unzip sas problems: input and unzip files by batch on SAS Programming. 07-15-2020 04:58 PM
- Posted Re: unzip sas problems: input and unzip files by batch on SAS Programming. 07-15-2020 04:28 PM
- Posted Re: unzip sas problems: input and unzip files by batch on SAS Programming. 07-15-2020 01:31 PM
- Posted unzip sas problems: input and unzip files by batch on SAS Programming. 07-15-2020 01:07 PM
- Got a Like for Re: Change the table format, not transpose though.. 11-14-2019 05:46 PM
-
Posts I Liked
Subject Likes Author Latest Post 1 1 1 -
My Liked Posts
Subject Likes Posted 1 07-11-2019 12:45 PM 2 11-14-2019 04:16 PM 2 07-30-2019 02:48 PM 1 07-16-2019 12:02 PM
08-03-2020
02:04 AM
thanks for your input, I answered this above. We normally do run the code in Oracle and the process is quite quick..
... View more
08-03-2020
02:02 AM
Yes, currently I run the code on a lower security level, so I can still connect to oracle. However, we do have a special safe environment and all the data were imported manually from various resources. The data will be imported through .gz files into sas and run locally only under SAS.. So I am testing my code now and will move to safe environment after everything runs smoothly..I hope this clarifies.
... View more
08-03-2020
01:58 AM
thanks. In the process of testing code now, I am dealing an error message. Will get back to you.
... View more
08-02-2020
11:47 AM
1, hynp.flat is the source table; so label, dt, num are all from the source table. att.list_low5_&c_year b is a label cohort generated from hynp.flat; 2, &s_date does not change and is a certain date for each macro; 3, No, these filters are set, there is no dynamic in WHERE;
... View more
08-02-2020
03:48 AM
Hi everyone, Previous, when using SAS, we can have pass through to run the query in Oracle, this process is quite smooth. However, due to a special nature of safe environment, I have to run all the codes in SAS only. It runs incredibly slow now, even for a small set for data b. Here is the code, and I will put some labels. I did some searches, but frankly not sure which will be the best approach like index( in this case will > than 34% unique labels) /hash?). I tried: 1) data step to extract all related data, then join; take too long and too much spaces; 2) use subquery, not by join. use filter as ---- label in (select distinct label in att.list_low5_&c_year), run quite slow as well. Any recommendation from advanced users? thank you very much. proc sql; create table att.total_list_&p_year as select distinct a.label, num, datepart(dt) as dt from hynp.flat a (********************* data in Oracle, not sortable, 80 millions records each year) inner join att.list_low5_&c_year b (******************* records from thousands to up to 4millions, this macro run ten circles) on a.label = b.label where dt >= &s_date and dt <='31Mar2019'd and prov = .. and tp = .. and pec = .. and pnum ^= .. and cd < .. and vcd not in (.....) and vloc not in (....) and tm not between ... and ... and tm not in (...) and num not in (...) order by a.label,dt desc, num; quit;
... View more
08-01-2020
02:50 PM
Thanks everyone for your input. In the end, I did use macro. 1, generate a cohort for the first year first; 2, then use the macro and almost the same code (need some changes to join the cohort with database though) to generate the cohort for the last 9 years. 3, then label and put them together to give me the whole 10 year list. The most easiest way for coding will be: 1) extract all the data for last 10 years; 2) then rank the visits or service date.. However, the extraction will take too long for the first step and is not practical in my case. thanks everyone.
... View more
07-28-2020
08:03 PM
Well, I think I understand both of your points now. However, I just want to point out just in case if I did not explain what I need clearly. 1, we cannot create a large dataset, without set the client cohort from previous year, we are going to deal with the complete database for 10 year, so it's not practical; Most importantly, I need the list from a specific year (a, let's 2017) to run the previous year (a-1, 2016) results, but date is changing for each run and should include all the years after the new a-1, 2016 to the year 2019 . 2, Let's say I got 4 millions patients in 2019, create a special list with 3 millions patients in 2019 as list_2019; 3, then use list_2019, run the similar codes, but time from 2018-2019 two years, generate a list 2 millions list_2018; 4, then use list_2018, time from 2017-2019; generate a list 1.5 millions; .... 10, list_2011, date 2010-2019, generate 200K patients. regardless, thank you both for the discussion.
... View more
07-28-2020
06:40 PM
I just did copied the codes for 10 times and made very minor changes for each set. The coding does little time like 10 mins, which is great. I am not sure if I misunderstood both of you. I say 'smart way' means the code is short and uses whatever short macro/ loop with only one set of main program, like simple macro.. I normally use the combination of sas/sql (under oracle), so don't know lots of commands under pure SAS.
... View more
07-28-2020
05:34 PM
Yeah, I understand you, just feel it's not a smart way to do it. I will do this if I could not get the answer here or figure out a way..
... View more
07-28-2020
05:25 PM
Hi everyone, I have not used do loop very much. however I have the following codes/logic need to be developed, please kindly help. Let's say, 1, By running programs 1, 2, 3, I created a cohort about 3 million of clients for my cohort_2019 ; 2, then I use this cohort_2019, run the same program, 1, 2, 3, (need to change the dates though, should also in the loop) create another cohort_2018; ...... Create back to cohort_2010; I personally would do it without 'do loop' if it's less than 4 circles. For 10, I think it's too much. As I lack experience in do loop, so I ask your kind help. thanks.
... View more
07-17-2020
02:41 PM
Thank you very much Tom. I tested run your (step by step) code below as well. I found that the step-by-step codes give me file name as full name with 'path'. However, in this solution code, the first step data all the filenames, but the data removed all the path to these file. so in second step, I added the path as fname= '&path\'||filename. Everything works perfectly.
... View more
07-15-2020
04:58 PM
Sorry, it's under Safe Environment, so I could not copy anything out. Regardless data want; or data libname.want; the error message is the same as I wrote in last reply.
... View more
07-15-2020
04:28 PM
Thanks for your advice Tom. yeah, not zip file, but .gz file. First step indeed generate a list of files I want. The problem is second step. 1, All the data store in 'DATA' folder, which I only have access to read, not write; 2, No matter I create a folder and use libname , or just under sas work file. The error message is always: ERROR: Open failure for C:\WINDOWS\SYSTEM32\filename during attempt to create a local file handle. so I am not sure if this is a path problem (which does not link to orginial 'DATA' folder) or it's a unzip problem. thanks.
... View more
07-15-2020
01:31 PM
filename fromzip ZIP "C:\Logs\SEGuide_log.10168.txt.gz" GZIP;
data logdata;
infile fromzip; /* read directly from compressed file */
input date : yymmdd10. time : anydttme. ;
format date date9. time timeampm.;
run; Hi Reeza, thanks for you advice. 1, the above code is is similar to what I used. I will try proc import. 2, I thought about it, if there is no better choice, macro is the only way I guess.
... View more
07-15-2020
01:07 PM
Hi everyone, I got hundreds of file need to unzip. There are two major problems; 1, input: right now, 'input;' does not work, only work when you specify the variables and length and format, this causes lots of trouble, as we have to re-define all the variables and format. Especially when dealing with different table, it's a disaster. I hope to have a simple command to universally treat input; 2, Any way to unzip a batch of files? Got hundreds, batch by batch and I don't want to do it one by one. Thanks.
... View more