About JackieO

JackieO · ‎04-08-2019

Thank you all for your help. Consider this case closed. 🙂 Jackie O

JackieO · ‎04-05-2019

Chris and Tom, Thank you for the quick responses. My SAS application is executed within a korn-shell script. Looks like I can control permissions using umask as you have suggested. If this doesn't work for me, I will explore macros, but have minimal experience working with them. Still would love to know why a single file created works perfectly with the PERMISSION option for the FILE statement, while multiple files using FILEVAR does not. At least not for me. 🙂 Again...thank you! Jackie O.

JackieO · ‎04-04-2019

Hello, I hadn't seen the first reference, but had the second. Thank you for your help. Doing a "find" on both yields nothing concerning "PERMISSION". Perhaps I was unclear concerning my problem. I just recently began using "FILEVAR". I can create the external files, but only with my profile settings for file permissions. Those external files need different permissions for group. Too many to chmod. Hoped to modify the permissions while the files are being generated. Someone else in my group will be using these files when running an editing application. Jackie O

JackieO · ‎04-03-2019

I need to create several external files and establish their permissions as u=rwx, g=rwx, and o=r-x. I can accomplish this easily if an output file isn't created using "FILEVAR". Can I not do this using "FILEVAR"? Please see the attached code. What is the proper way? Jackie O.

JackieO · ‎01-11-2019

One final reply to Tom, I am almost done with my project(99% complete). Wouldn't have been without your expert advise. I have read more and learned more in the past two weeks than I have in 30 years. I was a bit offended at first, after reading initial responses, but quickly found that you were absolutely right in your comments. I am a better programmer for it. Thank you! Jackie O

JackieO · ‎01-05-2019

Tom, All I've done is think about how best to accomplish this. I'll try to explain further. 1) Some, but not all of our data makes it through to our Oracle tables. We have an edit routine that validates data reported and generates element errors found. Those with the highest level "FATAL" are restricted from staging for our Oracle ETL process that populates all of our data tables. These data are held back until these types of errors are corrected. So to answer your question, no. I cannot just correct bad SSNs reported in our Oracle database. 2) We require total replacement files when institutions submit data to us. Yes, we have multiple files submitted by every institution during reporting cycle deadlines. This is why I read in files called "I" files to ensure the most recent data sent. Often the SSN changes go back many years. After a while on Unix, the time stamp just has a year and not month, day and time. 3) Institutions should not be re-assigning numbers. The first 3 digits of assigned SSNs must contain the institution code (such as "110"). The other 6 digits are assigned consecutively. If enough time has transpired, I guess they could re-use. Even the more important to free up an assigned SSN when it gets changed to a valid SSN. 4) The update table does include institution code. I have found that some of these assigned SSNs can actually be a valid SSN if searched system wide. This is why I want to make changes only to data submitted by the given institution. We have an institution code of "146". At another institution say "110" there could well be a valid SSN 146XXXXXX. I have to check both institution code and old SSN prior to changing to the new SSN. 5) A flag on the changed rows would be a good thing. OK. I tried your code. Getting errors on the: INFILE IN FILEVAR = in_file_name LRECL = 250 truncover end=eof; Expecting something other than in_file_name after the first = sign. Reading to find out why. Sorry for just now getting back with you. Winter storm came through. Work was closed and power was out at my house. Thank you for hanging in there with me. Jackie O.

JackieO · ‎01-02-2019

Tom, So in SUBSTR(PUT(yr,4.),3)) the 3 means start at that position and stop at the end? Used to have to put how many places to include. I get it. Thank you again! At most the number of files for a record type (S is demographic) would be #years X #semesters X #institutions (9 X 3 X 46 = 1,242). The table containing the SSN corrections has no academic year or semester within. Institutions often enroll students under assigned numbers if they fail to have a valid SSN. Later on those students do obtain valid SSNs and to report persistence, we need to have those SSNs consistent throughout admission to the institution. So to make things easier on the schools, we search through all our data and update the SSNs for consistency. We do have most of our data stored in our Oracle database. That is...the data that has passed our edit checks. The in_file_name files contain both the data that passed and the data that failed. So...I go there to make my corrections. I hope to generate a list of the actual files that have been changed so that they can be reprocessed through our ETL and broadcast into our Oracle tables. BTW...the reference to 7 million are the possible number of records and not the possible number of files. I'm going to play around with this tonight. I so do appreciate all your help!!! Thank you, Jackie O.

JackieO · ‎01-02-2019

Tom, Let me digest your suggested code. Got some new things in there as far as I am concerned. Need to research the SUBSTR(PUT...) combination. Also I have concerns bringing all those records across at the same time. Your data step file_list could be as large as 7 million records depending on how many institutions are making SSN changes. Not so much the S demographic records, but the enrollment records which will be another group of files to be corrected later in the program. Questions? Your third line of code is just establishing variables and their respective lengths? You create in_file_name in the first data step, but reference it again in the data step current. How is scope not a player here? Thank you, Tom Jackie O

JackieO · ‎01-02-2019

Anyone? Is it possible within a DATA STEP to read both an external file and results from a SQL query? I have attached my code. I have two SQL queries. One determines which institutions have to be processed. The second contains SSN changes. The goal: 1. Out of 46 institutions, process only those with changes found in the first query. 2. Only process semester and years impacted by the changes( SUM,FALL,SPRING AND 20102011 to 20182019). 3. Output the external files in entirety, but with the SSN changes made. I am certain with all your experiences that you know of a better way. Sometimes I equate posting a question to walking before a firing squad. I am a bit sensitive. However, Tom opened my eyes to SAS 9.4 and I so thank him. I first learned SAS in 1985 and my coding is antiquated at best. Trying new things with version 2 of this project. Thank you in advance, Jackie O. Jackie O.

JackieO · ‎01-01-2019

Tom, Sorry to not have responded sooner. Happy New Year BTW! And...thank you! You are correct. My code is convoluted. I learned SAS the way my former supervisor taught me way back in 1985. I am the only programmer doing SAS in my office and have not taken time to update my "skills". Have been trying new things today. Thank you for the "prodding". Started a new version of my program. Trying to use loops (i,j,k) to iterate through years, semester, and institutions (in that order). How would you increment a loop index if not consecutive numbering? Example of my institution codes are '110','111','120','340'. Is there a way to use an IN('110','111') to update my index. I am attaching my new attempt. Jackie O

JackieO · ‎12-31-2018

I thought I was on to my answer, but have struggled all day and gotten no further with resolution to my problem. I will simplify in hopes that you will have an answer. 1. My first data step reads in records from a student demographic file and sorts it with another file that marks ssns needing to be corrected. A flag is set to 'Y' to indicate that the file actually was modified and all records (changed/unchanged) are written out to a new file containing "changed" in the name. 2. My second data step reads in records from a student course file and again sorts it with the file that marks ssns needing to be corrected. The flag in step #1 is initialized to 'N'. It is set to 'Y' again per verbiage in step #1. 3. My third data step reads in records from a student degree awarded file and again sorts it with the file that marks ssns needing to be corrected. This file may or may not contain the students' ssns corrected in steps #1,2. Not every student in the demographic/course files receive degrees. The problem is that I need to know what the flag has been set to in steps #1 and/or #2. This will be used in step #3 to generate a degree file to be processed with the demographic and course files. Our ETL will have problems if all three files are not processed together. I hope this is clear enough. If not, I can expand on my explanation. Many thanks in advance! Jackie O.

JackieO · ‎12-31-2018

Good morning all, I want to thank those who replied to my post. I thought I had attached code to better explained my problem. Oh well, I've been reading all weekend. I think I have found my answer...%GLOBAL. Will test out today. Closing this post. Happy New Year to all. Maybe will "see" ya next year. Jackie O.

JackieO · ‎12-27-2018

The attached code processes multiple file types (S, E and D) and corrects bad reporting of SSN data. Our ETL process requires file types S, E and D to be run through together. My problem is that S and E will have changes, while D may not. How do I "track" what has happened in preceding data steps for other file types.? Like have a variable "outside" that is updated within these data steps. The above code just covers one year and semester. We will be correcting SSN data to present (where needed). I need to generate a corresponding D "change" file even though nothing has been changed. I am trying to generate a list of changed files that my data tech can process through our edit application to stage data for the ETL. Hope I have explained this well enough. Thank you for any assistance you give to me. Jackie O.

JackieO · ‎12-05-2018

Reeza! It works! It works! Thank you again for EVERYTHING!!! SEE... NOTE: The file OUT is: Filename=/uds/data/newdata/20102011/SUM/I110/110sum1011s_ssn_changes.dat, Owner Name=jackie,Group Name=uds, Access Permission=-rw-rw----, Last Modified=05Dec2018:15:56:08 NOTE: The file OUT is: Filename=/uds/data/newdata/20102011/SUM/I146/146sum1011s_ssn_changes.dat, Owner Name=jackie,Group Name=uds, Access Permission=-rw-rw----, Last Modified=05Dec2018:15:56:08 NOTE: 9492 records were written to the file OUT. The minimum record length was 222. The maximum record length was 222. NOTE: 10279 records were written to the file OUT. The minimum record length was 222. The maximum record length was 222. Have a great evening!!!!! JackieO

JackieO · ‎12-05-2018

Reeza, I am sorry to be so irritating to you. I don't understand your method either. Guess I will keep trying to make that one line work that contains the catt and catx functions. I guess they are functions. New to me, too. Anyway...I really do appreciate all your help. Have a better day and thank you, JackieO

Online Status	Offline
Date Last Visited	‎04-08-2019 10:07 AM

Re: External output file permissions

Re: External output file permissions

Re: External output file permissions

External output file permissions

Re: Track value of same variable across multiple data steps

Re: Track value of same variable across multiple data steps

Re: Track value of same variable across multiple data steps

Re: Track value of same variable across multiple data steps

Re: Track value of same variable across multiple data steps

Re: Track value of same variable across multiple data steps

Re: How to reference a local variable that is part of a FILE statement

Re: How to reference a local variable that is part of a FILE statement

Re: External output file permissions

Re: External output file permissions

Re: External output file permissions

External output file permissions

Re: Track value of same variable across multiple data steps

Re: Track value of same variable across multiple data steps

Re: Track value of same variable across multiple data steps

Re: Track value of same variable across multiple data steps

Re: Track value of same variable across multiple data steps

Re: Track value of same variable across multiple data steps

Re: Track value of same variable across multiple data steps

Re: Track value of same variable across multiple data steps

Track value of same variable across multiple data steps

Re: How to reference a local variable that is part of a FILE statement

Re: How to reference a local variable that is part of a FILE statement