About Shakir_Juolay

Shakir_Juolay · ‎09-16-2020

Thank You Doug. It was a new learning for me that the benefits of a categorical variable in logistic regression will only be seen when used with an interval variable. I know my sample size is small and hence the differences in Training and Validation sets. But if my sample size was large enough (say 1000 -> 700 Training and 300 Validation) then are the differences in Training and Validation in terms overall target percentage and target percentage split by the different level of a categorical variable acceptable for model validation. In other words, I am trying to ask should Training and Validation Sets have same/similar target percentage JUST for the entire set or ALSO for different levels of categorical variables.

Shakir_Juolay · ‎09-13-2020

Consider the following hypothetical situation, in which I have two question regarding the validity of the split between training and validation set. I am trying to build a logistic regression model with only one categorical explanatory variable X for a binary target Y, event being Y = 1. Below is the distribution for Y and X in Raw Table Y # % 0 97 53% 1 85 47% Total 182 100% X # % A 62 34% B 24 13% C 91 50% D 5 3% Total 182 100% I am doing a 70:30 split for training and validation set. Below is the distribution for Y and X in Training Table Y # % 0 67 53% 1 60 47% Total 127 100% X # % A 44 35% B 17 13% C 64 50% D 2 2% Total 127 100% Below is the distribution for Y and X in Validation Table Y # % 0 30 55% 1 25 45% Total 55 100% X # % A 18 33% B 7 13% C 27 49% D 3 5% Total 55 100% Question 1: Given the above numbers for Raw, Training and Validation Tables is the split between Training and Validation a valid one for building a model? If yes, then below are the cross tabulation numbers. Cross Tabulation in Raw Table X Y # %Y=1 in X A 0 27 A 1 35 56% B 0 11 B 1 13 54% C 0 57 C 1 34 37% D 0 2 D 1 3 60% Cross Tabulation in Training Table X Y # %Y=1 in X A 0 17 A 1 27 61% B 0 8 B 1 9 53% C 0 41 C 1 23 36% D 0 1 D 1 1 50% Cross Tabulation in Validation Table X Y # %Y=1 in X A 0 10 A 1 8 44% B 0 3 B 1 4 57% C 0 16 C 1 11 41% D 0 1 D 1 2 67% Question 2: If answer to Question 1 is yes, then given the cross tabulation numbers above and differences in %Y=1 in X (event rate) for different levels of X in Raw, Training and Validation Tables (like for X=A 56% in Raw, 61% in Training and 44% in Validation) is the split between Training and Validation a valid one for building a model?

Shakir_Juolay · ‎04-30-2020

I would like to understand mathematically how the Cumulative Lift is adjusted for Prior Probabilities by SAS Enterprise Miner in the Regression Node for Logistic Regression. I got the formula for adjusting posterior probabilities over here https://documentation.sas.com/?docsetId=emxndg&docsetTarget=p1vqpbjwoo4bv7n1sw77e0z64xxs.htm&docsetVersion=15.1&locale=en

Shakir_Juolay · ‎12-05-2019

Please share output of the below 1) PROC CONTENTS DATA=ALERTCSE.Alert_Cases; RUN; 2) PROC PRINT DATA=ALERTCSE.Alert_Cases (OBS=10); RUN;

Shakir_Juolay · ‎12-04-2019

What I need is mentioned in this paper https://support.sas.com/resources/papers/proceedings11/050-2011.pdf

Shakir_Juolay · ‎11-25-2019

I want to create a SAS Stored Process using Enterprise Guide, that I will call from Excel using SAS Add-In and it will take the data from an Excel range as input that I will specify when prompted to. I would like to know if the above is feasible or not, if yes, then how?

Shakir_Juolay · ‎10-22-2018

Dear Tom, Thank you for sharing this option with me. I will play around with it. I was wondering if this will apply to Teradata as the documentation says Data source: Oracle I agree with you on "any non-standard (non 7bit) ASCII codes in your LATIN1 data will take more than one byte to store in the SAS variable". At least for the columns that I create using LATIN I know the variable is ASCII and can fit in 7 bits.

Shakir_Juolay · ‎10-21-2018

Dear Kurt, I see what you mean by SAS tries to "play it safe" with regards to defined lengths. There was a gap in my understanding of UTF-8 and how many bytes are used by it for storage. When I read the Efficiency section of https://en.wikipedia.org/wiki/Comparison_of_Unicode_encodings I see that characters with code points with 8 bits i.e. U+0080 to U+00FF are stored in UTF-8 in 2 bytes and not one. And characters with code points with 16 bits are stored with 3 bytes. Teradata LATIN CHAR(n) needs 8 bits per n hence in SAS UTF-8 the length becomes 2n, and Teradata UNICODE CHAR(n) needs 16 bits per n hence in SAS UTF-8 the length becomes 3n. Now I store English characters in Teradata LATIN which only need 7 bits so I can use dbsastype to change Teradata LATIN CHAR(n) to SAS UTF-8 length n. I use Teradata UNICODE only for Arabic characters, which fit in UTF-8 with 16 bits so I can use dbsastype to change Teradata UNICODE CHAR(n) to SAS UTF-8 length 2n. It appears using dbsastype, I will have to do column by column, is there a universal SAS OPTION to do all columns together? Else I write a SAS macro leveraging Teradata dictionary under DB DBC.* and dbsastype. Thank You Kurt.

Shakir_Juolay · ‎10-16-2018

Also can we somehow configure SAS and/or Teradata Client on the SAS server, so that for var1 the length in SAS is 5, since its LATIN and 1 byte per character is enough for LATIN.

Shakir_Juolay · ‎10-16-2018

This is what I did data test_char_length; set d_dmin.test_char_length; put var1 $10.; put var1 $hex20.; put var2 $15.; put var2 $hex30.; run; This is the log 15 data test_char_length; 16 set d_dmin.test_char_length; 17 put var1 $10.; 18 put var1 $hex20.; 19 put var2 $15.; 20 put var2 $hex30.; 21 run; home 686F6D65202020202020 home 686F6D652020202020202020202020 NOTE: There were 1 observations read from the data set D_DMIN.test_char_length. NOTE: The data set WORK.TEST_CHAR_LENGTH has 1 observations and 3 variables. NOTE: DATA statement used (Total process time): real time 0.16 seconds cpu time 0.03 seconds Also this post https://community.teradata.com/t5/Database/UTF-8-or-UTF-16/m-p/61680 says unicode in Teradata is always UTF-16. Is it possible in SAS for one column to be LATIN and another one to be UTF8 or UTF16 in the same Table.

Shakir_Juolay · ‎10-16-2018

I have created a Teradata table as below create multiset table d_test.test_char_length ( contract_id int ,var1 varchar(5) character set latin ,var2 varchar(5) character set unicode )unique primary index(contract_id) ; Inserted a dummy row as below insert into d_test.test_char_length values (123,'home','home'); Below is the SAS code to read the Teradata table and PROC CONTENTS on the table in Teradata and in SAS. data test_char_length; set d_test.test_char_length; run; proc contents data=d_test.test_char_length; run; proc contents data=test_char_length; run; Below is the output 10:14 Tuesday, October 16, 2018 1 The CONTENTS Procedure Data Set Name D_TEST.test_char_length Observations . Member Type DATA Variables 3 Engine TERADATA Indexes 0 Created . Observation Length 0 Last Modified . Deleted Observations 0 Protection Compressed NO Data Set Type Sorted NO Label Data Representation Default Encoding Default Alphabetic List of Variables and Attributes # Variable Type Len Format Informat Label 1 contract_id Num 8 11. 11. contract_id 2 var1 Char 10 $10. $10. var1 3 var2 Char 15 $15. $15. var2 10:14 Tuesday, October 16, 2018 2 The CONTENTS Procedure Data Set Name WORK.TEST_CHAR_LENGTH Observations 1 Member Type DATA Variables 3 Engine V9 Indexes 0 Created 10/16/2018 10:22:00 Observation Length 40 Last Modified 10/16/2018 10:22:00 Deleted Observations 0 Protection Compressed NO Data Set Type Sorted NO Label Data Representation SOLARIS_X86_64, LINUX_X86_64, ALPHA_TRU64, LINUX_IA64 Encoding utf-8 Unicode (UTF-8) Engine/Host Dependent Information Data Set Page Size 65536 Number of Data Set Pages 1 First Data Page 1 Max Obs per Page 1632 Obs in First Data Page 1 Number of Data Set Repairs 0 Filename /saswork/SAS_work0E0F00005108_sas-compute/SAS_workD74B00005108_sas-compute/test_char_length.sas7bdat Release Created 9.0401M4 Host Created Linux Inode Number 4063249 Access Permission rw-r--r-- Owner Name unxsrv File Size 128KB File Size (bytes) 131072 Alphabetic List of Variables and Attributes # Variable Type Len Format Informat Label 1 contract_id Num 8 11. 11. contract_id 2 var1 Char 10 $10. $10. var1 3 var2 Char 15 $15. $15. var2 My question is why the length of var1 (LATIN encoding in Teradata) in SAS is 10 instead of 5 and var2 (UNICODE encoding in Teradata) is 15 instead of 5. I was expecting var1 to be 5 and var2 to be 10 in SAS. Regards, Shakir

Shakir_Juolay · ‎12-06-2017

That I do.

Shakir_Juolay · ‎12-06-2017

Dear @SASKiwi, Do you mean I have a fellow SAS user with a PC that remains on Windows 10 64-bit? I am the first one.

Shakir_Juolay · ‎12-06-2017

My work laptop (64-bit processor) needs to be migrated from Windows 10 32-bit to Windows 10 64-bit. I have several SAS EG projects with embedded codes only. Will these projects work fine as-is when I reinstall SAS EG on the 64-bit Windows. There is no change on the sever side. The above also applies to MS Excel Add-in. SAS EG Version: 7.13 (7.100.3.5408) (32-bit) SAS Add-In for Microsoft Office: 7.13 (7.100.3.5386) (32-bit) Server Side SAS Version: 9.04.01M4P110916 Server Side Operating System: LIN X64 Also will the SAS depo that I currently have work fine and install on 64-bit, or do I need a new depo? Can I verify this?

Shakir_Juolay · ‎10-18-2017

Dear @nhvdwalt, @JuanS_OCS, @Damo, The issue is intermittent. I raised a ticket with SAS Support. And they suggested 1. Close all SAS and Microsoft Office applications, including Outlook. 2. Navigate to the SASAddinForMicrosoftOffice\x.x\ folder. (where x.x is your version of the SAS Add-in). 3. Right click on the switcherutility.exe and choose "run as admin". 4. Select Excel, but not Outlook. 5. Complete the process. Now, test by doing the following: 1. Open Excel (to check Disable items first) 2. Go to File -> Options -> Add-Ins 3. At the bottom of the window, in the drop down menu 'Manage' select Disabled Items -> Click Go 4. Does SAS Add in appear here, if it does click the enable button. Next, check Com Add-ins: 5. Go to File -> Options -> Add-Ins 6. At the bottom of the window, in the drop down menu 'Manage' select COM Add ins -> Click Go 7. Check the box for SAS Add In and Select ok Now close Excel and re-open. Is the SAS tab present? Though I could not run the SwitcherUtility as not admin rights, enabling it from Disabled Add-Ins and checking it from COM Add-Ins helps.

Online Status	Offline
Date Last Visited	‎12-19-2022 08:34 AM

Re: Quality Check of Training and Validation Set

Quality Check of Training and Validation Set

Prior Probabilities and Cumulative Lift

Re: date

Re: Input Excel Range Selection to SAS Stored Process

Input Excel Range Selection to SAS Stored Process

Re: Length of character variable of Teradata table in SAS

Re: Length of character variable of Teradata table in SAS

Re: Length of character variable of Teradata table in SAS

Re: Length of character variable of Teradata table in SAS

Re: SAS Excel Add-in load issue

Re: SAS Enterprise Miner environment file

Re: Quality Check of Training and Validation Set

Quality Check of Training and Validation Set

Prior Probabilities and Cumulative Lift

Re: date

Re: Input Excel Range Selection to SAS Stored Process

Input Excel Range Selection to SAS Stored Process

Re: Length of character variable of Teradata table in SAS

Re: Length of character variable of Teradata table in SAS

Re: Length of character variable of Teradata table in SAS

Re: Length of character variable of Teradata table in SAS

Length of character variable of Teradata table in SAS

Re: Windows 10 Migration from 32 Bit to 64 Bit

Re: Windows 10 Migration from 32 Bit to 64 Bit

Windows 10 Migration from 32 Bit to 64 Bit

Re: SAS Excel Add-in load issue

SAS Analytics Explorers