About NickVe

NickVe · ‎04-15-2020

This 2D binning article looks indeed promosing workaround to generate a regular grid from my irregularly spaced data. This could then be combined with the G3GRID method proposed above. Note however that this is still a workaround and technically not a solution to my problem. The reason for rejecting the previous methods are that they do not answer the question: "proc GLM" provides a "fit", not an "interpolation" as requested "RSREG" provides a "fit", not an "interpolation" as requested G3GRID only works for "regular grids", not irregularily shaped data as requested "PROC MI" seems only to do missing imputation, not "interpolation" "PROC LOESS" provides a "fit", not an "interpolation" as requested "PROC KRIGE" performs krigging, with is again a "fit", not an "interpolation" as requested The closest solution to my problem I have found seems to be the "proc discrim" method, which I use to make a 1NN interpolation. The advantage is that this meets the criteria of my question: I can provide a "train" dataset of (X0, Y0, Z0) values I can easily provide new values (X, Y) and obtain the corresponding (Z) values This works for data that is not on a rectangular grid For me personally this is the most viable solution at the moment. Unfortunately this only fits the nearest point, and does not perform linear interpolation. I understand that I could write my own program in SAS to solve this issue, however that was not the question. I wanted to avoid "re-inventing the wheel" by working with the optimized SAS procedures as much as possible. My belief was that these "interpolation functionalities" would also be available in SAS since they are available in for example MATLAB, R, Python and C++ (see examples).

NickVe · ‎04-15-2020

If I read the documentation correctly, "proc GLM" only provides a "linear fit", not an "interpolation" as is asked in the original question. Please correct me if I misread this.

NickVe · ‎04-14-2020

Dear all, Thank you for the many responses. My hope was that there would be an easy method that constructed an interpolation(x, y) function (as is the case in the examples in Python, R, MATLAB and C++ shown in the original question). If I read the different responses correctly, this is not possible at all, am I correct with this? The popular G3GRID method that is suggested does not seem to meet my criteria. The reason is that I have irregularly spaced datapoints that I wish to determine. This appears to be impossible using this method. The MI method seems promising at first sight. However I only see examples where the imputation is done by either the "mean value" or by a "regression result". The documentation of this method does not discuss "interpolation". My current "quick and dirty" workaround is by using the "proc discrim" procedure, to essentially do 1-nearest neighbour interpolation. This gives me the value of the nearest point as the "interpolation". This is a workable solution at the moment for me, although it is far from ideal. As a recap: I have a series of regularly spaced (x0, y0, z0) values. I also have a (very large) series of irregularly spaced (x, y) values for which I want to determine the corresponding z values.

NickVe · ‎03-31-2020

Hi PGStats, I indeed saw the G3GRID procedure in my search, and this is in our licence. However after reading the documentation I only found examples to make graphs. My goal is to add the z-value to a table of (x,y) values. Where the z-value is determined by an interpolation of a given (x, y, z) table. I am not sure if the G3GRID procedure can do this?

NickVe · ‎03-31-2020

Hi Reeza, Thank you for the suggestion! My personal drawback on this, is that I have a list of numbers (x, y, z) which we know to be exact. The goal is to estimate the z value for a new point (x,y). The function should pass trough the original numbers (x, y, z), as they are exact, and interpolate these number linearly (or in a smoothed way).

NickVe · ‎03-31-2020

Just to clarify on what I mean with bilinear. With bilinear I am aiming to the meaning that it has on WikiPedia (https://en.wikipedia.org/wiki/Bilinear_interpolation). This means that I have a list of (x,y,z) values, and that I am looking for some function that can fill in the missing z-values if I provide the missing z-values if I provide a list of (x,y) values within the range of the original provided values.

NickVe · ‎03-31-2020

Hi PaigeMiller, With bilinear I am aiming to the meaning that it has on WikiPedia (https://en.wikipedia.org/wiki/Bilinear_interpolation). This means that I have a list of (x,y,z) values, and that I am looking for some function that can fill in the missing z-values if I provide the missing z-values if I provide a list of (x,y) values within the range of the original provided values. Thank you for the advice, I will adjust my question to avoid all unclarities.

NickVe · ‎03-30-2020

Is it possible to do bilinear interpolation in SAS? For interpolation in one dimension I found that this can be easily done using the "proc expand" or a "proc transreg" (link: https://support.sas.com/kb/24/560.html). However, for the case where I have 2 "input variables (x,y)" and one "output variable (z)" I have not found a good procedure. Are there any good procedures to do bilinear interpolation in SAS? Additional clarification on what I mean with "bilinear interpolation" With bilinear I am aiming to the meaning that it has on WikiPedia (https://en.wikipedia.org/wiki/Bilinear_interpolation). This means that I have a list of (x,y,z) values, and that I am looking for some function that can fill in the missing z-values if I provide the missing z-values if I provide a list of (x,y) values within the range of the original provided values. Interpolation methods that would be a great fit for my purpose: bilinear interpolation: https://en.wikipedia.org/wiki/Bilinear_interpolation bicubic interpolation: https://en.wikipedia.org/wiki/Bicubic_interpolation nearest-neighbours interpolation: https://en.wikipedia.org/wiki/Nearest-neighbor_interpolation Although the latter one I personally do not prefer, it could also provide a solution to my problem. Examples from other programming languages Examples of the function I am looking for can also be found in other programming languages. All the examples accept a list of (X, Y, Z) values, together with a list of (X0, Y0) values for which the (Z) values are then determined so these can be added to a table. MATLAB example (interp2): https://nl.mathworks.com/help/matlab/ref/interp2.html Python example (interp2d): https://docs.scipy.org/doc/scipy-0.14.0/reference/generated/scipy.interpolate.interp2d.html R example (interp2): https://www.rdocumentation.org/packages/pracma/versions/1.9.9/topics/interp2 C++ example (libInterpolate): https://github.com/CD3/libInterpolate

NickVe · ‎02-19-2020

As a follow up: Apparently this is a bug in Enterprise-Miner triggerd by the "ensemble model" node. If your target is numeric you can get multiple length statements in the optimized code.

NickVe · ‎02-12-2020

Hey CrhisNZ, No problem, I would have probably answered the same if I did not know E-miner. I am already very thankful for the information you have provided. I share your concern considering turning off the warings. I would rather solve the problem than the symptoms since this is better for maintainability. I will check in with SAS support as well, maybe they might have a solution for this issue. I will keep this thread open for one more week. If there is no new answer in this time I will mark your answer as the solution since it essentially solves my problem.

NickVe · ‎02-10-2020

Hi ChrisNZ, I completely agree with you, this was actually also what I meant with the question "Might it be possible to fix the variable length in E-Miner itself ?". The variable I_TARGET is a variable that is generated by SAS itself in E-Miner for the model evaluation. Now I have been looking for some time where I could configure this variable in SAS, but unfortunately I have not found it yet. Do you perhaps know where I could set this variable length in E-Miner?

NickVe · ‎01-23-2020

When I try to use the SAS optimized code in my datastep, i get the warning below: "WARNING: Length of character variable I_TARGET has already been set. " When looking at the sas file itself, I see that the optimized code has three different length statements for the variable I_TARGET, each with a different value for the length ($32, $16 and $32). Is there an easy way to fix this issue? The obvious solution is of course to take out two of the length statements which effectively fixes the warning. However I prefer not to do this since the models from E-Miner will be frequently updated and re-trained, meaning that the optimized code will be overwritten each time. This solution is therefore not suitable for automation purposes. Might it be possible to fix the variable length in E-Miner itself ? If this issue can not be solved easily, can I turn off warnings on variable length for that particular data step somehow?

NickVe · ‎12-23-2019

Thanks for the tip, I will have a look at this!

NickVe · ‎12-23-2019

Thanks! This is indeed a much cleaner solution to the problem.

NickVe · ‎12-23-2019

I am currently trying to write a small SAS macro that does the following: The macro reads the input ID values from an input table: "input_table". The ID values are used to query an oracle database for the variable "TARGET". The macro is shown below. Whenever I run the macro, the filtering on ID does not seem to work and the proc sql return an empty table. I can not get my head around what might be going wrong, all help is welcome! My current solution is using an inner join, which does the job. However, the SQL solution is strongly preferred for efficiency reasons. QUESTION: Why is the Proc SQL not selecting records based on the list "id_list"? %macro query_DB_from_table(input_table = , output_table = ); /* PART 1: Get IDs from the input table */ %local id_list; proc sql noprint; select ID into: id_list separated by "' , '" from &input_table; quit; /* PART 2: Query the Oracle Database */ proc sql noprint; create table &output_table as select ID, TARGET from ORACLE_DB where ID in (%str(')%bquote(&id_list)%str(')) order by ID; quit; %mend query_DB_from_table;

Online Status	Offline
Date Last Visited	‎05-05-2020 12:48 PM

Re: Bilinear interpolation in SAS

Re: Bilinear interpolation in SAS

Re: Bilinear interpolation in SAS

Re: Bilinear interpolation in SAS

Re: Bilinear interpolation in SAS

Re: Bilinear interpolation in SAS

Re: Bilinear interpolation in SAS

Bilinear interpolation in SAS

Re: SAS E-Miner optimized code causes warnings when used in the SAS da...

Re: SAS E-Miner optimized code causes warnings when used in the SAS da...

Re: Bilinear interpolation in SAS

Re: SAS E-Miner optimized code causes warnings when used in the SAS da...

Re: Proc SQL with a macro in where statement to select on ID's

Re: Bilinear interpolation in SAS

Re: Bilinear interpolation in SAS

Re: Bilinear interpolation in SAS

Re: Bilinear interpolation in SAS

Re: Bilinear interpolation in SAS

Re: Bilinear interpolation in SAS

Re: Bilinear interpolation in SAS

Bilinear interpolation in SAS

Re: SAS E-Miner optimized code causes warnings when used in the SAS da...

Re: SAS E-Miner optimized code causes warnings when used in the SAS da...

Re: SAS E-Miner optimized code causes warnings when used in the SAS da...

SAS E-Miner optimized code causes warnings when used in the SAS datast...

Re: Proc SQL with a macro in where statement to select on ID's

Re: Proc SQL with a macro in where statement to select on ID's

Proc SQL with a macro in where statement to select on ID's