yoanbolduc Tracker

Re: Repeat value set to blank

yoanbolduc — Wed, 12 Jun 2013 21:03:33 GMT

Hi SASPhile,

proc sort data=inputds out=sortedds;

by id acct1;

data outputds;

set sortedds;

lag_acct1 = lag(acct1);

if ^first.id then

if acct1 = lag_acct1 then acct1 = .;

drop lag_acct1;

run;

And you do the same for acct2.

If the acct1 and acct2 pair doesn't matter, you could simply produce 2 output datasets.

I hope this helps.

Yoan

Re: lag function issue

yoanbolduc — Wed, 12 Jun 2013 20:25:12 GMT

Hi,

You did it right, but apparently this approach doesn't work in all cases. The test I had done was simpler. I just looked for a different approach but Astounding did beat me with a better solution for your case.

In the case you needed to use the input statement instead of set, here is what I came up with (which is not as elegant as the solution proposed by Astounding).

data work.toto;

input value;

retain lag_value;

if missing(value) then do;

value = lag_value+1;

end;

lag_value = value;

datalines;;;

;;;

run;

So basically I reproduced the functionnality of the lag function using a retain statement. This one does work with input but is not very elegant.

Re: Need help with coding/syntax In SAS Enterprise Guide, more details inside.

yoanbolduc — Wed, 12 Jun 2013 20:13:55 GMT

Hi SEG,

I think we would need a little more details about where you execute your code. If you are executing it on a Workspace server, your SAS administrator should be able to activate logging on that workspace server so that your session activity is logged on the server.

Other than that, any Base SAS code that you write should run without problem. You simply have to make sure that you include it in a "Code node". There is one exception it's what we call X commands and this is what you are trying to do in the code you copied in your post. X Commands have to be allowed for the Workspace server that you are working on for any calls to the system function or the X statement to work. Your administrator will most probably object to change this setting so you may have so arguing to do to convince him that it is required to make your program work.

If you are NOT working on a workspace server, which means you are running your code locally and using EG to submit SAS to a local SAS installation. Then any Base SAS code should run fine in a code node.

For the log part, I am sorry I don't have enough experience using EG on a daily basis to help you with that aspect.

Yoan

Re: lag function issue

yoanbolduc — Wed, 12 Jun 2013 20:02:06 GMT

Hi AllSoEasy,

I might be wrong but my understanding of how lag works is that the value for the next iteration is established when you call the lag function in the current iteration. So what is means in practice is that when you do "lag_flat_rate = lag(Flat_Rate);" at your second observation, the current value of "Flat_Rate" is put in the buffer, that is missing. So even if the value of "Flat_Rate" is changed later in the current iteration, that buffer used by the lag function is not updated. Adding a second "lag_flat_rate = lag(Flat_Rate);" after the value has been updated will give you the result you expect.

I hope this helps you.

Yoan

Re: How to best install SAS products with these hardware specs

yoanbolduc — Wed, 12 Jun 2013 18:42:18 GMT

Hi Jared,

I understand the situation with your IT dept. Keep in mind, Linux is not only for server applications, it can be used as a desktop operating system too and a very great one actually.

OS and Apps don't require that much I/O so this is not where I would invest the most. A good 512 GB SSD will outperform any 2 drive RAID 0 setup in any application. You need at least 4 HD in RAID 0 to match a good 512 GB SSD. A good rule of thumb is: SSD for speed, HD for volume.

Going RAID 1 for storage is very good for data security. But you will get poor write performance and very low capacity compared to a comparable RAID 5 setup. The only issue with RAID 5 is that you will need AT LEAST 3 disks. Also, RAID 5 does not take random IO as well as RAID 1 or RAID 0 but with SAS you'll get mostly sequentiel IO so you don't have to worry about this.

Here is something to think about (please keep in mind that this is oversimplified): you have 16 cores, so you should aim for 800 MB/s of IO throughput. This is a lot but your SSD should give you close to 500 MB/s and it is safe to expect 170-200 MB/s from a 3 disk RAID 5 with a good controller. So if you plan your workload correctly, you should be able to keep this CPU busy!

Yoan

Re: Regarding Auto exec

yoanbolduc — Wed, 12 Jun 2013 18:26:58 GMT

Hi Jason,

If you are in a server environment with a metadata server you can define multiple Workspace server, each with it own sasv9.cfg and autoexec.sas file. Using metadata security you'll be able to restrict access to each of those workspace server to specific groups or users (departments and individuals).

This is the most efficient approach I found to support departments with different configurations.

I hope this helps.

Yoan

Re: Table Lock

yoanbolduc — Wed, 12 Jun 2013 18:22:48 GMT

Hi ksuman,

As everybody mentionned, this is very typical of a multi-user SAS environment. The idea that Tom proposed of using a database engine would probably be the best approach. The Base SAS engine is a very poor choice for multi-user environment as it relies on the operating system file locking mechanism.

If you want to keep using the Base SAS engine, you can use the fuser command on any UNIX-like operating system to list active locks on a file. It is easy to build a small script that will kill the locking processes but this is a very drastic approach.

I hope this helps a little,

Yoan

Re: How to best install SAS products with these hardware specs

yoanbolduc — Wed, 12 Jun 2013 18:11:11 GMT

Hi Jared,

Looked at the setup you are proposing. I am a little puzzled by your hardware decisions. I find this to be a pretty "big" machine to still use a client version of Windows or even Windows at all. I see you want to use Visual Studio so this is probably the reason why you opted for Windows. If you are flexible on the operating system choice, I would recommend installing Linux instead of Windows.

For your disk system, have you considered using a RAID 5 made of maybe 4 or 5 of thos 1 TB 10 KRPM disks? You'll get data protection and increase IOPS. You have to keep in mind that SAS usually requires about 50-75 MB/s of I/O per CPU core to ensure all CPU cores are "busy". As for your other disks, having 2 SSDs (one for OS/Apps and one for work) would be the best. If you don't want to purchase 2 SSDs, I would put the OS and Apps on a HD and SASWORK on the SSD. Yes, your SSD's lifespan will be much shorter but the performance gain you'll observe are definitely worth it. Also, as you seam to understand, SAS does mostly sequential I/O so this help "save" your SSD.

Read-Ahead and Write-Behind is useful. But multi-pathing only applies if you are using a SAN which does not seams to be your case.

For the procs, I'm not sure what the answer is. I rarely see people using the desktop version of SAS anymore so I can hardly tell, best thing would be to check with your sales rep.

I hope this helps a little.

Yoan