A few thoughts... I'm sure you've already tried some of these. 1. Make sure to turn on the maximum performance statistics available with OPTIONS FULLSTIMER; You could set that in a program node. Also, make sure the project log is turned on so you can review all the log info at once. In EG 4.1 that's under File / Properties / Project Log. Consulting the project log with FULLSTIMER set will give you an idea of what steps are taking the most resources in terms of I/O, memory, CPU time, and real time. 2. Indexes can improve the performance of subsetting and joins. Since you're rebuilding the data every day, it might not be worth the overhead to recreate indexes, but if you do a lot of subsequent subsetting etc. it might be worth a try. 3. Joins may give you better performance that data step merges. Of course, you may be doing joins already. 4. Pre-sorting intelligently can sometimes reduce or eliminate insufficient memory messages. I'm thinking about procedures like PROC MEANS with a CLASS statement or PROC REPORT. Of course, sorting takes a lot of resources. 5. If you're firing off many separate types of analysis that are grouped by the same set of columns, pre-sorting based on that set of columns can reduce EG overhead. 6. Assuming you can get the insufficient memory problems under control, what's wrong with just scheduling your monster project to run at 4 AM each day? Ultimately, all the coding enhancements in the world will only get you so far since you're processing such a large volume of data. After that, you'd really have to upgrade your hardware and/or find a better storage medium for this much data (e.g. SPD file, Teradata, etc.). Again, many of these suggestions you may have tried and/or not apply to your project. Maybe one of these will help! What steps, specifically, are generating the insufficient memory messages? PROC SQL? PROC REPORT? ... View more

Bill, I would say the way you did it is the correct solution. Defining the prompt in the prompt manager makes sure that EG has the prompt available for use throughout the project. Getting those values used in the appropriate spot, and getting the appropriate prompt window, comes from associating the prompt with specific tasks, queries, or programs. I agree it is a little work, but think about how much time you're saving down the road by getting the prompt rather than having to manually edit the code each time! ... View more

Replied in the other thread you posted this on... ... View more

I found a reference to a program called ESS which seems to do something like this (search for ESS in the pdf): http://www2.sas.com/proceedings/sugi29/142-29.pdf The paper referenced is here: http://www.bepress.com/cgi/viewcontent.cgi?article=1002&context=uwbiostat I didn't have time for a read of what the conversion involves, so apologies if it isn't on track for what you want. Another resource that could be really valuable is the following, from the authors of the Little SAS Book. This is a paper about coming to SAS from SPSS: http://support.sas.com/publishing/bbu/companion_site/62272.pdf ... View more

I think you'd have to write code to do this -- I can't think of a feature to export a fixed-width file. You could use the DATA step as follows (simple example): data _null_; set sashelp.class; file "C:\test.txt"; put @1 height @10 name; run; ... View more

Hi Joshua, We are going to roll out an EG administration course for EG 4.2 some time next year. If you're using EG 4.1, there are already a series of e-Lectures available on this topic. There's some discussion in the EG 4.1 lectures on using a Metadata repository, though not everything is predicated on this. First lecture: https://support.sas.com/edu/schedules.html?id=417&ctry=US All five lectures can be found under SAS Platform Administration at: https://support.sas.com/edu/elearning.html?ctry=us&productType=electure The course and e-Lectures really focus on the software, not the hardware. In terms of hardware specs and monitoring, I'd definitely contact SAS and see what their thoughts are. Your sales rep would be a good starting point and could give you an idea of what resources are available. ... View more

In WRS 4.2 (not sure about previous versions) you can select File / E-mail. This will send a URL of the report to whomever you specify. http://support.sas.com/documentation/cdl/en/citug/61084/HTML/default/p0pek9xbqgt0gin1iaoaqmo9q9y2.htm Is that what you're looking for? ... View more

I think you may be in luck, if your underlying data is an OLAP cube. You can create a custom data item based on a relative time function, which allows OLAP data to display things like differences, percent change, and rolling totals in a time dimension. Search this documentation for "rolling total" and see if that will work for you: http://support.sas.com/documentation/cdl/en/citug/61084/HTML/default/n1irtj68ymv2o9n1l7z5oe7v0ber.htm ... View more

I haven't done a group break distribution personally, but I know it can be done. Here's the documentation; it starts talking about group breaks in step 12. http://support.sas.com/documentation/cdl/en/citug/61084/HTML/default/p0ssvy4ny3bg9mn1kr7cxepb2dc3.htm A word of warning: these are instructions for group breaks that are defined on a report section in WRS with a specific information map data source. You can still use the distribute report wizard on a report containing only a stored process. HOWEVER, if you had some sort of code in your stored process which defined groups, you'd have to write more code to distribute by those groups, probably using FILENAME EMAIL statements. So, the easy distribution for group breaks only works if you built the report based on an information map in WRS and selected hierarchy or category data items for the group breaks. ... View more

I'm not a stat person, so I'm going to punt on this... Try SAS Tech Support (it's free to all SAS customers!) at: http://support.sas.com/techsup/contact/. I'm not sure whether you should have them route you to a Stat expert or an EG expert -- I would probably tell them you have a stat question since that's the tougher part here. Perhaps someone else on this forum can provide advice, which would be excellent. I know enough about sampling to say with confidence that I don't know how to calculate standard deviation on the whole population based on just a sample. 🙂 ... View more

Couple thoughts: 1. Map a drive on the server to your local computer, then reference that drive (Z: or whatever) when you export. 2. You might also try the Download Data Files to PC task. That's saved me a couple times for doing this sort of thing. It's a new task in EG 4.2, although I believe it was available for EG 4.1 as a custom task. ... View more

My 2 cents on the subject: EG is nice because it has so much flexibility built into it. The disadvantage is that there's a definite learning curve to get used to creating projects, working with tasks, etc. A stored process would remove the learning curve aspect as users are asked a series of questions about what they want. The disadvantage is if your customer wanted something outside the pre-defined prompts for the stored process, you'd have to modify the stored process for each new request. And really, if the stored process has more than 3-6 prompts built in, it's going to start to feel really cumbersome to use. In terms of OLAP, that does let you drill down in a dynamic way. For a large data mart however, you may start to push the OLAP server and the I/O between it and the underlying data source as you go into lower levels of detail if there are many, many rows being analyzed and returned. So, it can be done, but performance-wise it may not be a good decision. Really depends on what you need returned, how much presummarization is in the cube, and the performance of your hardware and network. Another alternative would be to design reports in SAS Web Report Studio that feature report object linking. For example, you could click on a particular value in a crosstab table, and it would take you to another report (or report section) that was filtered based on where you clicked. This could be useful if the customer always wants the same general sorts of detail reports. One last alternative would be to use the Visual Data Explorer in the portal. This lets you select data items and create filters in an easy-to-use web-based environment. If the only part of EG you really needed was the query builder, the visual data explorer could be a nice alternative. Good luck! ... View more

Agreed with Patrick: I think (especially in EG 4.1) your only choices may be to turn off the automatic display of data sets and/or use PROC DATASETS or PROC SQL to delete some of the intermediary tables so they don't show up in the process flow. For example: proc sql; drop table work.test1, sasuser.query1; quit; In EG 4.2, there's a pop-up box to switch between output data sets from code or a task. That helps a little, but still isn't quite what you're asking for. ... View more

My suggestion would be exactly what you mention: create a new column using a PUT function and your user-defined format. IMO, I don't think you're missing anything simpler. ... View more

Ah, I see what you're saying. I'm playing around right now and am blanking on a simple way to do this. So, here's my best attempt -- I'll post again if I figure out something simpler! 1. New plan: let's use two Transpose Tasks instead of Split Columns. For the first Transpose task, Transpose variable is: Teachernm. Group analysis by variables are: name, dob. In the Options pane, set a prefix of Teachnm_course. The transpose task will automatically handle the numbering (1,2,3,...). 2. Second transpose task, Transpose variable is: school. Group analysis by variables are: name, dob. In the Options pane, set a prefix of school_course. 3. Use a Query Builder to join these two tables. Join based on name, dob. Drag all the columns into the select data area except the columns named SOURCE or LABEL. That should work. Again, if I figure out a better way to do it, I'll let you know. The other solution I came up with involved Transpose, then Stack Columns, then a Query Builder, then Split Columns. Not pretty! 🙂 ... View more