DATA Step, Macro, Functions and more

SAS 9.4 incredibly slow to import/sort Stata data file

Accepted Solution Solved
Reply
Occasional Contributor
Posts: 14
Accepted Solution

SAS 9.4 incredibly slow to import/sort Stata data file

[ Edited ]

Hi there,

 

My computer was recently upgraded to windows 10, so I changed over to SAS 9.4. Originally I had Windows 7 and was using SAS 9.3. I'm trying to import and sort a 2GB stata dataset that used to load in about 1-2 minutes with Windows7/SAS 9.3, Now it takes over 8 minutes. And then after it's imported, trying to merge this with any other dataset takes significantly more time. 

I'm using data steps, not proc SQL since I'm old school and haven't converted over yet Smiley Happy Could this be the problem? Any suggestions?

Thank you!

Jen


Accepted Solutions
Solution
‎05-15-2018 03:03 PM
Community Manager
Posts: 3,378

Re: SAS 9.4 incredibly slow to import/sort Stata data file

That's different and might not be related.  The SASUSER.PROFILE contains your SAS settings for things like key mappings, window layout, etc.

 

If it's not present or corrupted, SAS creates a new one. See the doc about how that works.

View solution in original post


All Replies
SAS Super FREQ
Posts: 4,171

Re: SAS 9.4 incredibly slow

Strange. I think we need more details. Posting your code would be good. For example, are you sorting by numerical variables or character? Are the characters variables encoded in WLATIN1 or something else? 

Community Manager
Posts: 3,378

Re: SAS 9.4 incredibly slow

The log would help too.  I assume you're using PROC IMPORT with DBMS=dta.

 

Your SAS session encoding might be a factor -- if SAS has to transcode from Latin1 to UTF-8.  Check the encoding with

 

PROC OPTIONS group=languagecontrol; run;

And is it the Import step or the Sort step that takes longer -- or both?

Super User
Posts: 13,298

Re: SAS 9.4 incredibly slow to import/sort Stata data file

Are you reloading Stata data that you had imported previously or is this a new data set?

I am asking because it almost sounds like you might be reading the same data source repeatedly which sounds inefficient.

 

Other things about your environment might have an impact. When you upgraded the SAS version did you move to a server version from stand-alone (or vice versa).

How much free memory do you have? I might be that your Windows10 set up isn't allowing as much memory for SAS if you upgraded OS software but no hardware.

Is network connectivity involved any where in the process?

Occasional Contributor
Posts: 14

Re: SAS 9.4 incredibly slow to import/sort Stata data file

Thanks everyone, something was going on with Windows 10, it was storing my temporary programs in my C drive, which used up all my memory. I didn't encounter these problems at all with Windows 7, so instead of working with IT who know nothing about SAS, I just decided to go back to Windows 7-seemed like the quickest, easiest solution for now. 

Community Manager
Posts: 3,378

Re: SAS 9.4 incredibly slow to import/sort Stata data file

Thankfully you didn't have to revert to Windows XP Smiley Wink

 

In future, one thing to check is the TEMP and/or TMP environment variable, which you can control in System Properties, Environment Variables settings.

 

tmp.png

 

Most apps, including SAS, take their cue for the "temp space" root from one of those environment variables.

 

up_temp.png

 

 

Your upgrade to Windows 10 might have affect that, or it might have changed the %USERPROFILE% location which in turn affects the Temp space.

Occasional Contributor
Posts: 14

Re: SAS 9.4 incredibly slow to import/sort Stata data file

Posted in reply to ChrisHemedinger
Yes! I'm having issues with the user profile! I didn't realize I was getting these error messages- any idea what's going on?:
NOTE: SASUSER.PROFILE.CATALOG does not exist and is being created from
NOTE: SASUSER.PROFBAK.CATALOG.

NOTE: SASUSER.PROFBAK.CATALOG has been copied to SASUSER.PROFILE.CATALOG.




Solution
‎05-15-2018 03:03 PM
Community Manager
Posts: 3,378

Re: SAS 9.4 incredibly slow to import/sort Stata data file

That's different and might not be related.  The SASUSER.PROFILE contains your SAS settings for things like key mappings, window layout, etc.

 

If it's not present or corrupted, SAS creates a new one. See the doc about how that works.

☑ This topic is solved.

Need further help from the community? Please ask a new question.

Discussion stats
  • 7 replies
  • 151 views
  • 0 likes
  • 4 in conversation