DATA Step, Macro, Functions and more

Disappearing Permanent Dataset

Accepted Solution Solved
Reply
Regular Contributor
Posts: 199
Accepted Solution

Disappearing Permanent Dataset

 

I am using a DATA step to create a new, permanent SAS dataset from a .csv file. 

 

The log indicates the dataset was successfully created: "NOTE: The data set SURVEY.CSCSP_2016 has 2864 observations and 141 variables." The dataset appears in the SURVEY library. 

 

When I run a PROC CONTENTS on the newly created dataset, it disappears from the SURVEY library and the Log states: " ERROR: File SURVEY.CSCSP_2016.DATA does not exist."

 

I have never experienced this before. Any clues?

 

Thanks! 

 

PROC CONTENTS DATA=survey.cscsp_2016 VARNUM;
RUN;


Accepted Solutions
Solution
‎05-25-2017 11:09 AM
Super User
Posts: 19,820

Re: Disappearing Permanent Dataset

Note that when you show the datasets in the library it didn't disappear, it was renamed somehow with a 2 and then a 3, which means it thinks you already have a copy in that folder somewhere. I think that it's only working when you run once is timing - before the rename happens. 

 

And once that 2 is there it's not a valid SAS name which may be why it's not showing up in the library. You could try changing VALIDVARNAME option to see it shows up in the library. 

 

option validvarname=any;

 

I don't know what's causing it to create that 2/3. Did you set up any automated back ups or anything else new on your Mac?

Is this a cloud space, ie Dropbox or Google drive that may be doing automatic versioning?

 

 

1. Check your library if you have the dataset in there already. If you do, erase it and see if you replicate the issue. 

 

2. Try the following before your programs and retry. 

 

options replace;

3. Can you replicate it with another dataset, what happens with the following, or if you add the SLEEP pause step in between your data import and proc contents. 

data survey.class;
set sashelp.class;
run;

data _null_;
sleep(10);
run;

proc contents data=survey.class;
run;

 

4. Have you modified any of the settings - changing your work location, options etc that are in your autoexec?

 

 

 

View solution in original post


All Replies
Contributor
Posts: 22

Re: Disappearing Permanent Dataset

That is odd. Are you sure the survey libref is not being changed between creating the dataset and running the proc contents? If you are in the Windows environment, you can look at the survey library in explorer right after creating the dataset and right before running the proc contents. You can also see the directory assigned to the library in the explorer window.

Super User
Posts: 19,820

Re: Disappearing Permanent Dataset

Post the full log that shows this please. If it happens as you've stated and you can replicate it you should contact Tech Support.

Regular Contributor
Posts: 199

Re: Disappearing Permanent Dataset

1 OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;
55
56 PROC CONTENTS DATA=survey.cscsp_2016 VARNUM;
ERROR: File SURVEY.CSCSP_2016.DATA does not exist.
57 RUN;
 
NOTE: Statements not processed because of errors noted above.
NOTE: PROCEDURE CONTENTS used (Total process time):
real time 0.02 seconds
cpu time 0.02 seconds
 
NOTE: The SAS System stopped processing this step because of errors.
58
59 OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;
71
Super User
Posts: 7,809

Re: Disappearing Permanent Dataset

That is only the log of the proc contents. Where is the step creating the dataset?

---------------------------------------------------------------------------------------------
Maxims of Maximally Efficient SAS Programmers
Regular Contributor
Posts: 199

Re: Disappearing Permanent Dataset

[ Edited ]
Posted in reply to KurtBremser
 
Contributor
Posts: 22

Re: Disappearing Permanent Dataset

I don't see anything unusual with your code.

Here are some troubleshooting ideas:

Will proc contents work if you resubmit the libname statment? Do you see the file in the directory?

if you just do a simple data step to create the dataset, does that work?

libname survey "/folders/myfolders/Cannabis";

data survey.cscsp_2016;

   a = 1;

 run;

 

proc contents data = survey.cscsp_2016 varnum;

run;

 

Super User
Posts: 19,820

Re: Disappearing Permanent Dataset

You need to show it back to back, ie the log with the input and proc contents in one series. Otherwise we have no idea of what's been done in between. 99% of these issues are user errors, which is why we're being so picky.

 

You should fix your import first by the way - that's far from a clean log. 

 

Regular Contributor
Posts: 199

Re: Disappearing Permanent Dataset

[ Edited ]

The PROC CONTENTS runs successfully when I run it concurrently w/ the LIBNAME and PROC CONTENTS steps, but not when I run it AFTER running the LIBNAME and DATA steps (Syntax below).

 

If I run the entire SAS program successfully, I get the "ERROR: File SURVEY.CSCSP_2016.DATA does not exist." error in the subsequent SAS program using that data set.

 

 Updated Log (6:30pm PST 05-23-2017)
1 OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;
NOTE: ODS statements in the SAS Studio environment may disable some output features.
56
57 LIBNAME survey "/folders/myfolders/Cannabis";
NOTE: Libref SURVEY was successfully assigned as follows:
Engine: V9
Physical Name: /folders/myfolders/Cannabis
58
59 * Reading in the original data from a .csv file;
60 * This file does not contain the hand-coded variables;
61
62
63 DATA survey.cscsp_2016;
64
65 infile "/folders/myfolders/Cannabis/cannabis_use_survey_data.csv"
66 delimiter = ','
67 MISSOVER
68 DSD
69 firstobs=2 ;
70
71 informat participant_id $4. ;
72 informat redcap_survey_identifier $1. ;
73 informat survey_timestamp anydtdtm40. ;
74 informat eligible best32. ;
75 informat generalhealth best32. ;
76 informat qol best32. ;
77 informat physicalhealth best32. ;
78 informat mentalmoodthink best32. ;
79 informat social best32. ;
80 informat socialrole best32. ;
81 informat everydayfunction best32. ;
82 informat emotionalprobs best32. ;
83 informat fatigue best32. ;
84 informat pain best32. ;
85 informat usertype___1 best32. ;
86 informat usertype___2 best32. ;
87 informat usertype___3 best32. ;
88 informat usertype___4 best32. ;
89 informat legalstatus $1. ;
90 informat conditions___1 best32. ;
91 informat conditions___2 best32. ;
92 informat conditions___3 best32. ;
93 informat conditions___4 best32. ;
94 informat conditions___5 best32. ;
95 informat conditions___6 best32. ;
96 informat conditions___7 best32. ;
97 informat conditions___8 best32. ;
98 informat conditions___9 best32. ;
99 informat conditions___10 best32. ;
100 informat conditions___11 best32. ;
101 informat conditions___12 best32. ;
102 informat conditions___13 best32. ;
103 informat conditions___14 best32. ;
104 informat conditions___15 best32. ;
105 informat conditions___16 best32. ;
106 informat conditions___17 best32. ;
107 informat conditions___18 best32. ;
108 informat conditions___20 best32. ;
109 informat conditions___21 best32. ;
110 informat conditions___22 best32. ;
111 informat conditions___19 best32. ;
112 informat other $53. ;
113 informat effectsoverall best32. ;
114 informat effectsoverall2_568 best32. ;
115 informat effectsoverall2_990 best32. ;
116 informat effectsoverall2_9902_34d best32. ;
117 informat effectsoverall2_9902_a12 best32. ;
118 informat effectsoverall2_9902_2_2fe best32. ;
119 informat effectsoverall2_9902_2_9df best32. ;
120 informat effectsoverall2_9902_2_ecc best32. ;
121 informat effectsoverall2_9902_2_292 best32. ;
122 informat effectsoverall2_9902_2_39a best32. ;
123 informat effectsoverall2_9902_2_258 best32. ;
124 informat effectsoverall2_9902_2_360 best32. ;
125 informat effectsoverall2_9902_2_902 best32. ;
126 informat effectsoverall2_9902_2_a2b best32. ;
127 informat effectsoverall2_9902_2_265 best32. ;
128 informat effectsoverall2_9902_2_384 best32. ;
129 informat effectsoverall2_9902_2_f58 best32. ;
130 informat effectsoverall2_9902_2_567 best32. ;
131 informat sleepimprovement best32. ;
132 informat pdimprove best32. ;
133 informat additionaluse $1. ;
134 informat shopping___1 best32. ;
135 informat shopping___2 best32. ;
136 informat shopping___3 best32. ;
137 informat shopping___4 best32. ;
138 informat shopping___5 best32. ;
139 informat shopping___6 best32. ;
140 informat shopping___7 best32. ;
141 informat shopping___8 best32. ;
142 informat shopping___9 best32. ;
143 informat shopping___10 best32. ;
144 informat shopping___11 best32. ;
145 informat shopping___12 best32. ;
146 informat shopping___13 best32. ;
147 informat shopping___14 best32. ;
148 informat shopping___15 best32. ;
149 informat shopping___16 best32. ;
150 informat shopping___17 best32. ;
151 informat shopping___18 best32. ;
152 informat method best32. ;
153 informat inhaleonset $1. ;
154 informat additionalmethods___1 best32. ;
155 informat additionalmethods___2 best32. ;
156 informat additionalmethods___3 best32. ;
157 informat additionalmethods___4 best32. ;
158 informat additionalmethods___5 best32. ;
159 informat additionalmethods___6 best32. ;
160 informat additionalmethods___7 best32. ;
161 informat additionalmethods___8 best32. ;
162 informat additionalmethods___9 best32. ;
163 informat edibleduration $1. ;
164 informat smokeduration $1. ;
165 informat edible_duration $1. ;
166 informat source___1 best32. ;
167 informat source___2 best32. ;
168 informat source___3 best32. ;
169 informat source___4 best32. ;
170 informat source___5 best32. ;
171 informat source___6 best32. ;
172 informat underground___1 best32. ;
173 informat underground___2 best32. ;
174 informat underground___3 best32. ;
175 informat underground___4 best32. ;
176 informat underground___5 best32. ;
177 informat quantity best32. ;
178 informat inhale best32. ;
179 informat frequency best32. ;
180 informat etoh best32. ;
181 informat tobacco best32. ;
182 informat otherdrugs best32. ;
183 informat listdrugs $345. ;
184 informat addict best32. ;
185 informat toomuch best32. ;
186 informat addictive best32. ;
187 informat problem $1. ;
188 informat describeproblem $1. ;
189 informat aa best32. ;
190 informat etohsub best32. ;
191 informat illegalsub best32. ;
192 informat rxsub best32. ;
193 informat rxsublist $450. ;
194 informat age best12. ;
195 informat gender best32. ;
196 informat job best32. ;
197 informat country1 $24. ;
198 informat state1 $24. ;
199 informat prov1 $24. ;
200 informat uk $1. ;
201 informat relationship best32. ;
202 informat ethnic best32. ;
203 informat education best32. ;
204 informat income best32. ;
205 informat privatecode $8. ;
206 informat additionalinfo $1236. ;
207 informat survey_complete best32. ;
208 informat country $24. ;
209 informat province $25. ;
210 informat zip $10. ;
211 informat respondent_data_complete best32. ;
212
213 format participant_id $4. ;
214 format redcap_survey_identifier $1. ;
215 format survey_timestamp datetime. ;
216 format eligible best12. ;
217 format generalhealth best12. ;
218 format qol best12. ;
219 format physicalhealth best12. ;
220 format mentalmoodthink best12. ;
221 format social best12. ;
222 format socialrole best12. ;
223 format everydayfunction best12. ;
224 format emotionalprobs best12. ;
225 format fatigue best12. ;
226 format pain best12. ;
227 format usertype___1 best12. ;
228 format usertype___2 best12. ;
229 format usertype___3 best12. ;
230 format usertype___4 best12. ;
231 format legalstatus $1. ;
232 format conditions___1 best12. ;
233 format conditions___2 best12. ;
234 format conditions___3 best12. ;
235 format conditions___4 best12. ;
236 format conditions___5 best12. ;
237 format conditions___6 best12. ;
238 format conditions___7 best12. ;
239 format conditions___8 best12. ;
240 format conditions___9 best12. ;
241 format conditions___10 best12. ;
242 format conditions___11 best12. ;
243 format conditions___12 best12. ;
244 format conditions___13 best12. ;
245 format conditions___14 best12. ;
246 format conditions___15 best12. ;
247 format conditions___16 best12. ;
248 format conditions___17 best12. ;
249 format conditions___18 best12. ;
250 format conditions___20 best12. ;
251 format conditions___21 best12. ;
252 format conditions___22 best12. ;
253 format conditions___19 best12. ;
254 format other $53. ;
255 format effectsoverall best12. ;
256 format effectsoverall2_568 best12. ;
257 format effectsoverall2_990 best12. ;
258 format effectsoverall2_9902_34d best12. ;
259 format effectsoverall2_9902_a12 best12. ;
260 format effectsoverall2_9902_2_2fe best12. ;
261 format effectsoverall2_9902_2_9df best12. ;
262 format effectsoverall2_9902_2_ecc best12. ;
263 format effectsoverall2_9902_2_292 best12. ;
264 format effectsoverall2_9902_2_39a best12. ;
265 format effectsoverall2_9902_2_258 best12. ;
266 format effectsoverall2_9902_2_360 best12. ;
267 format effectsoverall2_9902_2_902 best12. ;
268 format effectsoverall2_9902_2_a2b best12. ;
269 format effectsoverall2_9902_2_265 best12. ;
270 format effectsoverall2_9902_2_384 best12. ;
271 format effectsoverall2_9902_2_f58 best12. ;
272 format effectsoverall2_9902_2_567 best12. ;
273 format sleepimprovement best12.;
274 format pdimprove best12.;
275 format additionaluse $1. ;
276 format shopping___1 best12. ;
277 format shopping___2 best12. ;
278 format shopping___3 best12. ;
279 format shopping___4 best12. ;
280 format shopping___5 best12. ;
281 format shopping___6 best12. ;
282 format shopping___7 best12. ;
283 format shopping___8 best12. ;
284 format shopping___9 best12. ;
285 format shopping___10 best12. ;
286 format shopping___11 best12. ;
287 format shopping___12 best12. ;
288 format shopping___13 best12. ;
289 format shopping___14 best12. ;
290 format shopping___15 best12. ;
291 format shopping___16 best12. ;
292 format shopping___17 best12. ;
293 format shopping___18 best12. ;
294 format method best12. ;
295 format inhaleonset $1. ;
296 format additionalmethods___1 best12. ;
297 format additionalmethods___2 best12. ;
298 format additionalmethods___3 best12. ;
299 format additionalmethods___4 best12. ;
300 format additionalmethods___5 best12. ;
301 format additionalmethods___6 best12. ;
302 format additionalmethods___7 best12. ;
303 format additionalmethods___8 best12. ;
304 format additionalmethods___9 best12. ;
305 format edibleduration $1. ;
306 format smokeduration $1. ;
307 format edible_duration $1. ;
308 format source___1 best12. ;
309 format source___2 best12. ;
310 format source___3 best12. ;
311 format source___4 best12. ;
312 format source___5 best12. ;
313 format source___6 best12. ;
314 format underground___1 best12. ;
315 format underground___2 best12. ;
316 format underground___3 best12. ;
317 format underground___4 best12. ;
318 format underground___5 best12. ;
319 format quantity best12. ;
320 format inhale best12. ;
321 format frequency best12. ;
322 format etoh best12. ;
323 format tobacco best12. ;
324 format otherdrugs best12. ;
325 format listdrugs $345. ;
326 format addict best12. ;
327 format toomuch best12. ;
328 format addictive best12. ;
329 format problem $1. ;
330 format describeproblem $1. ;
331 format aa best12. ;
332 format etohsub best12. ;
333 format illegalsub best12. ;
334 format rxsub best12. ;
335 format rxsublist $450. ;
336 format age best12. ;
337 format gender best12. ;
338 format job best12. ;
339 format country1 $24. ;
340 format state1 $24. ;
341 format prov1 $24. ;
342 format uk $1. ;
343 format relationship best12. ;
344 format ethnic best12. ;
345 format education best12. ;
346 format income best12. ;
347 format privatecode $8. ;
348 format additionalinfo $1236. ;
349 format survey_complete best12. ;
350 format country $24. ;
351 format province $25. ;
352 format zip $10. ;
353 format respondent_data_complete best12. ;
354
355 input
356 participant_id $
357 redcap_survey_identifier $
358 survey_timestamp
359 eligible
360 generalhealth
361 qol
362 physicalhealth
363 mentalmoodthink
364 social
365 socialrole
366 everydayfunction
367 emotionalprobs
368 fatigue
369 pain
370 usertype___1
371 usertype___2
372 usertype___3
373 usertype___4
374 legalstatus $
375 conditions___1
376 conditions___2
377 conditions___3
378 conditions___4
379 conditions___5
380 conditions___6
381 conditions___7
382 conditions___8
383 conditions___9
384 conditions___10
385 conditions___11
386 conditions___12
387 conditions___13
388 conditions___14
389 conditions___15
390 conditions___16
391 conditions___17
392 conditions___18
393 conditions___20
394 conditions___21
395 conditions___22
396 conditions___19
397 other $
398 effectsoverall $
399 effectsoverall2_568 $
400 effectsoverall2_990
401 effectsoverall2_9902_34d $
402 effectsoverall2_9902_a12 $
403 effectsoverall2_9902_2_2fe $
404 effectsoverall2_9902_2_9df $
405 effectsoverall2_9902_2_ecc
406 effectsoverall2_9902_2_292 $
407 effectsoverall2_9902_2_39a
408 effectsoverall2_9902_2_258
409 effectsoverall2_9902_2_360
410 effectsoverall2_9902_2_902
411 effectsoverall2_9902_2_a2b
412 effectsoverall2_9902_2_265 $
413 effectsoverall2_9902_2_384
414 effectsoverall2_9902_2_f58 $
415 effectsoverall2_9902_2_567
416 sleepimprovement $
417 pdimprove $
418 additionaluse $
419 shopping___1
420 shopping___2
421 shopping___3
422 shopping___4
423 shopping___5
424 shopping___6
425 shopping___7
426 shopping___8
427 shopping___9
428 shopping___10
429 shopping___11
430 shopping___12
431 shopping___13
432 shopping___14
433 shopping___15
434 shopping___16
435 shopping___17
436 shopping___18
437 method
438 inhaleonset $
439 additionalmethods___1
440 additionalmethods___2
441 additionalmethods___3
442 additionalmethods___4
443 additionalmethods___5
444 additionalmethods___6
445 additionalmethods___7
446 additionalmethods___8
447 additionalmethods___9
448 edibleduration $
449 smokeduration $
450 edible_duration $
451 source___1
452 source___2
453 source___3
454 source___4
455 source___5
456 source___6
457 underground___1
458 underground___2
459 underground___3
460 underground___4
461 underground___5
462 quantity
463 inhale
464 frequency
465 etoh
466 tobacco
467 otherdrugs
468 listdrugs $
469 addict
470 toomuch
471 addictive
472 problem $
473 describeproblem $
474 aa
475 etohsub
476 illegalsub
477 rxsub
478 rxsublist $
479 age
480 gender
481 job
482 country1 $
483 state1 $
484 prov1 $
485 uk $
486 relationship
487 ethnic
488 education
489 income
490 privatecode $
491 additionalinfo $
492 survey_complete
493 country $
494 province $
495 zip $
496 respondent_data_complete
497 ;
498 RUN;
 
NOTE: The infile "/folders/myfolders/Cannabis/cannabis_use_survey_data.csv" is:
Filename=/folders/myfolders/Cannabis/cannabis_use_survey_data.csv,
Owner Name=root,Group Name=vboxsf,
Access Permission=-rwxrwx---,
Last Modified=23May2017:18:25:04,
File Size (bytes)=1346807
 
NOTE: 2864 records were read from the infile "/folders/myfolders/Cannabis/cannabis_use_survey_data.csv".
The minimum record length was 222.
The maximum record length was 3695.
NOTE: The data set SURVEY.CSCSP_2016 has 2864 observations and 141 variables.
NOTE: DATA statement used (Total process time):
real time 0.35 seconds
cpu time 0.25 seconds
 
 
499
500 PROC CONTENTS DATA=survey.cscsp_2016 VARNUM;
501 RUN;
 
NOTE: PROCEDURE CONTENTS used (Total process time):
real time 0.44 seconds
cpu time 0.42 seconds
 
 
502
503 OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;
515
 
Respected Advisor
Posts: 4,173

Re: Disappearing Permanent Dataset

[ Edited ]

@jcorroon

The two most likely causes for this disappearance are:

 

A) You run your code against two different SAS servers

--> make sure you execute against the same server

 

B) You run your code in two different sessions

--> make sure you run both steps in the same session OR add the libname statement from your import step also the code with your Proc Contents.

 

And as a first test:

Copy your Proc contents code into the same code window where you've got your import code and run everything together. If this still doesn't work then please post the code and log of this execution so we can see everything together.

 

Regular Contributor
Posts: 199

Re: Disappearing Permanent Dataset

<your import code>
Could you please clarify?
Respected Advisor
Posts: 4,173

Re: Disappearing Permanent Dataset

@jcorroon

"Could you please clarify?"

The data step you've posted (infile / input)

 

Super User
Posts: 19,820

Re: Disappearing Permanent Dataset


jcorroon wrote:
<your import code>
Could you please clarify?

There's no way to know what you're referring to here. Your 'IMPORT CODE' is the data step that you use to read the CSV file. It's obviously incorrect since you have all the NOTES in your log. So you should fix this. 

 

You're statement that if you run them back to back means everything is working as expected. You have something else going on. 

 

You should talk to someone on site or someone familiar with your installation to see. It looks like you're working on SAS UE, but I know that isn't an issue because I constantly use permanent data sets. So without knowing exactly what you're doing  ¯\_(ツ)_/¯

 

Explain what you're doing step by step and include your log. And fix that import first, trying to wade through that is difficult and makes it easy to miss errors/notes that you should deal with. 

 

Regular Contributor
Posts: 199

Re: Disappearing Permanent Dataset

@Reeza 

 

<it's obviously incorrect since you have all the NOTES in your log. So you should fix this.>

 

It's fixed and re-posted.

 

<Explain what you're doing step by step>

 

I have a SAS program w/ a LIBNAME statement, a DATA step and a PROC CONTENTS step in one window. When I run this entire program, it executes succesfully. The log is posted above. However, when I then try to use the resulting data set (SURVEY.CSCSP_2016.DATA) in any other SAS program, I get an error in the Log: ERROR: File SURVEY.CSCSP_2016.DATA does not exist. I get the same error in the PROC CONTENTS step when I run the steps in this program incrementally (i.e. LIBNAME + DATA step, then PROC CONTENTS). 

 

The SAS dataset appears in the SURVEY library when the DATA step is executed, but then disappears when the subsequent SAS program is run (e.g. PROC CONTENTS). And, a new SAS data set is created in the folder on my HD with an incremental number assigned to the data set name (i.e. cscsp_2016 2.sas7bdat, cscsp_2016 3.sas7bdat, cscsp_2016 4.sas7bdat, etc.) for each time the DATA step is re-run.

 

This syntax ran w/o problem for 2 months. All of the sudden it is choking.

 

 

 

 

 

Super User
Posts: 19,820

Re: Disappearing Permanent Dataset

Are you by any chance using SAS UE in multiple windows/tabs?

☑ This topic is solved.

Need further help from the community? Please ask a new question.

Discussion stats
  • 29 replies
  • 231 views
  • 9 likes
  • 5 in conversation