topic Re: Disappearing Permanent Dataset in SAS Programming

Disappearing Permanent Dataset

_maldini_ — Tue, 23 May 2017 18:03:45 GMT

I am using a DATA step to create a new, permanent SAS dataset from a .csv file.

The log indicates the dataset was successfully created: "NOTE: The data set SURVEY.CSCSP_2016 has 2864 observations and 141 variables." The dataset appears in the SURVEY library.

When I run a PROC CONTENTS on the newly created dataset, it disappears from the SURVEY library and the Log states: " ERROR: File SURVEY.CSCSP_2016.DATA does not exist."

I have never experienced this before. Any clues?

Thanks!

PROC CONTENTS DATA=survey.cscsp_2016 VARNUM;
RUN;

Re: Disappearing Permanent Dataset

sschleede — Tue, 23 May 2017 18:17:59 GMT

That is odd. Are you sure the survey libref is not being changed between creating the dataset and running the proc contents? If you are in the Windows environment, you can look at the survey library in explorer right after creating the dataset and right before running the proc contents. You can also see the directory assigned to the library in the explorer window.

Re: Disappearing Permanent Dataset

Reeza — Tue, 23 May 2017 19:20:33 GMT

Post the full log that shows this please. If it happens as you've stated and you can replicate it you should contact Tech Support.

Re: Disappearing Permanent Dataset

_maldini_ — Tue, 23 May 2017 20:33:25 GMT

1 OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;

56 PROC CONTENTS DATA=survey.cscsp_2016 VARNUM;

ERROR: File SURVEY.CSCSP_2016.DATA does not exist.

57 RUN;

NOTE: Statements not processed because of errors noted above.

NOTE: PROCEDURE CONTENTS used (Total process time):

real time 0.02 seconds

cpu time 0.02 seconds

NOTE: The SAS System stopped processing this step because of errors.

59 OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;

Re: Disappearing Permanent Dataset

Kurt_Bremser — Tue, 23 May 2017 20:40:37 GMT

That is only the log of the proc contents. Where is the step creating the dataset?

Re: Disappearing Permanent Dataset

_maldini_ — Wed, 24 May 2017 01:28:49 GMT

Re: Disappearing Permanent Dataset

sschleede — Tue, 23 May 2017 21:04:49 GMT

I don't see anything unusual with your code.

Here are some troubleshooting ideas:

Will proc contents work if you resubmit the libname statment? Do you see the file in the directory?

if you just do a simple data step to create the dataset, does that work?

libname survey "/folders/myfolders/Cannabis";

data survey.cscsp_2016;

a = 1;

run;

proc contents data = survey.cscsp_2016 varnum;

run;

Re: Disappearing Permanent Dataset

Reeza — Tue, 23 May 2017 21:24:15 GMT

You need to show it back to back, ie the log with the input and proc contents in one series. Otherwise we have no idea of what's been done in between. 99% of these issues are user errors, which is why we're being so picky.

You should fix your import first by the way - that's far from a clean log.

Re: Disappearing Permanent Dataset

Patrick — Tue, 23 May 2017 21:41:00 GMT

@_maldini_

The two most likely causes for this disappearance are:

A) You run your code against two different SAS servers

--> make sure you execute against the same server

B) You run your code in two different sessions

--> make sure you run both steps in the same session OR add the libname statement from your import step also the code with your Proc Contents.

And as a first test:

Copy your Proc contents code into the same code window where you've got your import code and run everything together. If this still doesn't work then please post the code and log of this execution so we can see everything together.

Re: Disappearing Permanent Dataset

_maldini_ — Wed, 24 May 2017 01:28:02 GMT

The PROC CONTENTS runs successfully when I run it concurrently w/ the LIBNAME and PROC CONTENTS steps, but not when I run it AFTER running the LIBNAME and DATA steps (Syntax below).

If I run the entire SAS program successfully, I get the "ERROR: File SURVEY.CSCSP_2016.DATA does not exist." error in the subsequent SAS program using that data set.

Updated Log (6:30pm PST 05-23-2017)

1 OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;

NOTE: ODS statements in the SAS Studio environment may disable some output features.

57 LIBNAME survey "/folders/myfolders/Cannabis";

NOTE: Libref SURVEY was successfully assigned as follows:

Engine: V9

Physical Name: /folders/myfolders/Cannabis

59 * Reading in the original data from a .csv file;

60 * This file does not contain the hand-coded variables;

63 DATA survey.cscsp_2016;

65 infile "/folders/myfolders/Cannabis/cannabis_use_survey_data.csv"

66 delimiter = ','

67 MISSOVER

68 DSD

69 firstobs=2 ;

71 informat participant_id $4. ;

72 informat redcap_survey_identifier $1. ;

73 informat survey_timestamp anydtdtm40. ;

74 informat eligible best32. ;

75 informat generalhealth best32. ;

76 informat qol best32. ;

77 informat physicalhealth best32. ;

78 informat mentalmoodthink best32. ;

79 informat social best32. ;

80 informat socialrole best32. ;

81 informat everydayfunction best32. ;

82 informat emotionalprobs best32. ;

83 informat fatigue best32. ;

84 informat pain best32. ;

85 informat usertype___1 best32. ;

86 informat usertype___2 best32. ;

87 informat usertype___3 best32. ;

88 informat usertype___4 best32. ;

89 informat legalstatus $1. ;

90 informat conditions___1 best32. ;

91 informat conditions___2 best32. ;

92 informat conditions___3 best32. ;

93 informat conditions___4 best32. ;

94 informat conditions___5 best32. ;

95 informat conditions___6 best32. ;

96 informat conditions___7 best32. ;

97 informat conditions___8 best32. ;

98 informat conditions___9 best32. ;

99 informat conditions___10 best32. ;

100 informat conditions___11 best32. ;

101 informat conditions___12 best32. ;

102 informat conditions___13 best32. ;

103 informat conditions___14 best32. ;

104 informat conditions___15 best32. ;

105 informat conditions___16 best32. ;

106 informat conditions___17 best32. ;

107 informat conditions___18 best32. ;

108 informat conditions___20 best32. ;

109 informat conditions___21 best32. ;

110 informat conditions___22 best32. ;

111 informat conditions___19 best32. ;

112 informat other $53. ;

113 informat effectsoverall best32. ;

114 informat effectsoverall2_568 best32. ;

115 informat effectsoverall2_990 best32. ;

116 informat effectsoverall2_9902_34d best32. ;

117 informat effectsoverall2_9902_a12 best32. ;

118 informat effectsoverall2_9902_2_2fe best32. ;

119 informat effectsoverall2_9902_2_9df best32. ;

120 informat effectsoverall2_9902_2_ecc best32. ;

121 informat effectsoverall2_9902_2_292 best32. ;

122 informat effectsoverall2_9902_2_39a best32. ;

123 informat effectsoverall2_9902_2_258 best32. ;

124 informat effectsoverall2_9902_2_360 best32. ;

125 informat effectsoverall2_9902_2_902 best32. ;

126 informat effectsoverall2_9902_2_a2b best32. ;

127 informat effectsoverall2_9902_2_265 best32. ;

128 informat effectsoverall2_9902_2_384 best32. ;

129 informat effectsoverall2_9902_2_f58 best32. ;

130 informat effectsoverall2_9902_2_567 best32. ;

131 informat sleepimprovement best32. ;

132 informat pdimprove best32. ;

133 informat additionaluse $1. ;

134 informat shopping___1 best32. ;

135 informat shopping___2 best32. ;

136 informat shopping___3 best32. ;

137 informat shopping___4 best32. ;

138 informat shopping___5 best32. ;

139 informat shopping___6 best32. ;

140 informat shopping___7 best32. ;

141 informat shopping___8 best32. ;

142 informat shopping___9 best32. ;

143 informat shopping___10 best32. ;

144 informat shopping___11 best32. ;

145 informat shopping___12 best32. ;

146 informat shopping___13 best32. ;

147 informat shopping___14 best32. ;

148 informat shopping___15 best32. ;

149 informat shopping___16 best32. ;

150 informat shopping___17 best32. ;

151 informat shopping___18 best32. ;

152 informat method best32. ;

153 informat inhaleonset $1. ;

154 informat additionalmethods___1 best32. ;

155 informat additionalmethods___2 best32. ;

156 informat additionalmethods___3 best32. ;

157 informat additionalmethods___4 best32. ;

158 informat additionalmethods___5 best32. ;

159 informat additionalmethods___6 best32. ;

160 informat additionalmethods___7 best32. ;

161 informat additionalmethods___8 best32. ;

162 informat additionalmethods___9 best32. ;

163 informat edibleduration $1. ;

164 informat smokeduration $1. ;

165 informat edible_duration $1. ;

166 informat source___1 best32. ;

167 informat source___2 best32. ;

168 informat source___3 best32. ;

169 informat source___4 best32. ;

170 informat source___5 best32. ;

171 informat source___6 best32. ;

172 informat underground___1 best32. ;

173 informat underground___2 best32. ;

174 informat underground___3 best32. ;

175 informat underground___4 best32. ;

176 informat underground___5 best32. ;

177 informat quantity best32. ;

178 informat inhale best32. ;

179 informat frequency best32. ;

180 informat etoh best32. ;

181 informat tobacco best32. ;

182 informat otherdrugs best32. ;

183 informat listdrugs $345. ;

184 informat addict best32. ;

185 informat toomuch best32. ;

186 informat addictive best32. ;

187 informat problem $1. ;

188 informat describeproblem $1. ;

189 informat aa best32. ;

190 informat etohsub best32. ;

191 informat illegalsub best32. ;

192 informat rxsub best32. ;

193 informat rxsublist $450. ;

194 informat age best12. ;

195 informat gender best32. ;

196 informat job best32. ;

197 informat country1 $24. ;

198 informat state1 $24. ;

199 informat prov1 $24. ;

200 informat uk $1. ;

201 informat relationship best32. ;

202 informat ethnic best32. ;

203 informat education best32. ;

204 informat income best32. ;

205 informat privatecode $8. ;

206 informat additionalinfo $1236. ;

207 informat survey_complete best32. ;

208 informat country $24. ;

209 informat province $25. ;

210 informat zip $10. ;

211 informat respondent_data_complete best32. ;

212

213 format participant_id $4. ;

214 format redcap_survey_identifier $1. ;

215 format survey_timestamp datetime. ;

216 format eligible best12. ;

217 format generalhealth best12. ;

218 format qol best12. ;

219 format physicalhealth best12. ;

220 format mentalmoodthink best12. ;

221 format social best12. ;

222 format socialrole best12. ;

223 format everydayfunction best12. ;

224 format emotionalprobs best12. ;

225 format fatigue best12. ;

226 format pain best12. ;

227 format usertype___1 best12. ;

228 format usertype___2 best12. ;

229 format usertype___3 best12. ;

230 format usertype___4 best12. ;

231 format legalstatus $1. ;

232 format conditions___1 best12. ;

233 format conditions___2 best12. ;

234 format conditions___3 best12. ;

235 format conditions___4 best12. ;

236 format conditions___5 best12. ;

237 format conditions___6 best12. ;

238 format conditions___7 best12. ;

239 format conditions___8 best12. ;

240 format conditions___9 best12. ;

241 format conditions___10 best12. ;

242 format conditions___11 best12. ;

243 format conditions___12 best12. ;

244 format conditions___13 best12. ;

245 format conditions___14 best12. ;

246 format conditions___15 best12. ;

247 format conditions___16 best12. ;

248 format conditions___17 best12. ;

249 format conditions___18 best12. ;

250 format conditions___20 best12. ;

251 format conditions___21 best12. ;

252 format conditions___22 best12. ;

253 format conditions___19 best12. ;

254 format other $53. ;

255 format effectsoverall best12. ;

256 format effectsoverall2_568 best12. ;

257 format effectsoverall2_990 best12. ;

258 format effectsoverall2_9902_34d best12. ;

259 format effectsoverall2_9902_a12 best12. ;

260 format effectsoverall2_9902_2_2fe best12. ;

261 format effectsoverall2_9902_2_9df best12. ;

262 format effectsoverall2_9902_2_ecc best12. ;

263 format effectsoverall2_9902_2_292 best12. ;

264 format effectsoverall2_9902_2_39a best12. ;

265 format effectsoverall2_9902_2_258 best12. ;

266 format effectsoverall2_9902_2_360 best12. ;

267 format effectsoverall2_9902_2_902 best12. ;

268 format effectsoverall2_9902_2_a2b best12. ;

269 format effectsoverall2_9902_2_265 best12. ;

270 format effectsoverall2_9902_2_384 best12. ;

271 format effectsoverall2_9902_2_f58 best12. ;

272 format effectsoverall2_9902_2_567 best12. ;

273 format sleepimprovement best12.;

274 format pdimprove best12.;

275 format additionaluse $1. ;

276 format shopping___1 best12. ;

277 format shopping___2 best12. ;

278 format shopping___3 best12. ;

279 format shopping___4 best12. ;

280 format shopping___5 best12. ;

281 format shopping___6 best12. ;

282 format shopping___7 best12. ;

283 format shopping___8 best12. ;

284 format shopping___9 best12. ;

285 format shopping___10 best12. ;

286 format shopping___11 best12. ;

287 format shopping___12 best12. ;

288 format shopping___13 best12. ;

289 format shopping___14 best12. ;

290 format shopping___15 best12. ;

291 format shopping___16 best12. ;

292 format shopping___17 best12. ;

293 format shopping___18 best12. ;

294 format method best12. ;

295 format inhaleonset $1. ;

296 format additionalmethods___1 best12. ;

297 format additionalmethods___2 best12. ;

298 format additionalmethods___3 best12. ;

299 format additionalmethods___4 best12. ;

300 format additionalmethods___5 best12. ;

301 format additionalmethods___6 best12. ;

302 format additionalmethods___7 best12. ;

303 format additionalmethods___8 best12. ;

304 format additionalmethods___9 best12. ;

305 format edibleduration $1. ;

306 format smokeduration $1. ;

307 format edible_duration $1. ;

308 format source___1 best12. ;

309 format source___2 best12. ;

310 format source___3 best12. ;

311 format source___4 best12. ;

312 format source___5 best12. ;

313 format source___6 best12. ;

314 format underground___1 best12. ;

315 format underground___2 best12. ;

316 format underground___3 best12. ;

317 format underground___4 best12. ;

318 format underground___5 best12. ;

319 format quantity best12. ;

320 format inhale best12. ;

321 format frequency best12. ;

322 format etoh best12. ;

323 format tobacco best12. ;

324 format otherdrugs best12. ;

325 format listdrugs $345. ;

326 format addict best12. ;

327 format toomuch best12. ;

328 format addictive best12. ;

329 format problem $1. ;

330 format describeproblem $1. ;

331 format aa best12. ;

332 format etohsub best12. ;

333 format illegalsub best12. ;

334 format rxsub best12. ;

335 format rxsublist $450. ;

336 format age best12. ;

337 format gender best12. ;

338 format job best12. ;

339 format country1 $24. ;

340 format state1 $24. ;

341 format prov1 $24. ;

342 format uk $1. ;

343 format relationship best12. ;

344 format ethnic best12. ;

345 format education best12. ;

346 format income best12. ;

347 format privatecode $8. ;

348 format additionalinfo $1236. ;

349 format survey_complete best12. ;

350 format country $24. ;

351 format province $25. ;

352 format zip $10. ;

353 format respondent_data_complete best12. ;

354

355 input

356 participant_id $

357 redcap_survey_identifier $

358 survey_timestamp

359 eligible

360 generalhealth

361 qol

362 physicalhealth

363 mentalmoodthink

364 social

365 socialrole

366 everydayfunction

367 emotionalprobs

368 fatigue

369 pain

370 usertype___1

371 usertype___2

372 usertype___3

373 usertype___4

374 legalstatus $

375 conditions___1

376 conditions___2

377 conditions___3

378 conditions___4

379 conditions___5

380 conditions___6

381 conditions___7

382 conditions___8

383 conditions___9

384 conditions___10

385 conditions___11

386 conditions___12

387 conditions___13

388 conditions___14

389 conditions___15

390 conditions___16

391 conditions___17

392 conditions___18

393 conditions___20

394 conditions___21

395 conditions___22

396 conditions___19

397 other $

398 effectsoverall $

399 effectsoverall2_568 $

400 effectsoverall2_990

401 effectsoverall2_9902_34d $

402 effectsoverall2_9902_a12 $

403 effectsoverall2_9902_2_2fe $

404 effectsoverall2_9902_2_9df $

405 effectsoverall2_9902_2_ecc

406 effectsoverall2_9902_2_292 $

407 effectsoverall2_9902_2_39a

408 effectsoverall2_9902_2_258

409 effectsoverall2_9902_2_360

410 effectsoverall2_9902_2_902

411 effectsoverall2_9902_2_a2b

412 effectsoverall2_9902_2_265 $

413 effectsoverall2_9902_2_384

414 effectsoverall2_9902_2_f58 $

415 effectsoverall2_9902_2_567

416 sleepimprovement $

417 pdimprove $

418 additionaluse $

419 shopping___1

420 shopping___2

421 shopping___3

422 shopping___4

423 shopping___5

424 shopping___6

425 shopping___7

426 shopping___8

427 shopping___9

428 shopping___10

429 shopping___11

430 shopping___12

431 shopping___13

432 shopping___14

433 shopping___15

434 shopping___16

435 shopping___17

436 shopping___18

437 method

438 inhaleonset $

439 additionalmethods___1

440 additionalmethods___2

441 additionalmethods___3

442 additionalmethods___4

443 additionalmethods___5

444 additionalmethods___6

445 additionalmethods___7

446 additionalmethods___8

447 additionalmethods___9

448 edibleduration $

449 smokeduration $

450 edible_duration $

451 source___1

452 source___2

453 source___3

454 source___4

455 source___5

456 source___6

457 underground___1

458 underground___2

459 underground___3

460 underground___4

461 underground___5

462 quantity

463 inhale

464 frequency

465 etoh

466 tobacco

467 otherdrugs

468 listdrugs $

469 addict

470 toomuch

471 addictive

472 problem $

473 describeproblem $

474 aa

475 etohsub

476 illegalsub

477 rxsub

478 rxsublist $

479 age

480 gender

481 job

482 country1 $

483 state1 $

484 prov1 $

485 uk $

486 relationship

487 ethnic

488 education

489 income

490 privatecode $

491 additionalinfo $

492 survey_complete

493 country $

494 province $

495 zip $

496 respondent_data_complete

497 ;

498 RUN;

NOTE: The infile "/folders/myfolders/Cannabis/cannabis_use_survey_data.csv" is:

Filename=/folders/myfolders/Cannabis/cannabis_use_survey_data.csv,

Owner Name=root,Group Name=vboxsf,

Access Permission=-rwxrwx---,

Last Modified=23May2017:18:25:04,

File Size (bytes)=1346807

NOTE: 2864 records were read from the infile "/folders/myfolders/Cannabis/cannabis_use_survey_data.csv".

The minimum record length was 222.

The maximum record length was 3695.

NOTE: The data set SURVEY.CSCSP_2016 has 2864 observations and 141 variables.

NOTE: DATA statement used (Total process time):

real time 0.35 seconds

cpu time 0.25 seconds

499

500 PROC CONTENTS DATA=survey.cscsp_2016 VARNUM;

501 RUN;

NOTE: PROCEDURE CONTENTS used (Total process time):

real time 0.44 seconds

cpu time 0.42 seconds

502

503 OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;

515

Re: Disappearing Permanent Dataset

_maldini_ — Tue, 23 May 2017 23:18:08 GMT

<your import code>
Could you please clarify?

Re: Disappearing Permanent Dataset

Patrick — Tue, 23 May 2017 23:32:05 GMT

@_maldini_

"Could you please clarify?"

The data step you've posted (infile / input)

Re: Disappearing Permanent Dataset

_maldini_ — Tue, 23 May 2017 23:37:04 GMT

See my recent response to @Reeza:

"The PROC CONTENTS runs successfully when I run it concurrently w/ the LIBNAME and PROC CONTENTS steps, but not when I run it AFTER running the LIBNAME and DATA steps (Syntax below).

If I run the entire SAS program successfully, I get the "ERROR: File SURVEY.CSCSP_2016.DATA does not exist." error on the subsequent step (DATA, PROC FREQ, whatever)."

I posted the log of this execution in the response to @Reeza

Thanks!

Re: Disappearing Permanent Dataset

Reeza — Tue, 23 May 2017 23:46:43 GMT

@_maldini_ wrote:
<your import code>
Could you please clarify?

There's no way to know what you're referring to here. Your 'IMPORT CODE' is the data step that you use to read the CSV file. It's obviously incorrect since you have all the NOTES in your log. So you should fix this.

You're statement that if you run them back to back means everything is working as expected. You have something else going on.

You should talk to someone on site or someone familiar with your installation to see. It looks like you're working on SAS UE, but I know that isn't an issue because I constantly use permanent data sets. So without knowing exactly what you're doing ¯\_(ツ)_/¯

Explain what you're doing step by step and include your log. And fix that import first, trying to wade through that is difficult and makes it easy to miss errors/notes that you should deal with.

Re: Disappearing Permanent Dataset

Patrick — Wed, 24 May 2017 03:42:45 GMT

@_maldini_

Then I suggest that you physically check in the folder where your table should reside (path used for the libname definition) if there is a file called CSCSP_2016.sas7bdat

If that's not the case then there must be some process removing this file.

IF the file still exists then make sure that you're connecting with the same user than you had creating the file and that you connect to the same server. If all that's met and things should still not work then something very strange must be going on in your environment.

Also: Add right before the failing Proc Contents the following statement and check in the log that the libref still points to the correct folder location.

libname SURVEY list;

Re: Disappearing Permanent Dataset

_maldini_ — Wed, 24 May 2017 01:38:30 GMT

@Reeza

<it's obviously incorrect since you have all the NOTES in your log. So you should fix this.>

It's fixed and re-posted.

I have a SAS program w/ a LIBNAME statement, a DATA step and a PROC CONTENTS step in one window. When I run this entire program, it executes succesfully. The log is posted above. However, when I then try to use the resulting data set (SURVEY.CSCSP_2016.DATA) in any other SAS program, I get an error in the Log: ERROR: File SURVEY.CSCSP_2016.DATA does not exist. I get the same error in the PROC CONTENTS step when I run the steps in this program incrementally (i.e. LIBNAME + DATA step, then PROC CONTENTS).

The SAS dataset appears in the SURVEY library when the DATA step is executed, but then disappears when the subsequent SAS program is run (e.g. PROC CONTENTS). And, a new SAS data set is created in the folder on my HD with an incremental number assigned to the data set name (i.e. cscsp_2016 2.sas7bdat, cscsp_2016 3.sas7bdat, cscsp_2016 4.sas7bdat, etc.) for each time the DATA step is re-run.

This syntax ran w/o problem for 2 months. All of the sudden it is choking.

Re: Disappearing Permanent Dataset

Reeza — Wed, 24 May 2017 01:39:45 GMT

Are you by any chance using SAS UE in multiple windows/tabs?

Re: Disappearing Permanent Dataset

_maldini_ — Wed, 24 May 2017 01:41:35 GMT

Yes. But unless something has changed, this syntax worked w/o a hitch for months...

Re: Disappearing Permanent Dataset

Reeza — Wed, 24 May 2017 01:44:54 GMT

I suspect you somehow are running different instances of SAS here for some reason.

Re: Disappearing Permanent Dataset

Reeza — Wed, 24 May 2017 02:03:30 GMT

So, I opened two different windows of SAS UE and cannot replicate your issue.

Given what you've shown so far, I suspect you're doing something incorrectly but we can't see what.

I would restart your browsers and make sure you're working on the same instance of SAS.

Make sure Virtual Box has only the latest version of SAS UE.

If the issue persists and you're certain you're not doing anything incorrectly, then post back with FULL details. If you think it's not relevant post it anways. Record a video if you have to.

You can try contacting SAS Tech Support, but I suspect this would be considered out of scope.