sas metadata server backup failed

[woo]

If someone can please comment,

We are running SAS 9.4 on Linux. Have below error in metadata server log, also, we don't see successful backup from sas management console,

 :sas2 - Metadata Server Backup Not Executed Due To I/O Errors

what would be the most common cause for below error?

[nhvdwalt]

Do you have enough disk space where you are writing the backups to ?

[woo]

yes, we checked that already, we do have enough space. specific directory is about 70% usage.

[nhvdwalt]

Has this ever worked ?

Can you maybe publish more of the log if you can ?

Are you running this backup manually from SMC ?

[woo]

It is auto scheduled back up runs everyday at 1 am. it has stopped today with error mentioned earlier. some more errors are like,

Journal file calculatedSize=16 and lDataSize=0 and calculatedSize should never exceed lDataSize! Assuming calculation is correct and resetting lDataSize to match.

Journal file calculatedSize=1528 and lDataSize=1544 and the two should always be the same!

Crash recovery apparently failed.

An error occurred while attempting to Recover the Journal File after I/O failures.

Journal reload has failed.

[nhvdwalt]

Is it possible to restore to yesterday’s backup in order to isolate this
set as problematic ?

[woo]

we can always do but just trying to find the root cause. Also, wanted to see if someone came across similar error and if can add any comment. Thanks...

[nhvdwalt]

No, unfortunately haven't seen this before.

My only suggestions would be to:

1.) Roll back the Metadata repository to at least get you back to a stable state

2.) Trace the metadata server logs line by line to see where the inconsistencies originated from

[jklaverstijn]

Have you attempted a metadata Analyze&Repair from SAS Management Console? If not have a go and let us know what happened.

Regards,

- Jan.

[JuanS_OCS · Accepted solution]

Hello @woo,

I have seen this before.

First, you can try to delete the journal backup file (then xml on your backup directory), so it will be recreated. Only CON, you won't be able to rollback that previous with the SMC anymore, since it will get a new registry of backups.

Also, perhaps you could give a look into this Note, although I doubt that will be your problem http://support.sas.com/kb/56/230.html

Finally, when you can turn SAS services down (evening or patch time), perhaps you can try to extend information in the logs on the metadata server, start it and run a manual backup and see if there is any additional information. When you get the logs, you should rollback the changes on the log configuration and restart tmetadata server again.