SAS Deployment Backup problem - error code 681

[MarkESmith]

Hey all,

Running SAS on a Linux machine in a non-clustered setup.

Just got a SAS Alerts email from our SAS System. The following is the text contained in the e-mail:

A problem occurred in executing a "Deployment Backup and Recovery" operation.

Operation Status : NOT_STARTED

This was a scheduled backup.

It is recommended to take another clean backup by running sas-backup after recovering from the error situation.

Got Exception as follows:

 Backup Errors 

       Error Entry:

             Status Code: 681

             Name: ERROR_CODE_GENERAL_IO_FAILURE

             Category: GENERAL

             Classification: ERROR

             Date and Time Of Error: 2020-02-09T01_00_01

             Log Name: session.log

             Log File Location: /opt/sas94/config/Lev1/Backup/Logs/2020-02-09T01_00_00/session/session

             Operation Type: session

             Originating Tier: Host: sas Configuration Directory: /opt/sas94/config/Lev1

             Message: An error occurred while trying to do some disk operation.

             Stack Trace: Unable to copy file /opt/sas94/config/Lev1/Backup/Vault/BackupHistory.xml to /opt/sas94/config/Lev1/Backup/dropzone/config/BackupHistory.xml

After reading the following thread, I decided to try and rename the BackupHistory.xml file in three locations and manually re-run the backup from the Administrator web page - but the error unfortunately still occurs.

https://communities.sas.com/t5/Administration-and-Deployment/SAS-Backup-failed/td-p/431689

Here is the log from the aforementioned log location:

2020-02-09 14:16:16,791 [main] INFO com.sas.svcs.backup.server.handler.ServiceSessionHandler - Drop Zone : /opt/sas94/config/Lev1/Backup/dropzone
2020-02-09 14:16:16,791 [main] INFO com.sas.svcs.backup.server.handler.ServiceSessionHandler - History & config file base is /opt/sas94/config/Lev1/Backup/Vault
2020-02-09 14:16:16,792 [main] INFO com.sas.svcs.backup.server.handler.ServiceSessionHandler - Is Start Session : false
2020-02-09 14:16:16,792 [main] INFO com.sas.svcs.backup.server.handler.ServiceSessionHandler - Synchronizing up..
2020-02-09 14:16:16,792 [main] INFO com.sas.svcs.backup.server.handler.ServiceSessionHandler - Is primary node : true
2020-02-09 14:16:16,792 [main] INFO com.sas.svcs.backup.server.handler.ServiceSessionHandler - Node is primary. Ending the BackupHistory & Backup Configuration maintenance for the service session.
2020-02-09 14:16:16,792 [main] INFO com.sas.svcs.backup.server.handler.ServiceSessionHandler - Trying to copy file : /opt/sas94/config/Lev1/Backup/dropzone/config/BackupHistory.xml to /opt/sas94/config/Lev1/Backup/Vault/BackupHistory.xml
2020-02-09 14:16:16,792 [main] INFO com.sas.svcs.backup.server.handler.ServiceSessionHandler - Copying history file from /opt/sas94/config/Lev1/Backup/dropzone/config/BackupHistory.xml to /opt/sas94/config/Lev1/Backup/Vault/BackupHistory.xml
2020-02-09 14:16:16,792 [main] ERROR com.sas.svcs.backup.server.handler.ServiceSessionHandler - Unable to copy file /opt/sas94/config/Lev1/Backup/dropzone/config/BackupHistory.xml to /opt/sas94/config/Lev1/Backup/Vault/BackupHistory.xml
2020-02-09 14:16:16,793 [main] ERROR com.sas.svcs.backup.server.BackupServerSession - An exception while running the command: Unable to copy file /opt/sas94/config/Lev1/Backup/dropzone/config/BackupHistory.xml to /opt/sas94/config/Lev1/Backup/Vault/BackupHistory.xml
2020-02-09 14:16:16,811 [main] ERROR com.sas.svcs.backup.server.BackupDriver - An exception while running the BackupServer executable, Exiting..... : Unable to copy file /opt/sas94/config/Lev1/Backup/dropzone/config/BackupHistory.xml to /opt/sas94/config/Lev1/Backup/Vault/BackupHistory.xml
Error Code: 681, Stack Trace: Unable to copy file /opt/sas94/config/Lev1/Backup/dropzone/config/BackupHistory.xml to /opt/sas94/config/Lev1/Backup/Vault/BackupHistory.xml
at com.sas.svcs.backup.server.handler.ServiceSessionHandler.copyFile(ServiceSessionHandler.java:191)
at com.sas.svcs.backup.server.handler.ServiceSessionHandler.syncUp(ServiceSessionHandler.java:118)
at com.sas.svcs.backup.server.handler.ServiceSessionHandler.execute(ServiceSessionHandler.java:100)
at com.sas.svcs.backup.server.handler.ServiceSessionHandler.execute(ServiceSessionHandler.java:33)
at com.sas.svcs.backup.server.BackupServerSession.execute(BackupServerSession.java:107)
at com.sas.svcs.backup.server.BackupDriver.main(BackupDriver.java:131)
2020-02-09 14:16:16,812 [main] ERROR com.sas.svcs.backup.server.BackupDriver - [Ljava.lang.StackTraceElement;@63dc6270
2020-02-09 14:16:16,812 [main] INFO com.sas.svcs.backup.server.BackupDriver - Exiting with status code: 681
~

So it looks like the BackupHistory.xml file can't be copied and that's the root of the problem - but why? Is there anything else I can try to get this to work?

[Kurt_Bremser]

Look at the filesystem for /opt/sas94/config/Lev1/Backup/ to check if it is running out of space. Also make sure that the user running the backup has write permission there.

[MarkESmith]

The filesystem that this location is mounted on has 99GB of available storage space, so I don't think it's a matter of running out of space. 

The user who owns the Backup directory has rwx permission, so shouldn't be a permissions problem. 

[MarkESmith · Accepted solution]

Figured out the problem - just in case anyone has this same problem in the future, here's what my problem was. The backup files in the following location had the wrong owner (root) and had an incorrect octal permission set:

/opt/sas94/config/Lev1/Backup/dropzone/config

Changed the ownership of all files within this path to the 'sasinst' user and changed the octal permission set of all files to '664' - after doing so, and manually running the backup, it completed successfully.