Network Disaster Recovery
This document describes action to be taken in the event of corruption of Escape data files as the result of network failure or ongoing network problems. Problems that can affect the integrity of your Escape system generally fall into two categories:
- Hardware failure - The most obvious is a power outage that causes a server failure during work hours. Others include intermittent router, network adapter, or network cabling problems.
- Software problems - Outdated server or client software is generally problematic. It is important to install service releases of these components as they become available.
Identifying Network and Escape System Problems
Sometimes it is easy to identify that a network catastrophe has occurred. If a server crashes, or the power goes out,
everyone knows immediately. But when a router goes bad, or cabling is damaged, it can be more difficult to identify. Whatever the cause, the symptoms Escape reports will generally be one of the following:
- "Red box" errors - the program crashes with a red box error message. The error will often be reported as invalid file, file not found, sharing violation, etc. If the file truly is broken or missing, all users will get the error when attempting to open the file. If the errors are the result of intermittent network problems, some users will not experience the errors. (Note: Review the .DMP and system log files for error messages.)
- Records appear to be missing - usually the result of broken key files or corrupted data files.
- Users are unable to add records to some files, but are able to change records. For example, the users can edit a vendor and change information, but cannot add a new vendor - receive a red box when they do.
- User workstations freeze or "hang" while attempting to sign into Escape or load records. No error is ever displayed on screen and pressing CTRL-BRK does not do anything.
- When a user attempts to edit one record, data from another appears, indicating data corruption. A user edits a requisitions and views line items. The line items that appear belong to a different requisition. Or, data is "shifted" on screen. For example, dollar amounts appear in text fields, etc.
Escape System Recovery
The following steps are a guideline to recovering from a network disaster, once you have determined and corrected the cause. The assumption here is that you believe that your Escape system has been compromised.
Always notify Escape Customer Care immediately when you suspect that network problems have affected the integrity of your system!
Lock Escape
Lock the system and exit all users. Verify at the server console that no Escape files are
opened by ANY users. Keep the system locked down until you are certain system integrity exists.
TIP: Search the Knowledge Base for any up-to-the-minute disaster recovery suggestions.
Copy Production to Recovery
Copy your production system to a "recovery" system location so you can test to determine the existence and nature of Escape system problems.
NOTE: Do not overwrite your existing Test system! We may need data from there to complete a recovery. You may also want to zip up your production system for extra protection if there are issues during recovery.
Test the system
In the Recovery system, have several users run a variety of tests within the software to insure the integrity of the system. View and edit requisitions, view accounts by different sort methods, view journal entries, etc. Run critical processes such as PO printing, AP check runs, Payroll calculations. Review system log and .DMP files. If you are confident that the Escape data files are intact, go to step 8. Otherwise, report your findings to Escape Customer Care and proceed with step 4.
Verify backups
Verify that you have a good full backup from the previous day. Tell users that you will be restoring backup and that they need to note all work done today, since they must redo that work.
Restore from backup
Restore your good backup to the Production location. Verify that the restore was successful, that all files were restored.
Verify directory structure, network rights and file attributes
Check File attributes or NetWare flagging in the \ESCAPE folder. Check Trustee Rights. Check user folder & file rights for the Escape data folders.
Check Escape user authorities and menus
Go into Escape System Manager module, lock this restored system, and verify that user authorities, system setup and user menus match those in the Recovery system database. (Changes may have been made since the backup.)
Run Escape utilities
Run the utilities outlined in the subsequent section Escape System Utilities.
Copy Production to Recovery again
Copy the restored and/or repaired system to Recovery before performing any further tests.
Repeat tests
Repeat the test procedures you performed in step 3 above. If problems persist, call Escape Technology immediately. If you are satisfied that your network is stable and the Escape system is repaired, go to the next step.
Phase rollout
Allow a few system users, representative of various departments back into the system. Monitor the system closely. Check logs and .DMP files often. If the consensus is that the system is stable, continue rollout.
Make and verify backup
At the end of the day, make a full Escape backup, verify it, and store it out of your backup rotation.
- Take a well deserved break!
Escape System Utilities
The following utilities should be run in the event of a system corruption due to network problems.
TIP: Complete instructions for these utilities can be found in the Troubleshooting section of the
Escape Finance Setup and Maintenance Guide.
Run CHKACCTS Utility
Run the Escape utility CHKACCTS on the current fiscal year to verify the integrity of your account-related data files. This will read and report errors in all files, including accounts, account balance, account index and extract files, and account key files. When the utility is finished, review the *.ERR files in the \ESCAPE\FY sub-directory. For the 2004 FY, check \ESCAPE\2004.
Run FIXUSERS Utility
Run FIXUSERS to clear stranded records from Escape data files. To run this utility, start from the \ESCAPE\COMMON directory and type the following: \ESCAPE\UTIL\FIXUSERS.EXE.
This utility creates its own error log called FIXUSERS.AUD. You will see information about the process displayed on the screen. This same information will be recorded to the AUD file. If there are any error messages print out FIXUSERS.AUD and notify Escape Technology.
Run FIXACCTS Utility
Once you are sure of the integrity of the accounts files and the JEHDR and JEDTL key files, run FIXACCTS from the Chart module. FIXACCTS will read through the posted journal entries and update account balances based on the journal entries. (For more information about FIXACCTS, please see the Finance Setup and Maintenance Guide, Troubleshooting section.)
Run SCANJE & SCANJDTL Utilities
Run the Escape utilities SCANJE and SCANJDTL to search for partially posted journal entries. Partially posted journal
entries are very rare and only occur when there are power or system problems during the posted process. Since the above utilities are powerful and can affect your database integrity, please consult with Escape Customer Care before using them. (For more information about SCANJE and SCANJDTL, please see the Finance Setup and Maintenance Guide, Troubleshooting section.)
Run a Trial Balance Report
Trial balance reports should be run on a regular basis while all users are out of the system. After system problems, it is important to run a trial balance while all users are out of Escape, to verify the integrity of the database. Please see the Batch Processing FYI for more information about running this report.
|
