Lucent #5 ESS Routine Operations and Maintenance Procedures

Purpose

This Routine Operations and Maintenance Manual contains both descriptive material and detailed procedures for routine operations and maintenance of the Lucent #5 ESS switch.  This manual covers the 5E2(2) through 5E8 software releases.  As the #5 ESS switch continues to evolve, this manual will be reissued to cover future software releases.

This manual is primarily intended for telephone company personnel who schedule or perform routine maintenance for the #5 ESS switch.

Alarm System Management

General

The alarm system management is used to set or clear manual or automatic alarm release, to assign level and labels on scan points, and to provide alarm control.

Alarm Release

The critical and major alarms in an exchange can be released manually or automatically.  When the alarm mode is manual, the alarms have to be released by pressing the ALM RLS function key at the Master Control Center (MCC) video terminal.  If the alarm mode is set to automatic release, the alarms will be released in 5 seconds by the system.

Automatic or manual alarm release can also be activated by means of the menu commands 800 and 801 at the MCC Pages 105/106 - BLDG/POWER & ALARM CNTRLS.  Minor alarms will be released automatically all the time, independent of the alarm release mode.

Alarm Level and Label Assignments

Alarm level and label assignments can be given to building or miscellaneous alarms.  The assignments are shown at the MCC Pages 105/106 - BLDG/POWER & ALARM CNTRLS and Page 119 - MISCELLANEOUS ALARMS.  The alarm level can be major or minor.

The label can consist of a maximum of nine upper-case characters, that is, letters, digits, spaces, plus, or minus signs.  When a label has been assigned, the report data will show the old and new label.  Once level and label are filled in, they are protected from loss when the system is booted.

Alarm Controls

Some alarms can be inhibited or allowed manually from alarm reporting.  These alarms are as follows:

When a building power alarm is inhibited, the respective indicator at the MCC Page 105/106 will show the abbreviation "INH" in reverse video.  Also the words "BLDG INH" in the SUMMARY STATUS AREA will be backlighted.  To inhibit or allow building power alarms, one can use the appropriate input message or the menu command (shown at the MCC Page 105/106).

The external sanity monitor power alarm indicator is shown at the MCC Page 116 - MISCELLANEOUS.  Once the alarm is inhibited, the "INHIBIT" indicator will be in reverse video.  Also this alarm can be inhibited or allowed by either the input message or the menu command at the appropriate MCC page.

The MSGS, TMS, and ONTC fan or fan fuse alarms can be inhibited by the system.  When such an alarm is inhibited, this is shown at MCC Page 115 - COMMUNICATION MODULE SUMMARY.  The reason an alarm will be inhibited is because the respective scan point is chattering.  After solving the problem, the alarm can be allowed again by entering the input message:

If Man-Machine Language (MML):

ALW:ALM,MSGS=b;
or
ALW:ALM,TMS=b;
or
ALW:ALM,ONTC=b;

If Program Documentation Standard (PDS):

ALW:ALM,MSGS b!
or
ALW:ALM,TMS b!
or
ALW:ALM,ONTC b!

Where: b = Unit Number 0 or 1.

The miscellaneous alarms can be allowed or inhibited manually by means of the appropriate input message or the menu command at the MCC Page 119 - MISCELLANEOUS ALARMS.  When the alarm is inhibited, the "INH" indicator will backlight.  Also the "INHIBIT" indicator of MISCELLANEOUS ALARMS at MCC Page 116 will be in reverse video.  The miscellaneous frame fuse alarm can be inhibited by the system once the scan point is chattering.  After solving the problem, the alarm can be allowed by entering the following input message:

If MML:

ALW:ALM,MFFUSE=b;

If PDS:

ALW:ALM,MFFUSE b!

Where: b = Miscellaneous Frame Fuse Unit Number 0 or 1.

When this alarm is inhibited, the abbreviation "INH" is displayed in reverse video at the right of MISC FRAME FUSE ALARM.




File and File System Management

Introduction

Maintaining the file system includes the following:

General

This subject lists and briefly describes the input messages and reports used in managing files and the file system.  These messages and responses are used in performing various actions such as reporting the contents of a file or directory and copying a file from an active disk to an offline or out-of-service disk.

The input/output messages are categorized according to the function performed on the file or file system and how they are used (type).  The different functions and the corresponding tables are as follows:

The three types of input/output messages are routine, troubleshooting, and permanent.  Routine messages are used periodically to determine actual or potential troubles in the system.  The troubleshooting messages are used to clear troubles and will not cause permanent changes to the system.  The messages labeled permanent can cause permanent changes in the file structure or process structure of the system.  Permanent messages should be used upon concurrence with the local technical assistance organization or as directed in a software update.

A detailed description of these input messages can be found in AT&T 235-600-700 (formerly, IM-5D000-01) and a detailed description of these output messages can be found in AT&T 235-600-750 (formerly, OM-5D000-01).

Checking File System Resources

The REPT FS output message warns that a file system is about to run out of space.  Immediate action must be taken to provide more space in the file system.  Contact your technical assistance organization to determine the files that can be removed to provide file system space (for example, software update history).

File system free space can be monitored and action can be taken before the free space is used up.  The command OP:STATUS:FREEDISK reports the free space and free i-nodes in the mounted file systems.  Monitor these numbers regularly to determine if space or i-nodes are being depleted.

Running the file system block audit may increase the free space in a file system if the audit finds lost resources that it can recover.  Note that no error is reported when lost free disk blocks are recovered.

The allocation of disk space to contiguous files is limited to the size of the largest set of contiguous free disk blocks in the file system.  In a fragmented file system, this may be considerably less than total free space.  The compaction audit can be used to increase the size of the contiguous free disk space.

-----------------------------------------------------------------------------------------
File System Control - Table A

Message/Response Name           Type                Description
-----------------------------------------------------------------------------------------
ALW:FSYS-ACCESS                 Permanent           Changes access permissions of files.
-----------------------------------------------------------------------------------------
ALW:FSYS-MOUNT                  Permanent           Mounts an unmounted file system so 
                                                    that the system can access files in
			                            the file system.
-----------------------------------------------------------------------------------------
CLR:FSYS-OWNER                  Permanent           Changes the owner of a file.  All
                                                    operating system files are owned
			              	            by root.
-----------------------------------------------------------------------------------------
CLR:FSYS-DIR                    Permanent           Removes a directory from the file
                                                    system.
-----------------------------------------------------------------------------------------
CLR:FSYS-FILE                   Permanent           Removes a file from the file system.
-----------------------------------------------------------------------------------------
COPY:FSYS-CFILE                 Permanent           Copies a file to a contiguous area
                                                    of the disk.  Contiguous files are
			             	            transferred from disk to main memory
						    more quickly than regular files.
-----------------------------------------------------------------------------------------
COPY:FSYS-FILE                  Permanent           Makes another copy of a file in a
                                                    different directory.  The copied file
			            		    has the same name as the original file.
-----------------------------------------------------------------------------------------
IN:FSYS-DIR                     Permanent           Creates a new directory in the file 
                                                    system.
-----------------------------------------------------------------------------------------
OP:ST-DISKUSE                   Troubleshooting     Reports the number of blocks contained
                                                    in all files and directories within 
						    each specified directory or file name.
-----------------------------------------------------------------------------------------
OP:ST-FILESYS                   Routine             Lists all currently mounted file systems,
                                                    the directory under which they are 
						    mounted, and the time they were mounted.
-----------------------------------------------------------------------------------------
OP:ST-FREEDISK                  Routine             Lists the mounted file systems with the
                                                    number of free blocks and free i-nodes.
						    Used to determine if file system space is
						    being depleted.
-----------------------------------------------------------------------------------------
OP:ST-LISTDIR                   Routine             Reports the contents of a specific
                                                    directory or file.
-----------------------------------------------------------------------------------------
REPT:FILESYS                    Troubleshooting     This autonomous report warns that a 
                                                    file system within the central processor
						    is about to run out of space.
-----------------------------------------------------------------------------------------
Note:  The message/response names as presented in these tables are not intended to be typed
       into the system.  These names are provided so the user can reference these messages
       in AT&T 235-600-700, Input Message Manual, and AT&T 235-600-750, Output Message Manual,
       for the correct syntax.
-----------------------------------------------------------------------------------------
-End-
-----------------------------------------------------------------------------------------
File Modification and Retrieval - Table B

Message/Response Name           Type                Description
-----------------------------------------------------------------------------------------
IN:F-APND-A                     Permanent           Appends a line to an ASCII file.
-----------------------------------------------------------------------------------------
IN:F-DEL                        Permanent           Deletes one or more lines from an
                                                    ASCII file.
-----------------------------------------------------------------------------------------
IN:F-REPL                       Permanent           Replaces lines in an ASCII file with
                                                    user supplied lines.
-----------------------------------------------------------------------------------------
DUMP:F-ALL                      Troubleshooting     Prints the contents of an ASCII file
                                                    on the Receive-Only Printer (ROP).
-----------------------------------------------------------------------------------------
DUMP:F-DATA                     Troubleshooting     Prints the contents of a file in the
                                                    specified format.
-----------------------------------------------------------------------------------------
DUMP:F-PARTL                    Troubleshooting     Prints one or more lines of an ASCII
                                                    file.
-----------------------------------------------------------------------------------------
-End-


-----------------------------------------------------------------------------------------
File Transfer and Backup - Table C

Message/Response Name           Type                Description
-----------------------------------------------------------------------------------------
COPY:ACTDISK                    Permanent           Copies a file from an active disk to an
                                                    offline or out-of-service disk.
-----------------------------------------------------------------------------------------
COPY:CPOOSF                     Permanent           Copies a file from an out-of-service
                                                    disk to an active disk.
-----------------------------------------------------------------------------------------
COPY:CPSPDISK                   Permanent           Copies a specific partition or a list of
                                                    partitions from one of the system disks
						    to an active spare disk.
-----------------------------------------------------------------------------------------
COPY:PTN:ALL                    Permanent           Copies one set of partitions into a
                                                    corresponding set of partitions.  This
						    message is used to recover mutilated disk
						    partitions from backup disk partitions
						    and to generate partition backup copies.
-----------------------------------------------------------------------------------------
COPY:TAPE-IN                    Permanent           Copies files from a magnetic tape 
                                                    containing full or relative pathnames
						    and header information, and places them
						    in their respective directories.  The
						    message can also print a table of contents
						    of the tape.
-----------------------------------------------------------------------------------------
COPY:TAPE-OUT                    Permanent          Copies one or more files to a magnetic
                                                    tape along with relative pathnames
						    and header information.
-----------------------------------------------------------------------------------------
-End-

-----------------------------------------------------------------------------------------
Maintenance Tools - Table D

Message/Response Name           Type                Description
-----------------------------------------------------------------------------------------
OP:ST-SUM                       Routine             Calculates a checksum for a given file
                                                    and prints the number of blocks in the
						    file.
-----------------------------------------------------------------------------------------
VFY:TAPE                        Routine             Verifies the readability of information
                                                    on system tapes and the consistency of
						    corresponding hash sums.
-----------------------------------------------------------------------------------------
-End-

Copying Files to Nonactive Disks - Utility Requirements

The active/nonactive disk copy utility copies a file from an active to a nonactive disk.  The file can be a regular file, a contiguous file (type C or x), or a block device file (type b which consists of a partition or a file system).

The use this facility, the user needs to specify the following:

Emergency Dump

On disk, a partition is reserved for emergency dump.  When there has been data written in this partition, an autonomous report (REPT EMERGENCY DUMP PARTITION FULL) will be printed.  When data has been written in the emergency dump partition, the emergency dump status flag will be set.  Due to the status flag, the previously mentioned report will be printed periodically.  When the flag has been set, no other emergency dump can be written within the next 12 hours.  Therefore, an input message in present to clear the status flag.  This message must only be used when the dumped data has been saved.  As soon as the status flag has been cleared, the emergency dump partition is marked empty.  The message to clear the status flag is CLR:EMERDUMP;.  Before saving the data, the status has to be investigated.  This is done by means of the input message OP:EMERSTAT;.  This will result in a report indicating on which disk, MHD 0 or 1, the data has been dumped, how many bytes have been written, and the hexadecimal address of each segment written.  To save the dumped data, an emergency dump and be performed.




Log File Handling

General

Hardware and software errors will generate error reports.  These reports will either be printed on the ROP, collected in the log file, or both.  The message class of the report is decisive whether a report will either be logged, printed or both.

In software release 5E2(1), when there are reports of a certain message class, which are normally logged, the user can change the log option to log and print.  This can be done by means of the input message SET:LPS.  If there are reports of a message class, which have only the print option, the user can change those reports to print and log with the same message previously mentioned.  To change the options to their original value, the input message CLR:LPS,MSGCLS=ALL; can be used.  It is not possible to change a message class from the original print option to a log-only option or vice versa.

In software releases 5E2(2) through 5E7, the user can change the log or print option with the input message CHG:LPS?.  This will direct the output to the DAYLOG file or print the data at the devices specified in the Equipment Configuration Database (ECD) for that message class.  The OP:LPS? message can be used to determine the current log and print status of a message class.

On MCC display Page 110 - SYSTEM INHIBITS, the poke command 411 can be entered to set all applicable message classes to log and print.

Log files can be used to do the following:

Log files are determined in the classdef and device forms in the ECD.  All log files are located in the directory /etc/log, if 5E2(2) through 5E5, or /var/log, if 5E6 or later.  The pathname for each log file is defined in the particular device form.

Size of Log Files

To prevent a file system overflow, the files are limited in size.  When a log file is first created, the file will be called XXXXX1, where XXXXX stands for the log filename.  When half of the disk space is used, the contents of XXXXX1 is copied into XXXXX0, and the most recent information will be stored in XXXXX1 again.  When all disk space is used, the "1-part" is again copied into the "0-part" which will overwrite the old information by then and the "1-part" will be filled again with the most recent information.  Each log file entry has time of day information, real-time clock values, and sequence numbers.

The log files must be dumped at regular intervals to avoid losing the contents of the files.  It is advisable to dump the files once a day.

The available space in the log file for operations on recent change data, RCLOG, can be obtained by using the OP:AVAILLOG; input message.


Types of Log Files

There are several log files present in the exchange.  Most of them are related to the Administrative Module (AM).  Table E shows the log files related to the AM.  Table F shows the log files related to recent change and equipment data an input messages.

-----------------------------------------------------------------------------------------
Review of the Log Files Related to the AM - Table E

Log File           Description
-----------------------------------------------------------------------------------------
CONFLOG            Configuration management log file.  This file contains a record
                   of each error detected in a hardware unit.  Activation of storing
                   the information in CONFLOG is done by the ALW:CNFLG message.
-----------------------------------------------------------------------------------------
ERLOG              Error interrupt log file.  This file contains Control Unit (CU)
                   error interrupts, except memory related ones.
-----------------------------------------------------------------------------------------
MEMLOG             Memory history log file.  This file contains the supplementary data for
                   memory error interrupts.  This log file will be used to locate transient
		   memory failures.
-----------------------------------------------------------------------------------------
IODRVLOG           Input/output driver log file.  This file contains the error
                   reports associated with the input/output driver and disk driver.
-----------------------------------------------------------------------------------------
PMLOG              Postmortem log file.  This file contains the postmortem dumps.
-----------------------------------------------------------------------------------------
SPLLOG             Spooler output log file.  This file contains the Spooler Output
                   Process (SOP) failure printouts.
-----------------------------------------------------------------------------------------
SIMLOG             System integrity monitor log file.  This file contains errors detected
                   by the system integrity monitor, usually dealing with resource overload
		   conditions.
-----------------------------------------------------------------------------------------
CMONLOG            Maintenance monitor log file.  This file contains a record of
                   terminated and restarted maintenance interface processes.
-----------------------------------------------------------------------------------------
DAYLOG             Daylog file.  This file contains output messages from the AM
                   as well as Switching Module (SM), Communication Module (CM), 
		   and other areas of the switch.  It is used to debug software 
		   faults and has detailed information that is mot required for 
		   routine office operation.  This file is a binary file in 
		   5E2(2) through 5E5, and an ASCII file in 5E6 and later.  The
		   method used to dump the file is dependent upon the software release.
-----------------------------------------------------------------------------------------
-End-















-----------------------------------------------------------------------------------------
Log Files Used for Recent Change Data and Input Messages - Table F

Log File           Description
-----------------------------------------------------------------------------------------
RCLOG              Operations on Recent Change (RC) log files.  This files contains
                   the changes made by operations on RC data.  When an insert is 
	           made, the log file will contain the new data.  When a deletion
		   has taken place, the old data will be stored in the log file.  
		   To inhibit logging of operations on RC data, the command 612
		   at the MCC, Page 110 - SYSTEM INHIBITS, must be keyed in.
		   Note that unlogged operations on RC data will be lost after a 
		   boot.  When a RC log file reaches 100% in use, the major
		   alarm will be set off and the LOG FILE FULL message will be
		   printed on the ROP, indicating that the RC/Vs are locked out until
		   a backup is done.
-----------------------------------------------------------------------------------------
ECDLOG             Equipment configuration data log file.  This file contains
                   all the changes made in the equipment configuration database.
		   The old as well as the new data will be kept.
-----------------------------------------------------------------------------------------
CMDLOG             Command log file.  This file contains all the input messages
                   entered in the exchange together with the dialogue number
		   and the person identity or the teletype number (dependent on
		   the kind of authority chosen).
-----------------------------------------------------------------------------------------
-End-

Log File Dumps

To preserve the log file information for later use, the contents of log files can be dumped.  This can be done by using the OP:LOG? input message.  With this input message, several options are provided to print a particular part of a log file instead of the whole log file.  The keyword option or the type option can be used to look for transient errors.  For example, if the maintenance person wants to retrieve only the error reports from the CONFLOG, the input message will be:

OP:LOG;LG="CONFLOG":TYPE=0291,DEVICE="xxx";

If only the fault reports are required:

OP:LOG;LG="CONFLOG":TYPE=0801,DEVICE="xxx";

must be entered.  The xxx means the logical output device to which the output should be routed (for example, rop0).  Refer to AT&T 235-600-700, Input Message Manual, for complete information about the OP:LOG command.

To dump the contents of the MEMLOG file, the input message OP:MEMERRS must be entered.

An entire log file can be dumped on tape for later investigation.  This can be achieved by using the input message: OP:LOG?

Use of the Day Log File

The day log file is kept for the manufacturer to locate severe software faults.  The contents of this log file have a hexadecimal format in 5E2(2) through 5E5 software releases, and an ASCII format in 5E6 and later software releases.  The maintenance person in the exchange can make a dump of this log file, if necessary, by using the appropriate input message for the software release installed.

Dump of Day Log File - 5E2(2) through 5E5:  A dump of this log file can be made using the DUMP:DAYLOG? input message for the day log file of the AM or for a particular switching module.

The size of the day log file is such that in normal operations the log file will be able to contain all logged information for that day.  When an entry is overwriting another entry which is less than 24 hours old, a minor alarm will be generated.  This alarm is also called the 24 hour log lost alarm.  The day log file can be inhibited by means of command 608 on MCC Page 110 - SYSTEM INHIBITS.  If this command is used, no 24 hour log lost alarm will be generated when an entry is overwritten within 24 hours.

When an entry is made, an autonomous report will indicate how many and what kind of reports are entering the file.

Dump of Day Log File - 5E6 and Later:  A dump of the day log file can be made using the OP:LOG? input message described previously in the "Log File Dumps" section.

Hourly Plant Reports

The hourly plant report contains data regarding originating, incoming, terminating and outgoing calls, call connect setup troubles, and reflects the maintenance effect on traffic during the past hour.

If enabled, the hourly plant report is automatically printed.  It can be enabled by using the ALW:PLNTHR input message.  It can also be requested at any time.  The OP:PLNTHR input message can be used to print the last complete hourly plant report.  The INH:PLNTHR input message can be used to inhibit the printing of the hourly plant report.

The report is sent the the MCC, the SCCS, and the SCCS/NAC channel.

For more detailed information on the hourly plant report, refer to AT&T 235-070-100.

24 Hour Plant Report

The 24 hour plant report contains data regarding originating, incoming, terminating and outgoing calls, call connect setup troubles, and reflects the maintenance effect on traffic during the past 24 hours.

The 24 hour plant report is generated and issued by the #5 ESS switch once-a-day at 02:00:00 or 02:07:00 [software release 5E2(2)].  The report is sent to the MCC, the SCCS, and the SCCS/NAC channel and is saved for the next 24 hours to fulfill requests.

For more detailed information on the 24 hour plant report, refer to AT&T 235-070-100.

Program Update

Program update is the process of activating orderly program changes in the switching equipment software.  The changes are made to a particular software release and/or software release issue to solve a system problem.

The types of program updates available are as follows:

Software Update

General

In-service offices receive most official software changes in the form of software updates.  The software update originates as a fix for a problem within the software release.

Four external interfaces are employed to provide for the generation, distribution, and activation of software updates.  These interfaces (Figure 1) are as follows:

Programmer Support System (PSS):  The PSS originates software updates.  After a software update has been assembled, tested, and approved at the PSS, a software update identification number is assigned.  The software update is then transmitted to the Software Change Administration and Notification System (SCANS) for distribution.

SCANS:  The SCANS is an AT&T time-shared computer system for orderly software update distribution.  Maintenance personnel who subscribe to SCANS may access SCANS daily to receive and record the software updates.  The SCANS also lists any software updates that have been canceled or changed.  Refer to AT&T 190-306-010 and PA-591152 for SCANS procedures.  For Switching Control Center System (SCCS) application, see PA-1P139, Section 12.

SCCS:  Using a 1200-baud dial-up terminal, the SCCS has the capability of remote software update activation.  The SCCS accesses SCANS and triggers the delivery of a software update using the program update subsystem.  Then the received software update is remotely activated from the SCCS via the maintenance channel.

CSCANS:  Another external interface which provides for the distribution of software updates only in offices that are so equipped is the Customer Service Computer Access Network System (CSCANS) interface.  The CSCANS is a Regional Bell Operating Company (RBOC) owned and operated computer system for automated software update distribution.  Subscribing offices may access CSCANS to receive software updates.

Under normal operating conditions, the software update distribution point is SCANS.  However, in an emergency (such as a SCANS outage), a software update can be transmitted from the PSS over a data link directly to an office.  Maintenance personnel at the SCCS or local office must make a verbal request to the Regional Technical Assistance Center (RTAC).  The field update coordinator then sets up an emergency data link from the PSS to the switch and manually transmits the software update (after maintenace personnel have primed the switch for reception of the software update files).  Procedures for activating the software update are not altered.

The software update activation responsibility between AT&T and the Operating Telephone Company (OTC) is as follows:

Software Update Format

The software update format, illustrated in Figure 2, consists of at least four files:


Header File:  The header file contains the necessary information for maintaining the integrity of the software update.  The information consists of the software update number, software release(s) affected, sequence number, name, size, and checksum of each file in the software update.  This information is used by the verification process to verify each software update before activation.

Message File:  The message file contains the commands necessary to install the software update, plus any special instructions required.  Figure 3 shows an example of a PDS message file.

Binary Update File(s):  The binary update file contains the binary data for a file targeted for the update.  For nonkillable processes, these take the form of minimal object (.m) files.  For killable processes, these are updated object files for target processes.

The SCANS Information File:  The SCANS information file contains the software update number, software release(s) affected, general descriptive information, name, and size of each file to be updated, associated software update(s), Customer Assistance Request (CAR) numbers, and the name of the person to contact in case of trouble.

SCCS and Master Control Center Interface

The software updates are normally activated remotely by the SCCS.  Communication between the SCCS and the program update subsystem is over the maintenance channel.  The local office can be unattended.

Software updates may also be activated locally at the Master Control Center (MCC) video terminal.  If the software updates are to be requested from SCANS by the local office, a 1200-baud terminal [other than the Maintenance Cathode Ray Terminal (MCRT)] must be present.  The terminal must be full duplex, capable of printing at least 80 characters per line and must have a 212A-type data set.  If the software updates are to be loaded into the switch from a tape, the 1200-baud terminal is not needed.

General Format for Activation of Software Updates Received from SCANS

Reception from SCANS

To receive software updates, first a dial-up link must be established with SCANS.  When the dial-up link is completed, the proper login, password, and subpassword is used to gain access to the SCANS database.  When access is obtained, a listing is requested of all items from SCANS.  This listing is reviewed, and all applicable software updates are stored.

The office storage space required for each software update binary data package is provided in the software change size section of the software update.  Prior to data transmission, it must be determined that sufficient file space is available in the directory on disk where the software updates are to be stored.  If file space is insufficient, memory audit and space reclamation techniques are used to create space, or the software updates are requested in stages (one or two at a time) as space allows.

When enough file space is available, the office is primed for binary data package delivery by using the IN:REMOTE:START! message.  During priming, file space is reserved and a transaction identifier (ID) is established.  This transaction ID provides additional security at the application level in the file transfer process.

The office will receive data from SCANS for up to 24 hours after it is printed unless the data session is manually terminated using the IN:REMOTE:STOP! message.  If the SCANS does not begin to send data within the 24 hour period, the data session times out and an IN REMOTE ERROR message is displayed at the SCCS (or MCC).  After a time-out, the office will need to be primed again.

After priming, a delivery request is made to SCANS for the binary data packages.  This request, which is usually made from a dial-up terminal between SCANS and the SCC, issues a Binary Overwrite (BOW) command to SCANS.  The BOW command includes the #5 ESS switch office name, the identification of the software update, and a transaction ID.

As soon as a port is available, SCANS sets up a 4-wire dial-up link to the target office.  This link is used for delivery of the binary data packages.  Assuming no equipment failures or unusually high demand, SCANS will establish this link within 24 hours of the delivery request.

Using the access login and the transaction ID, SCANS accesses the AT&T 3B20 computer.  The SCANS then establishes a 4800-b/s BX.25 data link to the target office.  The binary update files, along with the header and message files, are then transmitted to the target office to be placed on disk in the software update directory (/etc/bwm) of a storage partition.

Upon successful reception of the binary overwrite files, the IN REMOTE STOPPED message is dumped at the SCCS workstation and/or MCC video terminal to indicate that the binary files have been received and loaded into /etc/bwm.

Assuming that the binary update files have been received and verified, the BX.25 datalink is automatically terminated.

The software updates received from SCANS are the same software updates generated at the PSS.  The SCANS does not alter the internal structure or format of any software update.  The SCANS information file is not required by the program update subsystem.  If this file is sent along with the BOW, it is ignored.

Reception from CSCANS

To receive software updates from CSCANS, a dial-up link must first be established with CSCANS.  When the dial-up link is completed, the proper login and passwords must be used to gain access to the CSCANS database, for those offices that are so equipped.  For subscribing offices, follow local CSCANS procedures for accessing the CSCANS database and requesting applicable software updates.

The office storage space required for each software update binary data package is provided in the software change size section of the software.  Prior to data transmission, it must be determined that sufficient file space is available in the directory on disk where the software updates are to be stored.  If the file space is insufficient, memory audit and space reclamation techniques are used to create space, or the software updates are requested in stages (one or two at a time) as space allows.

The office will receive data from CSCANS at any time after it is primed and the CSCANS database is updated.  Note that the office will normally not need to be primed again.  The office may be primed again, if desired for security or other reasons, through use of the UPD:INITPW:PASSWD="xxxxxx",KEY="yy"! message.  It is important to note that the CSCANS database must also be updated accordingly with the newly chosen password and key.

After priming, a delivery request is made to CSCANS for the binary data packages.  Follow local CSCANS procedures for requesting a binary data package.

Upon successful reception of the binary overwrite files, a UPD:CSCANS message is dumped at the SCCS workstation and/or MCC video terminals to indicate that the binary files have been received and loaded into /etc/bwm.

Assuming that the binary update files have been received and verified, the BX.25 data link is automatically terminated.

Verification

A process is provided by the program update subsystem to verify software updates before overwriting the resident software release.  When the system receives the verification command, a check is made to confirm the existence and correctness of all files and associated checksums.

If a software update error is detected through verification, the software update in question should be requested again from SCANS.  If the software update fails verification a second time, the Electronic Switching Assistance Center (ESAC) and the AT&T RTAC should be notified.

Activation

Note:  This section is a general description of a software update activation.  Except as noted, the description was written for the 5E2(2) specific software release.  Later software releases allowed for enhanced program update capabilities through the use of a menu-driven craft interface.

After the software updates have been successfully verified, an executable message file for a specific software update is created.  The message file contains an instruction set for activating a particular software update.  The UPD:BWMNO"a"! message contains a software update number that corresponds to a specific message file and the pathname of the message file.  Only one executable message file can be executed in the system at any one time.  The previous executable message file is overwritten each time the UPD:BWMO"a"! message is input.

Note:  The software updates must be activated in sequential order.

The executable message file contains four user accessible sections.  Only three sections are used for normal installation.  The fourth section is used for emergency backout.  Once an executable message file has been created for a specific software update, that software update can be installed by entering forms of the UPD:EXEC"a"! message as follows:

Apply:  The APPLY section is used to place an update into a temporary mode.  It is executed by entering the UPD:EXEC"a";CMD APPLY! message.

Soak:  The SOAK section contains the recommended soak interval for a software update.  A soak interval is a period of time when the software update is tested and observed for proper operation.  Although it is possible to execute the SOAK section by using the UPD:EXEC"a";CMD:SOAK! message, each step of the SOAK section, where applicable, should be performed on a manual basis to allow close observation of the soak interval.

Official:  The OFFICIAL section is used to make the temporary update permanent following successful completion of the soak interval.  It is executed by entering the UPD:EXEC"a";CMD OFFICIAL! message.

Backout:  The BACKOUT section is used only when it becomes necessary to back out of a temporary software update.  It is executed by entering the UPD:EXEC"a";CMD BKOUT! message.  Before software release 5E5, when an update had been made permanent, it could not be backed out.  The 5E5 software release allowed for just the last official software update to be backed out.  With the 5E6 software release, this was enhanced to allow for up to three official software updates to be backed out.

The APPLY, OFFICIAL, and BACKOUT sections each contain one or more executable messages.  The SOAK section may or may not contain messages.  A copy of the current executable message file may be obtained by using the UPD:BWMNO"b";LIST ALL! message.

The normal progression for software update activation is to execute the APPLY section first, followed by the SOAK section, and finally the OFFICIAL section.

If the system is rebooted when temporary updates are in place (other than switching module updates), the temporary disk file is thrown away, and the system boots from the official file, effectively backing out the update(s).

When a temporary update is made permanent, an updated version of the file is built in the same directory as the original version.  A windowless move then takes place to effectively make the new version official.  The in-core memory is not touched when an update is made official since the update has already been installed there.

The software updates can also be activated by maintenance personnel at the SCCS or MCC by printing the message file of the applicable software updates and manually installing the required messages.

Backout

During the soak interval, the temporary software update is observed to ensure proper performance.  If the software update does not perform properly at any time during the soak interval, it should be backed out using the UPD:EXEC"a";CMD BKOUT! message.  The BACKOUT section backs out the designated change, plus any subsequently installed temporary changes; that is, the backout process can only delete changes in the exact reverse order of application.  A sequential list of changes that are in the temporary states may be obtained using the UPD:DISPLAY;TEMP! message.

Emergency Fix

General

Regular program updates are performed in a timely and orderly fashion through software updates.  Upexpected problems with the software release can occur that require immediate correction, not allowing time for the normal software update development and issue processes.  These immediate corrections are known as emergency fixes.  Emergency fixes are accomplished on a word-by-word basis under the direction of the AT&T Product Engineering Control Center.

Emergency fixes are assigned a sequential craft number similar to the software update number.  The program update subsystems provides emergency fixes with the same status and processes as software updates (that is, make temporary, backout, make permanent).  Emergency fixes specify the address to be changed, the new data to be inserted, and the old data to be matched.  Emergency fixes are also known as address data couplets.

SCCS and MCC Interface

As with software updates, most emergency fixes are activated remotely by the SCCS.  Communication between the SCCS and the program update subsystem is via the maintenance channel.  The local office can be unattended during the activation of the fix.  Emergency fixes may also be activated locally through the MCC.

General Format for Emergency Fix Activation

Activation

The LOAD:UPNM....! message causes a temporary change to be made at the specified location.  After a suitable test and soak period, the UPD:UPNM....;OFC! message makes the change permanent, and normal backout procedures can no longer be used.

Backout

Normal backout can be accomplished only while the fix is in a temporary state.  The backout procedure can be implemented using the UPD:BKOUT;UPNM a! message.  This message backs out the designated change, plus any subsequently installed changes, because the backout process can only delete changes in the exact reverse order of application.  The UPD:DISPLAY;TEMP! message provides a sequential list of changes that are in a temporary state.

Note:  The UPD:BKOUT;UPNM a! message only restores the words specified in the change.  If memory other than that specified is this change is mutilated, a system BOOT may be required to restore the switching system to normal.

Space Reclamation

Reclaim Space in Software Update Storage Directory /etc/bwm

As software updates are brought into /etc/bwm and activated, the space available in /etc/bwm for future software update storage is gradually reduced.  To avoid running out of space, the file space occupied by software updates which have been made permanent should be cleared.  Such space in /etc/bwm is cleared using the CLR:BWM:"b"! message.  A listing of permanent updates should be obtained using the UPD:DISPLAY;OFC! message and compared to the contents of /etc/bwm using the OP:STATUS:LISTDIR,DN "/etc/bwm"! message.

Reclaim Patch Space

Update functions are installed in patch space.  After an update has been soaked and made permanent, the old functions and decision functions are normally no longer needed.  These should be removed from execution, and the space reclamation is accomplished through the use of the UPD:AUD! message.  From 5E4 to 5E6, the space reclamation is accomplished through the use of the UPD:RECLAIM! message.  In 5E7 and later software releases, the space reclamation is performed automatically when the update is made permanent.

Error Conditions

Error Table 1 and Error Table 2 contain listings of possible error conditions that may be encountered during file transfer.  Should any error conditions arise during update activation that are not listed in the table, refer to the error condition listing in AT&T 235-600-700, Input Message Manual, or AT&T 235-600-750, Output Message Manual, for the particular message in question.

Enhanced Program Update Capabilities

Release 2 of the 5E2(2) software release provides enhanced program update capabilities through the use of a menu-driven craft interface.  This interface provides a user friendly program update procedure which simplifies software update installation.  These enhancements eliminate the need for lengthy input messages that must be entered precisely.

When a software update package is received, four types of files are included.  One or more update files are included which contain modified or new functions in the form of an object file to fix a process.  A message file (Figure 3) is included which contains a set of craft commands required to install a given software update.  A header file is included which is used to verify the software update on site.  This file contains information such as target software release issue, file size, and checksum for all files in the software update.  Finally, a SCANS file contains a description of the problem heing corrected and administrative information used by the SCANS to determine to which offices the software update should be sent.

The message file provides commands and instructions used by an automated display mechanism to guide the craft through the process of installing the associated software update.  This process reduces the amount of time required for and the potential errors associated with manual message inputs.  The craft may examine the contents of the message file and monitor the status of an update transaction via video display pages.

The menu-driven craft interface provides software update installation menu pages and program update pages (Figure 4 through Figure 9).  Numbered commands, called pokes, are entered from these display pages to perform the desired procedures.  Refer to the Memory Alteration Procedures for detailed procedures using the menu-driven craft interface.

Software Update Installation

Warning:  In 5E6 and later software releases, Craft Interface Recovery Features (CIRF) provides the capability to recover the craft interface from craft lockout without affecting the call processing.  This feature will kill UNIX system processes including program-update processes; therefore, it should be used with great care.  IT IS STRONGLY RECOMMENDED NOT TO USE THIS FEATURE WHILE SOFTWARE UPDATE IS IN PROGRESS because this feature may cause software update application in a state which cannot be recovered by any means.

Note:  Starting with the 5E5 software release (or 5E4 software release if software update 880080 is installed), MCC Display Page 1940 - EASY BWM INSTALLATION (Figure 10) provides a simplified alternate method of installing software updates.  With this method, the user is required to enter a minimum number of pokes manually and is not required to constantly monitor the software update progress.  In the 5E6 software release, poke 9870 was changed from simply turning on the Stop After Soak (SAS) feature to allowing the user to toggle the SAS feature between being ON (so as to stop after the soak section) or OFF (so it will continue all the way to OFC).  Refer to AT&T 235-105-110 for details on how to use MCC Display Page 1940.  The MCC Display Page 1960 is still the standard method to load or backout software updates.

The craft can install a software update through the use of the software update installation menu (Figure 4 and Figure 5).  This menu is on MCC Display Page 1960.  The craft interface simplifies the task of installing a software update by directing the craft through the installation process.  The upper part of the display provides a list of menu items, each of which is identified by a poke number.  The lower part of the menu page provides a display window for sections of the message file.  The current section of the message file, called the working section, is displayed so it may be examined by the craft before execution.  If the working section does not fit the designated window, the craft can scroll forward and backward using the NEXT and PREVIOIUS window pokes.  The message file may be printed on the MCC printer by entering the PRINT poke provided on the menu page.  A RESPONSE line is provided on the software update installation page to provide response messages to the craft during software update installation.  Sequencing checks are made between software updates to ensure proper sequencing for all software updates in a #5 ESS switch.

To select a particular software update, the START software update poke is entered along with the desired software update number.  The menu page then displays the current status of the requested software update and sets the "BWM =" indicator to the requested software update name.

Prior to 5E5 software release, the message file is in a combined Program Documentation Standard (PDS) and Man-Machine Language (MML) format so it can be used in all #5 ESS switches.  The VERIFY poke automatically deletes messages from the message file which are not in the language format required for the specific office.  For 5E5 and later software releases, the PSD format is not used.  The necessary tool changes have been made in the 5E6 software release to provide only MML format in the message file.  Additionally, this poke checks the message file integrity.  The VERIFY poke must be executed before the software update can be applied to the system.  When this poke is completed successfully, an indicator is provided on the display to notify the craft that the software update was successfully verified.  The VERIFY poke may be executed as many times as desired.  However, if there was no change to the message file since the last VERIFY poke was executed and the VERIFY flag is set to COMPLETED, subsequent VERIFY commands are rejected.

When the software update has been successfully verified, the contents of the APPLY section of the message file are automatically displayed.  The action indicators (DISPLAY, PRINT, EXEC ALL, and EXEC NEXT) are then used in conjunction with the section indicators (APPLY, SOAK, OFC, BKOUT, and FILE) to perform the desired software update installation functions.  A single poke number is used to represent the action desired and the section of the message file to be acted upon.  The first two digits represent the action and the second two digits represent the section.

To apply the selected software update, the 9310 poke (EXEC ALL, APPLY) is used.  This poke causes all commands in the APPLY section of the message file to be executed automatically and in sequential order.  These commands may be executed one at a time using the 9410 (EXEC NEXT, APPLY) poke.  The 9410 poke must be used repetitively to execute all the commands in the APPLY section.  The 9310 poke may be used at any time after a 9410 poke in order to cause the execution of all remaining command lines in the APPLY section of the message file.

Subsequent to an EXEC poke, the RESET LINE poke may be used to reset the current command to be executed.  This poke is entered followed by a comma and a line number to which the execution pointer should be reset.  The specified line is then highlighted and may be executed again.  If the requested line number is in the next or previous display window, the RESET LINE poke causes the display window to be adjusted.

If an error occurs during the execution of a command line, a summary of the error is displayed on the RESPONSE line of the menu page.  The command line causing the error is displayed, and a detailed error message is printed on the MCC printer.  If the 9310 poke is being used when the error occurs, execution of the next command line (if any are still remaining) is automatically stopped.

The STOP EXEC poke (9560) may be used subsequent to the EXEC ALL poke to stop execution of the next command line in the section.

Updates intended for switching modules are automatically propagated to all operational switching modules.  Any switching modules which are isolated at the time of the update do not get updated.

When all command lines in the section are successfully executed, a "CMPL" indicator associated with the APPLY status is displayed, and the content of the SOAK section is automatically displayed.  This guides the user to the next step of the software update installation process.  This section of the message file contains the procedures required to test the software update and the required soak interval.

The DISPLAY poke (91XX) may be used to display selected sections of the message file.  The first ten lines of the section are displayed in the display window.  If more than ten lines exist for the selected section, the NEXT WINDOW and PREV WINDOW pokes may be used to scroll forward and backward through the section.  These pokes do not affect the execution of commands and may be used only after a DISPLAY or EXEC poke has been entered for a given software update.

The PRINT FILE F poke (9260), added in 5E4, can be used to print out any ASCII files associated with the currently installed software update.

If a failure occurs during the SOAK procedures, the software update must be backed out.  The BKOUT section of the message file is displayed by using the 9140 (DISPLAY, BKOUT) poke.  This section provides information to the craft and commands required to back out the given software update.  The 9340 (EXEC ALL, BKOUT) poke automatically executes all commands in the BKOUT section of the message file, one by one, and in sequence.  When the execution of these commands is completed successfully, the content of the APPLY section is again automatically displayed on the menu page.

When the software update has been successfully tested, it may be made official.  This means that the software release file in the official disk partition is updated.  Thus, the update can be saved accross system bootstraps.  However, reloading the system from the backup disk partitiion will destroy the update unless the updated software release file is copied to the backup partition.  The 9330 (EXEC ALL, OFC) poke is used to make the software update official.  This poke cannot complete until all command lines in the APPLY section have been completed successfully, and soak section is completed and timer is expired.

Figure 1 - Program Update Distribution


Figure 2 - Software Update Structure

Figure 3 - Example of a Message File (5E3)

Figure 4 - BWM Installation Menu Page 1960 (5E3 and Earlier)

Figure 5 - BWM Installation Menu Page 1960 (5E4 and Later)

Figure 6 - Program Update Menu Page 1950 (5E2 and Earlier)

Figure 7 - Program Update Menu Page 1950 (5E3 through 5E5)

Figure 8 - Program Update Menu Page 1950 (5E6)

Figure 9 - Program Update Menu Page 1950 (5E7)

Figure 10 - Easy BWM Installation Page 1940

Error Table 1 - File Transfer Error Conditions (SCANS Interface)

Error Table 2 - File Transfer Error Conditions (CSCANS Interface)