Day 36

TSM BA Client Error log and Schedule Log Overview

TSM Error Log and Schedule Log Configuration

You have to provide space for error log and schedule log for each BA client and any TDP clients separately. If you are multiple schedules for one node, it is recommended to use separate log files for each schedule so that you can easily troubleshoot when problem occurs. You have to make sure that the users have permission to edit the log files. Most of the problems in TDP are because of this issue only. 

TSM Client Error Log File

The following error logging options are available, you have to mention these parameters in dsm.sys file on UNIX and dsm.opt in windows platforms.

• ERRORLOGNAME: Use this option to specify the path and name of the file where Tivoli Storage Manager is to store information about errors that occur during processing.

errorlogname /tmp/dsmerror.log

• ERRORLOGRETENTION: Use this option to specify the number of days to keep entries in the error log, and whether to save the pruned entries. The error log is pruned when the first error is written to the log after a Tivoli Storage Manager session starts and after every schedule that runs. If the only Tivoli Storage Manager session that you run is the scheduler, and if you run it 24 hours a day, the error log might not be pruned according to your expectations. You must stop the session and restart it so that the log is pruned when the next error is written. It is suggested that you set your log to be kept for 7 days.

errorlogretention 400 s

• ERRORLOGMAX: This option specifies the maximum size of the error log, in megabytes.

errorlogmax 50

• If you change from ERRORLOGMAX to ERRORLOGRETENTION, all existing log entries are retained and the log is pruned by using the ERRORLOGRETENTION criteria.

• If you change from ERRORLOGRETENTION to ERRORLOGMAX, all records in the existing log are copied to the pruned log, the existing log is emptied, and logging begins under the new log-wrapping criteria.

• If you change the value of the ERRORLOGMAX option, the existing log is extended or shortened to accommodate the new size. If the value is reduced, the oldest entries are deleted to reduce the file to the new size.

• QUIET: This option prevents messages from showing on your screen during Tivoli Storage Manager processing. For example, when you run either the incremental or selective backup command, Tivoli Storage Manager shows information about each file that it backs up. Use the QUIET option to reduce the number of entries that you must look through. When you use the QUIET option, certain error information still shows on your screen, and messages are written to log files. If you do not specify QUIET, Tivoli Storage Manager uses the default option, VERBOSE.

TSM Schedule Log File

To specify the number of days to keep log file entries in the schedule log, you can use the SCHEDLOGRETENTION option. When you run the schedule command as a service, output is directed to the dsmsched.log file and to the application event log.

By default, the dsmsched.log file is in the current directory. You can change the path and file name by using the SCHEDLOGNAME option.

schedlogname /opt/tivoli/tsm/client/ba/bin64/dsmsched.log

schedlogmax 200

If you change from schedlogmax to schedlogretention, all existing log entries are retained and the log is pruned using the new schedlogretention criteria.

If you change from schedlogretention to schedlogmax, all records in the existing log are copied to the pruned log, dsmsched.pru, the existing log is emptied, and logging begins under the new log wrapping criteria.

If you change the value of the schedlogmax option, the existing log is extended or shortened to accommodate the new size. If the value is reduced, the oldest entries are deleted to reduce the file to the new size.

The amount of detail is determined by whether VERBOSE or QUIET is set in the dsm.opt file. After scheduled work is performed, you can check the schedule log to check if the schedule is successful or not.

Options to improving the client data movement

You can control the amount of data that travels between the client and server before the server commits the data and changes to the server database. This option changes the speed at which the client works. You can also control the amount of data that travels when files are batched together during backup or during a restore procedure.

The TXNBYTELIMIT client option specifies the number of kilobytes that the client program buffers before it sends a transaction to the server. The default is 25600 KB.

The TXNGROUPMAX option in the server options file indicates the maximum number of logical files (client files) that a client can send to the server in a single transaction. The default is 4096 files. Set the TXNGROUPMAX option in the dsmserv.opt file.

The TXNGROUPMAX option specifies the number of files that transfer as a group between a client and the server between transaction commit points. Using a larger value for the TXNGROUPMAX option affects the performance of client backup, archive, restore, and retrieve operations.

You can use the TXNGROUPMAX option to increase performance when Tivoli Storage Manager writes to tape. This can considerably increase performance when a user transfers multiple small files.

Either set the TXNGROUPMAX option by editing the dsmserv.opt file on the server, or you can use the setopt command. The range is 4 to 65000 for the maximum number of objects per transaction. The default is 4096.

After the txngroupmax number is reached, the client sends the files to the server, even if the transaction byte limit is not reached.The txnbytelimit client option controls the amount of data that travels between the client and server, before the server commits the data and changes to the server database.

Day 35

Configuring TSM Client options from TSM Server console (Cloptset)

What is TSM Client Option Set  (cloptset) ?

TSM client option set (cloptset) is the set of rules that can be assigned to one or more clients from the TSM server console. This feature is mainly used when you need to change some client option settings and you don't have permission to login to the client machine to edit the dsm.sys file. You also need to understand the difference & preferences between Client options file and Client option set before using this feature.

Client options file: An editable file that identifies the server and communication method and provides the configuration for backup, archive, hierarchical storage management, and scheduling.

Client option set: A group of options that are defined on the server and that the client nodes use in conjunction with client options files.

If you have an option in both the client option file (dsm.opt) and the central client option set, the central client option set can override the client option file by using the force option.

Client Option Sets Features

• Client option sets are defined on the server. You can centrally manage client option sets for multiple clients. When you create a client option set, you define a name for the option set and provide a description of the option set.

• The client option set is empty when it is first defined. Use client option sets to set options for groups of clients.

• Clients can only attach to one client option set, but you can use client option sets to centrally manage multiple clients.

Perform the following steps to create a client option set and have the clients use the option set

• Create the client option set with the define cloptset command. The define cloptset command defines the option set name and description. The name is up to 64 characters in length and should not include embedded blanks. The option set description is up to 255 characters in length and should be enclosed in quotation marks if it uses embedded blanks. To issue this command, the administrator needs to have system or unrestricted policy privilege.

            define cloptset qltbackup desc="settings for quality system"

• Add client options to the option set with the define clientopt command. After the option set is defined for Tivoli Storage Manager, options and their values must be defined for the option set. Use the define clientopt command to define the options and their values. Supply the options set name, option name, and option value. Optionally, you can supply, either or both, a sequence number and force value. 

           define clientopt qltbackup inclexcl "include /tsmsoft/ mgmt"

• Specify the clients that should use the option set with the register node or update node command. After defining a client option set and the client options, you can associate nodes with that set. Use either the CLOPTSET option on the register node command for new nodes, or on the update node command for existing nodes.

Client Option sets sequencing

The server assigns sequence numbers to the specified options automatically. Or you can specify the sequence number for order of processing. For example:

define clientopt backup inclexcl “include d:\plans” seqnumber=20

define clientopt backup inclexcl “include c:\patents” seqnumber=10

Sequence numbers identify the order of processing. You can use client option sets to specify additional options that might not be included in the option file (dsm.opt) for the client. You can specify which clients use the option set with the register node or update node commands. The client can use these defined options during a backup, archive, restore, or retrieve process.

The options defined in a client option set are a subset of the available client options. Options such as communication method are still stored on the client machine. An administrator can control these options centrally so that individual options are not overridden in the client machine.

The options are processed starting with the highest sequence number. Include-exclude statements in the server client option set have priority over the include-exclude statements in the local client option sets. The server include-exclude statements are always enforced and placed at the bottom of the include-exclude list and evaluated before the client include-exclude statements.

If the server option set has several include-exclude statements, the statements are processed starting with the highest sequence number. The client uses the query inclexcl command to view the include-exclude statements in the order that they are processed. The query inclexcl command also shows the source of each include-exclude statement. Use double-digit numbers, for example, 10 and 20, so that you have more space between sequence numbers to insert new statements.

Forcing an Client Option

The sequence number assigned to a client option is used when the same option is defined more than once. Options such as INCLUDE might be defined to a client option set many times, and the sequence is crucial.

You can use the FORCE option to specify options that cannot be overridden by the local option file for the client. To stop an option from being overridden by the client, define the option to the option set with FORCE=YES. The default is NO.

An administrator can use the FORCE parameter to specify whether a client node can override an option value. The default is NO. If FORCE=YES, the client cannot override the value. For example

define clientopt backup compression no force=yes

Copy, query, delete, or update a client option set

Client option sets can be copied, queried, deleted, or updated by using the copy, query, delete, and update cloptset commands. Individual client options within an option set are deleted by using the delete clientopt command. The sequence number of a client option is changed by using the update clientopt command. You can also create a file that has include-exclude statements and point a client to that file. This is convenient when you want to create one set of statements and point multiple clients at it.

Day 34

Configuring INCLUDE and EXCLUDE for managing backup and archive

How to exclude or include files from backup and archive ?

You can create an include-exclude list to exclude a specific file or groups of files from backup services, and to assign specific management classes to files. Specify your include-exclude list in your client options file dsm.opt on windows and dsm.sys file in UNIX. The include-exclude list can also go into a separate file, which is referred to by the inclexcl option. The include-exclude statements are not case-sensitive, if the file is not excluded, it is included.

If you do not create an include-exclude list, Tivoli Storage Manager considers all files for backup services and uses the default management class for backup and archive services. This is an optional task but useful in many scenarios.

The include-exclude list will be processed in the same order whether you work with it in the client GUI or if you open the client options file and use a text editor. The order of the rules are as follows

• The list is read from the bottom to the top

• Processing stops when a match is found

• By default, objects that are not excluded are included.

The include-exclude process is a bottom-up process. The last include-exclude statement is checked first. When the file matches the statement, exclude processing ends for that file. The include-exclude list is generated in the configuration options file. The exclude list in the options file are overridden by the include-exclude options on the command line.

You can use the query inclexcl command to display a list of include and exclude statements in the order they are examined when determining whether an object is to be included.

Excluding directories from backup

The EXCLUDE.DIR statement excludes a directory structure from backup and from being traversed during incremental backup. The EXCLUDE.DIR statements are read first, regardless of their position within the include-exclude list, and the excluded directories are removed from the list of objects that are available to process.

You can abbreviate the EXCLUDE.FILE statement to EXCLUDE. The statement excludes files from backup. A directory is traversed even if all the files from that directory are excluded. If a directory structure is excluded by using EXCLUDE.DIR, subdirectories in the excluded directory tree are not eligible for backup. Any INCLUDE statement that includes part of an excluded directory structure is bypassed at backup time.

The Tivoli Storage Manager backup-archive client builds an internal traverse tree before performing a backup. The EXCLUDE.DIR statement excludes a directory structure from the traverse tree. It prevents directories and directory attributes from being backed up.

Although a directory structure is excluded by using the EXCLUDE.DIR statement, subdirectories and files within the excluded directory structure can be explicitly backed up.

Consider the following example incl-excl list.

include /home/dir1/subdir2 

exclude.dir /home/dir1 

include /home/dir1/subdir1

In the above example, the include statement for the /home/dir1/subdir1 and /home/dir1/subdir2 directories is bypassed when an incremental backup is done because the subdirectory is excluded by use of an EXCLUDE.DIR statement.

dsmc> selective -subdir=yes /home/* 

dsmc> selective /home/dir1/subdir1/*

But with the above statements, the first selective backup command backs up all files except files that are in the excluded directory (home/dir1). This is because the EXCLUDE.DIR statement applies to subdirectories. The second selective backup command backs up files in the /home/dir1/subdir1 directory. This is an explicit backup of an excluded directory that overrides the EXCLUDE.DIR statement.

Enabling client compression and encryption by include-exclude statements

CLIENT COMPRESSION: The compression option compresses files before you send them to the server.

CLIENT ENCRYPTION: You can encrypt the data that goes to the server during a backup or archive operation by using standard encryption.

If you want to include specific files or groups of files for compression, encryption, and adaptive subfile backup processing during a backup or archive operation, you must set the compression option to yes to enable processing for the following actions

• Compression
• Encryption

Tivoli Storage Manager processes EXCLUDE.DIR and other include-exclude statements first. Afterward, Tivoli Storage Manager processes the following statements

• include.compression
• include.encrypt

For example, consider the following include-exclude list

exclude c:\lab\file.txt 

include.compression c:\lab\file.txt 

include.encrypt c:\lab\file.txt

Tivoli Storage Manager examines the exclude c:\lab\file.txt statement first, determines that c:\lab\file.txt is excluded from backup processing and that the file is not a candidate for compression or encryption processing.

Binding Files or Directories to another management class using include-exclude list

In the include-exclude list, you can establish files that are included in or excluded from backup processing. The include statement serves two purposes. One purpose is to specify exceptions to the exclude list. The other purpose is to associate a management class with a file or group of files. The include statement is also used during archiving to determine the management class. The exclude statement is not checked during the archive processing. 

Unless you have the EXCLUDE.DIR statement, directory objects are always included in the backup, even when all the files within the directory are excluded. You can assign a management class for a file or file group by using an INCLUDE statement in your client options file. For example, to associate all of the files in the sales directory with a management class named project, enter the following text

include c:\tsm\project\sales\* project

Wild cards and special characters

The include-exclude list uses metacharacters to select files to include or exclude. Some metacharacters differ, depending on the client platform. These metacharacters are to specify wildcard processing. You can use the metacharacters or wild cards in the command line to specify the file specification on most commands.

? The match one character finds any single character except the directory separator; it does not find the end of the string.

* The match-all character.

\...\  Windows: The match-n character matches zero or more directories.

/.../  UNIX: The match-n character matches zero or more directories.

- The character-class range includes characters from the first character to the last one specified.

[ ]  The open and close character-class characters allow the enumeration of a character class.