Control-M Workload Archiving Configuration

Control-M Workload Archiving is a Control-M add-on that enables you to archive job data in a secure, central repository when jobs finish execution. The Workload Archiving Server archives Control-M/Server job logs, Agent job output, and a small amount of job metadata from Control-M/EM in a separate PostgreSQL or Oracle database for a defined period of time, based on the active Workload Archiving policy.

Workload Archiving policies determine the job data that is archived based on some of the following main criteria:

  • Data Type: Job logs or output.

  • Retention Period: The number of days, months, or years to archive the data.

  • System Type: Distributed systems or z/OS mainframes.

  • The Workload Archiving process receives Control-M/EM authorization data to apply Control-M/EM user authorization rules. You do not need to define additional authorizations, as described in User and Role Authorizations.

  • (z/OS only) To collect job output and logs, you must define the security user ARCUSER with the View LOG and View SYSOUT permissions.

  • (z/OS only) To enable the efficient performance of log archiving, you must activate and ensure periodic updates of the IOA Log Index component, as described in IOA Log Index.

The following procedures describe how to start up and configure Workload Archiving settings:

The following utilities enable you to configure Workload Archiving:

Starting Up the Workload Archiving Server

This procedure describes how to start up the Workload Archiving Server from the CCM.

Begin

  • From the Components Tree in the left pane, right-click the Workload Archiving Server component, select Desired State, and then select Up.

    The Workload Archiving Server starts up.

Shutting Down the Workload Archiving Server

This procedure describes how to shut down the Workload Archiving Server from the CCM.

Begin

  • From the Components Tree in the left pane, right-click the Workload Archiving Server component, select Desired State, and then select Down.

    The Workload Archiving Server shuts down.

Deleting Data from the Workload Archiving Server

This procedure describes how to delete data—including jobs, output, and logs—from the Workload Archiving server. You can delete data that belongs to a specific rule or to data that matches specific command criteria.

Begin

  • Type one of the following commands:

    • Rule Name: arc_cleanup -user <user> -password <password> -rule <rule_name>

    • Data Attributes: arc_cleanup -user <user> -password <password> -job_name <job_name> -job_name_exceptions <job_name_exceptions> -ctm <Control-M/Server name> -ctm_exceptions <Control-M/Server name exceptions> -job_state <OK|NOTOK> -app <Application name> -app_exceptions <appNameExceptions> -subapp <Sub-application_name> -subapp_exceptions <Sub-application_exceptions> -folder <folder_name> -folder_exceptions <folder_name_exceptions> -lib <library> -lib_exceptions <library_exceptions>

    arc_cleanup Parameters

    The following table describes the mandatory arc_cleanup parameters.

    Parameter

    Description

    -user

    Defines the username.

    You can use * and ? wildcards, as described in Pattern-Matching Strings.

    -password

    Defines the user password.

    Rules:

    • Wildcards: You can use * and ? wildcards, as described in Pattern-Matching Strings.

    • ! Characters: Passwords that contain ! characters must be enveloped in " and a \ must precede the !.

    The following command uses an ! character in its password and deletes all archived jobs that contain the string AA, except for job names that begin with the letter B:

    arc_cleanup –user <user> -password "Passw\!rd" –job_name "*AA*" –job_name_exceptions "B*"

Defining a Workload Archiving Policy

This procedure describes how to define a Workload Archiving policy that enables you to determine which data to archive and how long to store it in the Workload Archiving database.

Begin

  1. From the icon, select Configuration.

    The Configuration domain opens.

  2. From the drop-down list, select Workload Archiving.

    The Workload Archiving tab appears.

  3. Click Add Rule.

    The Add Archive Policy Rule dialog box appears.

  4. In each field, type or select the required value, as described in Workload Archiving Policy Rule Parameters.

  5. Click Add.

    The Workload Archiving policy rule appears in the table.

Workload Archiving Policy Rule Parameters

The following table describes the Workload Archiving policy rule parameters.

Attribute

Description

Activate Rule

Determines whether to activate a Workload Archiving policy rule.

Name

Defines the Workload Archiving policy name.

Description

Defines a free-text description of the Workload Archiving policy.

Archive Data

Determines which of the following data to archive:

  • Logs and Outputs

  • Logs

  • Outputs

Retention Period

Determines the number of days, months, or years to archive the data defined in Archive Data.

Platform

Determines which of the following platforms archive the data:

  • All

  • Distributed systems

  • z/OS

Job Type

Determines the job type of the jobs to archive (for example, OS jobs).

This attribute is not relevant for z/OS.

Job Status

Determines whether to archive jobs with the following status:

  • All: Applies the rule to jobs with all status types.

  • Ended OK: Applies the rule only to jobs that Ended OK.

  • Ended Not OK: Applies the rule only to jobs that Ended Not OK.

Output Limit

Determines the maximum amount of storage on the Workload Archiving Server to reserve for the job output.

Range:

  • 1-100 MB

  • 1-999 KB

Trim Output

Discards all or part of the job output file if it exceeds the amount defined in Output Limit.

Valid Values:

  • Do not archive output when it exceeds the limit: Discards the entire output file.

  • Trim output from the start when it exceeds the limit: Removes part of the data from the beginning of the last output file.

  • Trim output from the end when it exceeds the limit: Removes part of the data from the end of the last output file.

Control-M/Server

Filters the jobs on the Control-M/Server that you define.

You can use * and ? wildcards, as described in Pattern-Matching Strings.

Job Name

Filters the jobs to be archived.

You can use * and ? wildcards, as described in Pattern-Matching Strings.

Application

Filters the jobs in the Applications that you define.

You can use * and ? wildcards, as described in Pattern-Matching Strings.

Sub-application

Filters the jobs in the Sub-applications that you define.

You can use * and ? wildcards, as described in Pattern-Matching Strings.

Folder Name

Filters the jobs in the folders that you define.

The folders are defined in the following format:

<SMART_Folder_Name>/<Sub-folder_Name>

You can use * and ? wildcards, as described in Pattern-Matching Strings.

The following SMART folders and sub-folders are defined in a Workspace:

  • SMART folder A_SMART contains sub-folder a_Sub.

  • SMART folder B_SMART contains sub-folder b_Sub.

  • SMART folder C_SMART contains sub-folder b_Sub.

The Folder parameter is defined as follows:

  • A_SMART: Archives the A_SMART SMART folder.

  • B_SMART/b_Sub: Archives the b_Sub sub-folder in the B_SMART SMART folder.

  • */b_Sub: Archives both b_Sub sub-folders, in the B_SMART and C_SMART SMART folders.

Folder Library

(z/OS only) Filters the folder libraries for z/OS.

You can use * and ? wildcards, as described in Pattern-Matching Strings.

Configuring the Workload Archiving Cyclic Log Collection

This procedure describes how to configure how often Workload Archiving archives cyclic job logs and outputs. This can reduce the load on resources, such as CPU usage and disk space.

Begin

  1. From the Control-M/EM host, navigate to the following directory:

    <Control-M/EM_Home>\archive\config

  2. Open the SystemParameters.xml file and locate the cyclic.log.collection.interval.minutes system parameter.

  3. Change the parameter value to reflect how often, in minutes, Workload Archiving archive cyclic job logs and outputs.

    If you enter a value of 0, Workload Archiving archives a cyclic job log and output after every job execution.

Configuring Retention Parameters to Prevent Loss of Archive Data

This procedure describes how to check and adjust retention parameters, which enables you to prevent data loss when the Workload Archiving server is down and cannot archive job data. You must ensure that data is retained in the Control-M components over the period of time that begins when the Workload Archiving server goes down and ends after Workload Archiving completes retrieval of the stored job data on the components when the server is back up.

Before You Begin

  • Set aside additional free database space on Control-M/EM and Control-M/Server and additional disk space on each Agent to accommodate the extra space required for storing more days of job logs and output files.

Begin

  1. Do the following to update the retention parameters for Control-M/Server via the ctmsys utility:

    1. Run the ctmsys utility, as described in Running the ctmsys Utility.

    2. From the Main Menu, choose option 2, System Parameters.

    3. Type n to display the second page of parameters.

    4. Update the following parameters with larger values:

      • 6 - Maximum Days Retained by Control-M Log

      • 7 - Maximum Days to Retain Output Files

    5. Save your changes and exit the utility.

  2. To create a retention period that is longer than the default of 30 days, do the following:

    1. Update the GENERAL_CLEANUP_INTERVAL parameter on each of your Agents, as described in Configuring Agent System Parameters and Diagnostics Parameters.

    2. Update the following Control-M/EM system parameters, as described in Configuring Control-M/EM System Parameters:

  3. Do the following to verify that the Control-M/Server retention parameters take effect on all Agents:

    1. Open the CONFIG.dat file in the following directory and delete the SYSKEEPDAYS parameter if it is there:

      • UNIX/Linux: <Agent_home>/data/

      • Windows: <Agent_home>\Default\DATA

    2. From the CCM, right-click each Agent and select Properties.

      The Control-M/Agent Properties dialog box appears.

    3. From the Output tab, verify that the Days to retain OUTPUT files drop-down list is set to Default.

    4. Open the config.dat file in the following directory:

      • UNIX/Linux: <Control-M/Server_home>/ctm_server/data

      • Windows: <Control-M/Server_home>\ctm_server\data

    5. Add or update the following line:

      AGENTS_CLEANUP_IN_NEWDAY Y

    6. Restart the Control-M/Server.

  4. Update the following retention parameters in $EM_HOME/archive/config/SystemParameters.xml, on the Workload Archiving server, with the same values that you used in Step 1:

    • override.num.days.log.keep.in.server

    • override.num.days.output.keep.in.server

  5. Recycle the Workload Archiving server.

    The retention parameters are reconfigured.

Workload Archiving Command Line Utilities

The following table describes the Workload Archiving command line utilities.

Utility

Description

arc_start_server

Starts up the Workload Archiving server.

arc_stop_server

Shuts down the Workload Archiving server.

arc_server_state

Checks the status of the Workload Archiving server.

arc_kill_server

Terminates Workload Archiving server processes if they do not stop regularly.

arc_sql

Starts an SQL prompt in the Workload Archiving database.

arc_data_collector

Creates a package of OS and Workload Archiving logs and configuration for review by BMC Support.

arc_recollect_jobs

Attempts to recollect the job output or log after the Workload Archiving server fails to collect and exceeds the retry threshold.

To avoid resource overload on the Workload Archiving server, apply accurate search criteria when you run this script.