Product Knowledge Base

Space shortcuts

Page tree

Introduction

This article describes individual configuration parameters and provides examples of their use with monitors such as op5 and Zabbix and using of the mail und JMS interfaces.

Send notifications

Notify on error

SystemMonitorNotification / Notification / NotificationMonitor / @service_name_on_error is configured.

SystemMonitorNotification / Notification / NotificationObjects / JobChain
Error messages
Recovery messages
SystemMonitorNotification / Notification / NotificationObjects / Job
Error messages

Notify on success

SystemMonitorNotification / Notification / NotificationMonitor / @service_name_on_success is configured.

SystemMonitorNotification / Notification / NotificationObjects / JobChain
Success messages
SystemMonitorNotification / Notification / NotificationObjects / Job
Success messages

Configuration Editor

We recommend that the XML Editor is used generate monitoring configuration objects. This editor automatically uses an XSD Schema to generate configuration suggestions and validate configurations, and its use is intended to provide a significant reduction in the time required to develop and test a configuration.

XSD Schema locations

Configuration

JobScheduler

Activation of Monitoring Interface 

Note: 

  • JobScheduler releases before 1.13.1
    • SystemMonitorNotification_<MonitorSystem>.xml file(s) (see below) must be configured before activtion.
  • JobScheduler releases starting from 1.13.1
    • SystemMonitorNotification_<MonitorSystem>.xml file(s) (see below) must be configured before activtion or
    • The NOTIFICATION configuration was forwarded by the Joc Cockpit to the respective JobScheduler Master.

SystemMonitorNotification files

JobScheduler releases before 1.13.1

Location: <scheduler_data>/config/notification

FileDescription
SystemMonitorNotification_v1.0.xsd

The XML Schema file defines which values are allowed in your XML files for the JobScheduler monitoring.

That means that to configure the JobScheduler objects you want to monitor and the System Monitor you just have to modify your SystemMonitorNotification_<MonitorSystem>.xml files but not the XML schema file.

SystemMonitorNotification_<MonitorSystem>.xml

 Configuration file for each System Monitor.

  • Specifies the delivery way to System Monitor.
  • Specifies notification for error or success conditions
  • Specifies notification to measure performance of JobScheduler objects
 

SystemMonitorNotificationTimers.xml

Configuration file for all System Monitors.

  • Specifies notification to measure performance of JobScheduler objects

This file is optional and contains the definitions of the SystemMonitorNotification / Timer elements.

JobScheduler releases starting from 1.13.1

Note: Usage of the configuration files in the <scheduler_data>/config/notification folder has been deprecated.

Location:  <scheduler_data>/config/live/sos/.configuration/notification

FileDescription
SystemMonitorNotification_v1.0.xsd

The XML Schema file defines which values are allowed in your XML files for the JobScheduler monitoring.

notification.xml

 Configuration file for System Monitors:

  • Specifies the delivery way to System Monitor.
  • Specifies notification for error or success conditions
  • Specifies notification to measure performance of JobScheduler objects
  • this file is an automatically generated file and will be overridden when the configuration is deployed


SystemMonitorNotification Elements

The configuration element descriptions are organized into the following major categories:

ElementElement descriptionDescription
SystemMonitorNotificationTop Level ElementConfiguration for notifications to be sent to a system monitor.
NotificationRequired, multiple use allowed inside the SystemMonitorNotification elementSpecifies a system monitor notification that includes a command line invocation and the JobScheduler objects.
TimerOptional or multiple use allowed inside the SystemMonitorNotification elementPerformance measurement definition.
SystemMonitorNotification

JITL-230 - Getting issue details... STATUS

SystemMonitorNotification supports the following attributes:

AttributeJobScheduler releaseUsageDescription
system_idbefore 1.13.1required

System Monitor identifier.

See JobScheduler - Job Chains customization 

Note:

  • attribute system_id 

    • in case of the SystemMonitorNotificationTimers.xml the value of this attribute is not important and can have any value.

      • e.g.: timers


starting from 1.13.1required
  • If a configuration is created by JOC Cockpit the value of the attribute system_id will be ignored.
    • The SystemNotifier job chain will use MonitorSystem as identifier instead.
Example 
<SystemMonitorNotification system_id="op5">
...


SystemMonitorNotification / Notification

Notification supports the following attributes:

AttributeUsageDescription
nameoptionalNotification description

 

Example 
...
<Notification name="Mail: on failed job">
...

 

The following elements may be nested inside a Notification element:

ElementElement descriptionDescription
NotificationMonitorRequired, only once inside the Notification elementSpecifies the System Monitor interface that is being used for messages: either by a Plug-in Interface or by command line invocation
NotificationObjectsRequired, only once inside the Notification elementSpecifies the Job Chain and the Timer definitions
SystemMonitorNotification / Notification / NotificationMonitor

The JobScheduler Interface Monitor can be used to monitor the messages for the 3 use cases:

  • error case
    • an error has occurred / been recovered during a job chain / job execution
      • the service_name_on_error setting is responsible for this monitoring case
  • success case
    • a job chain / job ends successfully
      • the service_name_on_success setting is responsible for this monitoring case
  • performance check (see Timer)
    • usually the service_name_on_error setting is responsible for this monitoring case but the performance check will also work if only the service_name_on_success setting has been defined.


In addition, the service_name_on_error / service_name_on_success attributes have the following meaning:

  • NotificationInterface
    • The setting must match the corresponding service name in the System Monitor such as Nagios or op5.
  • NotificationCommand
    • Freely selectable, has no further meaning than to identify a notification.
  • NotificationMail
    • Freely selectable, has no further meaning than to identify a notification.
  • NotificationJMS
    • The setting must match the corresponding queue/topic name in the JMS Server.

Note:

  • attributes service_name_on_error and service_name_on_success
    • at least one of these attributes must be configured
    • both attributes can be configured together
    •  the use of this settings must be unique within one SystemNotification

NotificationMonitor supports the following attributes:

AttributeUsageDescription
service_name_on_errorOptional

See explanation above.

service_name_on_successOptional

See explanation above.

service_status_on_errorOptional

This setting specifies the service status code for error messages.

Default: CRITICAL

service_status_on_successOptional

This setting specifies the service status code for success messages

Default: OK

Example 
<!-- Example 
op5 NSCA Status: 
0 - OK 
1 - WARNING 
2 - CRITICAL 
3 - UNKNOWN --> 
... 
<!-- Sending errors as CRITICAL (default) --> 
<NotificationMonitor service_name_on_error="JobScheduler Monitoring Errors"> 
... 
<!-- Sending occurred errors as WARNING --> 
<NotificationMonitor service_name_on_error="JobScheduler Monitoring Errors" service_status_on_error="1"> 
...

One of the following elements must be nested inside a NotificationMonitor element:

ElementElement descriptionDescription
NotificationInterfaceOptional or only once inside the NotificationMonitor elementNSCA plug-in Interface to be executed for System Monitor notification
NotificationCommandOptional or only once inside the NotificationMonitor elementCommand line to be executed for System Monitor notification
NotificationMailOptional or only once inside the NotificationMonitor elementMail interface to be executed for System Monitor notification
NotificationJMSOptional or only once inside the NotificationMonitor elementJMS interface to be executed for System Monitor notification


SystemMonitorNotification / Notification / NotificationMonitor / NotificationInterface

NSCA plug-in Interface to be executed for System Monitor notification.

NotificationInterface supports the following attributes:

AttributeUsageDescription
monitor_hostRequiredThis setting specifies the host name or IP address of the System Monitor host.
monitor_portRequiredThis setting specifies the TCP port that the System Monitor will listen to.
monitor_passwordOptional

This setting specifies the password

  • for NSCA the password is configured in the ncsa.cfg file.
monitor_connection_timeoutOptional

This setting specifies the connection timeout in ms.

Default: 5000

monitor_response_timeoutOptionalThis setting specifies the response timeout in ms.
monitor_encryptionOptional

This setting specifies that the communication with the System Monitor is encrypted. By default no encryption is used.

  • NONE             - no encryption
  • XOR            - XOR encryption
  • TRIPLE_DES - use of triple des algorithm for encryption
service_hostRequiredThis setting specifies the name of the host that executes the passive check. The name must match the corresponding setting in the System Monitor.
pluginOptional

Default:

    • JobScheduler releases before 1.11

      • com.sos.scheduler.notification.plugins.notifier.SystemNotifierSendNscaPlugin

    • JobScheduler releases starting from 1.11

      • com.sos.jitl.notification.plugins.notifier.SystemNotifierSendNscaPlugin

See : Message, Examples

Example 
...
<NotificationInterface monitor_host="monitor_host" monitor_port="5667" monitor_encryption="XOR" service_host="service_host"><![CDATA[
scheduler id=${MON_N_SCHEDULER_ID}, history id=${MON_N_ORDER_HISTORY_ID}, job_chain=${MON_N_JOB_CHAIN_NAME}(${MON_N_ORDER_ID}), step =${MON_N_ORDER_STEP_STATE}, error=${MON_N_ERROR_TEXT}
]]></NotificationInterface>
...

System Monitor: Opsview

In case you are using OpsView as the monitoring tool, the plugin used in NotificationInterface will not work, since OpsView supports 20 types of encryption, which are not supported by this plugin. You can use it only in case that no encryption is used (type: NONE) .

Instead, you should use the XML element NotificationCommand and indicate the exact command to send passive checks to an OpsView server from a remote machine (see example for op5 or example for zabbix).


SystemMonitorNotification / Notification / NotificationMonitor / NotificationCommand

Command line to be executed for System Monitor notification.

NotificationCommand supports the following attributes:

AttributeUsageDescription
pluginOptional

Default:

  • JobScheduler releases before 1.11
    • com.sos.scheduler.notification.plugins.notifier.SystemNotifierProcessBuilderPlugin

  • JobScheduler releases starting from 1.11

    • com.sos.jitl.notification.plugins.notifier.SystemNotifierProcessBuilderPlugin


See : Message, Examples

Example 
...
<NotificationCommand><![CDATA[
echo scheduler id=${MON_N_SCHEDULER_ID}, history id=${MON_N_ORDER_HISTORY_ID}, job_chain=${MON_N_JOB_CHAIN_NAME}(${MON_N_ORDER_ID}), step =${MON_N_ORDER_STEP_STATE}, error=${MON_N_ERROR_TEXT} > D://errors.txt
]]></NotificationCommand>
...
SystemMonitorNotification / Notification / NotificationMonitor / NotificationMail

JS-1388 - Getting issue details... STATUS

The Mail interface to be executed for System Monitor notification.

The Mail interface reads the following values from the configuration files:

  • config/factory.ini
    • Section spooler
      • log_mail_from
      • log_mail_to
      • log_mail_cc
      • log_mail_bcc
      • smtp
      • mail_queue_dir
      • mail_queue_only
    • Section smtp
      • mail.smtp.user
      • mail.smtp.password
      • mail.smtp.port
      • mail.smtp.connectiontimeout
      • mail.smtp.timeout
  • config/private/private.conf
    • joc.url

NotificationMail supports the following attributes:

AttributeUsageDescription
content_typeOptional

Content type of the e-mail.

Possible values:

  • text/plain
  • text/html.


Default: text/html

charsetOptional

Charset of the e-mail.


Default: ISO-8859-1

encodingOptional

Encoding of the e-mail.

Possible values:

  • 7bit
  • Quoted-Printable
  • Base64.


Default: 7bit

priorityOptional

Priority of the e-mail.

Possible values:

  • Highest
  • High
  • Normal.


Default: Normal

pluginOptional

Java class of the plugin implementation (extends com.sos.jitl.notification.plugins.notifier.SystemNotifierCustomPlugin)


Default: com.sos.jitl.notification.plugins.notifier.SystemNotifierSendMailPlugin

The following elements can be nested inside a NotificationMail element:

ElementElement descriptionDescription
FromOptional or only once inside of the NotificationMail element  E-mail address of the account that sends e-mail.
ToOptional or only once inside of the NotificationMail elementE-mail address of the recipient(s) of a notification e-mail.
CCOptional or only once inside of the NotificationMail elementE-mail address of the recipient(s) of a carbon copy notification e-mail.
BCCOptional or only once inside of the NotificationMail elementE-mail address of recipient(s) of a blind carbon copy notification e-mail.
SubjectRequired, only once inside of the NotificationMail elementSubject of an e-mail notification.
BodyRequired, only once inside of the NotificationMail elementBody of an e-mail notification.
SystemMonitorNotification / Notification / NotificationMonitor / NotificationMail / From

E-mail address of the account that sends the e-mail.

The mail notification interface uses the value of the log_mail_from entry (configuration file config/factory.ini) when this element is not set.

 

SystemMonitorNotification / Notification / NotificationMonitor / NotificationMail / To

E-mail address of the recipient(s) of a notification e-mail.

When this element

  • is not set
    • log_mail_to will be used
  • is set
    •   log_mail_to, log_mail_cc, log_mail_bcc are not used

 

SystemMonitorNotification / Notification / NotificationMonitor / NotificationMail / CC

E-mail address of the recipient(s) of a carbon copy notification e-mail.

When this element

  • is not set
    • log_mail_cc will be used (if the NotificationMail/To element is not defined - see above)
  • is set
    •   log_mail_cc, log_mail_bcc are not used
SystemMonitorNotification / Notification / NotificationMonitor / NotificationMail / BCC

E-mail address of recipient(s) of a blind carbon copy notification e-mail.

When this element

  • is not set
    • log_mail_bcc will be used (if the NotificationMail/To or NotificationMail/CC elements are not defined - see above)
  • is set
    •   log_mail_bcc are not used

 

SystemMonitorNotification / Notification / NotificationMonitor / NotificationMail / Subject

Subject of an e-mail notification.

The Subject can contain the JobScheduler Monitoring Interface variables.

See : Message, Examples

Example 
...
<Subject><![CDATA[JobScheduler notification: ${SERVICE_MESSAGE_PREFIX}, job executed with errors: ${MON_N_JOB_NAME}]]></Subject>
...

 

SystemMonitorNotification / Notification / NotificationMonitor / NotificationMail / Body

Body of an e-mail notification.

The Body can contain the JobScheduler Monitoring Interface variables.

See : Message, Examples

Example 
<!DOCTYPE html>
<html lang="en">
<head>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>
    <style>.tg  {border-collapse:collapse;border-spacing:0;border-color:#aaa;}.tg td{font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;border-color:#aaa;color:#333;background-color:#fff;}.tg th{font-family:Arial, sans-serif;font-size:14px;font-weight:bold;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;border-color:#aaa;color:#fff;background-color:#f38630;}</style>
    <title>JobScheduler Notification</title>
</head>
<body>
<table class="tg">
    <tr>
        <th colspan="4">Error</th>
    </tr>
    <tr>
        <td>Code:</td><td>${MON_N_ERROR_CODE}</td>
        <td>Messsage</td><td>${MON_N_ERROR_TEXT}</td>
    </tr>
    <tr>
        <th colspan="4">JobScheduler</th>
    </tr>
    <tr>
        <td>JobScheduler ID</td><td>${MON_N_SCHEDULER_ID}</td>
        <td>Agent URL</td><td>${MON_N_AGENT_URL}</td>
    </tr>
    <tr>
        <th colspan="4">Order</th>
    </tr>
    <tr>
        <td>Order ID</td><td><a href="${JOC_HREF_ORDER}">${MON_N_ORDER_ID}</a></td>
        <td>Order Title</td><td>${MON_N_ORDER_TITLE}</td>
    </tr>
    <tr>
        <td>Job Chain Name</td><td><a href="${JOC_HREF_JOB_CHAIN}">${MON_N_JOB_CHAIN_NAME}</a></td>
        <td>Job Chain Title</td><td>${MON_N_JOB_CHAIN_TITLE}</td>
    </tr>
    <tr>
        <td>Job Name</td><td><a href="${JOC_HREF_JOB}">${MON_N_JOB_NAME}${MON_N_JOB_NAME}</a></td>
        <td>Job Title</td><td>${MON_N_JOB_TITLE}</td>
    </tr>
    <tr>
        <th colspan="4">Task History</th>
    </tr>
    <tr>
        <td>Task ID</td><td>${MON_N_TASK_ID}</td>
        <td>Time elapsed</td><td>${MON_N_TASK_TIME_ELAPSED}</td>
    </tr>
    <tr>
        <td>Start Time</td><td>${MON_N_TASK_START_TIME}</td>
        <td>End Time</td><td>${MON_N_TASK_END_TIME}</td>
    </tr>
</table>
</body>
</html>


SystemMonitorNotification / Notification / NotificationMonitor / NotificationJMS

JITL-280 - Getting issue details... STATUS

JMS Interface to be executed for System Monitor notification.

Note: the provider-specific queue or topic name will be defined with the service_name_on_error / service_status_on_succes attribute of the parent SystemMonitorNotification / Notification / NotificationMonitor element.

NotificationJMS supports the following attributes: 

AttributeUsageDescription
client_idOptional

The client identifier for this connection.

See: Connection.setClientID

destinationOptional

A Destination object encapsulates a provider-specific address.

Possible values:

  • Queue
  • Topic.

See: Destination


Default: Queue

acknowledge_modeOptional

Session acknowledgment mode.

Possible values: 

  • Session.AUTO_ACKNOWLEDGE
  • Session.CLIENT_ACKNOWLEDGE
  • Session.AUTO_ACKNOWLEDGE

See: Session


Default: Session.CLIENT_ACKNOWLEDGE

delivery_modeOptional

Delivery mode.

Possible values: 

  • DeliveryMode.NON_PERSISTENT
  • DeliveryMode.PERSISTENT

See:


Default: DeliveryMode.PERSISTENT

priorityOptional

The producer's default priority.

See: MessageProducer.setPriority


Default: 4

time_to_liveOptional

Sets the default length of time in milliseconds from its dispatch time that a produced message should be retained by the message system.

See: MessageProducer.setTimeToLive

Possible values:

  • <n> - duration in milliseconds
    • e.g.: 60000
  • <n>s - duration in seconds
    • e.g.: 30s
  • <n>m - duration in minutes
    • e.g.: 30m
  • <n>h - duration in hours
    • e.g.: 24h
  • <n>d - duration in days
    • e.g.: 30d
  • <n>w - duration in weeks
    • e.g.: 4w
  • Combined values (with blank space as separator):
    • e.g.: 1w 1d 1h

Default: 0

pluginOptional

Java class of the plugin implementation (extends com.sos.jitl.notification.plugins.notifier.SystemNotifierCustomPlugin)


Default: com.sos.jitl.notification.plugins.notifier.SystemNotifierSendJMSPlugin

 

One of the following elements must be nested inside a NotificationJMS element:

ElementElement descriptionDescription
ConnectionFactoryOptional or only once inside the NotificationJMS elementSpecifies use of a JMS ConnectionFactory implementation
ConnectionJNDIOptional or only once inside the NotificationJMS elementSpecifies use of a JNDI properties file to create a JNDI IntialContextFactory


JMS message:

ElementElement descriptionDescription
MessageRequired, only once inside of NotificationJMS elementBody of a JMS notification

 

SystemMonitorNotification / Notification / NotificationMonitor / NotificationJMS / ConnectionFactory

Specifies use of a JMS ConnectionFactory implementation.

ConnectionFactory supports the following attributes:  

AttributeUsageDescription
java_classRequired

Java class of the JMS ConnectionFactory

e.g.: org.apache.activemq.ActiveMQConnectionFactory

user_nameOptionalThe caller's user name
passwordOptionalThe caller's password

 The following element can be nested inside a ConnectionFactory element:

ElementElement descriptionDescription
ConstructorArgumentsOptional or only once inside of ConnectionFactory element  

 

SystemMonitorNotification / Notification / NotificationMonitor / NotificationJMS / ConnectionFactory / ConstructorArguments

The following elements can be nested inside a ConstructorArguments element:

ElementElement descriptionDescription
ArgumentRequired, multiple use allowed inside the ConstructorArguments element  JMS ConnectionFactory constructor argument

 

Example: ActiveMQConnectionFactory with 1 constructor argument 
...
<ConnectionFactory java_class="org.apache.activemq.ActiveMQConnectionFactory">
    <ConstructorArguments>
        <Argument type="java.lang.String"><![CDATA[tcp://localhost:61616]]></Argument>
    </ConstructorArguments>
</ConnectionFactory>
...

 

Example: ActiveMQConnectionFactory with 3 constructor arguments 
...
<ConnectionFactory java_class="org.apache.activemq.ActiveMQConnectionFactory">
    <ConstructorArguments>
        <Argument type="java.lang.String"><![CDATA[my_user_name]]></Argument>
        <Argument type="java.lang.String"><![CDATA[my_password]]></Argument>
        <Argument type="java.lang.String"><![CDATA[tcp://localhost:61616]]></Argument>
    </ConstructorArguments>
</ConnectionFactory>
...

 

SystemMonitorNotification / Notification / NotificationMonitor / NotificationJMS / ConnectionFactory / ConstructorArguments / Argument

 Argument supports the following attributes: 

Attribute
Usage
Description
typeRequired

Java type of a constructor argument.

Possible values:

  • boolean
  • double
  • float
  • int
  • long
  • short
  • java.lang.Boolean
  • java.lang.Double
  • java.lang.Float
  • java.lang.Integer
  • java.lang.Long
  • java.lang.Short
  • java.lang.String


Default: java.lang.String

The value of the constructor argument will be stored as CDATA node.

Example 
...
<Argument type="java.lang.String"><![CDATA[tcp://localhost:61616]]></Argument>
...

 

SystemMonitorNotification / Notification / NotificationMonitor / NotificationJMS / ConnectionJNDI

Specifies use of a JNDI properties file to create a JNDI IntialContextFactory.

See: Connecting to the JMS Server by Using JNDI

ConnectionJNDI supports the following attributes:  

AttributeUsageDescription
fileRequiredPath of the JNDI properties file
lookup_nameOptional

Name to lookup JMS connection factory objects


Default: ConnectionFactory

 

Example: content of the JNDI properties file for ActiveMQ 
java.naming.factory.initial=org.apache.activemq.jndi.ActiveMQInitialContextFactory
java.naming.provider.url=tcp://localhost:61616

 

SystemMonitorNotification / Notification / NotificationMonitor / NotificationJMS / Message

Body of a JMS notification.

See : Message, Examples


SystemMonitorNotification / Notification / NotificationObjects

One of the following elements must be nested inside a NotificationObjects element:

ElementElement descriptionDescription
Job Optional or multiple use allowed inside the NotificationObjects element  Restricts notifications for order or standalone jobs 
JobChainOptional or multiple use allowed inside the NotificationObjects elementRestricts notifications for job chains
TimerRefOptional or multiple use allowed inside the NotificationObjects elementRestricts notifications for performance checks (Timer)
MasterMessageOptional or only once inside the NotificationObjects element

JS-1837 - Getting issue details... STATUS

Includes problems detected by a JobScheduler Master, e.g. database connection lost.

TaskWarningOptional or only once inside the NotificationObjects element

JS-1837 - Getting issue details... STATUS  

 Includes job execution warning messages. 

TaskIfLongerThanOptional or only once inside the NotificationObjects element

JITL-522 - Getting issue details... STATUS  

Includes the feature to send notification in case that the execution of job requires a longer duration than expected.

TaskIfShorterThanOptional or only once inside the NotificationObjects element

JITL-522 - Getting issue details... STATUS

Includes the feature to send notification in case that the execution of job requires a shorter duration than expected.

Example 
<SystemMonitorNotification system_id="op5"> 
  <Notification> 
    <NotificationMonitor service_name_on_error="JobScheduler Monitoring Errors"> 
      ... 
    </NotificationMonitor> 
    <NotificationObjects>
      <!-- Send the job error, occurrent in the "test/my_job" order job, to the "JobScheduler Monitoring Errors" service. -->        
      <Job name="test/my_job" /> 
      <!-- Send the job chain error, occurrent in the "test/my_jobchain" job chain, to the "JobScheduler Monitoring Errors" service. --> 
      <JobChain name="test/my_jobchain" /> 
    </NotificationObjects> 
 </Notification> 
</SystemMonitorNotification>  


SystemMonitorNotification / Notification / NotificationObjects / Job 

This element specifies the order-controlled or standalone jobs for which notifications are being sent to a system monitor.

Support for standalone jobs starting with JITL-400 - Getting issue details... STATUS .


Job supports the following attributes:

AttributeUsageDescription
notifications

Optional

Integer

Specifies the number of transfers the same notification to a System Monitor.

Default: 1

scheduler_idOptional

Notifications are restricted to the JobScheduler instance with the given identification. By default notifications will be sent for all JobScheduler instances that log into the same database.

Regular expression can be used.

nameOptional

Job name including possible folder names.

Regular expression can be used.

JITL-353 - Getting issue details... STATUS

return_code_from Optional Restricts notifications for jobs for a particular return code range. 
return_code_from Optional  Restricts notifications for jobs for a particular return code range.  
Example 
...
<Job notifications="2" name="test/my_job"/>
...
<Job scheduler_id="scheduler_4444" />
...
<Job scheduler_id="scheduler_4444" name="test/my_.*" />
... 
<Job name="test/my_job" return_code_from="5"/> 
...  
<Job name="test/my_job" return_code_to="10"/>
...  
<Job name="test/my_job" return_code_from="5" return_code_to="5"/>  
...


SystemMonitorNotification / Notification / NotificationObjects / JobChain

Specifies the job chains for which notifications are being sent to a system monitor.
The element can be repeatedly used to specifiy a number of job chains.

Default behaviour for repeatedly failed job chain steps: when an error reoccurrs in the same job node for which a notification has already been sent then this order state is considered being previously notified and no new notification will be sent.
See child element NotifyRepeatedError.

JobChain supports the following attributes:

AttributeUsageDescription
notifications

Optional

Integer

Specifies the number of transfers the same notification to a System Monitor.

Default: 1

scheduler_idOptional

Notifications are restricted to the JobScheduler instance with the given identification. By default notifications will be sent for all JobScheduler instances that log into the same database.

Regular expression can be used.

nameOptional

Job chain name including possible folder names.

Regular expression can be used.

JITL-353 - Getting issue details... STATUS

return_code_from Optional Restricts notifications for job chains  for a particular return code range. 
return_code_from Optional  Restricts notifications for job chains for a particular return code range.  
step_fromOptionalRestricts notifications for job chains to a sequence of job nodes that are specified with the step_from and step_to attributes.
step_toOptionalRestricts notifications for job chains to a sequence of job nodes that are specified with the step_from and step_to attributes.
excluded_stepsOptionalSpecifies the steps which will be excluded from the analyzing (separated by semicolon)

The following element can be nested inside a JobChain element:

ElementElement descriptionDescription
NotifyRepeatedErrorOptional or only once inside the JobChain element  Send notifications for all errors that occur, do not suppress errors for repeatedly failed executions.
Example 
...
<JobChain notifications="2" name="test/my_jobchain"/>
...
<JobChain scheduler_id="scheduler_4444" />
...
<JobChain scheduler_id="scheduler_4444" name="test/my_.*" />
... 
<JobChain name="test/my_jobchain" return_code_from="5"/> 
... 
<JobChain name="test/my_jobchain" return_code_to="10"/>
... 
<JobChain name="test/my_jobchain" return_code_from="5" return_code_to="5"/>  
...
<JobChain name="test/my_jobchain" step_from="200"/>
...
<JobChain name="test/my_jobchain" step_to="500"/>
...
<JobChain name="test/my_jobchain" step_from="300" step_to="300"/>
...
<JobChain name="test/my_jobchain" excluded_steps="200;300"/>
...
<JobChain name="test/my_jobchain">
  <NotifyRepeatedError />
</JobChain>
...
SystemMonitorNotification / Notification / NotificationObjects / JobChain / NotifyRepeatedError

JITL-534 - Getting issue details... STATUS

Send notifications for all errors that occur, do not suppress errors for repeatedly failed executions.

One of the following elements can be nested inside a NotifyRepeatedError element:

ElementElement descriptionDescription
NotifyByInterventionOptional or only once inside the NotifyRepeatedError element  Send notifications for errors that occur due to repeated failed executions if the restart was caused by manual intervention.
NotifyByPeriodOptional or only once inside the NotifyRepeatedError element Send notifications for errors that occur due to repeatedly failed executions if a configurable period of time is exceeded.
Example 
...
<JobChain name="test/my_jobchain">
  <NotifyRepeatedError>
      <NotifyByIntervention />
  </NotifyRepeatedError>
</JobChain>
...
<JobChain name="test/my_jobchain">
  <NotifyRepeatedError>
      <NotifyByPeriod period="5h 30m" />
  </NotifyRepeatedError>
</JobChain>
...
<JobChain name="test/my_jobchain">
  <NotifyRepeatedError>
      <NotifyByIntervention />
      <NotifyByPeriod period="2h" />
  </NotifyRepeatedError>
</JobChain>
...
SystemMonitorNotification / Notification / NotificationObjects / JobChain / NotifyRepeatedError / NotifyByIntervention

Send notifications for errors that occur due to repeated failed executions if the restart was caused by manual intervention.

SystemMonitorNotification / Notification / NotificationObjects / JobChain / NotifyRepeatedError / NotifyByPeriod

Send notifications for errors that occur due to repeatedly failed executions if a configurable period of time is exceeded.

NotifyByPeriod supports the following attributes:

AttributeUsageDescription
period

Required

The period between notifications is calculated from the time of the last failed execution for which a notification has been sent and the time of the current failed execution.

Possible values:

  • <n> - duration in seconds
    • e.g.: 60
  • <n>m - duration in minutes
    • e.g.: 30m
  • <n>h - duration in hours
    • e.g.: 24h
  • <n>d - duration in days
    • e.g.: 30d
  • <n>w - duration in weeks
    • e.g.: 4w
  • Combined values (with blank space as separator):
    • e.g.: 2h 30m 30s
SystemMonitorNotification / Notification / NotificationObjects / TimerRef

TimerRef supports the following attributes:

AttributeUsageDescription
notifications

Optional

Integer

Specifies the number of transfers the same notification to a System Monitor.

Default: 1

refRequiredCorresponds with Timer name setting defined in the SystemMonitorNotification / Timer element
notify_on_error

Optional

Boolean

Send timer check notification when the configured job chain contains the error notifications.

Default: false

Example 
<SystemMonitorNotification system_id="op5"> 
  <Notification> 
    <NotificationMonitor service_name_on_error="JobScheduler Monitoring Error"> 
      ... 
    </NotificationMonitor> 
    <NotificationObjects> 
     <!-- 
     Send the job chain error, occurring in the "test/my_jobchain" job chain, to the "JobScheduler Monitoring Errors" service. 
     --> 
     <JobChain name="test/my_jobchain" /> 
    </NotificationObjects> 
  </Notification>   
 
  <Notification> 
    <NotificationMonitor service_name_on_error="JobScheduler Monitoring Performance"> 
      ... 
    </NotificationMonitor> 
    <NotificationObjects> 
      <!-- 
      Sends the performance check error, occurring in the "test/my_jobchain" job chain, to the "JobScheduler Monitoring Performance" service. 
      Sends the performance check error to the "JobScheduler Monitoring Performance" service will be ignored when the "test/my_jobchain" has the job chain error (default notify_on_error = false). 
      --> 
      <TimerRef ref="my_timer" /> 
    </NotificationObjects> 
 </Notification>   
 
 <Timer name="my_timer"> 
    <TimerJobChain name="test/my_jobchain" /> 
 </Timer> 
</SystemMonitorNotification> 


SystemMonitorNotification / Notification / NotificationObjects / MasterMessage

JS-1837 - Getting issue details... STATUS

MasterMessage includes problems detected by a JobScheduler Master, e.g. database connection lost.

Requirements:

  •   ./config/factory.ini configuration file:

    • mail_queue_only=true

    • mail_queue_dir setting specifies a directory to store the JobScheduler mails

    • mail_on_warning=true

    • mail_on_error=true

MasterMessage supports the following attributes:

AttributeUsageDescription
scheduler_idOptional

Notifications are restricted to the JobScheduler instance with the given identification. By default notifications will be sent for all JobScheduler instances that log into the same database.

Regular expression can be used.

notifications


Optional

Integer

Specifies the number of transfers the same notification to a System Monitor.

Default: 1


SystemMonitorNotification / Notification / NotificationObjects / TaskWarning

JS-1837 - Getting issue details... STATUS

TaskWarning includes job execution warning messages.

Requirements:

  •  ./config/factory.ini configuration file:

    • mail_queue_only=true

    • mail_queue_dir setting specifies a directory to store the JobScheduler mails

    • mail_on_warning=true

TaskWarning supports the following attributes:

AttributeUsageDescription
scheduler_idOptional

Notifications are restricted to the JobScheduler instance with the given identification. By default notifications will be sent for all JobScheduler instances that log into the same database.

Regular expression can be used.

notifications


Optional

Integer

Specifies the number of transfers the same notification to a System Monitor.

Default: 1


SystemMonitorNotification / Notification / NotificationObjects / TaskIfLongerThan

JITL-522 - Getting issue details... STATUS

TaskIfLongerThan includes the feature to send notification in case that the execution of job requires a longer duration than expected.

Requirements:

  • The Job configuration contains the warn_if_longer_than setting.  
  • ./config/factory.ini configuration file:

    • mail_queue_only=true

    • mail_queue_dir setting specifies a directory to store the JobScheduler mails

    • mail_on_warning=true

TaskIfLongerThan supports the following attributes:

AttributeUsageDescription
scheduler_idOptional

Notifications are restricted to the JobScheduler instance with the given identification. By default notifications will be sent for all JobScheduler instances that log into the same database.

Regular expression can be used.

notifications


Optional

Integer

Specifies the number of transfers the same notification to a System Monitor.

Default: 1

 

SystemMonitorNotification / Notification / NotificationObjects / TaskIfShorterThan

  JITL-522 - Getting issue details... STATUS

 TaskIfShorterThan includes the feature to send notification in case that the execution of job requires a shorter duration than expected.

 Requirements:

  • The Job configuration contains the warn_if_shorter_than setting.  

  • ./config/factory.ini configuration file:

    • mail_queue_only=true

    • mail_queue_dir setting specifies a directory to store the JobScheduler mails

    • mail_on_warning=true

 TaskIfShorterThan supports the following attributes:

AttributeUsageDescription
scheduler_idOptional

Notifications are restricted to the JobScheduler instance with the given identification. By default notifications will be sent for all JobScheduler instances that log into the same database.

Regular expression can be used.

notifications


Optional

Integer

Specifies the number of transfers the same notification to a System Monitor.

Default: 1


SystemMonitorNotification / Timer 

The following elements must be nested inside a Timer element:

ElementElement descriptionDescription
TimerJob Optional or multiple use allowed inside the Timer element 

Restricts notifications for order-controlled or standalone jobs.

JITL-401 - Getting issue details... STATUS  

TimerJobChainOptional or multiple use allowed inside the Timer elementRestricts notifications for job chains
MinimumOptional or only once inside the Timer elementMinimum required execution time for job chains or selected job nodes. Allows script code to be executed that returns the minimum execution time in seconds.
MaximumOptional or only once inside the Timer elementMaximum allowed execution time for job chains or selected job nodes. Allows script code to be executed that returns the maximum execution time in seconds.
Example 
<SystemMonitorNotification system_id="op5"> 
  ... 
  <Timer name="my_timer_1"> 
    <TimerJobChain name="test/my_jobchain_1" />
    <TimerJob name="test/my_job_1" /> 
    <Maximum><Script language="javascript"><![CDATA[1000]]></Script></Maximum> 
  </Timer> 
 
  <Timer name="my_timer_2"> 
    <TimerJobChain name="test/my_jobchain_2" /> 
    <TimerJobChain name="test/my_jobchain_3" /> 
    <Minimum><Script language="javascript"><![CDATA[500]]></Script></Minimum> 
    <Maximum><Script language="javascript"><![CDATA[1000]]></Script></Maximum> 
  </Timer> 
</SystemMonitorNotification> 

Timer supports the following attributes:

AttributeUsageDescription
 nameRequired

Corresponds to Timer used in the SystemMonitorNotification / Notification / NotificationObjects / TimerRef element.

The name must be unique across all timers definitions.

Example 
...
<Timer name="my_timer">
... 


SystemMonitorNotification / Timer / TimerJob

JITL-401 - Getting issue details... STATUS

TimerJob supports the following attributes: 

AtributeUsageDescription
 scheduler_idOptional

Notifications are restricted to the JobScheduler instance with the given identification. By default notifications will be sent for all JobScheduler instances that log into the same database.

Regular expression can be used.

 nameOptional

Job name including possible folder names.

Regular expression can be used.


 

Example 
...
<TimerJob scheduler_id="scheduler_4444" /> 
... 
<TimerJob scheduler_id="scheduler_4444" name="test/my_.*" /> 
... 
<TimerJob name="test/my_job"/> 
...

 

SystemMonitorNotification / Timer / TimerJobChain

TimerJobChain supports the following attributes:

AttributeUsageDescription
 scheduler_idOptional

Notifications are restricted to the JobScheduler instance with the given identification. By default notifications will be sent for all JobScheduler instances that log into the same database.

Regular expression can be used.

 nameOptional

Job chain name including possible folder names.

Regular expression can be used.

JITL-353 - Getting issue details... STATUS

 step_fromOptionalRestricts checks for job chains to a sequence of job nodes that are specified with the step_from and step_to attributes.
 step_toOptionalRestricts checks for job chains to a sequence of job nodes that are specified with the step_from and step_to attributes.
Example 
...
<TimerJobChain scheduler_id="scheduler_4444" /> 
... 
<TimerJobChain scheduler_id="scheduler_4444" name="test/my_.*" /> 
... 
<TimerJobChain name="test/my_jobchain" step_from="200"/> 
... 
<TimerJobChain name="test/my_jobchain" step_to="500"/> 
... 
<TimerJobChain name="test/my_jobchain" step_from="300" step_to="300"/>
...


SystemMonitorNotification / Timer / Minimum

The following elements must be nested inside a Minimum element:

ElementElement descriptionDescription
ScriptRequired, only once inside the Minimum elementScript code in one of the supported languages
Example 
...
<Timer name="my_timer">
  ...
  <Minimum><Script language="javascript"><![CDATA[1000]]></Script></Minimum>
</Timer>
... 


SystemMonitorNotification / Timer / Maximum

The following elements must be nested inside a Maximum element:

ElementElement descriptionDescription
ScriptRequired, only once inside the Maximum elementScript code in one of the supported languages
Example 
...
<Timer name="my_timer">
  ...
  <Maximum><Script language="javascript"><![CDATA[1000]]></Script></Maximum>
</Timer>
...


SystemMonitorNotification / Timer / Minimum|Maximum / Script

Script supports the following attributes:

AttributeUsageDescription
 languageRequired

Script language name

Supported languages:

  • javascript
  • ECMAScript 

 The Script element can contain:

  • a fixed value
  • a calculation based on the job/order parameters
Fixed value

A fixed value is the time allowed in seconds for the specific Minimum or Maximum definition

Example (fixed value) 
...
  <Script language="javascript"><![CDATA[1000]]></Script>
...
Calculation

The calculation is to result in the time in seconds for the specific Minimum or Maximum definition.

This example calculates the execution time depending on the ${file_size} parameter that was set by a specific job (see the example below)´.

Example (calculation) 
...
  <Script language="javascript"><![CDATA[                     
    function my_calculate(){                         
      var fileSize              = new java.lang.Double(${file_size});                         
      var timerExpiryFactor     = 0.0025;                         
      var timerExpiryTolerance  = timerExpiryFactor*0.1;                         
      var timerExpiry           = new java.lang.Double(timerExpiryFactor+timerExpiryTolerance);                         
      timerExpiry               = timerExpiry*fileSize;                     
      return timerExpiry;                     
    }                         
    my_calculate();
  ]]></Script>
...


This example job calculates and creates a new order parameter file_size.

To store the parameters into database (table SCHEDULER_MON_RESULTS) :

  • set the scheduler_notification_result_parameters parameter (see job documentation jobs/JobSchedulerNotificationStoreResultsJob.xml)
  • set the StoreResultsJobJSAdapterClass as monitor
    • JobScheduler releases before 1.11
      • com.sos.scheduler.notification.jobs.result.StoreResultsJobJSAdapterClass

    • JobScheduler releases starting from 1.11

      • com.sos.jitl.notification.jobs.result.StoreResultsJobJSAdapterClass
Example (order job) 
<?xml version="1.0" encoding="ISO-8859-1"?> 
<job  title="Sample Job with Store Result Monitor" order="yes" stop_on_error="no" tasks="1">     
  <params>
     <!--
     set the scheduler_notification_result_parameters parameter
     -->         
    <param name="scheduler_notification_result_parameters" value="file_size"/>     
  </params>     
  
  <!--
  calculate and create the new order parameter if necessary
  -->
  <script language="java:javascript"><![CDATA[             
      function spooler_process(){                                  
        var order    = spooler_task.order;                 
        var params   = spooler.create_variable_set();                 
        params.merge(spooler_task.params);                 
        params.merge(order.params);                      
        
        // parameter scheduler_file_path was set in the previous job chain step
        var file     = new java.io.File(params.value("scheduler_file_path"));                 
        var fileSize = file.length()/1024;                 
        order.params.set_var("file_size",fileSize.toString());                          
      return true;             
      }]]>     
   </script>          
 
   <!-- 
   set the StoreResultsJobJSAdapterClass as a monitor
   -->     
   <monitor  name="notification_monitor" ordering="1">         
     	<!-- JobScheduler releases before 1.11
	<script java_class="com.sos.scheduler.notification.jobs.result.StoreResultsJobJSAdapterClass" language="java"/>
        -->
   
        <!-- JobScheduler releases starting from 1.11 --> 
        <script java_class="com.sos.jitl.notification.jobs.result.StoreResultsJobJSAdapterClass" language="java"/> 
   </monitor>

   <run_time /> 
</job> 


Message

Usage

The Message can be configured on the following parent nodes as a CDATA element :

  • SystemMonitorNotification / Notification / NotificationInterface
  • SystemMonitorNotification / Notification / NotificationCommand
  • SystemMonitorNotification / Notification / NotificationMail 
    • Subject
    • Body
  • SystemMonitorNotification / Notification / NotificationJMS / Message

The Message can contain: 

  • Fixed values
  • Variables

Example: <![CDATA[ scheduler id = ${MON_N_SCHEDULER_ID}  ]]>

Variables

All variables (except OS environment variables) must be defined by using of the

${<variable name>}

syntax.

Note:

  • Syntax for the JobScheduler version 1.10.6 and higher. Syntax for the JobScheduler version 1.10.4, 1.10.5 (see below) is still supported.
  • Syntax for the JobScheduler version 1.10.4, 1.10.5:  {<variable name>}
  • Syntax for the JobScheduler previous versions: %<variable name>%

The order of the substitution the variables values is:

  1. Table variables.
  2. Service variables.
  3. JOC Cockpit variables.
  4. OS environment variables.

Table variables

 Table of the history of steps of processed orders / jobs.

NameDescription  Note
${MON_N_ID}
Unique notification id
${MON_N_SCHEDULER_ID} Id of the JobScheduler
${MON_N_TASK_ID}Id of the JobScheduler task 
${MON_N_STEP} Consecutive number of the order step
${MON_N_ORDER_HISTORY_ID} Id of the JobScheduler order 
${MON_N_JOB_CHAIN_NAME} Name of the job chain of the order 
${MON_N_JOB_CHAIN_TITLE}Title of the job chain of the order  
${MON_N_ORDER_ID} Unique (within the job chain) id of the order 
${MON_N_ORDER_TITLE} Title of the order 
${MON_N_ORDER_START_TIME} Timestamp of the start of the order
${MON_N_ORDER_END_TIME} Timestamp of the end of the order
${MON_N_ORDER_TIME_ELAPSED} The time or difference in seconds between a beginning time and an ending time of the order
${MON_N_ORDER_STEP_STATE} State of the order inside the job chain
${MON_N_ORDER_STEP_START_TIME}Timestamp of the start of the order step 
${MON_N_ORDER_STEP_END_TIME} Timestamp of the end of the order step 
${MON_N_ORDER_STEP_TIME_ELAPSED}The time or difference in seconds between a beginning time and an ending time of the order step 
${MON_N_JOB_NAME}Name of the job 
${MON_N_JOB_TITLE}Title of the job
${MON_N_JOB_CRITICALITY}User defined job criticality

Available with release 1.13.3

JOC-850 - Getting issue details... STATUS

${MON_N_TASK_START_TIME}Timestamp of the job task start 
${MON_N_TASK_END_TIME} Timestamp of the job task end
${MON_N_TASK_TIME_ELAPSED} The time or difference in seconds between a beginning time and an ending time of the job task
${MON_N_RECOVERED} 

0 = dependent of the ${MON_N_ERROR} - ok or error was not recovered,

1 = error was recovered  


${MON_N_RETURN_CODE}Return code number 
${MON_N_AGENT_URL}JobScheduler Agent uri 

Available with release 1.11.2

JITL-351 - Getting issue details... STATUS

${MON_N_CLUSTER_MEMBER_ID} ID of the Scheduler cluster member (only used in cluster mode)

Available with release 1.11.2

JITL-352 - Getting issue details... STATUS

${MON_N_ERROR}

0 = ok

1 = error 


${MON_N_ERROR_CODE} Exception-code of the job error 
${MON_N_ERROR_TEXT}Exception message of the job (that processed the order) 
${MON_N_CREATED} Timestamp of the notification initial record 
${MON_N_MODIFIED}Timestamp of the latest changes to this notification record 

 

Example 
 scheduler id = ${MON_N_SCHEDULER_ID}, history id = ${MON_N_ORDER_HISTORY_ID}, job_chain = ${MON_N_JOB_CHAIN_NAME}(${MON_N_ORDER_ID}), error = ${MON_N_ERROR_TEXT}

 Table of the history of notifications sent to a system monitor.

NameDescriptionNote
${MON_SN_ID}Unique system notification id 
${MON_SN_NOTIFICATION_ID}

Reference to the SCHEDULER_MON_NOTIFICATIONS.ID  table


${MON_SN_CHECK_ID}

Reference to the SCHEDULER_MON_CHECKS.ID   table


${MON_SN_SYSTEM_ID} 

Reference to the element attribute

SystemMonitorNotification / @system_id

defined in the XML configuration file


${MON_SN_SERVICE_NAME}

Reference to one of both element attributes

  • SystemMonitorNotification / Notification / NotificationMonitor / @service_name_on_error
  • SystemMonitorNotification / Notification / NotificationMonitor / @service_name_on_success

defined in the XML configuration file 


${MON_SN_OBJECT_TYPE}

NotificationObject type

  • 0
    • JobChain
  • 1
    • Job
  • 10
    • TaskIfLongerThan
  • 11
    • TaskIfShorterThan
  • 12
    • TaskWarning
  • 20
    • MasterMessage
  • 100
    • dummy code for interlal using 
  • 10,11,12,20
    • Available with release 1.12.9
      • JITL-522 - Getting issue details... STATUS
      • JS-1837 - Getting issue details... STATUS 
${MON_SN_OBJECT_TYPE_NAME}
See ${MON_SN_OBJECT_TYPE} above.
  • JobChain
  • Job
  • TaskIfLongerThan
  • TaskIfShorterThan
  • TaskWarning
  • MasterMessage
Available with release 1.12.9

JITL-522 - Getting issue details... STATUS

JS-1837 - Getting issue details... STATUS 

${MON_SN_TITLE}
Titel
  • Job, JobChain
    • empty
  • MasterMessage, TaskWarning, TaskIfLongerThan, TaskIfShorterThan
    • E-Mail subject generated by the JobScheduler Master
Available with release 1.12.9

JITL-522 - Getting issue details... STATUS

JS-1837 - Getting issue details... STATUS 

${MON_SN_RETURN_CODE_FROM} 

Reference to the element attribute

SystemMonitorNotification / Notification / NotificationObjects / JobChain / @return_code_from

defined in the XML configuration file


${MON_SN_RETURN_CODE_TO}

Reference to the element attribute

SystemMonitorNotification / Notification / NotificationObjects / JobChain / @return_code_to

defined in the XML configuration file 


${MON_SN_STEP_FROM}

Reference to the element attribute

SystemMonitorNotification / Notification / NotificationObjects / JobChain / @step_from

defined in the XML configuration file 


${MON_SN_STEP_TO}

Reference to the element attribute

SystemMonitorNotification / Notification / NotificationObjects / JobChain / @step_to

defined in the XML configuration file 


${MON_SN_STEP_FROM_START_TIME}Timestamp for the start of the order step 
${MON_SN_STEP_TO_END_TIME}Timestamp for the end of the order step  
${MON_SN_STEP_TIME_ELAPSED} The elapsed time or the difference in seconds between the start and end times of the order step  
${MON_SN_NOTIFICATIONS}

Reference to element attribute

SystemMonitorNotification / Notification / NotificationObjects / JobChain / @notifications

defined in the XML configuration file  


${MON_SN_CURRENT_NOTIFICATION}Number of notifications that already sent to a System Monitor
${MON_SN_MAX_NOTIFICATIONS}

0 = notifications counter was not reached

1 = notifications counter was reached, all configured notifications were sent


${MON_SN_ACKNOWLEDGED}

0 = not acknowledged

1 = acknowledged 


${MON_SN_RECOVERED}

0 = recovery not sent

1 = recovery sent


${MON_SN_SUCCESS}

0 = Notification onError, SystemMonitorNotification / Notification / NotificationMonitor / @service_name_on_error

1 = Notification onSuccess, SystemMonitorNotification / Notification / NotificationMonitor / @service_name_on_success


${MON_SN_CREATED}Timestamp of the initial system notification record  
${MON_SN_MODIFIED}Timestamp of the latest changes to this system notification record  
Example 
 step from = ${MON_SN_STEP_FROM}, step to = ${MON_SN_STEP_TO}, notification = ${MON_SN_CURRENT_NOTIFICATION} (of ${MON_SN_NOTIFICATIONS})

 Table of the history of executed checks (Timer)

NameDescription
${MON_C_ID}Unique check id  
${MON_C_NOTIFICATION_ID}

Reference to table SCHEDULER_MON_NOTIFICATIONS.ID   

${MON_C_NAME}

Reference to element attribute

SystemMonitorNotification / Timer / @name

defined in the XML configuration file 

${MON_C_STEP_FROM} 

Reference to element attribute

SystemMonitorNotification / Timer / TimerJobChain / @step_from

defined in the XML configuration file

 
${MON_C_STEP_TO}

Reference to element attribute

SystemMonitorNotification / Timer / TimerJobChain / @step_to

defined in the XML configuration file

${MON_C_STEP_FROM_START_TIME} Timestamp of the start of the order step  
${MON_C_STEP_TO_END_TIME} Timestamp of the end of the order step  
${MON_C_STEP_TIME_ELAPSED} The time or difference in seconds between a beginning time and an ending time of the order step   
${MON_C_CHECK_TEXT} Message of the check  
${MON_C_CREATED}Timestamp of the check initial record  
${MON_C_MODIFIED}Timestamp of the latest changes to this check record 
Example 
 timer name = ${MON_C_NAME}, text = ${MON_C_CHECK_TEXT}


Service variables
NameDescription
${SERVICE_NAME}

Current service name. One of both element attributes:

  • SystemMonitorNotification / Notification / NotificationMonitor / @service_name_on_error
  • SystemMonitorNotification / Notification / NotificationMonitor / @service_name_on_success
${SERVICE_STATUS}

Current service status. One of both element attributes or default: 

  • SystemMonitorNotification / Notification / NotificationMonitor / @service_status_on_error
  • SystemMonitorNotification / Notification / NotificationMonitor / @service_status_on_success
  • default CRITICAL error
  • default OK       success
${SERVICE_MESSAGE_PREFIX}

Message prefix

  • SUCCESS     success (JobScheduler releases from 1.10.11, 1.12.8, 1.13)
  • ERROR       error
  • RECOVERED     error recovery
  • TIMER       performance check
Example 
 service name = ${SERVICE_NAME}


JOC Cockpit variables

JS-1388 - Getting issue details... STATUS

Note:

  • the JOC Cockpit variables will be substituted only when the NotificationMail interface is used.

Requirement:

  •  config/private/private.conf configuration file is active and contains the configured joc.url entry.


NameDescription
${JOC_HREF_JOB_CHAIN}Link href to the job chain object in the JOC Cockpit
${JOC_HREF_ORDER}Link href to the order object in the JOC Cockpit
${JOC_HREF_JOB}Link href to the job object in the JOC Cockpit

 

Example 
<a href="${JOC_HREF_JOB_CHAIN}">${MON_N_JOB_CHAIN_NAME}</a>
<a href="${JOC_HREF_ORDER}">${MON_N_ORDER_ID}</a>
<a href="${JOC_HREF_JOB}">${MON_N_JOB_NAME}</a>

 

OS environment variables 

 All existing OS environment variables can be defined by message using the syntax %<variable name>% (Windows) or $<variable name> (Unix).

Example Windows 
 %TEMP%/test.exe


Notification environment variables

The default SystemNotifierProcessBuilderPlugin plugin used by the SystemMonitorNotification / Notification / NotificationCommand element sets the following variables as environment variables:

  1. Table variables
  2. Service variables

These variables can be used when the NotificationCommand calls the notification client - not directly but via a shell script that makes the logical implementation for sending the notification messages.

Table variables

All table variables (see Table variables explanation) are set as environment variables with the prefix:

  • SCHEDULER_MON_TABLE_

e.g.:

  • SCHEDULER_MON_TABLE_MON_N_ID
  • SCHEDULER_MON_TABLE_MON_N_SCHEDULER_ID
  • ...
Service variables
NameDescription

SCHEDULER_MON_SERVICE_NAME

Current service name. One of both element attributes:

  • SystemMonitorNotification / Notification / NotificationMonitor / @service_name_on_error
  • SystemMonitorNotification / Notification / NotificationMonitor / @service_name_on_success

SCHEDULER_MON_SERVICE_STATUS

Current service status. One of both element attributes or default:

  • SystemMonitorNotification / Notification / NotificationMonitor / @service_status_on_error
  • SystemMonitorNotification / Notification / NotificationMonitor / @service_status_on_success
  • default CRITICAL error
  • default OK       success

SCHEDULER_MON_SERVICE_MESSAGE_PREFIX

  • ERROR      error                           
  • RECOVERED    error recovery                            
  • TIMER             performance check 

SCHEDULER_MON_SERVICE_COMMAND

 Content of the SystemMonitorNotification / Notification / NotificationCommand after substitution

  

Sample NotificationCommand Unix. Script file (/tmp/command.sh). 
1) configured command in the SystemMonitorNotification_<MonitorSystem>.xml file
<NotificationCommand><![CDATA[/tmp/command.sh]</NotificationCommand>
 
2) content of the /tmp/command.sh file
#! /bin/sh 
# Note: "> /tmp/command_output.txt" is used to simulate the starting of the notification client
#
echo "$SCHEDULER_MON_SERVICE_NAME:$SCHEDULER_MON_SERVICE_STATUS:$SCHEDULER_MON_SERVICE_MESSAGE_PREFIX history id = $SCHEDULER_MON_TABLE_MON_N_ORDER_HISTORY_ID" > /tmp/command_output.txt
 
Sample NotificationCommand Windows. Script file (C:/temp/command.cmd). 
1) configured command in the SystemMonitorNotification_<MonitorSystem>.xml file
<NotificationCommand><![CDATA[C:/Temp/command.cmd]</NotificationCommand>
 
2) content of the C:/Temp/command.cmd file
rem Note: "> C:/Temp/command_output.txt" is used to simulate the starting of the notification client
rem
echo %SCHEDULER_MON_SERVICE_NAME%:%SCHEDULER_MON_SERVICE_STATUS%:%SCHEDULER_MON_SERVICE_MESSAGE_PREFIX% history id = %SCHEDULER_MON_TABLE_MON_N_ORDER_HISTORY_ID% > C:/Temp/command_output.txt
 

Examples

Message on error
scheduler id=${MON_N_SCHEDULER_ID}, history id=${MON_N_ORDER_HISTORY_ID}, job_chain=${MON_N_JOB_CHAIN_NAME}(${MON_N_ORDER_ID}), step=${MON_N_ORDER_STEP_STATE}, error=${MON_N_ERROR_TEXT}            


Message on success 
scheduler id=${MON_N_SCHEDULER_ID}, history id=${MON_N_ORDER_HISTORY_ID}, job_chain=${MON_N_JOB_CHAIN_NAME}(${MON_N_ORDER_ID}), steps(${MON_SN_STEP_FROM} to ${MON_SN_STEP_TO}), order time elapsed = ${MON_N_ORDER_TIME_ELAPSED}s           


Message on performance check (Timer) 
name = ${MON_C_NAME}, scheduler id=${MON_N_SCHEDULER_ID}, history id=${MON_N_ORDER_HISTORY_ID}, job_chain=${MON_N_JOB_CHAIN_NAME}(${MON_N_ORDER_ID}), steps(${MON_C_STEP_FROM} to ${MON_C_STEP_TO}), check = ${MON_C_CHECK_TEXT}           


Examples System Monitoring
NotificationInterface ( Nagios / OP5 )

The following is an except from an XML file used to notify a specific System Monitor (op5 Monitor) via the NotificationInterface:

SystemMonitorNotification_OP5.xml 
 ...
<!--
monitor_host            The hostname or ip address of System Monitor host 
monitor_port            The TCP port that the System Monitor would listen to
monitor_encryption      Encryption algorithm
service_host            The host that executes the passive check. The name must match the corresponding setting in the System Monitor
{MON_N_SCHEDULER_ID}    See explanation "Table variables"
...
-->
<NotificationInterface monitor_host="monitor_host" 
                       monitor_port="5667" 
                       monitor_encryption="XOR" 
                       service_host="service_host"><![CDATA[
scheduler id=${MON_N_SCHEDULER_ID}, history id=${MON_N_ORDER_HISTORY_ID}, job_chain=${MON_N_JOB_CHAIN_NAME}(${MON_N_ORDER_ID}), step =${MON_N_ORDER_STEP_STATE}, error=${MON_N_ERROR_TEXT}
]]></NotificationInterface>
...
NotificationCommand ( Nagios / OP5 )

The following is an except from an XML file used to notifying a specific System Monitor (op5 Monitor) via the NotificationCommand:

SystemMonitorNotification_OP5.xml 
... 
<!--
service_host               The host that executes the passive check. The name must match the corresponding setting in the System Monitor.
monitor_host               The hostname or ip address of System Monitor host.
{SERVICE_NAME}             See explanation "Service variables"
{SERVICE_STATUS}           See explanation "Service variables"
{SERVICE_MESSAGE_PREFIX}   See explanation "Service variables"
{MON_N_SCHEDULER_ID}       See explanation "Table variables"
...
NotificationCommand after substitution (error case):
<![CDATA[echo service_host:JobScheduler Monitoring Errors:2:ERROR scheduler id=scheduler_4444, history id=123, job_chain=test/my_jobchain(order_id), step=100, error=error occurred | D:\nsca\send_nsca.exe -H monitor_host -c D:\nsca\send_nsca.cfg -d : ]]>
 
NotificationCommand after substitution (recovery case): 
<![CDATA[echo service_host:JobScheduler Monitoring Errors:0:RECOVERED scheduler id=scheduler_4444, history id=123, job_chain=test/my_jobchain(order_id), step=100, error=error occurred | D:\nsca\send_nsca.exe -H monitor_host -c D:\nsca\send_nsca.cfg -d : ]]> 
 
NotificationCommand after substitution (success case):  
<![CDATA[echo service_host:JobScheduler Monitoring Success:0:SUCCESS scheduler id=scheduler_4444, history id=123, job_chain=test/my_jobchain(order_id), step=100, error= | D:\nsca\send_nsca.exe -H monitor_host -c D:\nsca\send_nsca.cfg -d : ]]>  
 
-->
<NotificationMonitor service_name_on_error="JobScheduler Monitoring Errors" 
                     service_name_on_success="JobScheduler Monitoring Success" 
                     service_status_on_error="2" 
                     service_status_on_success="0">
  <NotificationCommand><![CDATA[echo service_host:${SERVICE_NAME}:${SERVICE_STATUS}:${SERVICE_MESSAGE_PREFIX} scheduler id=${MON_N_SCHEDULER_ID}, history id=${MON_N_ORDER_HISTORY_ID}, job_chain=${MON_N_JOB_CHAIN_NAME}(${MON_N_ORDER_ID}), step=${MON_N_ORDER_STEP_STATE}, error=${MON_N_ERROR_TEXT} | D:\nsca\send_nsca.exe -H monitor_host -c D:\nsca\send_nsca.cfg -d : ]]>
  </NotificationCommand>  
</NotificationMonitor>

...
NotificationCommand ( Nagios / Opsview )

The following is an except from an XML file used to notifying a specific System Monitor (Opsview Monitor) via the NotificationCommand on Unix:

SystemMonitorNotification_Opsview.xml 
... 
<!--
service_host               The host that executes the passive check. The name must match the corresponding setting in the System Monitor. e.g- localhost
monitor_host               The hostname or ip address of System Monitor host.
{SERVICE_NAME}             See explanation "Service variables"
{SERVICE_STATUS}           See explanation "Service variables"
{SERVICE_MESSAGE_PREFIX}   See explanation "Service variables"
{MON_N_SCHEDULER_ID}       See explanation "Table variables"
...
NotificationCommand after substitution (error case):
<![CDATA[echo -e "localhost\tJobScheduler Monitoring Errors\t2\tERROR scheduler id=scheduler_4444, history id=123, job_chain=test/my_jobchain(order_id), step=100, error=error occurred\n" | /usr/local/nagios/bin/send_nsca -H monitor_host -c /usr/local/nagios/etc/send_nsca.cfg]]>
 
NotificationCommand after substitution (recovery case): 
<![CDATA[echo -e "localhost\tJobScheduler Monitoring Errors\t0\tRECOVERED scheduler id=scheduler_4444, history id=123, job_chain=test/my_jobchain(order_id), step=100, error=error occurred\n" | /usr/local/nagios/bin/send_nsca -H monitor_host -c /usr/local/nagios/etc/send_nsca.cfg]]> 
 
NotificationCommand after substitution (success case):  
<![CDATA[echo -e "localhost\tJobScheduler Monitoring Success\t0\tSUCCESS scheduler id=scheduler_4444, history id=123, job_chain=test/my_jobchain(order_id), step=100, error=\n" | /usr/local/nagios/bin/send_nsca -H monitor_host -c /usr/local/nagios/etc/send_nsca.cfg]]>  
 
-->
<NotificationMonitor service_name_on_error="JobScheduler Monitoring Errors" 
                     service_name_on_success="JobScheduler Monitoring Success" 
                     service_status_on_error="2" 
                     service_status_on_success="0">
  <NotificationCommand><![CDATA[echo -e "service_host\t${SERVICE_NAME}\t${SERVICE_STATUS}\t${SERVICE_MESSAGE_PREFIX} scheduler id=${MON_N_SCHEDULER_ID}, history id=${MON_N_ORDER_HISTORY_ID}, job_chain=${MON_N_JOB_CHAIN_NAME}(${MON_N_ORDER_ID}), step=${MON_N_ORDER_STEP_STATE}, error=${MON_N_ERROR_TEXT}\n" | /usr/local/nagios/bin/send_nsca -H monitor_host -c /usr/local/nagios/etc/send_nsca.cfg]]>
  </NotificationCommand>  
</NotificationMonitor>

...
 NotificationCommand ( Zabbix )

The following is an except from an XML file used to notify a specific System Monitor (Zabbix Monitor) and using NotificationCommand

SystemMonitorNotification_zabbix.xml 
... 
<!--
zabbix_sender            Zabbix sender installed on the JobScheduler host
localhost                Hostname of the zabbix server
Zabbix_server            JobScheduler Agent name(host name) that registred on Zabbix
samples.job1             Item key of zabbix (replace "/" to "." of JOB_NAME
${MON_N_ERROR_TEXT}      See explanation "Table variables"
-->
<NotificationCommand>
<![CDATA[zabbix_sender -z localhost -s zabbix_server -k samples.job1 -o ${MON_N_ERROR_TEXT}]]>
</NotificationCommand>
...


Examples Mail
NotificationMail content_type="text/html"
Mail: on failed job chain 
...
<NotificationMail content_type="text/html" charset="ISO-8859-1" encoding="7bit" priority="Normal">
    <From><![CDATA[jobscheduler@sos-berlin.com]]></From>
    <To><![CDATA[spam@sos-berlin.com]]></To>
    <Subject><![CDATA[JobScheduler notification: ${SERVICE_MESSAGE_PREFIX},  job chain executed with errors: ${MON_N_JOB_CHAIN_NAME}]]></Subject>
    <Body><![CDATA[<style type="text/css">
.tg  {border-collapse:collapse;border-spacing:0;border-color:#bbb;}
.tg td{font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;border-color:#bbb;color:#594F4F;background-color:#E0FFEB;}
.tg th{font-family:Arial, sans-serif;font-size:14px;font-weight:normal;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;border-color:#bbb;color:#493F3F;background-color:#9DE0AD;}
</style>
<table class="tg">
    <tr>
        <th colspan="4">Error</th>
    </tr>
    <tr>
        <td>Code:</td><td>${MON_N_ERROR_CODE}</td>
        <td>Messsage</td><td>${MON_N_ERROR_TEXT}</td>
    </tr>
    <tr>
        <th colspan="4">JobScheduler</th>
    </tr>
    <tr>
        <td>JobScheduler ID</td><td>${MON_N_SCHEDULER_ID}</td>
        <td>Agent URL</td><td>${MON_N_AGENT_URL}</td>
    </tr>
    <tr>
        <th colspan="4">Order</th>
    </tr>
    <tr>
        <td>Order ID</td><td><a href="${JOC_HREF_ORDER}">${MON_N_ORDER_ID}</a></td>
        <td>Order Title</td><td>${MON_N_ORDER_TITLE}</td>
    </tr>
    <tr>
        <td>Job Chain Name</td><td><a href="${JOC_HREF_JOB_CHAIN}">${MON_N_JOB_CHAIN_NAME}</a></td>
        <td>Job Chain Title</td><td>${MON_N_JOB_CHAIN_TITLE}</td>
    </tr>
    <tr>
        <td>Job Name</td><td><a href="${JOC_HREF_JOB}">${MON_N_JOB_NAME}${MON_N_JOB_NAME}</a></td>
        <td>Job Title</td><td>${MON_N_JOB_TITLE}</td>
    </tr>
    <tr>
        <th colspan="4">Order History</th>
    </tr>
    <tr>
        <td>Time elapsed</td><td>${MON_N_ORDER_TIME_ELAPSED}</td><td>&nbsp;</td><td>&nbsp;</td>
    </tr>
    <tr>
        <td>Start Time</td><td>${MON_N_ORDER_START_TIME}</td>
        <td>End Time</td><td>${MON_N_ORDER_END_TIME}</td>
    </tr>
    <tr>
        <th colspan="4">Order Step History</th>
    </tr>
    <tr>
        <td>State</td><td>${MON_N_ORDER_STEP_STATE}</td>
        <td>Time elapsed</td><td>${MON_N_ORDER_STEP_TIME_ELAPSED}</td>
    </tr>
    <tr>
        <td>Start Time</td><td>${MON_N_ORDER_STEP_START_TIME}</td>
        <td>End Time</td><td>${MON_N_ORDER_STEP_END_TIME}</td>
    </tr>
</table>]]></Body>
</NotificationMail>
...
Mail: on successful job chain 
...
<NotificationMail content_type="text/html" charset="ISO-8859-1" encoding="7bit" priority="Normal">
    <From><![CDATA[jobscheduler@sos-berlin.com]]></From>
    <To><![CDATA[spam@sos-berlin.com]]></To>
    <Subject><![CDATA[JobScheduler notification: job chain successfully completed: ${MON_N_JOB_CHAIN_NAME}]]></Subject>
    <Body><![CDATA[<style type="text/css">
.tg  {border-collapse:collapse;border-spacing:0;border-color:#aaa;}
.tg td{font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;border-color:#aaa;color:#333;background-color:#fff;}
.tg th{font-family:Arial, sans-serif;font-size:14px;font-weight:normal;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;border-color:#aaa;color:#fff;background-color:#f38630;}
</style>
<table class="tg">
    <tr>
        <th colspan="4">JobScheduler</th>
    </tr>
    <tr>
        <td>JobScheduler ID</td><td>${MON_N_SCHEDULER_ID}</td>
        <td>Agent URL</td><td>${MON_N_AGENT_URL}</td>
    </tr>
    <tr>
        <th colspan="4">Order</th>
    </tr>
    <tr>
        <td>Order ID</td><td><a href="${JOC_HREF_ORDER}">${MON_N_ORDER_ID}</a></td>
        <td>Order Title</td><td>${MON_N_ORDER_TITLE}</td>
    </tr>
    <tr>
        <td>Job Chain Name</td><td><a href="${JOC_HREF_JOB_CHAIN}">${MON_N_JOB_CHAIN_NAME}</a></td>
        <td>Job Chain Title</td><td>${MON_N_JOB_CHAIN_TITLE}</td>
    </tr>
    <tr>
        <th colspan="4">Order History</th>
    </tr>
    <tr>
        <td>Time elapsed</td><td>${MON_N_ORDER_TIME_ELAPSED}</td><td>&nbsp;</td><td>&nbsp;</td>
    </tr>
    <tr>
        <td>Start Time</td><td>${MON_N_ORDER_START_TIME}</td>
        <td>End Time</td><td>${MON_N_ORDER_END_TIME}</td>
    </tr>
</table>]]></Body>
</NotificationMail>
...
Mail: on failed job 
...
<NotificationMail content_type="text/html" charset="ISO-8859-1" encoding="7bit" priority="Normal">
    <From><![CDATA[jobscheduler@sos-berlin.com]]></From>
    <To><![CDATA[spam@sos-berlin.com]]></To>
    <Subject><![CDATA[JobScheduler notification: ${SERVICE_MESSAGE_PREFIX}, job executed with errors: ${MON_N_JOB_NAME}]]></Subject>
    <Body><![CDATA[<style type="text/css">
.tg  {border-collapse:collapse;border-spacing:0;border-color:#bbb;}
.tg td{font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;border-color:#bbb;color:#594F4F;background-color:#E0FFEB;}
.tg th{font-family:Arial, sans-serif;font-size:14px;font-weight:normal;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;border-color:#bbb;color:#493F3F;background-color:#9DE0AD;}
</style>
<table class="tg">
    <tr>
        <th colspan="4">Error</th>
    </tr>
    <tr>
        <td>Code:</td><td>${MON_N_ERROR_CODE}</td>
        <td>Messsage</td><td>${MON_N_ERROR_TEXT}</td>
    </tr>
    <tr>
        <th colspan="4">JobScheduler</th>
    </tr>
    <tr>
        <td>JobScheduler ID</td><td>${MON_N_SCHEDULER_ID}</td>
        <td>Agent URL</td><td>${MON_N_AGENT_URL}</td>
    </tr>
    <tr>
        <th colspan="4">Order</th>
    </tr>
    <tr>
        <td>Order ID</td><td><a href="${JOC_HREF_ORDER}">${MON_N_ORDER_ID}</a></td>
        <td>Order Title</td><td>${MON_N_ORDER_TITLE}</td>
    </tr>
    <tr>
        <td>Job Chain Name</td><td><a href="${JOC_HREF_JOB_CHAIN}">${MON_N_JOB_CHAIN_NAME}</a></td>
        <td>Job Chain Title</td><td>${MON_N_JOB_CHAIN_TITLE}</td>
    </tr>
    <tr>
        <td>Job Name</td><td><a href="${JOC_HREF_JOB}">${MON_N_JOB_NAME}${MON_N_JOB_NAME}</a></td>
        <td>Job Title</td><td>${MON_N_JOB_TITLE}</td>
    </tr>
    <tr>
        <th colspan="4">Task History</th>
    </tr>
    <tr>
        <td>Task ID</td><td>${MON_N_TASK_ID}</td>
        <td>Time elapsed</td><td>${MON_N_TASK_TIME_ELAPSED}</td>
    </tr>
    <tr>
        <td>Start Time</td><td>${MON_N_TASK_START_TIME}</td>
        <td>End Time</td><td>${MON_N_TASK_END_TIME}</td>
    </tr>
</table>]]></Body>
</NotificationMail>
...
Mail: on successful job 
...
<NotificationMail content_type="text/html" charset="ISO-8859-1" encoding="7bit" priority="Normal">
    <From><![CDATA[jobscheduler@sos-berlin.com]]></From>
    <To><![CDATA[spam@sos-berlin.com]]></To>
    <Subject><![CDATA[JobScheduler notification: job successfully completed: ${MON_N_JOB_NAME}]]></Subject>
    <Body><![CDATA[<style type="text/css">
.tg  {border-collapse:collapse;border-spacing:0;border-color:#aaa;}
.tg td{font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;border-color:#aaa;color:#333;background-color:#fff;}
.tg th{font-family:Arial, sans-serif;font-size:14px;font-weight:normal;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;border-color:#aaa;color:#fff;background-color:#f38630;}
</style>
<table class="tg">
    <tr>
        <th colspan="4">JobScheduler</th>
    </tr>
    <tr>
        <td>JobScheduler ID</td><td>${MON_N_SCHEDULER_ID}</td>
        <td>Agent URL</td><td>${MON_N_AGENT_URL}</td>
    </tr>
    <tr>
        <th colspan="4">Order</th>
    </tr>
    <tr>
        <td>Order ID</td><td><a href="${JOC_HREF_ORDER}">${MON_N_ORDER_ID}</a></td>
        <td>Order Title</td><td>${MON_N_ORDER_TITLE}</td>
    </tr>
    <tr>
        <td>Job Chain Name</td><td><a href="${JOC_HREF_JOB_CHAIN}">${MON_N_JOB_CHAIN_NAME}</a></td>
        <td>Job Chain Title</td><td>${MON_N_JOB_CHAIN_TITLE}</td>
    </tr>
    <tr>
        <td>Job Name</td><td><a href="${JOC_HREF_JOB}">${MON_N_JOB_NAME}${MON_N_JOB_NAME}</a></td>
        <td>Job Title</td><td>${MON_N_JOB_TITLE}</td>
    </tr>
    <tr>
        <th colspan="4">Task History</th>
    </tr>
    <tr>
        <td>Task ID</td><td>${MON_N_TASK_ID}</td>
        <td>Time elapsed</td><td>${MON_N_TASK_TIME_ELAPSED}</td>
    </tr>
    <tr>
        <td>Start Time</td><td>${MON_N_TASK_START_TIME}</td>
        <td>End Time</td><td>${MON_N_TASK_END_TIME}</td>
    </tr>
</table>]]></Body>
</NotificationMail>
...


JobScheduler - Store parameters to database

The Monitoring Interface provide functionality to store the job/order parameters of the specific jobs into database (table SCHEDULER_MON_RESULTS).

See explanation : Calculation

JobScheduler - Job Chains

The following job chains are provided and should be configured accordingly:

sos / notification / CheckHistory (JobScheduler releases before 1.11)

See <scheduler_install>/jobs/JobSchedulerNotificationCheckHistoryJob.xml

  • This is the main job that analyze the JobScheduler history tables und write results into the notification tables.
    • Job read all history entries for the job chains, configured in the SystemMonitorNotification XML files.
    • Job execute the performance checks for the defined Timers
  • Order Check
    • configure repeat interval for order run time, e.g. every two minutes.

sos / notification / CheckHistory (JobScheduler releases starting from 1.11)

  • Job chain removed

  • Set param sos.use_notification true (config/scheduler.xml)

JobScheduler version 1.11.x config/scheduler.xml 
...
<spooler>
	<config ...>
		<params>
			...
			<param name="sos.use_notification" value="true"/>
		...
</spooler>


sos / notification / SystemNotifier

See <scheduler_install>/jobs/JobSchedulerNotificationSystemNotifierJob.xml

  • Sends notifications to a specific System Monitor.
  • Order MonitorSystem
    • JobScheduler releases before 1.11
      • configures a repeat interval for the order run time that is not less than the interval that has been chosen for triggering the job chain sos/notification/CheckHistory

sos / notification / CleanupNotifications

See <scheduler_install>/jobs/JobSchedulerNotificationCleanupNotificationsJob.xml

  • Removes notifications that have expired. 
  • Order Cleanup
    • configure start time for order run time, e.g. 24:00

sos / notification / ResetNotifications

See <scheduler_install>/jobs/JobSchedulerNotificationResetNotificationsJob.xml

  • Some System Monitors may provide an "acknowledge" operation, that signaling has known problem.
  • Should an "acknowledge" operation have been performed for a specific service in the System Monitor then job chain ResetNotifications would stop JobScheduler from sending notifications for that service for errors that have already occurred.
  • Do not configure the order run time for this job chain, as job chain will be triggered by the System Monitor's "acknowledge" operation via add_order XML command.
Examples
Example ResetNotifications <add_order> XML command

The following example shows the XML command sent from a monitoring system to the JobScheduler to call the sos/notification/ResetNotifications job chain and set the relevant service name as acknowledged.

ResetNotifications OP5 add_order 
<add_order  job_chain   ="sos/notification/ResetNotifications"
            id          ="op5 JobScheduler Monitoring Error acknowledgement"
            title       ="op5 JobScheduler Monitoring Error acknowledgement">
    <params>
        <param name="service_name"  value="JobScheduler Monitoring Error" />
        <param name="system_id"     value="op5"/>
        <param name="operation"     value="acknowledge" />
     </params>
</add_order>

Key to the above code:

ElementAttributeValue
Description
add_order


XML Command to add the new order to the specified job chain on the JobScheduler.

job_chainsos/notification/ResetNotifications
Job chain path must correspond with the path of the ResetNotifications job chain installed on the JobScheduler.

id

Order identifier.

title

Order title.
param


3 following parameters must be set:

nameservice_name
JobScheduler Monitoring Error
Relevant service name to set all already occured service errors in JobScheduler Interface Monitor as acknowledged.

namesystem_idop5

System identification.

Corresponds with SystemMonitorNotification/@system_id setting in the  SystemMonitorNotification_<MonitorSystem>.xml configuration file.


nameoperationacknowledgeFixed value. Operation name to execute the acknowledgement in the JobScheduler Monitoring Interface.
Example ResetNotifications <add_order> XML command via Perl script for op5 monitor system

This example shows the integration of a Perl script into op5 monitor system that automatically sends the above XML command to the JobScheduler sos/notification/ResetNotifications job chain.

The "Acknowledgment" on the op5 Monitor side works as follows:

  1. Contact "acknowledgment" + Event Handler:
    • it first of all requires a contact, that receives the Notifications in the same way as the other contacts. However, an event notification for this contact is not received via Mail but an Event Handler, i.e.  an XML command will be executed instead of a mail being received. (Please see the next point, Notification Command.)
  2. The "svc_notify_ack_handle" Notification Command:
    • this command will always be executed for the services that are specified for the contact. This command is executed when the service status changes (for example, by a change from OK to Critical or Acknowledgment of an Error).
    • The command executes a check_acknowledge.pl script.
  3. The check_acknowledge.pl Script (see the example below): this script is executed by the command and first of all checks whether the command is a response to an Acknowledgment:
    • If the command is not a response to an Acknowledgment: then nothing happens
    • If the command is a response to an Acknowledgment      : then the script causes the JobScheduler to be contacted and sent am XML query, that instructs the JobScheduler to start a specific job chain (the sos/notification/ResetNotifications chain)

check_acknowledge.pl 
#!/usr/bin/perl -w

use strict;
use LWP::UserAgent;
use HTTP::Request::Common;
use Getopt::Long;
use vars qw($opt_H $opt_f $opt_s $opt_p $opt_t $opt_h);
use vars qw(%ERRORS &support);
my $host;
my $type;
my $service;
my $port;
my $timeout = 30;
our %ERROR;
%ERRORS =   (
            'OK'       => 0,
            'CRITICAL' => 2,
            'ERROR'    => 2,
            'UNKNOWN'  => 9,
              'WARNING'  => 1,
            );


sub print_help ();
sub print_usage ();

Getopt::Long::Configure('bundling');
GetOptions
   ("h"   => \$opt_h, "help"        => \$opt_h,
    "H=s" => \$opt_H, "hostname=s"  => \$opt_H,
    "f=s" => \$opt_f, 
    "s=s" => \$opt_s, "service=s"   => \$opt_s,
    "t=i" => \$opt_t, "timeout=i"   => \$opt_t,
    "p=i" => \$opt_p, "port=i"      => \$opt_p);

if($opt_h) {print_help(); exit 0;}

if($opt_H ) {
    if ( $opt_H =~ /([-.A-Za-z0-9]+)/ ) { $host = $opt_H;    }
    ($host) || print("Invalid host: $opt_H\n");
}
else{ print("Host name/address not specified\n");} 

if($opt_p ) {
    if ($opt_p =~ /([0-9]+)/) {    $port = $1 if ($opt_p =~ /([0-9]+)/);}
    ($port < 0 || $port > 65535) && print("Invalid Port: $opt_p\n");
}
else{ print("Port not specified\n");}

if ($opt_t) { $timeout = $opt_t; }

if( !$host || !$port ) { print_usage(); exit 1;}

#<add_order  job_chain   ="/sos/notification/ResetNotifications"
#            id          ="op5 JobScheduler Monitoring Error acknowledgement"
#            title       ="op5 JobScheduler Monitoring Error acknowledgement">
#    <params>
#        <param name="service_name"  value="JobScheduler Monitoring Error" />
#        <param name="system_id"     value="op5"/>
#        <param name="operation"     value="acknowledge" />
#     </params>
#</add_order>
my $message = "<?xml version=\"1.0\" encoding=\"ISO-8859-1\"?><add_order job_chain=\"/sos/notification/ResetNotifications\" id=\"op5 ".$opt_s." Acknowledegment\" title=\"op5 ".$opt_s." Acknwoledgement\"><params><param name=\"system_id\" value=\"MonitorSystem\"/><param name=\"service_name\" value=\"".$opt_s."\"/><param name=\"operation\" value=\"acknowledge\"/></params></add_order>";

if($opt_f=~m/ACKNOWLEDGEMENT/){
    send_request($message);
}
else{ print("Please set notification type to ACKNOWLEDGEMENT\n");}

sub send_request {
    my $message = shift;
  
    my $userAgent = LWP::UserAgent->new(agent => 'perl post');
    $userAgent->timeout($timeout);
  
    my $response = $userAgent->request(POST 'http://'.$host.':'.$port,Content_Type => 'text/xml',Content => $message);
    if ($response->is_success) {
        _report('OK', "OK: Service name: ".$opt_s."\nNotification type: ".$opt_f."\nRequest: ". $message."\n\nAnswer:\n".$response->as_string."\n"); 
    } 
    else {
        _report('ERROR',"ERROR: Service name: ".$opt_s."\nNotification type: ".$opt_f."\nRequest: ". $message."\n\nAnswer:\n".$response->error_as_HTML."\n");
    }
}

sub get_attribute_value {
    my ($attr_name, $elem_xml) = @_;
    $elem_xml =~ s/.*$attr_name\s*=\s*\"(.*?)\".*/$1/s;
    return $elem_xml;
}

sub get_state_elem {
    my $xml = shift;
    $xml =~ s/.*<spooler.*?>\s*<answer.*?>\s*(<state.*?>).*/$1/s;
    return $xml;
}       

sub print_help () {
   print $0. "\n";
   print "Copyright (c) 2015 SOS GmbH, info\@sos-berlin.com

This script tries to connect to given Job Scheduler

";
   print_usage();
   print "
-H, --hostname=HOST
   Name or IP address of host to check
-p, --port=INTEGER
   Port at host to check
-t, --timeout=INTEGER
   Timeout for HTTP connetion
-f =STRING
   Notification type, e.g. ACKNOWLEDGEMENT
-s, --service=STRING
   Service name, e.g. JobScheduler Errors
-h, --help
   This help
";
}

sub print_usage () {
   print  "Usage: $0 -H <host> -p <port> -f ACKNOWLEDGEMENT -s <service name> [-t <timeout>]\n";
}

sub _report { 
    print $_[1]; 
  if (defined($ERRORS{$_[0]})) { exit $ERRORS{$_[0]}; }
  else { exit 0; }
}

JobScheduler - Job Chains customization

The default name of the monitor system used in the configuration files and stored in the JobScheduler database is "MonitorSystem".

The default configuration can be changed to allow better customization of the monitoring systems used.

Example customization for the op5 system monitor:

  • <scheduler_install>/config/notification/SystemMonitorNotification_MonitorSystem.xml
    • rename this file to  SystemMonitorNotification_op5.xml
    • set system_id attribute to op5
      • e.g. <SystemMonitorNotification system_id="op5">
  • <scheduler_install>/config/live/sos/notification/SystemNotifier,MonitorSystem.order.xml
    • rename this file to SystemNotifier,op5.order.xml
    • set system_configuration_file attribute to SystemMonitorNotification_op5.xml
      • e.g. <param name="system_configuration_file" value="config/notification/SystemMonitorNotification_op5.xml"/>
  •  <scheduler_install>/config/live/sos/notification/ResetNotifications,AcknowledgeMonitorSystem.order.xml
    • rename this file to ResetNotifications,Acknowledgeop5.order.xml
    • set system_id attribute to op5
      • e.g. <param name="system_id" value="op5"/>

JobScheduler - Cluster

See Cluster Operation

In case of Cluster Operation please modify the job_chain element definition for all notification job chain files

  • add distributed="yes" attribute.
    • e.g: <job_chain distributed="yes" ...
  • remove orders_recoverable="no" attribute if exists

Following job chain files must be modified in the notification directory <scheduler_install>/config/live/sos/notification/:

  • CheckHistory.job_chain.xml
  • CleanupNotifications.job_chain.xml
  • ResetNotifications.job_chain.xml
  • SystemNotifier.job_chain.xml

Use Cases

Workflow Execution takes too long

Initial Situation

A Job Chain is triggered and it could not end, it hang in a step, taking longer than expected.

Problem

Execution time was too long

Handling

A timer for this Job Chain has been set and the System Monitor notified about it. The expiration times for the Job Chains are configured with enough time for processing. This is usually used for cases where the Job Chain could hang in a specific step.

Configuration

SFTP connection refused

Initial Situation

Consider a Job Chain that uses SFTP for transferring files. You have a setback configured in this step of the Job Chain, so that if the connection to the SFTP server fails, this step is retried after a specified time.

Problem

The SFTP server is not available anymore.

Handling

The System Monitor will be notified to the service related to the Job Chain with the message error. However, you don't want to have repeated notifications for a Job Chain when is an external factor, the connection to the SFTP Server, is producing the error.

Configuration

Thresholds

Initial Situation

Consider the situation where a workflow has to be executed successfully a specific number of times before a specific point in time. This means that a specific value has to be monitored in order to determine if this quote was reached.

Handling

A new History service is configured, so that the workflow executions (Job Chains in the JobScheduler vocabulary) send the information that they have been successfully executed to the System Monitor.

Configuration

Acknowledgment

Initial Situation

An alert for a Service has been sent to the System Monitor, which has sent a Mail to the Service Desk (Support Team) notifying them about the alert.

Handling

The problem is known to the Service Desk and they "acknowledge" the problem. The acknowledgment will cause the JobScheduler to be notified not to send any more notifications for this Service to the System Monitor until the Service has been recovered.

Configuration


Recoverable Errors

Initial Situation

You have a setback configured in one of the steps of the Job Chain, so that if the step execution fails, this step is retried after a specified time.

Problem

The step has ended with an error, but recovered after setback

Handling

If the error message has been sent to the System Monitor, in case of error recovery JobScheduler will automatically sent the recovery message on the same service with the same error message and the prefix RECOVERED.

Configuration

Change Management References

T Key Linked Issues Fix Version/s Status P Summary Updated
Refresh