Page History

...

This article describes individual configuration parameters and provides examples of their use with monitors such as op5 and Zabbix and using of the mail und JMS interfaces.

Send notifications

Notify on error

SystemMonitorNotification / Notification / NotificationMonitor / @service_name_on_error is configured.

SystemMonitorNotification / Notification / NotificationObjects / JobChain

Error messages

• will be sent:
  • when at the time of the sos / notification / SystemNotifier run, an order is in a job chain state(step) that has ended with an error
• will not be sent:
  • when after the last run of the sos / notification / SystemNotifier, an error has occured in an job chain state(step) but at the time of the current sos / notification / SystemNotifier run this order is in an other/next job chain state(step)
    • this kind of error is ignored because the order has continued to run
  • when a new error has reoccurred in the same job chain state(step) where a notification has already been sent
    • this order state is considered as notified and no new notification will be sent
    • e.g. an job chain state(step) has been restarted manually or by a setback
  • when a notification maximum has been reached
    • see  SystemMonitorNotification / Notification / NotificationObjects / JobChain / @notifications
  • when a job chain state(step) has been configured as excluded
    • see SystemMonitorNotification / Notification / NotificationObjects / JobChain / @excluded_steps
  • when the @step_from or @step_to settings have been configured and the job chain state(step) is out of the configured range
    • see SystemMonitorNotification / Notification / NotificationObjects / JobChain / @step_from | @step_to

  • when the @return_code_from or @return_code_to settings have been configured and the return code of the job chain state(step) is out of the configured range
    • see SystemMonitorNotification / Notification / NotificationObjects / JobChain / @return_code_from | @return_code_to


Recovery messages

• will be automatically sent using the same service name and message as the relevant error message:
  • when the error message of a job chain state(step) has already been sent and the order at the time of the current sos / notification / SystemNotifier run is in an other/next state(step)
    • e.g. the rerun of the error state(step) has been successfull and the order has been moved to the next job chain state(step)
    • Note: use ${SERVICE_STATUS} and ${SERVICE_MESSAGE_PREFIX} variables to differentiate between recovery and error message
      • See:
        • Message / Variables / Service variables
        • Notification environment variables / Service variables
        • Examples / Examples System Monitoring / NotificationCommand ( Nagios / OP5 )
• will not be sent:
  • when a job chain state(step) has recovered after the last run of the sos / notification / SystemNotifier but at the time of the current sos / notification / SystemNotifier run a new error in the other/next step has occured

SystemMonitorNotification / Notification / NotificationObjects / Job

Error messages

• will be sent:
  • when a job chain state(step) or standalone job (JobScheduler versions from 1.12) ends with an error
• will not be sent:
  • when a notification maximum has been reached
    • see SystemMonitorNotification / Notification / NotificationObjects / Job / @notifications
  • when the @return_code_from or @return_code_to settings have been configured and the return code of the job is out of the configured range
    • see SystemMonitorNotification / Notification / NotificationObjects / Job / @return_code_from | @return_code_to


Notify on success

SystemMonitorNotification / Notification / NotificationMonitor / @service_name_on_success is configured.

SystemMonitorNotification / Notification / NotificationObjects / JobChain

Success messages

• will be sent:
  • when an order is completed and the last job chain state(step) has no error
• will not be sent:
  • when a notification maximum has been reached
    • see SystemMonitorNotification / Notification / NotificationObjects / JobChain / @notifications
  • when the @return_code_from or @return_code_to settings have been configured and the return code of the job chain state(step) is out of the configured range

    • see SystemMonitorNotification / Notification / NotificationObjects / JobChain / @return_code_from | @return_code_to


SystemMonitorNotification / Notification / NotificationObjects / Job

Success messages

• will be sent:
  • when a job chain state(step) or standalone job (JobScheduler versions from 1.12) ends without error
• will not be sent:
  • when a notification maximum has been reached
    • see SystemMonitorNotification / Notification / NotificationObjects / Job / @notifications
  • when the @return_code_from or @return_code_to settings have been configured and the return code of the job is out of the configured range
    • see SystemMonitorNotification / Notification / NotificationObjects / Job / @return_code_from | @return_code_to


Configuration Editor

We recommend that the XML Editor is used generate monitoring configuration objects. This editor automatically uses an XSD Schema to generate configuration suggestions and validate configurations, and its use is intended to provide a significant reduction in the time required to develop and test a configuration.

...

• JobScheduler version 1.9.x, 1.10.x
  • see JobScheduler - Job Chains
• JobScheduler version 1.11.x, 1.12.x
  
  • Set param sos.use_notification true (config/scheduler.xml)
  • see JobScheduler - Job Chains

...

...

...

<SystemMonitorNotification system_id="op5"> 
  <Notification> 
    <NotificationMonitor service_name_on_error="JobScheduler Monitoring Errors"> 
      ... 
    </NotificationMonitor> 
    <NotificationObjects>
      <!-- Send the job error, occurrent in the "test/my_job" order job, to the "JobScheduler Monitoring Errors" service. -->        
      <Job name="test/my_job" /> 
      <!-- Send the job chain error, occurrent in the "test/my_jobchain" job chain, to the "JobScheduler Monitoring Errors" service. --> 
      <JobChain name="test/my_jobchain" /> 
    </NotificationObjects> 
 </Notification> 
</SystemMonitorNotification>  

SystemMonitorNotification / Notification / NotificationObjects / Job 

Note:

This element specifies the order-controlled or standalone jobs for which notifications are being sent to a system monitor.

...

...
<Job notifications="2" name="test/my_job"/>
...
<Job scheduler_id="scheduler_4444" />
...
<Job scheduler_id="scheduler_4444" name="test/my_.*" />
... 
<Job name="test/my_job" return_code_from="5"/> 
...  
<Job name="test/my_job" return_code_to="10"/>
...  
<Job name="test/my_job" return_code_from="5" return_code_to="5"/>  
...

SystemMonitorNotification / Notification / NotificationObjects / JobChain

JobChain supports the following attributes:

...

• set the scheduler_notification_result_parameters parameter (see job documentation jobs/JobSchedulerNotificationStoreResultsJob.xml)
• set the StoreResultsJobJSAdapterClass as monitor
  • JobScheduler version 1.9.x, 1.10.x
    • com.sos.scheduler.notification.jobs.result.StoreResultsJobJSAdapterClass

  • JobScheduler version 1.11.x, 1.12.x
    

    • com.sos.jitl.notification.jobs.result.StoreResultsJobJSAdapterClass

<?xml version="1.0" encoding="ISO-8859-1"?> 
<job  title="Sample Job with Store Result Monitor" order="yes" stop_on_error="no" tasks="1">     
  <params>
     <!--
     set the scheduler_notification_result_parameters parameter
     -->         
    <param name="scheduler_notification_result_parameters" value="file_size"/>     
  </params>     
  
  <!--
  calculate and create the new order parameter if necessary
  -->
  <script language="java:javascript"><![CDATA[             
      function spooler_process(){                                  
        var order    = spooler_task.order;                 
        var params   = spooler.create_variable_set();                 
        params.merge(spooler_task.params);                 
        params.merge(order.params);                      
        
        // parameter scheduler_file_path was set in the previous job chain step
        var file     = new java.io.File(params.value("scheduler_file_path"));                 
        var fileSize = file.length()/1024;                 
        order.params.set_var("file_size",fileSize.toString());                          
      return true;             
      }]]>     
   </script>          
 
   <!-- 
   set the StoreResultsJobJSAdapterClass as a monitor
   -->     
   <monitor  name="notification_monitor" ordering="1">         
     	<!-- JobScheduler version 1.9.x, 1.10.x
	<script java_class="com.sos.scheduler.notification.jobs.result.StoreResultsJobJSAdapterClass" language="java"/>
        -->
   
        <!-- JobScheduler version 1.11.x, 1.12.x --> 
        <script java_class="com.sos.jitl.notification.jobs.result.StoreResultsJobJSAdapterClass" language="java"/> 
   </monitor>

   <run_time /> 
</job> 

...

...

 ...
<!--
monitor_host            The hostname or ip address of System Monitor host 
monitor_port            The TCP port that the System Monitor would listen to
monitor_encryption      Encryption algorithm
service_host            The host that executes the passive check. The name must match the corresponding setting in the System Monitor
{MON_N_SCHEDULER_ID}    See explanation "Table variables"
...
-->
<NotificationInterface monitor_host="monitor_host" monitor_port="5667" monitor_encryption="XOR" name must match the corresponding setting in the System Monitor
{MON_N_SCHEDULER_ID}    See explanation "Table variables"
...
-->
<NotificationInterface monitor_host="monitor_host" 
                       monitor_port="5667" 
                       monitor_encryption="XOR" 
                       service_host="service_host"><![CDATA[
scheduler id=${MON_N_SCHEDULER_ID}, history id=${MON_N_ORDER_HISTORY_ID}, job_chain=${MON_N_JOB_CHAIN_NAME}(${MON_N_ORDER_ID}), step =${MON_N_ORDER_STEP_STATE}, error=${MON_N_ERROR_TEXT}
]]></NotificationInterface>
...

...

... 
<!--
service_host               The host that executes the passive check. The name must match the corresponding setting in the System Monitor.
monitor_host               The hostname or ip address of System Monitor host.
{SERVICE_NAME}             See explanation "Service variables"
{SERVICE_STATUS}           See explanation "Service variables"
{SERVICE_MESSAGE_PREFIX}   See explanation "Service variables"
{MON_N_SCHEDULER_ID}       See explanation "Table variables"
...
NotificationCommand after substitution (error case):
<![CDATA[echo service_host:JobScheduler Monitoring Errors:2:ERROR scheduler id=scheduler_4444, history id=123, job_chain=test/my_jobchain(order_id), step=100, error=error occurred | D:\nsca\send_nsca.exe -H monitor_host -c D:\nsca\send_nsca.cfg -d : ]]>
 
NotificationCommand after substitution (recovery case): 
<![CDATA[echo service_host:JobScheduler Monitoring Errors:0:RECOVERED scheduler id=scheduler_4444, history id=123, job_chain=test/my_jobchain(order_id), step=100, error=error occurred | D:\nsca\send_nsca.exe -H monitor_host -c D:\nsca\send_nsca.cfg -d : ]]> 
 
NotificationCommand after substitution (success case):  
<![CDATA[echo service_host:JobScheduler Monitoring Success:0:SUCCESS scheduler id=scheduler_4444, history id=123, job_chain=test/my_jobchain(order_id), step=100, error= | D:\nsca\send_nsca.exe -H monitor_host -c D:\nsca\send_nsca.cfg -d : ]]>  
 
-->
<NotificationMonitor service_name_on_error="JobScheduler Monitoring Errors" 
                     service_name_on_success="JobScheduler Monitoring Success" 
                     service_status_on_error="2" 
                     service_status_on_success="0">
  <NotificationCommand><![CDATA[echo service_host:${SERVICE_NAME}:${SERVICE_STATUS}:${SERVICE_MESSAGE_PREFIX}scheduler id=${MON_N_SCHEDULER_ID}, history id=${MON_N_ORDER_HISTORY_ID}, job_chain=${MON_N_JOB_CHAIN_NAME}(${MON_N_ORDER_ID}), step=${MON_N_ORDER_STEP_STATE}, error=${MON_N_ERROR_TEXT} | D:\nsca\send_nsca.exe -H monitor_host -c D:\nsca\send_nsca.cfg -d : ]]>
  </NotificationCommand>  
</NotificationMonitor>

...

...

• This is the main job that analyze the JobScheduler history tables und write results into the notification tables.
  
  • Job read all history entries for the job chains, configured in the SystemMonitorNotification XML files.
  • Job execute the performance checks for the defined Timers
• Order Check
  • configure repeat interval for order run time, e.g. every two minutes.

sos / notification / CheckHistory (JobScheduler version 1.11.x, 1.12.x)

• Job chain removed

• Set param sos.use_notification true (config/scheduler.xml)

...