How to use setbacks to make a job retry in the event an error

Introduction

This article show how you can make a job retry in event of errors and control this behavior.

Downloads

Required Version:

You will require JobScheduler Version 1.8 or newer to be able to run the example configuration files.

Download the file:

• setback.zip - configuration files

Instructions

• Unzip all files into the ./config/live folder of your JobScheduler installation.
• Open the JobScheduler Operating Center, JOC, in your browser using http://scheduler_host:scheduler_port
• Open the JOB CHAINS tab and enable Show orders.
• Find the job chain samples/setback/chain_setback.
• Find the order order1, click on the order to open the order pane and choose "Show log" in the order menu.

A window will open showing the log of the order. As the order isn't running yet, the window will show nothing more than a line giving some information about the Protocol and a line stating that the set_state is 'start'. Leave this log window open and move it to a place where you can see it along with the main browser window.

• Start the order by choosing "Start order now" from the Order menu.
  • Watch the log grow in the log window.
  • The order will run into the first job (setback) of the chain. This job actually uses a script to simulate an error and sets the order back into the Order Queue.
    You can see the Order Queue by clicking on the job to show the Job and Task History panes and select the Order Queue tab, which is alongside the Task History Tab.
  • You will now be able to follow the state of the order in the order queue through the three setbacks specified in the setback job.
    In the queue the order will be shown with a Next start of either:
    
    • Setback, plus the time left for the setback
    • Running since:
  • After the setback of 20 seconds has been completed, the JobScheduler will make the job process the order again. Like the last time the order will be setback.

On the third try, a successful completion of the job is simulated (representing, for example, that the network connection is working again). The order will then continue into the second job of the job chain.

How it works

The order has an initial order parameter count which is set to 1. The first job (setback) reads this parameter. If the parameter is not 3, it is increased and the job calls the order's setback() function (which is what a job should do in case of errors that might be temporary).

A setback interval is configured for a job - for example:

  <!-- after the first setback, try again every 20 seconds -->
  <delay_order_after_setback setback_count="1" delay="20"/>

  <!-- after the 100th setback, try again every 60 seconds -->
  <delay_order_after_setback setback_count="100" delay="60"/>

  <!-- after the 1000th setback, give up -->
  <delay_order_after_setback setback_count="1000" is_maximum="yes"/>

Calling setback() will cause the JobScheduler to retry processing this order or to give up if the maximum number of setbacks has been reached.

When the count parameter of the order has reached 3, success is simulated. The job does not setback the order this time and the order is able to run through the following jobs.

Sebacks in JOE

The JobScheduler's Object Editor, JOE provides a SetBack Form for the configuration of job setbacks. This form, together with the configuration listed in the code block above, is shown in the following screen shot: