How to manage complex conditions to check previous job executions and to start successor job chains

Scope

• Use Case

• Create Job Stream #1 

  • 1. Batch Job #1 runs only at 10pm. If the job fails to run successfully, is should be rerun up to a maximum 2 times.

  • 2. Batch Job #2 will run IF AND ONLY IF Job #1 completes successfully by 11pm and only if today is Wednesday.

  • 3. Batch Jobs #3 and #4 will run only if Job #2 ran successfully before 11.30pm and only if it’s before the 10th of the month (meaning the run date is < 10th OCT-2015 for example).

  • 4. Batch Job #5 will run after midnight and if Jobs #3 and #4 completed running within 45 minutes of their individual start times.

  • 5. Batch Job #6 will run only on Thursdays and Fridays of the week, and IF AND ONLY IF it’s after the 5th of the current month AND before 15th of the month (meaning the run date is > 5th OCT-2015 && < 15th OCT-2015 for example).

• Create Job Stream #2:

  • 6. Run Batch job # 7 all by itself in a separate Job Stream #2 during the 1st week of the 3rd month of every quarter in a calendar year.

• Combine the 2 Job Streams into a new Job Stream #3:

  • 7. Batch Job #8 and Batch Job #9 (in Jobs stream #3) will be automatically started and running at the same time IF AND ONLY IF Job #6 from Job Stream #1 and Job #7 from Job Stream #2 ran successfully.

• Create Job Stream # 4:

  • 8. Run Batch Job #10 (in Job Stream #4) only after the successful completion of Job #8 and #9 in Job Stream #3.

• Hints
  • This use case is derived from a complex workflow implemented with Control-M®.

Solution

• Download: final_chain.zip
• Extract the archive to a folder in your JobScheduler installation named ./config/live/issues.
• The archive will extract the files to a sub-folder workflow_condition. 
• Note that you can store the sample files in any folder you like: however, you will then have to adjust the job parameters accordingly.

Pattern

Implementation

Components

• The solution contains three job chains:
  • job_chain_A and job_chain_B represent the predecessor job chains that have no relevant dependencies with job_chain_Z.
  • job_chain_Z is the final job chain that checks if job_chain_A and job_chain_B have already been executed successfully.
• The solution implements a job check_predecessor_job_chain_Z that has been added at the start of job_chain_Z.
  • This job is parameterized with a number of parameters that start with the prefix "check_predecessor" and use an arbitrary suffix that is required in order to create unique parameter names. The value of each parameter is the final job of the respective job chain that is checked for successful execution:
    • Example:
      • check_predecessor_A = issues/final_chain/do_something_job_chain_A
      • check_predecessor_B = issues/final_chain/do_something_job_chain_B
    • Should the sample files not have been extracted to the ./config/live/issues/final_chain folder then please adjust the parameter value to point to the relevant directory.
  • This job implements a spooler_process()function that reads the parameters and checks the job history for successful execution of the jobs specified by the parameters.
  • Should all checks for previous successful execution provide a positive result then the current order will be moved to the next job chain node. Otherwise the current order is set back and will be repeated regularly according to the setback configuration for this job.

Usage

• Positive Check
  • Add an order to job_chain_A using JOC's Add Order context menu.
    • The order should pass through the job chain without errors.
  • Add an order to job_chain_B using JOC's Add Order context menu.
    • The order should also pass through the job chain without errors.
  • Add an order to job_chain_B using JOC's Add Order context menu.
• Negative Check
  • Modify the job do_something_job_chain_A to include an error, e.g. by adding the command exit 1 as the final line in the job script.
  • Add an order to job_chain_A:
    
    • The order should proceed with an error.
  • Add an order to job_chain_Z
    • The order should proceed with an error that is visible in the order log. This error will state that the predecessor job did not complete successfully.
    • The order will be set back and be repeated according to the setback intervals that have been specified for the job check_predecessor_job_chain_Z.
  • Removing the modification to job do_something_job_chain_A and adding an order to job_chain_A that then runs successfully should result in a successful run of job_chain_Z - either by a new order added to job_chain_Z or by the next execution of the previous order that has been set back.