Page History

Scope

• Use Case

• Create Job Stream #1 

  • 1. Batch Job #1 runs without pre-condition.

  • 2. Batch Job #2, #3, #4, #5, #6 will run if the respective predecessor Batch Job completed successfully.

• Create Job Stream #2:

  • 3. Batch Job #7 runs without pre-condition.

• Combine the 2 Job Streams into a new Job Stream #3:

  • 4. Batch Job #8 and Batch Job #9 (in Job Stream #3) will be automatically started and running at the same time IF AND ONLY IF Job Stream #1 and Job Stream #2 ran successfully.

• Create Job Stream # 4:

  • 5. Batch Job #10 (in Job Stream #4) runs only after the successful completion of Job #8 and #9 in Job Stream #3.

• Feature Availability
  

  • Display feature availability
    StartingFromRelease 1.10.2

    
    

  • Jira
    server SOS JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 6dc67751-9d67-34cd-985b-194a8cdc9602 key JITL-228

    
    

Solution

• Download: workflow_condition_job_chains.zip 
• Extract the archive to a folder in your JobScheduler installation named ./config/live/issues.
• The archive will extract the files to a sub-folder workflow_condition_job_chains. 
• Note that you can store the sample files in any folder you like.

Pattern

order1 [shape="ellipse",label="Order start_job_stream1\n start time: 10:00 every day ",fillcolor="violet"]
order2 [shape="ellipse",label="Order start_job_stream2\n start time: 12:00 every day ",fillcolor="violet"]
 
job_chain1 [label="Job Stream 1\nhas no relevant dependencies",fillcolor="orange"]
job_chain2 [label="Job Stream 2\nhas no relevant dependencies",fillcolor="orange"]
job_chain3 [label="Job Stream 3\ndepends on the execution of\nJob Stream 1 and Job Stream 2",fillcolor="orange"]
job_chain4 [label="Job Stream 4\ndepends on the execution of\nJob Stream 3",fillcolor="orange"]

job1 [label="Job 1\nConditions: none", fillcolor="lightskyblue"]
job2 [label="Job 2\nConditions: Job #1 completed successfully", fillcolor="lightskyblue"]
# job2 -> job1
job3 [label="Job 3\nConditions: Job #2 completed successfully", fillcolor="lightskyblue"]
# job3 -> job2
job4 [label="Job 4\nConditions: Job #3 completed successfully", fillcolor="lightskyblue"]
# job4 -> job3
job5 [label="Job 5\nConditions: Job #4 completed successfully", fillcolor="lightskyblue"]
# job5 -> job4
job6 [label="Job 6\nConditions: Job #5 completed successfully", fillcolor="lightskyblue"]
 
job7 [label="Job 7\nConditions: none", fillcolor="lightskyblue"]
 
job_splitter [label="Job Splitter\nConditions: Job Stream 1 and Job Stream 2\ncompleted successfully", fillcolor="lightskyblue"]
# job_splitter -> job_chain1
# job_splitter -> job_chain2
job8 [label="Job 8\nConditions: none", fillcolor="lightskyblue"]
job9 [label="Job 9\nConditions: none", fillcolor="lightskyblue"]
job_sync [label="Job Sync\nConditions: Splitted orders for Job 8 and Job 9\ncompleted successfully", fillcolor="lightskyblue"]
# job_sync -> job8
# job_sync -> job9
job_dispatcher [label="Job Dispatcher\nConditions: Job Sync completed successfully", fillcolor="lightskyblue"]
 
job10 [label="Job 10\nConditions: Job Stream 3 completed successfully", fillcolor="lightskyblue"]
# job10 -> job_chain3

order1 -> job_chain1 -> job1 -> job2 -> job3 -> job4 -> job5 -> job6 -> job_chain3
order2 -> job_chain2 -> job7 -> job_chain3
job_chain3 -> job_splitter
job_splitter -> job8 -> job_sync
job_splitter -> job9 -> job_sync
job_sync -> job_dispatcher -> job_chain4
job_chain4 -> job10

Implementation

Components

• Hints
  • The solution makes use of Scripting with Pre-/Post-Processing Monitors.
  • The term completed successfully as subsequently used includes that
    • if not specified otherwise the last run of a job chain assigned an order an end state that is not used as an error state in the job chain..
    • the last run of a job chain was successful (not any previous runs within the same period).
  • The conditions assigned to job chains are handled in a way that the successor job chain is executed if the conditoins can be met.
  • Except for job8 and job9 all jobs implement a simple "echo hello world" command.
• The solution contains four job streams with dependencies:
  • job_stream1 and job_stream2 represent two job chains that have to be executed successfully to make job_stream3 proceed.
  • job_stream3 splits the processing into parallel jobs (job8, job9).
  • job_stream4 is the final job chain that checks if job_stream3 did complete successfully.
• The solution implements the following job dependencies:
  • job_stream1
    • job1
      • runs without dependencies.
    • job2, job3, job4, job5, job6
      • run if their predecessor job completed successfully
    • job6
      • makes use of the Named Post-Processing Monitor start_job_stream (by a <monitor.use> element):
        • the Post-Processing Monitor will not start job chains if the job completes with a return code > 0 signalling an error or if one of the checks fails.
        • this Monitor checks if a number of jobs completed successfully (job_stream2): 
          • the job chain names are specified by use of the check_job_chains job parameter that is assigned a semicolon separated list of job chain names. 
          • job chain names can be specified with an absolute path (starting from the live folder) or with a path relative to the directory of this job.
        • if the checks are successful then this Monitor will start a number of job chains (job_stream3):
          • the job chain names are specified by use of the start_job_chains job parameter that is assigned a semicolon separated list of job chain names.
          • job chain names can be specified with an absolute path (starting from the live folder) or with a path relative to the directory of this job.
  • job_stream2
    • job7
      • makes use of the Named Post-Processing Monitor start_job_stream (by a <monitor.use> element) in the same way as job6.
      • the same job parameterization applies as for job6 with the exeception that the check is performed for job_stream1.
  • job_stream3
    • splitter
      • makes use of the Named Post-Processing Monitor start_job_stream (by a <monitor.use> element) in the same way as job6.
      • the same job parameterization applies as for job6 with the exeception that the check is performed for job_stream1 and job_stream2.
      • splits processing of the current order into parallel execution of jobs job8 and job9.
    • job8, job9
      • run without dependencies.
      • run in parallel.
    • sync
      • waits for job8 and job9 to complete successfully.
    • dispatcher
      • makes use of the Named Post-Processing Monitor start_job_stream (by a <monitor.use> element) in the same way as job6.
      • the same job parameterization applies as for job6 with the exception that no check is performed but 

  • job_stream4
    • job10
      • makes use of the Named Pre-Processing Monitor check_job_chain_history by a <monotor.use> element) in
        • this Monitor checks if a number of job chains completed successfully (job_stream3): 
          • the job chain names are specified by use of the check_job_chains job parameter that is assigned a semicolon separated list of job chain names. 
          • job chain names can be specified with an absolute path (starting from the live folder) or with a path relative to the directory of this job.
      • will be skipped and fail if the check fails. 

Usage

• Positive Check
  • Start the order start_job_stream1 using JOC's Start Order now context menu. Alernatively add a new order.
    • The order should pass through the job chain without errors.
  • Start the order start_job_stream2 using JOC's Start Order now context menu. Alternatively add a new order.
    • The order should also pass through the job chain without errors.
    • As the job chains job_stream1 and job_stream2 completed successfully the job chain job_stream3 will be kicked off by job7 of job_stream2.
    • Job chain job_stream3 runs job8 and job9 in parallel.
    • job9 in job_stream3 will take longer than job8 to complete and will start job chain job_stream4.
    • Job chain job_stream4 with job10 will complete successfully.
• Negative Check
  • Modify the job job1 in job chain job_stream1 to include an error, e.g. by adding the command exit 1 as the final line in the job script.
  • Start the order start_job_stream1 using JOC's Start Order now context menu. Alernatively add a new order.
    
    • The order should complete with an error having passed job1, no further jobs are executed.
  • Start the order start_job_stream2 using JOC's Start Order now context menu. Alernatively add a new order.
    • job7 will be executed succesfully, however, job_stream3 will not be started due to the fact that job_stream1 completed with error.

References

Change Management References

Documentation

• How to check the job chain history for previous job chain and order runs
• How to manage complex conditions to check previous job executions and to start successor job chains
• How to check the job history for previous job runs