Skip to end of metadata
Go to start of metadata

Scope

  • Use Case:
    • Consider the situation where:
      • A job chain is to start at a number of parallel job chains
      • Subsequent jobs in started job chains check if jobs from parallel chains completed successfully.
    • Usually a split & sync pattern is used for similar use cases, however, synchronizing includes that both jobs have to be completed.
    • Instead the current use case is about check and control if a job of a parallel chain has completed without forcing both jobs to run synchronously.

Solution

  • Download: controlParallelExecutionOfJobChainsExample.zip
  • Extract the archive to a folder in your JobScheduler installation named ./config/live.
  • The archive will extract the files to a sub-folder controlParallelExecutionOfJobChainsExample. 
  • Note that you can store the sample files in any folder you like.

Pattern

Implementation

Components

  • The solution contains six job chains:
    • JobChain1 runs Job1 and Job2 in parallel.
      • On successful completion of Job1 then JobChain2 is run
      • On successful completion of Job2 then JobChain3 is run
    • JobChain2 runs Job3 and Job4
    • JobChain3 runs Job5 and Job6
    • JobChain4 runs after successful completion of JobChain1
      • JobChain4 runs Job7, Job8, Job9
      • Job7 runs 1st
      • Job8 runs once Job7 and Job4 have run successfully
    • JobChain5 runs after successful completion of JobChain1
      • JobChain5 runs Job10, Job11 and Job12
      • Job10 runs 1st
      • Job11 runs once Job10, Job4 and Job6 have run successfully
      • Job12 runs once Job11 has completed successfully
    • JobChain6 runs once Job9 and Job12 have completed successfully

Usage

  • The first job split1 sets a number of parameters by use of a pre-processing Monitor for later control of successful completion of jobs. For each job to be controlled a pair of parameters is used:
    • The first parameter of the pair is a flag which is used for later checking if the dependent job is successfully completed.
      • The value of this parameter is initialised with "0" and is modified by the dependent jobs to "1".
      • In the above example the following parameters are used:

        • job4_successfully_completed=0 which will be checked by Job8 and Job11
        • job6_successfully_completed=0 which will be checked by Job11
        • job12_successfully_completed=0 which will be checked by Job13
    • The second parameter of the pair describes to which job chains the information about successful completion is broadcasted.
      • The example specifies the job chains for which a suspended order is continued if the current job completes successfully.
      • The current job chain  (here JobChain1) always is the first in the list.
      • The job chains can be specified by an absolute path or by a relative path to the current job chain.
      • In the above example the following values are used:

        • job4_commits_completion_to="/controlParallelExecutionOfJobChainsExample/JobChain1;JobChain4;JobChain5"
        • job6_commits_completion_to="/controlParallelExecutionOfJobChainsExample/JobChain1;JobChain5"
        • job12_commits_completion_to="/controlParallelExecutionOfJobChainsExample/JobChain1;JobChain4;JobChain6"
  • The dependent jobs (here Job4, Job6 and Job12) use a post-processing monitor that implements commands to resume the orders
    • in all job chains which are specified by the commits_completion_to parameter
    • where the value of the successfully_completed parameter is set to "1".
  • The jobs that wait for dependent jobs make use of a pre-processing monitor that checks the corresponding successfully_completed parameters.
    • These jobs use a task parameter dependent_jobs that specifies a list of dependent jobs.
    • In above example these are
      • Job8 : dependent_jobs="Job4"
      • Job11: dependent_jobs="Job4;Job6"
      • Job13: dependent_jobs="Job12"
  • The above described pre- and post-processing monitors include the attached control_parallel_execution_of_job_chains.js JavaScript like this:

    pre-processing monitor for split1 job
    <monitor  name="set_params_for_check_jobs" ordering="1">
            <script  language="java:javascript">
                <include  live_file="control_parallel_execution_of_job_chains.js"/>
                <![CDATA[
    function spooler_process_before() {
        
        //Specify the list of job chains for which Job4 commits that it has run successfully
        //These job chains can specify relative to the current job chain
        
        setParams4CheckingJobs( "Job4", ["Chain4", "Chain5"] );
    
        //Specify the list of job chains for which Job6 commits that it has run successfully
        //These job chains can specify relative to the current job chain
        
        setParams4CheckingJobs( "Job6", ["Chain5"] );    
        
        //Specify the list of job chains for which Job12 commits that it has run successfully
        //These job chains can specify relative to the current job chain
        
        setParams4CheckingJobs( "Job12", ["Chain4", "Chain6"] );
        
        return true;
    }
                ]]>
            </script>
        </monitor>
    post-processing monitor for jobs Job4, Job6 and Job12
    <monitor  name="commit_successfully_run" ordering="0">
            <script  language="java:javascript">
                <include  live_file="control_parallel_execution_of_job_chains.js"/>
                <![CDATA[
    function spooler_process_after(spooler_process_result) {
        return commit_successful_run(spooler_process_result);
    }
                ]]>
            </script>
        </monitor>
    pre-processing monitor for job Job8, Job11 and Job13
    <monitor  name="check_dependent_jobs" ordering="0">
            <script  language="java:javascript">
                <include  live_file="control_parallel_execution_of_job_chains.js"/>
                <![CDATA[
    function spooler_process_before() {
        return check_dependent_jobs();
    }
                ]]>
            </script>
        </monitor>

See also

 

 

Write a comment…