Scope
- Use Case
Create Job Stream #1
- 1. Batch Job #1 runs only at 10pm. If the job fails to run successfully, is should be rerun up to a maximum 2 times.
2. Batch Job #2 will run IF AND ONLY IF Job #1 completes successfully by 11pm and only if today is Wednesday.
3. Batch Jobs #3 and #4 will run only if Job #2 ran successfully before 11.30pm and only if it’s before the 10th of the month (meaning the run date is < 10th OCT-2015 for example).
4. Batch Job #5 will run after midnight and if Jobs #3 and #4 completed running within 45 minutes of their individual start times.
5. Batch Job #6 will run only on Thursdays and Fridays of the week, and IF AND ONLY IF it’s after the 5th of the current month AND before 15th of the month (meaning the run date is > 5th OCT-2015 && < 15th OCT-2015 for example).
Create Job Stream #2:
6. Run Batch Job #7 all by itself in a separate Job Stream #2 during the 1st week of the 3rd month of every quarter in a calendar year.
Combine the 2 Job Streams into a new Job Stream #3:
7. Batch Job #8 and Batch Job #9 (in Job Stream #3) will be automatically started and running at the same time IF AND ONLY IF Job #6 from Job Stream #1 and Job #7 from Job Stream #2 ran successfully.
Create Job Stream # 4:
8. Run Batch Job #10 (in Job Stream #4) only after the successful completion of Job #8 and #9 in Job Stream #3.
- Hint
- This use case is derived from a complex workflow implemented with Control-M®.
Solution
- Download: workflow_condition.zip
- Extract the archive to a folder in your JobScheduler installation named
./config/live/issues
. - The archive will extract the files to a sub-folder
workflow_condition.
- Note that you can store the sample files in any folder you like: however, you will then have to adjust the job parameters accordingly.
Pattern
Implementation
Components
- The solution makes use of Scripting with Pre-/Post-Processing Monitors and date calculation.
- The solution contains four job streams with dependencies:
job_stream1
andjob_stream2
represent two job chains that have to be executed successfully to makejob_stream3
proceed.job_stream3
splits the processing into parallel jobs (job8
,job9
).job_stream4
is the final job chain that checks ifjob8
andjob9
fromjob_stream3
did complete successfully.
- The solution implements the following job dependencies:
job_stream1
job1
- runs without dependencies.
- is configured for setback, i.e. will repeat iself in case of error up to two times.
job2
- implements an individual Pre-Processing Monitor to check if the current day is Wednesday and if the predecessor job
job1
completed successfully before 11:00pm. - will be skipped and the order moved to the next job node if the check fails.
- implements an individual Pre-Processing Monitor to check if the current day is Wednesday and if the predecessor job
job3
- implements an individual Pre-Processing Monitor to check if the current date is before the 10th of month if the predecessor job
job2
completed successfully before 11.30pm. - will be skipped and the order moved to the next job node if the check fails.
- implements an individual Pre-Processing Monitor to check if the current date is before the 10th of month if the predecessor job
job4
- implements the same check as
job3
.
- implements the same check as
job5
- implements an individual Pre-Processing Monitor to check if two predecessor jobs (
job3
,job4
) completed successfully and consumed less than 45 min. execution time. - will be skipped and the order moved to the next job node if the check fails.
- implements an individual Pre-Processing Monitor to check if two predecessor jobs (
job6
- implements an individual Pre-Processing Monitor to check if the current day is a Thursday or Friday and if the date is after the 5th and before the 15th of month.
- makes use of the Named Post-Processing Monitor
start_job_stream
(by a<monitor.use>
element):- this Monitor checks if a number of jobs completed successfully:
- the job names can be specified by use of the
check_jobs
job parameter that is assigned a semicolon separated list of job names. - job names can be specified with an absolute path (starting from the
live
folder) or with a path relative to the directory of this job.
- the job names can be specified by use of the
- if the checks are successful then this Monitor will start a number of job chains;
- the job chain names can be specified by use of the
start_job_chains
job parameter that is assigned a semicolon separated list of job chain names. - job chain names can be specified with an absolute path (starting from the
live
folder) or with a path relative to the directory of this job.
- the job chain names can be specified by use of the
- this Monitor checks if a number of jobs completed successfully:
- will be skipped and the order moved to the next job node if the check fails.
job_stream2
job7
job_stream3
job8
job9
job_stream4
- job10
- The solution implements a job
check_predecessor_job_chain_Z
that has been added at the start ofjob_chain_Z
.- This job is parameterized with a number of parameters that start with the prefix "check_predecessor" and use an arbitrary suffix that is required in order to create unique parameter names. The value of each parameter is the final job of the respective job chain that is checked for successful execution:
- Example:
check_predecessor_A = issues/final_chain/do_something_job_chain_A
check_predecessor_B = issues/final_chain/do_something_job_chain_B
- Should the sample files not have been extracted to the
./config/live/issues/final_chain
folder then please adjust the parameter value to point to the relevant directory.
- Example:
- This job implements a
spooler_process()
function that reads the parameters and checks the job history for successful execution of the jobs specified by the parameters. - Should all checks for previous successful execution provide a positive result then the current order will be moved to the next job chain node. Otherwise the current order is set back and will be repeated regularly according to the setback configuration for this job.
- This job is parameterized with a number of parameters that start with the prefix "check_predecessor" and use an arbitrary suffix that is required in order to create unique parameter names. The value of each parameter is the final job of the respective job chain that is checked for successful execution:
Usage
- Positive Check
- Add an order to
job_chain_A
using JOC'sAdd Order
context menu.- The order should pass through the job chain without errors.
- Add an order to
job_chain_B
using JOC'sAdd Order
context menu.- The order should also pass through the job chain without errors.
- Add an order to
job_chain_B
using JOC'sAdd Order
context menu.
- Add an order to
- Negative Check
- Modify the job
do_something_job_chain_A
to include an error, e.g. by adding the commandexit 1
as the final line in the job script. - Add an order to
job_chain_A
:- The order should proceed with an error.
- Add an order to
job_chain_Z
- The order should proceed with an error that is visible in the order log. This error will state that the predecessor job did not complete successfully.
- The order will be set back and be repeated according to the setback intervals that have been specified for the job
check_predecessor_job_chain_Z
.
- Removing the modification to job
do_something_job_chain_A
and adding an order tojob_chain_A
that then runs successfully should result in a successful run ofjob_chain_Z
- either by a new order added tojob_chain_Z
or by the next execution of the previous order that has been set back.
- Modify the job