Scope

• Job Streams are a means to implement dependencies with jobs. This feature improves existing forward dependencies (job chains) by introducing backward dependencies (conditions).
• Job Streams
• JobScheduler supports event handling using In Conditions and Out Conditions for jobs, see 

  Jira
  server SOS JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 6dc67751-9d67-34cd-985b-194a8cdc9602 key JOC-712

• Introducing job streams provides better support for backward dependencies with the JobScheduler. 
• The job streams bring a number of new objects to the user interface: 
  • EventsEvent
    • Events are tokens that are created on-the-fly to signal execution of jobs. Such Events can be used later on by successor jobs to check conditions that result in the current job being executed or not.
    • Global Events are the events that are created for the execution of jobs among any Master Cluster configuration. These are the tokens that are created on-the-fly to signal execution of jobs on another JobScheduler Master.
  • In ConditionsIn Condition
    • A job can have a number of In Conditions that control if the job will be executed. Therefore an In Condition includes a number of Expressions that are evaluated to become true or false. The result true will cause a job to start, the result false will not cause a job to start. Therefore a false result of an In Condition will put on hold the Job Stream, i.e. further execution of subsequent jobs.
    • Expressions in In Condition typically check the existence of Events. A number of Events can be evaluated including logical operators such as And, Or.
  • Out ConditionConditions
    • A job can have a number of Out Conditions that are checked after job execution. Out Condition include includes a number of Expressions and depending on the result of the check an Out Condition adds or deletes an Event that can be checked by In Conditions of later jobs.
    • Expressions in an Out Condition typically check the result of job execution, for example, checks the return code of job execution.
    Graphical representation:The
• A graphical representation of job streams Job Streams is available with JOC Cockpit to monitor job execution and to manage conditions and expressions..events.
  • Using the Job Streams view in JOC Cockpit offers three types of display
    • Image Added
    • The left symbol (highlighted) displays the graphical view that is used for the following explanations.
    • The middle symbol displays the card view.
    • the right symbol displays the list view similar to what you see from the Jobs view.

• For a good understanding of Job Streams, the user should

  • know

  The user should:

  • Know the technical terms like what is Job, Event and Job Chain, etc. and working of the JobScheduler and know how to work with JobScheduler Master and JOC Cockpit. 

  • Not not start with this article if he/she is you are new to the JobScheduler. 
  • Have the basic knowledge on how to create the jobs.

Motivation

• What is this feature good for? 
  • The end users can implement backward dependency for jobs using In Conditions, Out Conditions, write the expressions and can add or remove events. 
  • This feature allows graphical configuration of job dependencies. 
• What problem does it solve? 
• This feature was developed to add better backward dependencies to JobScheduler. 
  • Backward dependencies: The JobScheduler comes with a number of solutions to support backward dependencies for jobs and job chains. 
    • Check the job and job chain history for previously executed jobs and orders. 
    • Synchronize job chains. 
    • Job Streams. 
  • Forward dependencies: The JobScheduler comes with built-in features for implementing forward dependencies for jobs and orders. 
    • Basic dependencies are implemented by the Objects for the Jobs, Job Chains and Orders. 
    • Successor jobs can be made to depend on the code returned by the current job. 
    • Create successor orders for job chains depending on the return code of the current job chain node. 

Feature Availability

• •  In this case please start with the Product Knowledge Guides.
  • have basic knowledge of how to create jobs.

• Status
  subtle true colour Green title FEATURE AVAILABILITY STARTING FROM RELEASE 1.13.0

...

Objects used by Job Streams

The Job Streams include various feature feature introduces a number of objects:

• Jobs
  • Jobs are the basic unit for the processing of executable files (programs, scripts, commands, etc.).
  • Jobs are represented with a rectangular shape and grey background color. There is the The status of a job in is visible with the Job Stream. Each job status is represented with a different border color.
    • Pending - Yellow 
    • Running - Green
    • Stopped - Red

Image Modified

• Events

  • Events are tokens that are created on-the-fly to signal execution results of jobs. 
  • The JobScheduler supports two types of events:
    • Local Events
    • Global Events 
  • Events are represented by a hexagon. The An event has three statusthe following states:
    • Exist - If the event exists then the background color changes to grey and the border style depends on two factors:
      • Solid border → When the if the event is created by the a job in the same current Job Stream
      • Dotted Border → When if the event with the same name already exists in some other Job Streamis created by a job in another Job Stream than the current.
    • Not Exist -
    It
    • If the event does not exist then the background color is white.
      
      Image Modified
      
      

• Out ConditionConditions

  • Out Conditions include a number of Expressions that are evaluated after job execution and that result in true or false. If an Expression results in true then according to the Event List (Create Event or Delete Event) it creates and/or deletes the eventevents. If the Expression result is false then it puts on hold further execution of the job stream if these Events are used by In Conditions of successor jobs. 
  • Out Conditions are represented by right inclined parallelograms with a grey background color. The border color changes to green if the Expression results to true. The text color of the Expression also changes to green and black if the respective part of the Expression result is true or false.

Image RemovedImage Added

• In ConditionConditions

  In Condition 
  • In Conditions include a number of Expressions that can result in executing a Command to either start a job or to write a log file entry. If the Expression result is true then it executes the Command, and if the result is false then it does not execute the Command.
  • In Conditions are represented by left inclined parallelograms with a grey background color. The border color changes to green if the Expression results to true. The text color of the Expression changes to green and black if the Expression result is true or false respectively.
    

• Image Removed
   

• • 
    Image Added
    
    
  • In Conditions 
  • can be
  consumed and unconsumed.
  • consumed or not consumed:
  • The 
    • An 
    • In Condition is defined as being consumed when the job has once started by the result of In Condition. The consumed In Condition is represented by a white background color
    background
    • .
    The 
    • An In Condition is defined as
    unconsumed
    • being not consumed if it
    is
    • has not yet been used to start a job. The 
    unconsumed
    • not consumed In Condition is represented
    b
    • by a grey background color
    background
    • .

...

Image Added

• Off-Page-Connector

  • The Off-Page-Connector is an icon which is used to show the reference whether the defined In Condition or Out Condition is from the same Job Stream or is from whether there is a reference to another Job Stream then the current. The icon is represented by a pentagon at the top right corner of the In Condition or Out Condition.
  • The Off-Page-Connector is displayed on for an:
    • In Condition → When the out condition if an Out Condition that is not part of the current job stream creates an event that is consumed by an expression in the In Condition.
    • Out Condition → if an In Condition Condition that is not part of the actual current job stream .Out Condition → When the In Condition consumes an event that is created by an the Out Condition that is not part of the actual job stream.

Implementation 

Image Added

Implementation 

• Getting Started with Job Streams

• Job Streams View GUI Representation

  How to ...... Articles 

• How to enable Job Streams functionality

• How to add or edit Conditions to a job

• How to

  add or

  add or remove an Event from a Job Stream

• How to import and export Job Streams

• Valid

  Expressions for Conditions in Job Streams

  Import Export Job Streams

• Global Events

Use Cases

Use Case 1: Implement

...

simple Backward

...

Dependencies

• A simple Job Stream to implement a linear Backward Dependency may could look like this:

Image AddedImage Removed

Explanation

• The name of the Job Stream is job_stream1.
• The Use-Case job_stream1 is used to show a linear backward dependency.
• Here when the job1 runs it creates an event job1. The job2 gets executed automatically as it has an In Condition which checks whether the event job1 is created or not.
• Similarly the jobs job3 and job4 are started when the event job2 and job3 are created.
  

Use Case 2: Implement

...

the Split & Join Workflow Pattern

A simple Job Stream to implement the split & join workflow pattern might could look like this:

Image RemovedImage Added

Explanation

• The name of the Job Stream is test.
• The Job Stream includes two split operations, at first job1 is split into job2 and job3, and then job3 is split into job4 and job5.
• The job1, when executed if the Out Condition results to true then the Event job1 is created.

...

Use Case 3: Implement a Job Stream across Folders

A simple An example to implement a job stream across folders and Job Streamswith jobs from different folders.

Use Case 4: Use Job Streams for Job Monitoring

...

• The running job can be monitored in two ways:
  1. The border color of the job becomes green.
  2. The Task will be listed in the Running Task list and will be removed automatically once the job completed.

Image Added

• The Task Queue tab is used to monitor the tasks that are queued for future execution:

Image Added

• The Task History tab is used to monitor the history of tasks including information about successful or failed executions:

Image Added

Explanations

• A simple Job Stream to monitor the pending tasks may look like this:

Image Removed

Explanation

• The name of the Job Stream is js1.
• The Job Stream includes two jobs, " job1 " and " job2".
• The job1, when executed if and completed causes the Out Condition results  to result to true then and to create the Event eve1 is created.
• After that, if the In -Condition of the job2 results Condition of job2 results to true then it will set the job2 into to the pending state as the In - Condition of the job2 is configured with a for the command "startjob" with the parameters parameter "now + 3600".
• Therefore, the job2 will wait for it's start time to get execute after 3600 seconds later.
• During this period, the task will be shown displayed in the Task Queue list and will get be removed automatically once the job2 will be executedonce job2 will start.
• In this way, the tasks can be monitored whether it's in running state, pending state, got to be running, queued, failed or completed successfully.

...

S.No.	Job Name	Condition	Expression	Command
1.	job1	In Condition	-	-
		Out Condition	rc:0	eve1
2.	job2	In Condition	eve1	startjob: now + 3600
		Out Condition	rc:0	eve2

Roadmap

• This The Job Streams feature is subject to future improvement:

  • Done: User can add and remove Events from the graphical representation. 

  • Add We will add editing capabilities for jobs and job-related objects in order to replace the JOE 

• There will be a new tab named as "Job Streams" in the JOC Cockpit which will replace the JOE in the future. 

• This "Job Streams" contains the working and explanation of the Jobs and Job Chains. 

• User can create a Job by just drag and drop.  

• •  Job Editor.

  • The XML Editor will be migrated to JOC Cockpit.
  Provide export functionality to display (print) job streams in PDF, PNG
  •  

References

• Links to other articles
  • How to split and sync jobs in a job chain
  Change Management System
  Links  to JIRA

  • Jira
    server SOS JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 6dc67751-9d67-34cd-985b-194a8cdc9602 key JOC-712

Further Resources