JS7 JobScheduler

Space shortcuts

Page tree

Introduction

  • The system architecture describes:
    • system boundaries
    • products
    • platforms
    • setup scenarios

System Architecture

High Level Architecture

JS7 includes the following products:

  • The JOC Cockpit, which is used to manage the JS7 - Inventory of scheduling objects, to monitor and control workflow execution and to access the JS7 - History.
  • The Controller, which is used to orchestrate Agents, to deploy workflows and to pass orders for workflow execution to Agents.
  • Agents, which are dedicated to a Controller and are used to execute workflows and jobs. Agents can act autonomously for workflows that execute all included jobs with the same Agent.

JS7 products can be clustered:

  • JOC Cockpit and Controller can each implement a passive cluster with an active node and a standby node.
  • Agents can be operated standalone or as instances of a JS7 - Agent Cluster which implements active-passive or active-active scheduling.

Products

The products interact as described in the slide below:

  • The JOC Cockpit implements:
    • the user interface for browser access.
    • the JS7 - REST Web Service API which:
      • is connected to a database and which persistently stores the inventory and the history of workflow executions,
      • performs all operations for managing Controllers,
      • optionally is available from dedicated API Servers.
  • The Controller connects to Agents:
  • The Agents:
    • execute workflows and jobs,
    • return execution results and log output to the Controller.

Network Connections

Network connections are only established in the directions indicated.

  • Any connections are unilateral.
  • Connections can be secured by TLS/SSL certificates.

Cluster Architecture

The JS7 offers a 4-tier architecture with any JS7 products available for clustering, for details see JS7 - Cluster Architecture.

  • Each tier acts as a Cluster Watch in case that the underlying tier requires an arbitrator to decide about the active/standby role.
  • Each tier works asynchronously and does not rely on availability of the next tier. This includes absence of any single point of failure.
  • The Agent Cluster is represented from both a Director Agent and any number of Subagents that execute tasks.
    • This provides better autonomy to the Agent Cluster as Director Agents can be operated more close to Subagents in the network.
    • Agent Clusters can execute workflows autonomously provided that jobs and dependencies map to Agents in the same cluster.

Setup Scenarios

Consider attached slides JS7_JobScheduler_System_Architecture.pdf for further setup scenarios.

Cloud Setup

A high availability setup includes to cluster JOC Cockpit, Controller and Agents. Products are deployed to separate availability zones like this:


A hybrid approach suggests to use a cluster of JOC Cockpit instances and Controller instances from the Cloud. Agents can be integrated from the Cloud and from environments on premises:

On Premises Setup

A cluster of JOC Cockpit instances and Controller instances can be used like this:


Standalone Agents can be used and clustered Agents can be used for high availability like this:

Summary

  • Products
    • JOC Cockpit: manages the workflow and job inventory, monitors and controls workflow execution
    • Controller: orchestrates Agents, manages workflow execution
    • Agents: start workflows autonomously and return the execution history and log output
  • Operation
    • JOC Cockpit can be operated in the following modes:
      • as a standalone instance,
      • as a cluster with one active instance and any number of standby instances.
    • The Controller can be operated:
      • as a standalone instance,
      • as a cluster with one active instance and one standby instance.
    • Agents can be operated:
      • as a single instance,
      • as a passive cluster with one active instance and any number of standby instances,
      • as an active cluster with any number of active instances.
  • Network Connections
    • JOC Cockpit connects to Controller instances and connects to a database
    • Controllers connect to Agents
    • Network connections are firewall friendly, a single connection is established between each product
  • Platforms
    • JOC Cockpit: Container, Linux, Windows
    • Controller: Container, Linux, Windows
    • Agent: Container, Linux, Windows, any platform that provides a Java Virtual Machine
  • Setup Scenarios
    • Cloud
      • JOC Cockpit and Controller High Availability
      • Agent High Availability
      • Hybrid Use of Agents
    • On Premises
      • Standalone Server
      • Controller High Availability
      • Controller and JOC Cockpit High Availability
      • Multi-Client Capability
      • Agent High Availability

Resources



  • No labels