Introduction

Multiple JOC Cockpit instances can be operated as an active cluster, in conjunction with a load balancer, to achieve load balancing and high availability. Alternatively, redundancy can be achieved by installing several JOC Cockpits on separate servers "in parallel".

JOC Cockpit Clusters

FEATURE AVAILABILITY STARTING FROM RELEASE 1.12.2

With this configuration, multiple instances of the JOC Cockpit are installed on separate servers and configured to use the same reporting database. In addition, session sharing between the JOC Cockpit instances is activated with user sessions being stored in the database instead of in the JOC Cockpit instance. In addition the Shiro authentication and authorization information is shared between installations.

Fail-over is handled by a load balancer, which is placed in front of the JOC Cockpit servers. The load balancer handles all user requests via a single URL and ensures that the fail-over is transparent. Clustering also allows the HTTPS protocol to be used for communication between the user's browser and the load balancer.

Note that the load balancer has to be configured to forward session cookies when requests are forwarded from one JOC cockpit instance to another.

Cluster Configuration

Clustering is configured as follows:

  • Install two or more instances of the JOC Cockpit on separate servers.
  • All JOC Cockpit servers must use the same reporting database.
  • Add the following two lines of configuration items to the [main] section of the Shiro configuration for one of the instances:

    sessionDAO = com.sos.auth.shiro.SOSDistributedSessionDAO
    securityManager.sessionManager.sessionDAO = $sessionDAO

    This can be done most easily with a User Account that has the necessary permissions and using the Enable JOC Cluster button in the Manage Accounts / Main Section tab of the JOC Cockpit interface. See the Authentication and Authorization - Configuration article for more information about the Shiro configuration.

  • Connect a load balancer to all JOC Cockpit instances. The load balancer offers a single URL to all users and will forward any requests to one of the JOC Cockpit instances.
    • Consider that the load balancer should forward any headers of a request including session cookies.

Cluster Testing

You can test whether the cluster is working correctly with the following script:

This test is not related to use of a load balancer but checks if session handover works between two JOC Cockpit instances.

An example call for the script would be:

 ./joc_cockpit_cluster_check.sh -imy_server_4446 -uhttp://host1:4446 -rhttp://host2:4446 -aroot:root -t10

  • Arguments

    • -i: JobScheduler ID for the respective environment

    • -u: URL of JOC Cockpit on host1

    • -r: URL of JOC Cockpit on host2
    • -a: Account and Password for login separated by a colon ":"
    • -t: timeout
  • Explanations
    • The test script can be executed on any Unix host without pre-requisites (no JobScheduler installation required) as it makes use of the JOC Cockpit REST API.
    • The test script performs a login with JOC Cockpit on host1 (-u) and the specified user account (-a) for the specified JobScheduler instance (-i) and then sends a subsequent request to JOC Cockpit on host2 (-r) and performs a logout operation with JOC Cockpit on host2. If this works well, then the session handover between JOC Cockpit on host1 and JOC Cockpit on host2 is confirmed.
  • Sample Output of the Test Script
    • Sample output
      ACCESS_TOKEN=e182f136-0e4f-4103-885d-7186380a6f4d
      ----------------
      REST Web Service call for URL http://host1:4446 with access_token e182f136-0e4f-4103-885d-7186380a6f4d
      ----------------
      HTTP/1.1 200 OK
      Content-Type: application/json
      Cache-Control: no-cache, no-store, no-transform, must-revalidate
      Content-Length: 348
      Server: Jetty(9.3.11.v20160721)
      
      {"deliveryDate":"2018-03-26T08:24:55.292Z","jobscheduler":{"clusterType":{"_type":"PASSIVE","precedence":0},"host":"host1","jobschedulerId":"my_server_4446","port":40444,"startedAt":"2018-03-23T13:03:28Z","state":{"_text":"RUNNING","severity":0},"surveyDate":"2018-03-26T08:24:55.288Z","url":"http://host1:40444"}}
      ----------------
      REST Web Service call for URL http://host2:4446 with e182f136-0e4f-4103-885d-7186380a6f4d
      ----------------
      HTTP/1.1 200 OK
      Content-Type: application/json
      Cache-Control: no-cache, no-store, no-transform, must-revalidate
      Content-Length: 348
      Server: Jetty(9.3.11.v20160721)
      
      {"deliveryDate":"2018-03-26T08:24:55.967Z","jobscheduler":{"clusterType":{"_type":"PASSIVE","precedence":0},"host":"host1","jobschedulerId":"my_server_4446","port":40444,"startedAt":"2018-03-23T13:03:28Z","state":{"_text":"RUNNING","severity":0},"surveyDate":"2018-03-26T08:24:55.959Z","url":"http://host1:40444"}}
      ----------------
      Executing Logout
      ----------------
      HTTP/1.1 200 OK
      Content-Type: application/json
      Cache-Control: no-cache, no-store, no-transform, must-revalidate
      Content-Length: 115
      Server: Jetty(9.3.11.v20160721)
      
      {"enableTouch":true,"user":"root","accessToken":"","hasRole":false,"isAuthenticated":false,"isPermitted":false}


  • Checks

    • Check the file ./logs/JOCAuditLog.log of JOC Cockpit on host1 for an entry like this:
      • 2018-03-26 10:24:55,203 INFO  REQUEST: ./login - USER: root - PARAMS: {} - COMMENT: all - TIMESPENT: - - TICKET: -

        This entry confirms that a login operation for the specified account has been performed with JOC Cockpit on host1.

    • Check the file ./logs/JOCAuditLog.log of JOC Cockpit on host2 for an entry like this:
      • 2018-03-26 10:24:55,983 INFO  REQUEST: ./logout - USER: root - PARAMS: {} - COMMENT:  - TIMESPENT: - - TICKET: -

        This entry confirms that a logout operation for the specified account has been performed with JOC Cockpit on host2.

Redundant Installation of the JOC Cockpit

FEATURE AVAILABILITY STARTING FROM RELEASE 1.11

A number of JOC Cockpit instances running on separate servers can be used to achieve a degree of operational redundancy. As with a cluster, the JOC Cockpit instances are configured to use the same reporting database. However, each instance is addressed with its own URL and "fail-over" will involve the user manually changing the URL used to access the JOC Cockpit and re-logging in.

Installation of redundant JOC Cockpit instances follows standard installation procedures described in the JOC Cockpit - Installation article.