Introduction

• Consider the wording: a Controller can be operated as a Standalone Controller instance or as a Controller Cluster. The term Controller instance refers to the standalone instance or to a Controller Cluster member instance.
• The outage of a Controller instance will not stop the execution of workflows with Agents.
• However, if a workflow includes jobs that are executed with different Agents then the workflow will not be completed and will be put on hold as switching of Agents during workflow execution is performed by the Controller.
• Testing by SOS includes to perform tests for the scenario when a Controller is not available for 24 hours and the Agent executes any scheduled orders. Once the Controller instance is started again then job execution results are updated to the JOC Cockpit History and become visible with the GUI.
• For information about the behavior in case of outages see JS7 - FAQ - What happens to workflows in case of outage of a Controller?

Controller Cluster

If you operate a Controller Cluster then an automated fail-over takes place should the active Controller instance fail. A fail-over typically occurs within 3-5s. Should the standby Controller instance fail then this does not affect the active Controller instance. Running a JS7 high availability cluster gives you the relaxed option not to have to take immediate action if one of the instances fails. However, should you intend to immediately make available the failed Controller Cluster member instance then the steps explained below similarly apply to the failed instance.

The below troubleshooting hints are intended for users operating a Standalone Controller , the steps explained are not required for users operating a Controller Cluster.

Troubleshooting

The Controller is the component in JS7 that holds the job-related configurations and orchestrates the Agent. The outage of a Controller instance does not prevent execution of workflows having the jobs running on a single Agent. However, it affects execution of workflows that include jobs that are running on a number of Agents as switching of Agents during workflow execution is performed by the Controller. 

A Controller instance outage can be handled either by resolving the issue with the current Controller instance, e.g. by restarting, or by relocating the ./state directory to a new Controller instance. The journal of a Controller instance is stored in the ./state directory. To relocate a Controller instance to a new Controller instance location copy the journal files to the target Controller instance.

Relocating the Controller Instance's Journal

If ControllerInstance1 is facing an outage then follow the below steps to relocate the Controller instance's journal from ControllerInstance1 to a new ControllerInstance2:

1. Install ControllerInstance2. 
2. Copy the files from the ./state folder of ControllerInstance1 to the respective folder of ControllerInstance2. 
3. Start ControllerInstance2.
4. Consider that the Controller URL is not the same for ControllerInstance1 and ControllerInstance2. Therefore the URL has to be updated in the JOC Cockpit.
5. To change the Controller URL, log in to the JOC Cockpit.
6. From the main menu select the item "Manage Controllers/Agents".
7. Make sure you edit the existing ControllerInstance which is not in service.
8. Modify the "URL for JOC Cockpit" from the "Register Controller" dialogue box for the Controller instance that is not in service to point to the ControllerInstance2 URL.
9. When workflows are confirmed to work with ControllerInstance2 then drop the contents of the ./state directory of ControllerInstance1. This an important step as otherwise, when ControllerInstance1 is started it will forward past orders from its journal to available Agents. This would result in double job execution that might be harmful depending on the nature of your jobs.
   
   
   
   

Notes:

• If you use HTTPS connection for Controller instances then consider that ControlerInstance2 might need its own server authentication certificate. 
  • If ControllerInstance1 is operated for HTTPS then you can modify the protocol to HTTP and to point to a different host provided that this is in line with your security requirements.
  • The same applies vice versa if ControllerInstance1 is operated for HTTP and ControllerInstance2 is operated for HTTPS.
• If you intend to roll back from ControllerInstance2 to ControllerInstance1 then consider to apply the above steps respectively.