Connection Keep-Alive for Master and Agent

Scope

• JobScheduler Master and Classic and Universal Agents can maintain network connections by regularly sending keep-alive packets. This prevents connections across firewalls being broken if a long-running job should exceed allowed timeouts of firewalls or proxy servers.
• FEATURE AVAILABILITY STARTING FROM RELEASE 1.9

Related Issues

JS-1454 - Getting issue details... STATUS

JS-1456 - Getting issue details... STATUS

JS-1506 - Getting issue details... STATUS

Use Case

• Welche Rolle ist für den Fall verantwortlich? Wer will etwas? Engineering, Operating, Business
• Was ist das Ziel? Was ist das Mittel? Was ist die Motivation?

• Increasing awareness of security issues means that an increasing number of firewalls and proxy servers are to be found, for example within company intranets, where they can form part of the infrastructure used to create so-called network islands.
• This means that the interception of idle network connections between JobSchedulers - in particular between JobScheduler Masters and Agents - by firewalls or proxies can start to represent a significant problem.
• Idle connections occur when, for example, a long-running job is being processed by an Agent and the job does not perform any API calls, i.e. it does not create log output from time to time, meaning that data is not sent over the network.
• The feature described here uses so-called keep-alive packets that are sent across network connections at regular intervals
• If a long-running job does not perform any API calls, e.g. does not create log output from time to time, data will not be sent over the network and the connection may be intercepted - either politely or impolitely.
  • Polite interception means that the firewall sends a connection reset packet, impolite interception means that the firewall sends a drop packet. VERIFY WORDING

Implementierung

• Wie ist die Lösung implementiert? Mit welchen Mitteln?

• JobScheduler classic Agent can be configured to send a small keep-alive packet to JobScheduler Master to prevent connections from timing out by adding the following parameter to the <params> section of scheduler.xml:
  JobScheduler Master can be configured to send an empty command to the JobScheduler Agent to prevent connections from timing out by adding the following parameter to the <params> section of scheduler.xml:

  keep-alive parameter

  <params>
      <param name="scheduler.agent.keep_alive" value="300"/>
  </params>

• The value attribute sets the interval in seconds between keep-alive packets.

  Keep-alive packets will not be sent if the parameter is not set or if the value attribute is empty.

• The Master log shows the message SCHEDULER-727 Keep-alive package sent to Agent at info level that a keep-alive command has been successfully sent.
  • A duration lower than 30s is silently replaced by 30s.

Master / Agent Feature Compatibility

• Master sends keep-alive commands to Classic Agents (including and before Agent release 1.9) via TCP connections.
• Master and Universal Agent use only one connection (HTTP, HTTPS). As the Agent (starting with release 1.10) sends keep-alive packets, see JS-1456, no keep-alive packets have to be sent by the Master.

Workarounds for Older Versions

The idle connection can also be overcome by:

• regularly calling API methods (only in jobs using the JobScheduler's API),
• regularly logging to stdout (every log line to stdout is a log API call as well and makes use of the TCP connection),
• using one of the heartbeat scripts provided below.

Heartbeat script for Linux Agents

• Store the following script heartbeat.sh in the Agent's file system:

  Heartbeat script for Linux Agents

  trap 'kill $(jobs -p)' EXIT
  /path/to/heartbeat.sh &
  
  #your scriptcode here...

• Configure long running job scripts to run the heartbeat script.

Heartbeat script for Windows Agents

• Store the following heartbeat.cmd script in the Agent's file system:

  Heartbeat script for Windows Agents

  start /b C:\temp\heartbeat.cmd
  			
  @rem your scriptcode here...

• Configure long-running job scripts to run the heartbeat script.
• Don't use quotes in the start command, e.g. start /b "c:\my scripts\heartbeat.cmd".

Delimitation

• Abgrenzungen, was ein Feature nicht leistet, z.B. nur für bestimmten Plattformen verfügbar
• Funktionalität, die eben nicht bereitgestellt wird
• Limitation (technische Beschränkungen)
• Keep-alive packets are not created if Remote File Watching is performed by the Agent.
• Keep-alive packets are sent exclusively for running jobs.

Example

• Beispiel Code und Erläuterungen

  • Code Beispiel via Confluence Code Macro

• Beispiel Objekte

  • Ganze Objekte wie jobs, orders etc nur als Anhang

Change Management References

References

• Change Management References

  • JIRA Issues

• Documentation

  • XML Element in der Referenzdokumentation