Problem

Users might observe a situation where orders seem to have travelled through time - showing a more recent start time than end time in the JS7 - Order History:

Analysis

A possible reason for non-matching timestamps could lie in the fact that the servers involved in job scheduling do not use a the same system time.

• Typically in Unix and Windows environments a time service such as NTP is used to synchronize a server's clock from an external time server.
  • If the time service is not operational for some time then a server's clock will become more and more inaccurate.
  • Small delays of milliseconds can add up to a number of seconds and more after some time.
• The problem starts with a situation where servers used for the Controller and Agents no longer have their clocks synchronized:
  • When an order is added then the Controller knows the start time that an order is scheduled for. However, it's the Agent that reports completion of the last job in a workflow as the order's end time. JS7 - Resilience requires that an Agent can act autonomously and can report back the end time of an order. In a situation when the Agent cannot be contacted by its Controller, for example in case of network outages, the Agent will report back to the Controller at a later point in time. 
  • If the Controller's clock is behind the Agent's clock then an order that requires less time than the delay between both servers' clocks:
    • will show with the Order History a start time that is behind the end time,
    • will - even more confusingly - show non-matching timestamps in the order log.
  • Even in cases of smaller delays the results displayed in the Order History will not be accurate.
• This does not apply to start times and end times of jobs which are reported back by the same Agent.

Resolution

• It is essential that the servers used for JOC Cockpit and Controller use synchronized time.
  • In case of a JS7 - Controller Cluster the servers involved have to use synchronized clocks.
  • The same applies for the server operating the Cluster Watch Agent.
• It is recommended that synchronized clocks are used for servers which are operating Agents.
  • However, except for the confusion from logs and display of the history there will be no detrimental effect in the event of non-matching system times.

...