JOC Cockpit - Audit Log

Introduction

The Audit Log allows compliance and operations reporting requirements for job scheduling activities to be met by providing an automatic and secure record of all actions that users of the JobScheduler Web Service - including JOC Cockpit users and external applications - perform on job-related objects. The Audit Log lists the relevant JobScheduler objects - Jobs, Job Chains, Orders, etc. - and any actions that modify their status.The Audit Log is written by the JobScheduler Web Service as shown in the JOC Cockpit - Architecture page.

Scope of the Audit Log

The Audit Log is maintained for all job-related objects such as Jobs, Job Chains, Orders, etc., whose statuses are modified by users carrying out actions such as starting, stopping or pausing an Order or Job. It uses a format that is human and machine readable and fulfills the following main functions:

• provide a compliance-conform record in a simple file format that can be archived and handed over to a compliance officer,
• provide operational records of job scheduling activities for reporting purposes,
• provide an operational record that is readily accessible to operating staff,
• provide a second source of operational information if the database used to store operational records is not available.

Users of the JOC Cockpit with the necessary permissions can view the Audit Log in the JOC Cockpit by clicking the Audit Log link in the JOC Cockpit main menu as shown in the screenshot below. This view and, in particular the Reasons, Time Spent and Ticket Link information, is intended to provide users with more relevant operational information than is available from the History.

The Audit Log contains information about changes to JobScheduler objects initiated from the JOC Cockpit and by other applications using the JobScheduler Web Services. It does not include changes made to objects using the JobScheduler Object Editor, JOE, or changes made directly to objects on the file system using a text editor. 

Compliance, Reporting and Operational Features

• The Audit Log is written automatically by the JobScheduler Web Services:
  • Entries to the Audit log are made when an action that changes the state of a JobScheduler Object is initiated in the JOC Cockpit or other application accessing the JobScheduler Web Services.
  •  There is no mechanism for switching off the Audit Log.
• The Audit Log is written in two locations simultaneously:
  • In the DBMS used by the JobScheduler Web Services: 
    • This instance is used by the JOC Cockpit to allow operators to view the Audit Log information.
    • It is also used by the JOC Cockpit to generate a Microsoft Excel^®-compatible export file for reporting purposes.
  • In the Audit Log file:
    • It is intended that this instance is archived and used as a compliance record. System Administrators can make a copy of this instance available to operating staff if the DBMS should not be available. 
    • The Audit Log file is named JOCAuditLog.log and saved in the ./logs folder.
• All new entries will be added to the Audit Log - neither the log file not the DMBS records are overwritten.
  • System administrators have to arrange an archiving and housekeeping system for the Audit Log file according to relevant compliance laws and corporate governance.
  • It is the responsibility of system administrators to check compliance requirements such as the retention period for the information contained in the Audit Log file.
• The Audit Log is visible in the JOC Cockpit to:
  • Users with the appropriate permissions. The specification of user permissions is described in the Authentication and Authorization - Configuration article. 
    
    • The default authorization permissions for the Audit Log which are configured for the JOC Cockpit are listed in the Permissions Matrix shown in the Authentication and Authorization - Permissions for the JOC Cockpit Web Service article.  
  • System Administrators with access to the DBMS used by the JOC Cockpit or the file system where the Audit Log file is written.
• The Audit Log is read-only for all JOC Cockpit users - only System Administrators that have access to both the file system and to the database can modify the log. 

Key to the Audit Log Entries

• Created: the point in time when the action was performed.
• Account: the user account that caused the action.
• Request URL: the Web Service URL that was invoked by the JOC Cockpit. The URLs represent the object type and action, e.g. /orders/start.
• Job Chain, Order ID, Job: identifies the object for which the action was performed.
• Reasons: (can be mandatory or optional - see below) an explanation of why an action was carried out. Reasons are created either by selecting an item from a list of predefined reasons (configurable) or individual - i.e. free text input. A maximum of 2000 characters can be added to a free text reason.
• Time Spent: (optional) the number of minutes required for the action (including time for impact analysis) can be added.
• Ticket Link: (optional) a URL can be added that points to a ticket system, e.g. the SOS Ticket System, to identify the underlying ticket for the intervention.

Action Notes

Users carrying out actions are able to add an explanatory notes for each action. These notes are entered as part of the specification of an action and then saved in the Audit Log. They can then be read by all other JOC Cockpit users with the necessary permissions. The form for the 'Start Order At' action is shown as an example in the following screenshot:
 

Action Notes are made up of three elements as shown in the screenshot above:

• Reasons (mandatory or optional)
  • Predefined Reasons which are selected from a drop-down list and 
  • Individual Reasons are possible which are entered in a text field
• Time Spent (always optional):
  • in minutes
• Ticket Link (always optional):
  • has to start with http://

Mandatory and Optional Reasons

Mandatory Reasons

The  force_comments_for_audit_log flag can be set in the joc.properties file as shown in the following code box:

### This flag controls if the comment is required or not.
force_comments_for_audit_log = false

The joc.properties file is located in the ./joc_home/jetty_base/resources/joc directory. 

The default setting for force_comments_for_audit_log is false .

Either predefined or individual reasons can be set when force_comments_for_audit_log = true is set.

Optional Reasons

Users can configure in their profiles whether they wish to be offered the option of entering reasons for the actions they carry out. This is shown in the next screenshot:

If this option is not set and reasons are not mandatory, which is the default setting, then users will not be presented with the Enter a Reason function when the Start Job At or Start Order At functions.

Behavior of Start Order now and Start Task now 

The behavior of the Start Order now and Start Task now functions depends on whether or not Action Notes are enabled in the user's Profile setting and whether Action Note Reasons are mandatory or optional.

Predefined Reasons

A predefined set of reasons is delivered with the JOC Cockpit.

Predefined reasons are found in the ./joc_home/jetty_base/resources/joc directory, in the joc.properties file. 

The predefined reasons are listed in the following code box:

### Pre-defined comments
comments =  System maintenance; \
            Repeat execution; \
            Business requirement; \
            Restart failed execution; \
            Re-instantiate stopped object; \
            Temporary stop; \
            Change of JobScheduler object; \
            Rerun with parameter changes; \
            Change of external dependency; \
            Application deployment and upgrade

Predefined Reasons Syntax

• The ";" near the end of each line marks the end of the comments. The semicolon here functions as a delimiter and allows commas to be used in reasons.
• The "\" at the marks the end of a line.

Modifying Reasons Settings

System administrators are free to modify predefined reasons and the force_comments_for_audit_log flag to suit their own requirements.

The JOC Cockpit needs to be restarted before changes made to the joc.properties file will take effect.

Individual Reasons

Individual Reasons can be entered in the text field as shown in the screenshot above. 

UTF-8 characters are allowed.

Audit Log View

The arrow at the left hand side of each entry in the Audit Log view table allows more detailed information about the request submitted to the JobScheduler Web Services. The Request Body row shown in the following screenshot includes the Job Scheduler which the Order is to be processed on, as well as the Order Parameters and starting time (processing starting from State 20, starting immediately, etc),   

The Advanced Filter

In addition to the preset All, Today, Last hour, etc. filters listed at the top of the Audit Log view there is an Advanced Filter function which is opened with the Advanced Filter button at the top right of the view. Clicking this button opens the Advanced Filter function. Note that the preset Filters (All, Today, Last hour, etc.) work in parallel with the Advanced Filter. This means that if the Today preset filter is selected and then From 09:00:00 To 18:00:00 then all Audit Log entries started between 09:00 and 18:00 will be shown for all the days covered by the Audit Log.

The Search Button

The Search button allows a search of the results of the preset or Advanced Filters to be carried out. 

Audit Log Location

The Audit Log file is found in the ./joc_home/jetty_base/logs directory, in the JOCAuditLog.log file. If the file does not exist, it will be created with the first action modifying the status of an object that is carried out by a user.

Audit Log Sample

The following sample shows a number of Web Service requests that have been performed by two users aa_ito and root, who logged in at 14:13:47 using a different browser before aa_ito logged out:

2017-02-10 11:39:55,992 INFO  REQUEST: ./orders/add - USER: aa_ito - PARAMS: {"jobschedulerId":"jobscheduler_1.11","orders":[{"jobChain":"/tutorials/hello_world_job_chain","at":"now"}]} - COMMENT: Demo start for documentation
2017-02-10 11:43:05,831 INFO  REQUEST: ./jobs/start - USER: aa_ito - PARAMS: {"jobschedulerId":"jobscheduler_1.11","jobs":[{"job":"/standalone-jobs/test_sa_job1","at":"now"}]} - COMMENT: Demo start for documentation
2017-02-10 11:54:11,731 INFO  REQUEST: ./jobs/start - USER: aa_ito - PARAMS: {"jobschedulerId":"jobscheduler_1.11","jobs":[{"job":"/standalone-jobs/test_sa_job1","at":"now"}]} - COMMENT: null
2017-02-10 12:38:31,507 INFO  REQUEST: ./login - USER: aa_ito - PARAMS: {} - COMMENT: it_operator
2017-02-10 12:48:39,857 INFO  REQUEST: ./orders/start - USER: aa_ito - PARAMS: {"jobschedulerId":"jobscheduler_1.11","orders":[{"orderId":"hello_world_order","jobChain":"/tutorials/hello_world_job_chain","at":"now"}]} - COMMENT: null
2017-02-10 13:34:57,995 INFO  REQUEST: ./login - USER: aa_ito - PARAMS: {} - COMMENT: 
2017-02-10 13:35:03,391 INFO  REQUEST: ./login - USER: aa_ito - PARAMS: {} - COMMENT: it_operator
2017-02-10 13:35:49,186 INFO  REQUEST: ./orders/start - USER: aa_ito - PARAMS: {"jobschedulerId":"jobscheduler_1.11","orders":[{"orderId":"hello_world_order","jobChain":"/tutorials/hello_world_job_chain","at":"now"}]} - COMMENT: null
2017-02-10 13:36:13,905 INFO  REQUEST: ./jobs/start - USER: aa_ito - PARAMS: {"jobschedulerId":"jobscheduler_1.11","jobs":[{"job":"/standalone-jobs/test_sa_job1","at":"now"}]} - COMMENT: null
2017-02-10 13:53:50,184 INFO  REQUEST: ./orders/add - USER: aa_ito - PARAMS: {"jobschedulerId":"jobscheduler_1.11","orders":[{"jobChain":"/tutorials/hello_world_job_chain","state":"20","at":"now"}]} - COMMENT: Test Start from step 20
2017-02-10 13:56:05,459 INFO  REQUEST: ./orders/add - USER: aa_ito - PARAMS: {"jobschedulerId":"jobscheduler_1.11","orders":[{"jobChain":"/tutorials/hello_world_job_chain","at":"now"}]} - COMMENT: null
2017-02-10 14:02:30,467 INFO  REQUEST: ./login - USER: aa_ito - PARAMS: {} - COMMENT: it_operator
2017-02-10 14:09:38,471 INFO  REQUEST: ./orders/add - USER: aa_ito - PARAMS: {"jobschedulerId":"jobscheduler_1.11","orders":[{"jobChain":"/tutorials/hello_world_job_chain","state":"20","at":"now"}]} - COMMENT: Repeat execution
2017-02-10 14:10:36,839 INFO  REQUEST: ./orders/add - USER: aa_ito - PARAMS: {"jobschedulerId":"jobscheduler_1.11","orders":[{"jobChain":"/tutorials/hello_world_job_chain","state":"20","at":"now"}]} - COMMENT: Repeat execution
2017-02-10 14:13:47,553 INFO  REQUEST: ./login - USER: root - PARAMS: {} - COMMENT: all
2017-02-10 14:14:48,351 INFO  REQUEST: ./orders/add - USER: root - PARAMS: {"jobschedulerId":"jobscheduler_1.11","orders":[{"jobChain":"/tutorials/hello_world_job_chain","state":"20","at":"now"}]} - COMMENT: Repeat execution

Audit Log Explanation

• The Audit Log includes the Web Service URL, which allows the object type and operation to be identified. For example, ./orders/resume translates to the action of resuming an order that is identified by its order ID and Job Chain path, as specified in the PARAMS part of each log entry.

Audit Log Export

The Audit Log Export function, which is activated with the button with the arrow shown in the next screenshot, allows the Audit Log to be exported in a reporting-friendly format either in whole or filtered.

The filtered table as shown in Audit Log view is exported as a Microsoft Excel^® compatible .xls file. The next screenshot shows the file opened in Open Office Calc.

Note that the PARAMS part of records (see the Audit Log Sample section above) are not included in the export.

Audit Log Housekeeping & Archiving

As already noted the JOC Cockpit adds to the Audit Log on an ongoing basis.

System administrators need to implement their own housekeeping / log rotation and archiving system according to their business and compliance requirements.