Skip to end of metadata
Go to start of metadata

Purpose

  • The Cleanup Service is used to purge historic data that is no longer required. This includes data from the JS7 - History, the JS7 - Daily Plan history, the JS7 - Deployment history and the JS7 - Audit Log
    • The execution history and logs of orders and tasks is purged.
    • The history of submissions to the Daily Plan is purged.
    • The deployment history is purged.
    • The audit log is purged.
  • A purge of historic data is performed in order not to fill up the JS7 - Database and to guarantee overall performance at an ongoing basis, see JS7 - Database Maintenance.
  • For organizations that are subject to compliance requirements, different retention policies might apply e.g. when it comes to the period for which task logs and audit log entries should be stored. In this situation consider the default values mentioned below for operation of the Cleanup Service and adjust them to your needs. This includes the option not to use the Cleanup Service and to manually purge the JS7 database.

Cleanup Order and Task History

When an order is launched for a workflow then this will create an order log and a task log. The order log includes any output of jobs and instructions in a workflow, the task log is available for each job individually. 

Therefore the number of records in the order and task history depends on the number of workflows and jobs executed in a given period. The size of historic records depends on the size of output that is logged by your jobs.

Cleanup Daily Plan History

The JS7 - Daily Plan Service submits orders to any connected Controllers. The history of submissions allows tracking of the creation of orders, it is not related to the order and task history.

It is considered safe to purge entries from the Daily Plan History after orders have been completed.

Cleanup Deployment History

The Deployment History stores any objects that are subject to deployment, i.e. that have been forwarded to a Controller. The Deployment History allows tracking of the lifecycle of an object and allow re-deployment of objects from an older state.

It is safe to purge deployment history entries if you do not rely on falling back to an earlier version of an object, e.g. of a workflow.

The purge considers the number of versions per object that should be retained.

Cleanup Audit Log

The JS7 - Audit Log holds information about any operation performed which changes the state of an object, e.g. deploying objects, adding orders, etc. Depending on the number of deployments and interventions related to order execution, the audit log can grow to some considerable size.

Organizations which are subject to compliance requirements should check if retention periods before purging the audit log meet their needs.

Configuration

Location

  • The "Settings" menu is available from the JOC Cockpit GUI.

Configuration Items

SectionSettingDefaultRequiredPurpose
cleanuptime_zoneUTCnoSpecifies the time zone that is applied to the start time and period of the Cleanup Service

period
noSpecifies weekdays on which the Cleanup Service is performed. The first day of week is considered being Monday. When initially installing JS7 then the distribution includes the value: 1,2,3,4,5,6,7. Without this setting the Cleanup Service will not start.

period_begin01:00:00noSpecifies the start time of the Cleanup Service for the given period, e.g. on a daily basis.

period_end04.00:00noSpecifies the end of the period for which the Cleanup Service is allowed to run. If the Cleanup Service completes its task first then it will stop before the indicated time. However, it will not continue to run beyond the indicated time. 

batch_size1000noSpecifies the number of records that are purged within a single transaction. Increasing this value can improve performance, at the same time it will increase the risk of conflicts with concurrent transactions should parallel Services operate on the database.

order_history_age90dnoSpecifies the retention period for the execution history of orders and tasks. Any history entries older than the value specified will be purged.

order_history_logs_age90dnoSpecifies the retention period for orders and task logs. Any logs older than the value specified will be purged. Note that this value should not exceed the value of the cleanup_order_history_age setting as otherwise navigation to logs cannot be offered by the JOC Cockpit GUI.

file_transfer_history_age90dnoSpecifies the retention period for file transfer history entries. Any entries older than the value specified will be purged.

daily_plan_history_age30dnoSpecifies the retention period for the history of submissions with the Daily Plan. Any history entries older than the value specified will be purged.

deployment_history_versions10noSpecifies the number of versions per deployed object that should be retained. Such versions can be used to re-deploy an object from an earlier state. Any earlier deployed versions of deployed objects are removed.

audit_log_age90dnoSpecifies the retention period for the Audit Log. Any audit log entries older than the value specified will be purged.

monitoring_history_age1dnoSpecifies the retention period for entries in the Monitor view. As this suggests to be a tactical view longer retention periods are not recommended.

notification_history_age1dnoSpecifies the retention period for notifications, e.g. about job errors and warnings. As notifications are typically handled on the same day longer retention periods are not recommended.

Logging

  • The Cleanup Service logs general messages, warnings and errors to the joc.log file.
  • More detailed information in addition is logged to the Main Log service-cleanup.log file.
  • In addition to the Main Log detailed debug information is logged to the Debug Log service-cleanup-debug.log file.
  • For details see JS7 - Log Files and Locations


  • No labels
Write a comment…