Scope
- The Reporting Interface provides a data basis for individual reports:
- No standards reports are provided as they are not in scope of the Reporting Interface. Sample reports for MS Excel are provided that can be used as a starting point for individual reports.
- The Reporting Interface is used to provide facts and aggregations in a timely manner, e.g. for daily reporting.
Requirements
Technical Requirements
- Facts and Aggregations
- Implications
- The steps for collection of facts and aggregation of data are clearly separated and are provided by different jobs with individual start times:
- Facts
- are collected from the JobScheduler history and are stored to fact tables.
- can be collected multiple times per day.
- collection intervals can be freely configured.
- Aggregations
- are created for periods, e.g. daily, weekly, monthly, quarterly, yearly.
- are created exclusively based on fact tables.
- can be re-created any time (provided that the facts are available).
- Clustering
- User Story
- As an Application Manager I want to create reports across Cluster Members and Agents.
- Implications
- The history data of all JobScheduler instances that are logging to the same database are used by the Reporting Interface. The reporting jobs can be configured to collect reporting data for individual JobSchedulers specified by the Scheduler IDs.
- If multiple JobScheduler clusters are operated then then Reporting Interface can be used
- to collect history data from a number of JobScheduelr databases and
- to write reporting data into a common reporting database.
- Periods
- User Story
- As an Application Manager I would like to specify a period from..to for all reports.
- Implications
- The period of one day includes the start times of jobs and orders with respect to the UTC time zone.
- Parameterisation
- Date from, previous month
- Date to, for one month
- Mandators and Applications
- User Story
- As an Application Manager I would like to restrict reports to specific mandators and applications.
- Implications
- The rules for assigning mandators and applications to job objects are individual per user.
- A mapping mechanism is provided that includes look-up tables for individual mandators and applications.
- Mandators and applications are not included in fact tables.
- Parameterisation
- Mapping tables for the mapping of job objects to mandators and applications are provided.
- Job Reporting
- User Story
- As an Application Manager I want to restrict reports to specific jobs and job chains.
- Parameterisation
- Specification of jobs and job chains, e.g. by regular expressions.
- File Transfer Reporting
- Inclusion of file transfer history data requires use of the YADE Background Service.
- The jobs for file transfers are in scope of the job reporting.
- The Reporting Interface makes use of the tables for history data that are specific for YADE.
- Impact on Job Objects
- The Reporting Interface does not affect the configuration and execution of jobs.
- No modifications of existing jobs are required.
Functional Requirements
- Operational Reports
- Change Management Report: used by IT Operations
- User Story
- As an Application Manager I want to create reports of JobScheduler object changes over time across servers and agents.
- Implications
- All jobs, job chains and orders are specified that were subject to modification during the reporting period (created, modified, removed).
- Parameterisation
- Scheduling Load Report: used by IT Operations
- User Story
- As an Application Manager I want to know the periods during which the highest number of parallel jobs were executed.
- Implications
- Reports provide answers to the following questions:
- What is the maximum number of parallel tasks?
- Is the number of parallel task in a configuration sufficient?
Did any congestion of orders occur?What was the highest and longest consumption of CPU and memory?- This information requires to collect such data by a background process and to store them in a load history.
- This requirement is out of scope.
- Delimitation
- No information is provided on order congestion.
- No information is provided on CPU and memory load.
- Maintenance Window Report: used by IT Operations
- User Story
- As an Application Manager I want to find spaces for possible maintenance windows over a period of time, e.g. between 01.01.2014 to 30.06.2014
- Specify maintenance duration (hours) and optionally preferred days. Examples:
- show days of week with 4 hours possible maintenance window starting from 18:00
- show days of week with 8 hours possible maintenance window starting from 22:00 until 06:00 next days
- The results should point to periods
- that would completely fit for a maintenance window, i.e. no distinct job starts did occur.
- that would partially fit for a maintenance window, i.e. only a few job starts did occur, e.g. by specifying periods with less than 5 job executions. Job executions are not counted for jobs with a run-time that includes a relative repeat interval or absolute repeate interval.
- As an Application Manager I want to find spaces for possible maintenance windows between two dates, i.e. the zero or minimum activity time frame. JobScheduler house keeping job or some polling jobs can be ignored.
- Delimitation
- It is out of scope to consider
- orders that are not started for the first job node of a job chain.
- orders that would not pass all the nodes of a job chain.
- orders that would pass different sequences of job nodes based on control of exit codes.
- Questions
- How should unsuccessful orders by reported? Count successful executions only?
- How should orders be counted that were suspended (manual intervention required) or setback (automated intervention)?
- How should orders be counted that have been manually added?
- Parameterisation
- Optional: preferred weekdays, e.g. only for Thursday and Friday
- Optional: excluded weekdays, e.g. ignore Saturday and Sunday
- Mandatory: duration of Maintenance Window
- Optional: hour:minute for start of maintenance window
- Optional: hour:minute for end of maintenance window
- Performance Reporting: used by IT Operations for detection of performance changes
- User Story
- As an Application Manager I want to see changes in the performance of jobs and job chains over a period of time.
- As an Application Manager I want to see which jobs in a given period show differences in time consumption (shorter or longer execution time).
- As an Applicatin Manager I want to be aware of any deterioration of the execution time due to increase of data, e.g. due to missing indices in a database.
- Changes of the execution time for file transfers, e.g. this would help to detect transfers with files of different size by the same jobs.
- Implications
- Compare elapsed time for executions in the given time slot.
- Delimitation
- No periods are compared but changes to predecessor executions in the same period are compared.
- Parameterisation
- Number of previos job executions for performance comparison.
Compliance Reporting: used by Compliance Officer
- User Story
- As a compliance officer I want to know if passwords are configured in clear text for job configurations.
- Consideration of legal requirements, rules for governence and data protection.
- Implications
- Are any jobs in use that specify passwords in clear text?
- Specific jobs are not allowed to be started manually. The same applies to specific job chains.
- Delimitation
- Requirement is out of scope. Compliance is not a topic of JobScheduler reporting.
- SLA Reporting: used by Customer (OLA Reporting is addressed to IT Operations)
- User Story
- As a customer I want to receive the service level report as for each JobScheduler instance with that report showing the percentage of total executions, successful vs. failed executions.
- Delimitation
- The Reporting Interface does not provide SLA reports, instead it provides a collection of history data for individual SLA reports for a number of purposes, e.g.
- SLA Reports to check compliance with existing SLAs.
- SLA Reports to account for job executions.
- Distinct executions of the same jobs are counted. Excluded are repeatedly unsuccessful executions, e.g. for setbacks.
- Polling for file transfer jobs via setbacks should not be counted.
- Execution Summary Report: used by IT Operations
- User Story
- As an Application Manager I want to receive a summary execution report for a job or job chain that provides the start time, end time, elapsed time and job start cause.
- Implications
- The job start causes allow to distinguish jobs being
- started by start time
- manually started
- started by file watcher
- manually added
- started by API methods
- started by repeat interval
- started by resume operation (after suspend operation)
- started by setback
- The causes for job starts can aggregated for individual purposes, e.g. manually started jobs and manually added jobs could be combined. Such combinations are effected by use of mapping tables. If possible such mappings should include the mapping per mandator and per application.
- Aggregation Reports: used by different target groups
- User Story
- As a user I want to receive the categorywise reporting of jobs, i.e. executed every hour, 2 hour...., daily, weekly, monthly, yearly.
- As a customer I want to find catagorywise jobs per mandator or application group to calculate operational cost.
- As a user I want to crate monthly and weekly operational reports including the number of executions, successful vs failed executions, improvement or decline of performance.
- As an Application Manager I want to aggregate job executions and job durations per client for accounting purposes.
- Implications
- Aggregations are effected per period, e.g. daily, weekly, monthly, quarterly, yearly.
- Per aggregation period individual records are created.
- Optionally aggregations can be created per mandator and application, e.g. for accounting purposes.
Monitoring Reports- User Story
- As a user I want to know immediately after execution of a job or job chain if everything is ok and was executed in time.
- Delimitation
- This requirement is out of scope the Reporting Interface, as the reporting is executed with a delay and provides an interface but no complete reports.
- It is recommended to use the JobScheduler Monitoring Interface for timely notification to system monitors as this interface allows individual notification intervals to systems such as Nagios, op5, eMail etc..
Usability Requirements
- The Reporting Interface is provided without user interface.
- The Reporting Interface jobs can be configured by orders (parameters, start times) with the JOE GUI.
Operational Requirements
- The distribution of the Reporting Interface includes daily collection of facts and creation of aggregations.
- Aggregations are created for a configurable period (daily, weekly, monthly, quarterly, yearly).
- The Reporting Interface jobs can be re-started and can be repeatedly started at any time for any period.
- When creating facts then the Reporting Interface considers that such data may already be present or be partially present. Existing data are automatically removed and re-created.
- Aggregations can be re-calculated at any time for the respective period. Existing aggregation data are removed before re-calcuation.
Implementation
Job Reporting
- Reporting on all jobs is in scope of implementation
File Transfer Reporting
- Reporting on files transferred by jobs is not in scope of the first release of the Reporting Interface
{"serverDuration": 112, "requestCorrelationId": "22856e91c0b16ed9"}