Introduction

• Sizing is an important step when planning the installation and operation of JS7.
• The following guidelines are recommendations (best practices) that require to be mapped to the requirements of your environment.
• SOS recommends to decide on the desired target architecture in a first step and then to determine the sizing accordingly.

Architecture

• First an architecture decision has to be taken:
  • Use of JOC Cockpit and Controller standalone instances
  • Use of JOC Cockpit and Controller high-availability cluster instances
  • For details see JS7 - System Architecture
• Then the scheduling mode should be identified
  • Use of JS7 Agents
  • Use of JS7 - Agentless Scheduling

Scalability

The JS7 adjusts to both vertical and horizontal scaling, see JS7 - Performance:

• Vertical Scalability
  • There are no built-in limits as to the overall number jobs and number of parallel job executions per Controller and Agent.
  • A single Controller is tested for some 50 000 parallel tasks, a single Agent is tested for approx. 15 000 parallel tasks.
  • Adding more power to a single machine (CPU, Memory, SSD Disks) will improve performance as the components make use of any available cores.
• Horizontal Scalability
  • Splitting the job load across a number of Agents will push the number of parallel tasks.

Users have a choice to size for vertical or horizontal scalability.

Sizing

Server Resources

• The following indicators should be considered:
  • The overall number of workflows and jobs that will be available.
  • The number of jobs that will be executed in parallel.
• Number of Jobs
  • Small environments are assumed to use < 1 000 jobs, medium sized environments use < 20 000 jobs, larger environments use larger numbers.
• Parallelism of Jobs
  • Small environments run < 100 parallel jobs, medium sized environments run < 1 000 parallel jobs, larger environments run larger numbers of jobs.

Basic Recommendation

Overall number of jobs

• Impact
  • JS7 is designed to operate up to 1 000 000 overall job objects. However, for the sharing of duties users might split the load across multiple JS7 instances.
  • A high number of orders, workflows and jobs affects the responsiveness of the JOC Cockpit GUI to display the respective objects.
• Recommendations
  • For small and medium sized environments no specific measures are required.
  • For larger environments a number of measures are applied that are tailored to the user's needs. SOS provides Consulting Services to assist in designing workflows and jobs for performance.

CPU

• Impact
  • JOC Cockpit will use a small amount of CPU during idle times due to the fact that a number of background processes are running, see JS7 - Services. When load is increasing by parallel users or by parallel use of the JS7 - REST Web Service API then more threads will be used that map to processes and cores.
  • Controller and Agent will hardly use any CPU power during idle times. The Controller and Agent handle thread pools in way to balance CPU consumption across all available cores.
• Recommendations
  • Do not use a single core CPU system. Situations could occur when individual jobs might misbehave and consume more CPU than expected. A multi-core CPU system allows to have process resources ready in a situation when a single core CPU is blocked. The more cores are available the better performance results will be.

Memory

• Impact
  • The number of parallel orders and jobs increases memory consumption of an active Controller instance and the respective Agents.
    • JOC Cockpit can be operated from 128 MB heap size. This value should be increased if more users are working in parallel with JOC Cockpit.
    • The Controller can be operated starting from 128 MB memory. Higher parallelism of orders and jobs will require higher values, e.g. for 100 000 orders approx. 4 GB will be required.
    • The Agent has a footprint of about 100 MB memory. Similar to the Controller higher parallelism of orders and jobs will push this value. As a rule of thumb 2 GB main memory should be sufficient to execute 5 000 jobs in parallel on an Agent.
  • Consider the JS7 - FAQ - Which Java Options are recommended? article for default values and details how to specify memory consumption.
• Recommendations
  • Calculate 500 MB for JOC Cockpit for 5 to 10 parallel users. Calculate 2 GB each for a Controller and Agent if < 50 000 task executions per day and < 5000 parallel task executions are performed. You can reduce the values for Controller and Agent to 500 MB if < 10 000 task executions per day and < 1000 parallel task executions are performed.

Initial Installation

• File System
  • The size of installed files should not exceed approx. 250 MB for each JOC Cockpit, Controller and Agent.
• Database
  • The database initially can be used starting from 200 MB tablespace.

Ongoing Operation

File System

• Journal files
  • Journal files are used by Controller instances and Agent instances to store information about state transitions and log output. Typically such information is immediately forwarded from an Agent to the Controller and from the Controller to JOC Cockpit. However, in case of high load or outage of a component, for a example a Controller, then the Agent continues to execute jobs and stores such information in its journal. Journal files therefore can grow, technically to the max. size imposed by the OS. Journal files will shrink after JS7 components are recoupled. For most situations with <100 000 job executions per day and an outage period < 24h it is sufficient to assume 5 GB diskspace for journal files. 
• Log files
  • Log files by default are configured for log rotation to limit diskspace consumption and to restrict the number of files in use, see JS7 - Log Rotation.
    • All log files are controlled by Log4j2 and allow individual configuration.
    • By default diskspace consumption is limited to 5 GB and log files are retained for 30 days. This applies to each JOC Cockpit, Controller and Agent.
  • Recommendations
    • It is suggested not to assign less than 1 GB disk space for logs. Use of more than 5 GB disk space should be required only if the retention period for log files is prolonged.
    • When JS7 components cannot write to a log file then they will be blocked from further operation. To prevent such a situation add a disk space monitoring task to your system monitor.

Database

• Shared/Dedicated Database Server
  • The JS7 does not require a dedicated Database Server. At the same time in a shared scenario users should be aware of impacts if the Database Server is subject to high load.
  • If the JS7 faces insufficient database performance then this will slow down the performance of JOC Cockpit, preferably the JS7 - History might not be able to report job execution results and log output in good time. Practically speaking, if the job execution history typically is available within 1-2s after completion of a job then in case of high load of job executions and bad performance  of the database this can take longer, maybe minutes. However, such data will not be lost but are added with a delay.
•  Memory
  • Some DBMS allow to store tables in memory. This can improve the overall performance for frequently used tables. 
  • Recommendation
    • SOS recommends to leave this subject to your DBA. This is about database optimization and will result in a noticeable effect only in high performance environments.
• Tablespace
  • JS7
    • stores the job execution history in the database,
    • stores the task logs in a zipped format in the database,
    • can be configured to automatically purge the database for configurable retention periods of objects, see JS7 - Cleanup Service.
  • Recommendations
    • Calculate the increase in tablespace size based on the overall number of job executions. Assume 2 KB for job history and task log for each job execution. Add a factor for individual log output of scheduled scripts and programs.
    • A practical approach includes to closely monitor tablespace growth during the first few months of operation and then to adjust the configuration of the Cleanup Service to purge for shorter or longer retention periods.
    • As a rule of thumb do not start JS7 operation with a tablespace <200 MB.

Further Resources

• JS7 - Performance