JobScheduler Universal Agent - Running multiple instances

Running multiple instances of the JobScheduler Universal Agent

FEATURE AVAILABILITY STARTING FROM RELEASE 1.10.2

Multiple instances of the JobScheduler Universal Agent on can be operated the same computer. This allows, for example, jobs to be executed for different user accounts.

Multiple instances only require a full installation of a single Universal Agent to be carried out on each computer. Each instance only then requires the creation of a separate start script, and log and configuration files. Note that each Agent instance has to use a different port for listening to requests from a JobScheduler Master.

Universal Agent instance installation - an overview

The installation of a first Universal Agent is described in the JobScheduler Universal Agent - Installation & Operation article. 

Once the first Universal Agent has been installed:

• Find the jobscheduler_agent_instance.(sh|cmd)-example instance script template in the ./bin directory of the JobScheduler Agent installation.
  • Templates for Unix and for Windows for previous releases can be downloaded if required.
• Copy the template to jobscheduler_agent_<port>.(sh|cmd) where <port> is the port that the Agent is to listen on.
• The instance script includes settings for the environment variables and for calling the Agent start script (./bin/jobscheduler_agent.(sh|cmd)).
  • Configure the instance script with a text editor and set the environment variables as described in the following two sections of this article.
    
    • Note that the environment variables are described in slightly more detail in the JobScheduler Universal Agent - Installation & Operation article.
• You can start and stop etc. the JobScheduler Agent with the instance script and you will not have to use command line options such as -http-port, -ip-address, -kill-script and -java-options as they are specified by environment variables specified in the instance script.
• On Unix you can use the instance script as a service init script in /etc/init.d or corresponding locations for an automated start-up and shutdown.
• On Windows the instance script is used to create a service for an automated start-up and shutdown.
  

Manually creating and configuring a JobScheduler Universal Agent instance on Unix

• We assume that the JobScheduler Agent instance should listen to port 4441 for example (the default is 4445).

  create instance script for an Agent with port 4441

  > cp ./bin/jobscheduler_agent_instance.sh-example ./bin/jobscheduler_agent_4441.sh

• Edit the ./bin/jobscheduler_agent_4441.sh script.

  • All environment variables in the instance script are initially commented out -  the default values will be used unless they are configured.

  • If you want to set an environment variable then the line with the variable has to be uncommented and a value specified following the syntax:

    • ENVIRONMENT_VARIABLE=value

      SCHEDULER_HOME	This variable has to point to the installation path of the JobScheduler Universal Agent, e.g. /opt/sos-berlin.com/jobscheduler_agent. If the instance script is exclusively used in the current ./bin directory then this variable can remain unused.. This setting is required if the instance script is used as a service init script in /ect/init.d to find the Agent installation!
      SCHEDULER_USER	Specifies the user account that the JobScheduler Agent is operated for. This includes running jobs with the permissions of the specified user. Without setting this environment variable the user account that runs the start script is used. This setting is required if the instance script is used as a service init script in /ect/init.d to avoid that the Agent runs with the root user!
      SCHEDULER_HTTP_PORT	The http port of the Agent. This has to be the same value as the suffix in the instance script basename. In this example SCHEDULER_HTTP_PORT=4441 This option can be also used for indicating which network interfaces the JobScheduler Agent should listen to if you additionally specify a hostname or ip address. SCHEDULER_HTTP_PORT=localhost:4441
      SCHEDULER_HTTPS_PORT	The https port of the Agent. SCHEDULER_HTTPS_PORT=40441 This option can be also used for indicating which network interfaces the JobScheduler Agent should listen to if you additionally specify a hostname or ip address. SCHEDULER_HTTPS_PORT=123.456.789:40441
      SCHEDULER_DATA	Set the directory where the JobScheduler Agent has the configuration, logs, etc. This directory must be unique for each instance of the JobScheduler Agent. The default is: SCHEDULER_HOME/var_SCHEDULER_HTTP_PORT Make sure that the JobScheduler Agent user has read/write permissions.
      SCHEDULER_LOG_DIR	Set the directory where the JobScheduler Agent log files are created. The default location is SCHEDULER_HOME/logs. It is recommended to use the same log directory for all instances. According to the Linux Filesystem Hierarchy Standard (FHS) you can use for example SCHEDULER_LOG_DIR=/var/log/jobscheduler/agent Make sure that the SCHEDULER_USER has read and write permissions in SCHEDULER_LOG_DIR. If you use the same SCHEDULER_LOG_DIR for all instances then make sure that the SCHEDULER_USER of all instances has read and write permissions.
      SCHEDULER_PID_FILE_DIR	Set the directory where the JobScheduler Agent pid file is created. The default location is SCHEDULER_LOG_DIR. Make sure that the SCHEDULER_USER has read and write permissions in SCHEDULER_PID_FILE_DIR. If you use the same SCHEDULER_PID_FILE_DIR for all instances then make sure that the SCHEDULER_USER of all instances has read and write permissions.
      SCHEDULER_WORK_DIR	The default working directory of the JobScheduler Agent is SCHEDULER_HOME. You can specify a different working directory, e.g. $HOME.

  • According to the shell in use you can set the environment variables except SCHEDULER_HOME, SCHEDULER_HTTP_PORT and SCHEDULER_USER in a ~/.profile or ~/.bashrc file too.

  • Note

    An often-used minimum configuration is:

    ...
    SCHEDULER_HOME=/opt/sos-berlin.com/jobscheduler/jobscheduler_agent
    ...
    SCHEDULER_USER=sos
    ...
    SCHEDULER_HTTP_PORT=4441
    ...
    SCHEDULER_LOG_DIR=/var/log/sos-berlin.com/jobscheduler/agent
    ...

    Note that in this example configuration, the Universal Agent is being installed in a directory with restricted permissions, /opt/. In this case, even if the sos user is a member of sudoers, the  SCHEDULER_LOG_DIR variable should be set to a directory where the sos user is able to write without requiring elevated permissions. If this is not the case the agent will not be able to execute any tasks.

• If you want to install a JobScheduler Agent service for an automated start-up and shutdown then copy this instance script to /ect/init.d or corresponding locations.

  create a service init script set automated start-up and shutdown

  > cp ./bin/jobscheduler_agent_4441.sh /etc/init.d/jobscheduler_agent_4441 
  > # on SuSe or CentOS
  > chkconfig jobscheduler_agent_4441
  > # on Ubuntu or Debian
  > update-rc.d jobscheduler_agent_4441 default

Manually creating and configuring a JobScheduler Universal Agent instance on Windows

An installer is available for Windows that you can download here. The following steps explain how to manually create an instance.

• We assume that the JobScheduler Agent instance should listen to port 4441 for example (the default is 4445).

  create instance script for an Agent with port 4441

  > copy .\bin\jobscheduler_agent_instance.cmd-example .\bin\jobscheduler_agent_4441.cmd

• Edit the .\bin\jobscheduler_agent_4441.cmd.

• Find the  environment variables of the JobScheduler Universal Agent.

  • All environment variables are commented out in the instance script, i.e. the default values are used.

  • If you want to set an environment variable then the line with the variable has to be uncommented.

    SCHEDULER_HOME	This variable has to point to the installation path of the JobScheduler Universal Agent (e.g. C:\Program Files\sos-berlin.com\jobscheduler_agent). This variable is only required if this instance script is moved outside the folder where the start script .\bin\jobscheduler_agent.cmd is located. There is no reason on Windows to move this script to a different folder so this variable can normally remain unused.
    SCHEDULER_HTTP_PORT	The port of the Agent. It has to be the same value as the suffix in the instance script basename. In this example set /a SCHEDULER_HTTP_PORT=4441 This option can be also used for indicating which network interfaces the JobScheduler Agent should listen to if you specify a hostname or ip address in addition. set /a SCHEDULER_HTTP_PORT=localhost:4441
    SCHEDULER_HTTPS_PORT	The https port of the Agent. set /a SCHEDULER_HTTPS_PORT=40441 This option can be also used for indicating which network interfaces the JobScheduler Agent should listen to if you specify a hostname or ip address in addition. set /a SCHEDULER_HTTPS_PORT=123.456.789:40441
    SCHEDULER_DATA	Set the directory where the JobScheduler Agent has the configuration, logs, etc. This directory must be unique for each instance of the JobScheduler Agent. The following will create the directory  alongside a directory created by the Universal Agent Windows Installer: C:\ProgramData\sos-berlin.com\jobscheduler\agent_4441 Make sure that the JobScheduler Agent user has read/write permissions.
    SCHEDULER_LOG_DIR	Set the directory where the JobScheduler Agent log files are created. The default location is SCHEDULER_HOME/logs. It is recommended to use the same log directory for all instances. If the installation path of the JobScheduler Agent is located in C:\Program Files then due to the required permissions it is recommended to set this variable to a different folder, e.g. set SCHEDULER_LOG_DIR=C:\ProgramData\sos-berlin.com\jobscheduler\agent\logs The JobScheduler Agent is installed as a service. Make sure that the service account has read and write permissions in SCHEDULER_LOG_DIR. If you use the same SCHEDULER_LOG_DIR for all instances then make sure that all service accounts have read and write permissions.
    SCHEDULER_PID_FILE_DIR	Set the directory where the JobScheduler Agent pid file is created. The default location is SCHEDULER_LOG_DIR. The JobScheduler Agent is installed as a service. Make sure that the service account has read and write permissions in SCHEDULER_PID_FILE_DIR. If you use the same SCHEDULER_PID_FILE_DIR for all instances then make sure that all service accounts have read and write permissions.
    SCHEDULER_WORK_DIR	The default working directory of the JobScheduler Agent is SCHEDULER_HOME. You can set a different working directory, e.g. %USERPROFILE%.

  • create the service with the instance script

    > .\bin\jobscheduler_agent_4441.cmd install-service

  • The service runs now with the local system account. Specify a different user account for the service if required.
    
    • Open the Windows service panel for the service name SOS JobScheduler Agent -port=4441 and navigate to the tab "Log On", choose "This account" and add the account with the syntax DOMAIN\USER and the corresponding password.