Product Knowledge Base

Space shortcuts

Page tree

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 12 Next »

Introduction

The JobScheduler Universal Agent Batch Installer is a set of files that is used by a JobScheduler Master to install one or more Universal Agents on local and remote servers.

Article Scope

This article is one of a series describing how to configure the JobScheduler Universal Agent Batch Installer. It describes a simple use case for the Universal Agent Batch Installer - to install a single Universal Agent on a remote server. This article describes the installation procedure in detail and is intended as a "getting started" guide. The other articles in this series describe more complex use cases and are listed in the introductory JobScheduler Universal Agent - Batch Installation and Update of Agents article.

A general introduction to installation of the JobScheduler Universal Agent can be found in the JobScheduler Universal Agent - Installation & Operation article.

The JobScheduler Universal Agent - Batch Installation and Update of Agents article provides an overview of how the Batch Installer works. New users are strongly recommended to read this article before proceeding with the instructions presented here.

The Batch Installation - Reference Documentation article provides a detailed guide to configuration of the batch installer with, for example, full lists of parameters.

Installation Environment

This article describes the use of the JobScheduler Universal Agent Batch Installer on a Linux server, working from a Windows system.

Installation Procedure

General requirements:

  • The JobScheduler Master to be used for the installation of the Universal Agents should be up and running.
  • A Java Run-time Environment in version 1.8 or newer should already be installed on the system(s) the Agent is to be installed on.
  • We recommend that you use the SOS XML Editor to manage the configuration files required for the Batch Installer. More information about the installation and use of the XML editor will be provided later in this article.

1. Download and Extract the Batch Installer and Universal Agent Files

Files can be downloaded from http://www.sos-berlin.com/jobscheduler-downloads.

Unix Target Systems

  1. Download the "JobScheduler Universal Agent" tarball for Unix.
  2. Download the "Universal Agent Batch Installer".
    • This archive contains the following folders and sub-folders:
      • live/batch_install_universal_agent
        • This folder contains the Jobs and Job Chain used to carry out the installation.
          Extract these files to the the live folder of the JobScheduler Master: 
          • SCHEDULER_DATA/config/live.
      • batch_install
        • This folder contains the configuration files for the Batch Installer.
          Extract these files to the config directory of the JobScheduler Master: 
          • SCHEDULER_DATA/config.
  3. Copy the JobScheduler Universal Agent installation file to the SCHEDULER_DATA/config/batch_install folder.

Windows Target Systems

  1. Download the "JobScheduler Universal Agent" for Windows.
  2. Download the "Universal Agent Batch Installer".
    • This archive contains the following folders and sub-folders:
      • the live/batch_install_universal_agent folder
        • This folder contains the Jobs and Job Chain used to carry out the installation. 
          Extract these files to the live folder of the JobScheduler Master:
          • SCHEDULER_DATA/config/live.
      • the batch_install folder
        • This folder contains the configuration files for the Batch Installer.  
          Extract these files to the JobScheduler configuration directory:
          • SCHEDULER_DATA/config.
  3. Copy the JobScheduler Universal Agent installation file to the SCHEDULER_DATA/config/batch_install folder.

2. Configure the Transfer of the Universal Agent Installation Files and the Agent Installation

Hints

  • Configuring complex XML files with a text editor is often a tedious and error-prone procedure. To get around this, we recommend that you use our XML Editor to manage the configuration files required for the Batch Installer. The XML Editor automatically validates configuration files and therefore reduces the possibility of configuration errors.

  • After downloading the XML Editor, extract the contents of the downloaded zip file to a convenient folder (an installation routine is not required) and start the sos.xml.editor.exe program.

  • The  XML Editor is available for Windows.

  • An overview of the use of the XML Editor is provided in the Using the XML Editor section at the end of this article.
    A more detailed description of the use of the XML Editor can be found in the XML Editor - Operation article.
  • The JobScheduler Master comes with its own managed file transfer solution, YADE. YADE will be used here for transferring the files from the JobScheduler Master to the Agent servers.
    For more information about YADE, see the YADE product site.

2.1 Configure transfer of the Universal Agent installation files

The transfer of the Universal Agent installation files is configured in the yade_agent_batch_installer_settings.xml file. This file will be located in the SCHEDULER_DATA/config/batch_installer folder unpacked in step 1 above.

Two configuration elements are required for the transfer of the files required for each Universal Agent installation:

  • a Protocol Fragment that defines how the file transfer is to be carried out and
  • a Profile that defines the file transfer itself.
Protocol Fragment Configuration

A Protocol Fragment is required in the YADE configuration file for each installation. The easiest way to add a Protocol Fragment element using the XML Editor is to copy an already existing element and pasting it to the parent element - in this case <ProtocolFragments>.

Each Protocol Fragment has to have a name attribute that is unique to the YADE configuration file and is made up of <protocol>_<host>:<port> where:

  • <protocol> identifies the protocol to be used for making the transfer
  • <host> is the transfer target host name
  • <port> is the transfer target port number.

An example fragment name would be SFTP_192.11.0202:4441

A complete typical Protocol Fragment configuration would be (shown from the root Configurations element):

  • Configurations
    • Fragments
      • ProtocolFragments
        • SFTPFragment name="SFTP_192.11.0.202:4441"
          • BasicConnection
            • Hostname192.11.0.202
          • SSHAuthentication
            • Account sos
            • AuthenticationMethodPassword
              • Password ********
Profile Configuration

A separate Profile in the YADE configuration file is required for each Agent installation.

Each Profile has to have a profile_id attribute that is unique to the YADE configuration file and is made up of <host>:<port> where:

  • <host> is the transfer target host name and
  • <port> is the transfer target port number.

A typical profile_id attribute would be 192.11.0.202:4441.

A typical Profile configuration could contain the following elements:

  • The CopySource ... Directory element, which points to the batch_install folder which was unpacked as described in Section 1 of this article above.
  • The SFTPFragmentRef element, which contains the name of the SFTPFragment element configured in the Protocol Fragment Configuration section above.
  • The CopyTarget ... Directory, which points to a folder on the target server where the Account specified in the SSHAuthentication.Account element (configured in the Protocol Fragment Configuration section above) will be able to carry out the installation.

A complete typical Profile configuration for a single Agent would be (listed from the root Configurations element):

  • Configurations
    • Profiles
      • Profile profile_id="192.11.0.202:4441"
        • Operation
          • Copy
            • CopySource
              • CopySourceFragmentRef
                • LocalSource
              • SourceFileOptions
                • Selection
                  • FileSpecSelection
                    • FileSpec .*\.(sh|gz)$
                    • Directory C:\ProgramData\sos-berlin.com\jobscheduler\jobscheduler_1.12\config\batch_install
            • CopyTarget
              • CopyTargetFragmentRef
                • SFTPFragmentRef ref="SFTP_192.11.0.202:4441"
              • Directory /home/sos/temp/js_agent_batch_install

The following code block lists an example YADE configuration file for installing two agents. This file has therefore two SFTPFragment elements and two Profile elements.

YADE Configuration File 
<?xml version="1.0" encoding="utf-8"?>
<Configurations xsi:noNamespaceSchemaLocation="http://www.sos-berlin.com/schema/yade/YADE_configuration_v1.0.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <Fragments>
    <ProtocolFragments>
      <SFTPFragment name="SFTP_192.11.0.111:4445">
        <BasicConnection>
          <Hostname><![CDATA[192.11.0.111]]></Hostname>
        </BasicConnection>
        <SSHAuthentication>
          <Account><![CDATA[test]]></Account>
          <AuthenticationMethodPassword>
            <Password><![CDATA[xxxx]]></Password>
          </AuthenticationMethodPassword>
        </SSHAuthentication>
      </SFTPFragment>
      <SFTPFragment name="SFTP_192.11.0.95:4445">
        <BasicConnection>
          <Hostname><![CDATA[192.11.0.95]]></Hostname>
        </BasicConnection>
        <SSHAuthentication>
          <Account><![CDATA[test]]></Account>
          <AuthenticationMethodPassword>
            <Password><![CDATA[xxxx]]></Password>
          </AuthenticationMethodPassword>
        </SSHAuthentication>
      </SFTPFragment>
    </ProtocolFragments>
  </Fragments>
  <Profiles>
    <Profile profile_id="192.11.0.111:4445">
      <Operation>
        <Copy>
          <CopySource>
            <CopySourceFragmentRef>
              <LocalSource />
            </CopySourceFragmentRef>
            <SourceFileOptions>
              <Selection>
                <FileSpecSelection>
                  <FileSpec><![CDATA[.*\.(sh|gz)$]]></FileSpec>
                  <Directory><![CDATA[C:\Program Files (x86)\sos-berlin.com\jobscheduler\vg_4444\scheduler_data\config\batch_installer]]></Directory>
                </FileSpecSelection>
              </Selection>
            </SourceFileOptions>
          </CopySource>
          <CopyTarget>
            <CopyTargetFragmentRef>
              <SFTPFragmentRef ref="SFTP_192.11.0.111:4445" />
            </CopyTargetFragmentRef>
            <Directory><![CDATA[/home/test/tmp/]]></Directory>
          </CopyTarget>
        </Copy>
      </Operation>
    </Profile>
    <Profile profile_id="192.11.0.95:4445">
      <Operation>
        <Copy>
          <CopySource>
            <CopySourceFragmentRef>
              <LocalSource />
            </CopySourceFragmentRef>
            <SourceFileOptions>
              <Selection>
                <FileSpecSelection>
                  <FileSpec><![CDATA[.*\.(sh|gz)$]]></FileSpec>
                  <Directory><![CDATA[C:\Program Files (x86)\sos-berlin.com\jobscheduler\vg_4444\scheduler_data\config\batch_installer]]></Directory>
                </FileSpecSelection>
              </Selection>
            </SourceFileOptions>
          </CopySource>
          <CopyTarget>
            <CopyTargetFragmentRef>
              <SFTPFragmentRef ref="SFTP_192.11.0.95:4445" />
            </CopyTargetFragmentRef>
            <Directory><![CDATA[/home/test/tmp/]]></Directory>
          </CopyTarget>
        </Copy>
      </Operation>
    </Profile>
  </Profiles>
</Configurations>

The following files will be transferred from the JobScheduler Master to the Agent servers:

  • the JobScheduler Universal Agent installation file: jobscheduler_unix_universal_agent.<version>.tar.gz
  • a start script for every Agent instance: jobscheduler_agent_<port>.sh
  • an installation script: jobscheduler_universal_agent_install.sh
YADE Settings File for File Transfer

Note that the file shown above is in the YADE XML format and that YADE versions 1.10 and older require a .ini file to perform the transfer that is specified in the <globals><install_path><transfer><settings> element in the Batch Installer configuration file.

.ini file 
[protocol_fragment_sftp@SFTP_192.11.0.111:4445]
protocol                            = sftp
host                                = 192.11.0.111
user                                = test
ssh_auth_method                     = password
password                            = xxxx

[protocol_fragment_sftp@SFTP_192.11.0.95:4445]
protocol                            = sftp
host                                = 192.11.0.95
user                                = test
ssh_auth_method                     = password
password                            = xxxx

[192.11.0.111:4445]
operation                           = copy
source_protocol                     = local
file_spec                           = .*\.(sh|gz)$
source_dir                          = C:\Program Files (x86)\sos-berlin.com\jobscheduler\vg_4444\scheduler_data\config\batch_installer
target_include                      = protocol_fragment_sftp@SFTP_192.11.0.111:4445
target_dir                          = /home/test/tmp/

[192.11.0.95:4445]
operation                           = copy
source_protocol                     = local
file_spec                           = .*\.(sh|gz)$
source_dir                          = C:\Program Files (x86)\sos-berlin.com\jobscheduler\vg_4444\scheduler_data\config\batch_installer
target_include                      = protocol_fragment_sftp@SFTP_192.11.0.95:4445
target_dir                          = /home/test/tmp/

2.2 Configure the installation of the Universal Agents

The installation of the Universal Agents on the target servers is configured in the batch_installer_config_file.xml file in the XML Editor. This file will be located in the SCHEDULER_DATA/config/batch_installer folder unpacked in step 1 above.

  • Open your batch_installer_config_file.xml configuration file in the XML Editor.
    • Assign the scheduler_universal_agent_installations.xsd schema, which will also be located in the SCHEDULER_DATA/config/batch_installer folder.

The Batch Installer configuration file contains an <installation> element for each Agent instance to be installed. In addition, the <global> element can be used in a homogeneous environment to specify parameters that will be applied to all Agent installations. The majority of configuration elements can be specified as <global> and as <installation> elements. If a parameter is specified in both the <global> element and in an individual <installation> element then the setting for the individual element will overwrite the global setting.

The parameters to be configured are:

  • the <install_path> element, which points to a location where the Agent should be installed.
  • the <transfer><settings> element which points to the location where the YADE transfer configuration file that was configured in Section 2.1 above.
    • A typical value would be C:\ProgramData\sos-berlin.com\jobscheduler\jobscheduler_1.12\config\batch_install\yade_agent_batch_installer_settings.xml
  • the <ssh> elements (often partly configured as <global> elements):
    • <auth_method>: the authentication method to be used for the SSH connection.
    • <port>: SSH port.
    • <user>: SSH login account name.
    • <password>: Password for the SSH login account.
    • <auth_method>: SSH authentication method.
    • <host>: the host name of the target server.
  • the <installation_file> element - the name of the Universal Agent installation file. This filename contains the version of the Agent which you are installing and it is helpful for recognizing the version of the Agents that are installed.
    • For Unix systems this is usually jobscheduler_unix_universal_agent.<version>.tar.gz
  • the <agent_options> elements:
    • <scheduler_http_port>: the port the Agent is to use.
    • <scheduler_user>: the user name the Agent is to run under on the target system.
    • <scheduler_log_dir>: the directory to be used for the Agent Log files.
  • the <postprocessing> elements:
    • The post-processing commands for the Agent server's operating system. The default commands for an Ubuntu distribution will place the Agent start script under /etc/init.d and will setup the autostart for the Universal Agent:
      • echo ${ssh.sudo_password} | sudo -S cp ${install_path}/jobscheduler_agent/bin/jobscheduler_agent_${agent_options.scheduler_http_port}.sh /etc/init.d/jobscheduler_agent_${agent_options.scheduler_http_port}
      • echo ${ssh.sudo_password} | sudo -S update-rc.d jobscheduler_agent_${agent_options.scheduler_http_port} defaults
    • These two commands will be executed by the PerformInstall job node, which is the last job node in the batch installer job chain. If you want to add more commands, please add the call to the job node.

Post-processing commands

  • The SSH login user has to be set to root in order to execute these commands. Therefore the SSH login user should be included in sudoers.

  • Note that the values for the elements defined in the Batch Installer configuration file can be used for the postprocessing commands, for instance:

    • ${install_path}: the value for the <install_path> element

    • ${agent_options.scheduler_http_port}: the value for the <scheduler_http_port> element. If this element is not defined, the default value (4445) will be taken, as in the example full XML code listing below.

A complete typical installation configuration for a single Agent would be (shown from the root installations element):

  • installations lastRun="2018-01-02 13:01"
    • globals
      • install_path /opt/sos-berlin.com/jobscheduler
      • transfer
        • settings C:\ProgramData\sos-berlin.com\jobscheduler\jobscheduler_1.12\config\batch_install\yade_agent_batch_installer_settings.xml
      • ssh
        • auth_method password
        • port 22
      • installation_file jobscheduler_unix_universal_agent.1.12.0.tar.gz
    • installation lastRun="2018-01-02 13:01"
      • agent_options
      • ssh
        • user sos
        • password sos
        • sudo_password sos
        • host 192.11.0.202
      • postprocessing
        • command echo ${ssh.sudo_password} | sudo -S cp ${install_path}/jobscheduler_agent/bin/jobscheduler_agent_${agent_options.scheduler_http_port}.sh /etc/init.d/jobscheduler_agent_${agent_options.scheduler_http_port}
        • command echo ${ssh.sudo_password} | sudo -S update-rc.d jobscheduler_agent_${agent_options.scheduler_http_port} defaults

Note that in the above example two of the SSH elements (<auth_method> and <port>) are configured globally and four (<user>, <password>, <sudo_password> and <host>) are configured individually for each Agent.

Hints

  • Note that if the port is not specified with the <installations><installation><agent_options><scheduler_http_port> element the default value of 4445 will be applied to all Agent installations.

  • For a homogeneous environment parameters can be specified with the <global> element and applied to all Agent installations.

The following code block shows Batch Installer configuration file for installing two agents. This file has therefore two installation elements and fewer elements defined in the <globals> section than the typical installation configuration for a single Agent shown above.

The Batch Installer configuration file for the example can be downloaded here: batch_installer_config_file.xml.

Installer Configuration File 
<?xml version="1.0" encoding="utf-8"?>
<installations lastRun="2015-11-19 17:11">
  <globals>
    <transfer>
      <settings><![CDATA[C:\Program Files (x86)\sos-berlin.com\jobscheduler\vg_4444\scheduler_data\config\batch_installer\yade_agent_batch_installer_settings.ini]]></settings>
    </transfer>
	<installation_file><![CDATA[jobscheduler_unix_universal_agent.1.10.2.tar.gz]]></installation_file>
  </globals>
  <installation lastRun="2015-11-19 17:11">
    <install_path><![CDATA[/opt]]></install_path>
    <agent_options>
      <scheduler_http_port>4447</scheduler_http_port>
      <scheduler_user><![CDATA[test]]></scheduler_user>
      <scheduler_log_dir><![CDATA[/var/log/sos-berlin.com/jobscheduler/agent]]></scheduler_log_dir>
    </agent_options>
    <ssh>
      <password><![CDATA[test]]></password>
      <sudo_password><![CDATA[xxxx]]></sudo_password>
      <auth_method><![CDATA[password]]></auth_method>
      <host><![CDATA[192.11.0.111]]></host>
      <port><![CDATA[22]]></port>
    </ssh>
    <postprocessing>
      <command><![CDATA[echo ${ssh.sudo_password} | sudo -S cp ${install_path}/jobscheduler_agent/bin/jobscheduler_agent_${agent_options.scheduler_http_port}.sh /etc/init.d/jobscheduler_agent_${agent_options.scheduler_http_port}]]></command>
      <command><![CDATA[echo ${ssh.sudo_password} | sudo -S update-rc.d jobscheduler_agent_${agent_options.scheduler_http_port}]]></command>
    </postprocessing>
  </installation>
  <installation lastRun="2015-11-19 17:11">
    <install_path><![CDATA[/opt/]]></install_path>
    <agent_options>
      <scheduler_http_port>4445</scheduler_http_port>
      <scheduler_user><![CDATA[sos]]></scheduler_user>
      <scheduler_log_dir><![CDATA[/var/log/sos-berlin.com/jobscheduler/agent]]></scheduler_log_dir>
    </agent_options>
    <ssh>
      <user><![CDATA[sos]]></user>
      <password><![CDATA[xxxx]]></password>
      <sudo_password><![CDATA[xxxx]]></sudo_password>
      <auth_method><![CDATA[password]]></auth_method>
      <host><![CDATA[192.11.0.95]]></host>
      <port><![CDATA[22]]></port>
    </ssh>
    <postprocessing>
      <command><![CDATA[echo ${ssh.sudo_password} | sudo  -S cp ${install_path}/jobscheduler_agent/bin/jobscheduler_agent_${agent_options.scheduler_http_port}.sh /etc/init.d/jobscheduler_agent_${agent_options.scheduler_http_port}]]></command>
      <command><![CDATA[echo ${ssh.sudo_password} | sudo -S chkconfig jobscheduler_agent_${agent_options.scheduler_http_port}]]></command>
    </postprocessing>
  </installation>
</installations>

2.3 Configure the JobScheduler Jobs

Agent installation is carried out by the four Jobs in the universal_agent_installer Job Chain. Execution of this Job Chain is started by an Order that is generated by the AgentBatchInstaller Job. A more detailed description of the installation procedure can be found in the JobScheduler Universal Agent - Batch Installation and Update of Agents article.

Two parameters have to be set for the AgentBatchInstaller Job which generates the Order for the universal_agent_installer Job Chain.

Open the JobScheduler Master's SCHEDULER_DATA/live/batch_install_universal_agent folder in the JobScheduler Object Editor, JOE and edit the parameters for the AgentBatchInstaller Job:

  • Set the installation_definition_file Job parameter to the path that points to the Batch Installer configuration file.
    If the paths recommended in Step 1 of this procedure above were followed this will be:

    • SCHEDULER_DATA/config/batch_install/batch_installer_config_file.xml
  • Similarly, the installation_job_chain parameter should be set to:
    • /batch_install_universal_agent/universal_agent_installer
  • Leave the filter_install_host and filter_install_port parameters blank.
  • Leave the update parameter set to true.

3 Installation of the Agents

The Agents can be installed after the XML configuration files and JobScheduler Jobs have been configured as described above.

Agent installation is carries out by the four Jobs in the universal_agent_installer Job Chain. Execution of this Job Chain is started by an Order that is generated by the AgentBatchInstaller Job. A more detailed description of the installation procedure can be found in the JobScheduler Universal Agent - Batch Installation and Update of Agents article.

Carry out the following steps to start the AgentBatchInstaller Job:

  1. Sign in to the JOC Cockpit operating interface with a User Account that has the permissions necessary to start a Job.
  2. Open the "Jobs" view and open the "batch_installer_universal_agent" folder.
  3. Click on Action symbol (the ellipsis) alongside the AgentBatchInstaller Job and select the Start Task now option.
  4. This job will create the appropriate Orders for the JobScheduler Master as explained in the description of the Batch Installer Application and start execution of the Job Chain.
    1. Progress of the installation procedure can be followed either by clicking on the Expand Details button or by opening the same folder in the JOC Cockpit's Job Chains view, clicking on the Action symbol and selecting the Show Job Chain Details menu item.

 

Using the XML Editor

The XML Editor can be used to manage the Batch Installer configuration file and the YADE configuration file.

XML Editor for Batch Installer configuration file

An example Batch Installer configuration file for the installation of a single Agent is shown in the following screenshot:

XML Editor for YADE configuration file

An example YADE configuration file for the installation of a single Agent is shown in the next screenshot:

XML Editor for converting YADE configuration files (XML) into YADE setting files (.ini)

The "YADE Xml2Ini Converter" function in the XML Editor is available for converting YADE files from XML to the .ini format required by versions of YADE before 1.11. Clicking on the button of the same name will open the following form:

Note that it is not necessary to create a Profile for the Batch Installer, as conversion will take place automatically once the "Autotransform" option has been chosen. Every time a change in the XML file is saved the XML file will be automatically converted and saved in .ini format. 

A pop-up window will show every time a change is saved if the option "Show notification on autotransform" is chosen. 

  • No labels