The Agent Batch/Update solution comes with a job chain in JobScheduler and the job AgentBatchInstaller.
The Job AgentBatchInstaller reads a xml configuration file and creates orders for the installation job chain to setup/update agents. For each agent there is an entry in the configuration file.
The Parameters for the job chain steps will be set by the job AgentBachtInstaller while reading the xml input file.
FEATURE AVAILABILITY STARTING FROM RELEASE 1.10
The Batch Installer is available starting starting from release 1.10.2.
Universal Agent Batch Installer Job Chain
The job chain for update/installation of one Agent has the four nodes:
CheckPreRequisite
- Checking for the correct java version >= 1.8
- Checking whether JobScheduler Universal Agent already has been installed.
- If so
- Shutting down a probably running instance
- remove the installation
- If so
TransferInstallaionSetup
: To transfer the setup jar file to the hostThe files, that will be transerfered are specified in globals/transfer/file_spec=.*\.(sh|gz)$
- With this regular expression, these files will be transfered
- The JobScheduler Universal Agent: jobscheduler_unix_universal_agent.1.11.0-SNAPSHOT.tar.gz
- The instancescript: jobscheduler_agent_<port>.sh
- The installation scipt: jobscheduler_universal_agent_install.sh
PerformInstall
: To start the installation using SSH.- Before the installation starts, the preprocessor will create the instancescript.sh from the template jobscheduler_agent_instance_script.txt
- Then the just transfered installation script will be executed
- The installation script will install and start the JobScheduler Universal Agent
- Finally the postprocessing commands will be executed
- The postprocessing commands are stored in installation/postprocessing/command
- Three commands will be executed
- if you have more than three commands, please add the call in the job PerformInstall
- The default commands will setup the autostart for JobScheduler Universal Agent on an Ubuntu server
- echo ${ssh.sudo_password} | sudo cp ${install_path}/bin/jobscheduler_agent.sh /etc/init.d/jobscheduler
- echo ${ssh.sudo_password} | sudo update-rc.d jobscheduler defaults
Starting the update/installation
The update/installation procedure will be started by running the job AgentBatchInstaller.
The job AgentBatchInstaller
The job AgentBatchInstaller reads an input file and creates orders for each element <installation>
The element <installation> defines the source and the target of the installation files, the port for the JobScheduler Universal Agent and the ssh connection parameters. Please note
- For all elements you can define defaults in the globals section
- You can use variables for the values e.g. ${ssh.host}
Parameters
Parameter | Description | Example |
---|---|---|
installation_definition_file | The xml file that contains all parameters for the batch installation | c:\batch\batchinstall.xml |
filter_install_host | A host name if only one order should be created that matches the given host | myAgent |
filter_install_port | A port if only one order should be created that matches the given host | 4445 |
installation_job_chain | The name of the job chain for which the orders should be created | /batch_install_universal_agent/universal_agent_installer |
update | If true an order for all <instances> will be created. If false an order for <instances> that do not have the last_run attribute will be created | true |
Customizing the installation process
You can customize the installation process.
- Changing the script part in the ssh jobs CheckPreRequisite and PerformInstall e.g. if you want to check other prerequisites or execute some commands before or after the installation.
- Changing the installation script jobscheduler_universal_agent_install.sh
- Changing the template for instance start script jobscheduler_agent_instance_script.txt
Checking Prerequisites
The job CheckPreRequisite checks the java version, shutdown a probably running JobScheduler Universal Agent and removes the lib folder from the old installation.
PerformInstall
The job PerformInstall calls the installation script and executes the commands given in the <postprocessing> element.
The installation script
The installation script unzip the installation file and starts the JobScheduler Universal Agent
The installation input File
The installation input file contains a <installation> element for each JobScheduler Universal Agent instance.
Example for xml configuration file:
In the global entry, you specify the values which are used for all installation items. The installation items can overwrite these values. There is one installation item for each agent. You can make use of substitution in the values of the entries.
<command>echo ${ssh.sudo_password} | sudo cp ${install_path}/bin/jobscheduler_agent.sh /etc/init.d/jobscheduler</command>
Using the XML Editor
Installation of the Batch Installer for the JobScheduler Universal Agent
Files
The Zip file for the JobScheduler Universal Agent batch installer contains
Folder batch_install_universal_agent
Folder batch_install_universal_agent contains the configuration files for the job chain and jobs that will provide the batch installer for the JobScheduler Universal Agent
BatchInstaller.job.xml
Contains the configuration for the batch installer starter job. The job will read the batch installation file that contains an entry for each Agent and create an order for the batch installer job chain for each Agent that should be installed/updated
Parameters:
Name | Description |
---|---|
installation_definition_file | The file that contains the description for each agent, that should be installed/updated |
filter_install_host | If the input file should filtered, a host can be specified here for which the install/update should be executed |
filter_install_port | If the input file should filtered, a port can be specified here for which the install/update should be executed |
installation_job_chain | the full path of the job chain for which the orders should be generated |
update | if false, only entries will be executed that do not have the last_run attribute. |
CheckPreRequisite.job.xml
Contains the configuration for the job that checks the prerequisites.
The job is implemented by the JITL SSH job. The script that will be exexuted to check the prerequisites is defined in the script tag of the job. The order has the following parameters for this step.
Parameter | Description | XML |
---|---|---|
scheduler_ip_address | The ip address of the JobScheduler Universal Agents host | installation/agent_options/scheduler_ip_address |
scheduler_http_port | The port of the JobScheduler Universal Agent | installation/agent_options/scheduler_http_port |
host | The ssh ip address of the JobScheduler Universal Agents host | installation/ssh/host |
port | The ssh port of the JobScheduler Universal Agent | installation/ssh/port |
user | The ssh user of the JobScheduler Universal Agent | installation/ssh/user |
auth_method | The ssh auth method of the JobScheduler Universal Agent | installation/ssh/auth_method |
auth_file | The ssh auth file of the JobScheduler Universal Agent | installation/ssh/auth_file |
password | The ssh password of the JobScheduler Universal Agent | installation/ssh/password |
sudo_password | The sudo password for ssh user | installation/ssh/sudo_password |
install_path | The folder, where JobScheduler Universal Agent will be installed | installation/install_path |
- Checking for the correct Java version >= 1.8
- Checking whether JobScheduler Universal Agent already has been installed.
- If so
- Shutting down a probably running instance
- remove the installation
- If so
<?xml version="1.0" encoding="ISO-8859-1"?> <job title="Launch remote commands or executable files by SSH" order="yes" stop_on_error="no"> <description > <include file="jobs/JobSchedulerSSHJob.xml"/> </description> <params /> <script language="java" java_class="sos.scheduler.job.SOSSSHJob2JSAdapter"> <![CDATA[ echo .JobScheduler Universal Agent echo .checking prerequisites ${install_path} echo .checking Java Runtime Version >= 1.8? ...... JAVA_VER=$(java -version 2>&1 | sed 's/java version "\(.*\)\.\(.*\)\..*"/\1\2/; 1q') if [ "$JAVA_VER" > "18" ] then echo ..java version: $JAVA_VER ok else echo ..wrong Java version found echo ..expectet version is >= 1.8 echo ..installed version is $JAVA_VER exit 8 fi echo .checking whether JobScheduler Universal Agent is already installed? if [ -f ${install_path}/jobscheduler_agent/bin/jobscheduler_agent.sh ] then echo ..JobScheduler Universal Agent already installed echo ..shutting down JobScheduler Universal Agent ${install_path}/jobscheduler_agent/bin/jobscheduler_agent.sh stop echo ..removing installation from ${install_path} rm -Rf ${install_path}/lib else echo .. this is the first installation of JobScheduler Universal Agent fi exit $? ]]> </script> <run_time /> </job>
TransferInstallationSetup.job.xml
Contains the configuration for the job that transfers the installation files.
The job is implemented by the JITL YADE job. The order has the following parameters for this step.
Parameter | Description | XML |
---|---|---|
file_spec | a regular expression that defines the set of files that should be transfered | installation/transfer/file_spec |
target_host | the host to which the files should be transfered | installation/transfer/target/host |
target_port | the port according to the given protocol (e.g. 21 when protocol is ftp) | installation/transfer/target/port |
target_protocol | the protocol with wich the target will be transered (ftp, sftp, file, local) | installation/transfer/target/protocol |
target_user | the user (if the protocol needs one) with wich the target will be transered | installation/transfer/target/user |
target_password | the password (if the protocol needs one) with wich the target will be transered | installation/transfer/target/password |
target_dir | the location where the files will be copied to | installation/transfer/target/dir |
target_ssh_auth_method | the authentication method if the protocol is ssh. | installation/transfer/target/auth_method |
target_ssh_auth_file | the authentication file if the protocol is ssh and auth_method is publickey | installation/transfer/target/auth_file |
source_host | the host from where the files should be transfered | installation/transfer/source/host |
source_port | the port according to the given protocol (e.g. 21 when protocol is ftp) | installation/transfer/source/port |
source | the protocol with wich the source will be transered (ftp, sftp, file, local) | installation/transfer/source/protocol |
source_user | the user (if the protocol needs one) with wich the source will be transered | installation/transfer/source/user |
source_password | the password (if the protocol needs one) with wich the source will be transered | installation/transfer/source/password |
source_dir | the location from where the files will be copied | installation/transfer/source/dir |
source_ssh_auth_method | the authentication method if the protocol is ssh. | installation/transfer/source/auth_method |
source_ssh_auth_file | the authentication file if the protocol is ssh and auth_method is publickey | installation/transfer/source/auth_file |
<?xml version="1.0" encoding="ISO-8859-1"?> <job title="API Job for JobScheduler Advanced Data Exchange" order="yes" stop_on_error="no"> <description > <include file="jobs/jadeJob.xml"/> </description> <environment > <variable name="test" value="1"/> </environment> <script language="java" java_class="sos.scheduler.jade.JadeJob"/> <run_time /> </job>
installation_definition_file
<?xml version="1.0" encoding="ISO-8859-1"?> <job > <params > <param name="installation_definition_file" value="c:\temp\batchinstall.xml"/> <param name="filter_install_host" value=""/> <param name="filter_install_port" value=""/> <param name="installation_job_chain" value="/batch_install_universal_agent/universal_agent_installer"/> <param name="update" value="true"/> </params> <script language="java" java_class_path="" java_class="com.sos.jitl.agentbatchinstaller.JSUniversalAgentBatchInstallerJSAdapterClass"/> <run_time /> </job>
PerformInstall.job.xml
Contains the configuration for the job that executes the installation.
The job is implemented by the JITL SSH job. The script that will be exexuted to check the prerequisites is defined in the script tag of the job. The order has the following parameters for this step.
installation_definition_file
<?xml version="1.0" encoding="ISO-8859-1"?> <job > <params > <param name="installation_definition_file" value="c:\temp\batchinstall.xml"/> <param name="filter_install_host" value=""/> <param name="filter_install_port" value=""/> <param name="installation_job_chain" value="/batch_install_universal_agent/universal_agent_installer"/>01 <param name="update" value="true"/> </params> <script language="java" java_class_path="" java_class="com.sos.jitl.agentbatchinstaller.JSUniversalAgentBatchInstallerJSAdapterClass"/> <run_time /> </job>
Folder setup
The folder setup contains the files that will be transfered to the Agents server during the installation process
Installation script
Template for the instance script
The JobScheduler Universal Agent
Job Chain
Job Batch Installer
XSD for installation input file
You will find the xsd-schema for the configuration in config/scheduler_universal_agent_installations.xsd
Example for the installation input file
The files have the following structure
- installations
- global
- installation
installation
Example for xml configuration file:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <installations lastRun="2015-11-09 16:11"> <globals> <install_path>/home/test/temp/xxx</install_path> <transfer> <operation>copy</operation> <file_spec>.*\.(sh|gz)$</file_spec> <source> <host>localhost</host> <port></port> <protocol>local</protocol> <user></user> <password></password> <dir>C:\Temp\batch_install</dir> <ssh_auth_method>ssh_auth_method</ssh_auth_method> <ssh_auth_file></ssh_auth_file> </source> <target> <host>192.11.0.116</host> <port>22</port> <protocol>sftp</protocol> <user>test</user> <password>12345</password> <dir>/home/test/temp/</dir> <ssh_auth_method>password</ssh_auth_method> <ssh_auth_file></ssh_auth_file> </target> </transfer> <ssh> <host>192.11.0.116</host> <port>22</port> </ssh> </globals> <installation lastRun="2015-11-09 16:11"> <install_path></install_path> <agent_options> <java_home>java home</java_home> <java_options>java options</java_options> <scheduler_home>scheduler home</scheduler_home> <scheduler_http_port>4445</scheduler_http_port> <scheduler_ip_address>192.11.0.116</scheduler_ip_address> <scheduler_user>scheduler user</scheduler_user> <scheduler_log_dir>scheduler log dir</scheduler_log_dir> <scheduler_kill_script>scheduler kill script</scheduler_kill_script> <scheduler_pid_file_dir>scheduler pid file dir</scheduler_pid_file_dir> </agent_options> <transfer> <operation>copy</operation> <file_spec>.*\.(sh|gz)$</file_spec> <source> <host></host> <port></port> <protocol></protocol> <user></user> <password></password> <ssh_auth_method>ssh_auth_method</ssh_auth_method> <ssh_auth_file></ssh_auth_file> </source> <target> <host></host> <port></port> <protocol></protocol> <user></user> <password></password> <ssh_auth_method></ssh_auth_method> <ssh_auth_file></ssh_auth_file> </target> </transfer> <ssh> <user>test</user> <password>12345</password> <sudo_password>12345</sudo_password> <auth_method>password</auth_method> <auth_file></auth_file> <port>22</port> </ssh> <postprocessing> <command>echo ${ssh.sudo_password} | sudo cp ${install_path}/bin/jobscheduler_agent.sh /etc/init.d/jobscheduler</command> <command>echo ${ssh.sudo_password} | sudo update-rc.d jobscheduler defaults</command> </postprocessing> </installation> </installations
In the global entry, you specify the values which are used for all installation items. The installation items can overwrite these values. There is one installation item for each agent. You can make use of substitution in the values of the entries.
<command>echo ${ssh.sudo_password} | sudo cp ${install_path}/bin/jobscheduler_agent.sh /etc/init.d/jobscheduler</command>