You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 13 Next »

 

The Agent Batch/Update solution comes with a job chain in JobScheduler and the job AgentBatchInstaller.

The Job AgentBatchInstaller reads a xml configuration file and creates orders for the installation job chain to setup/update agents. For each agent there is an entry in the configuration file.

The Parameters for the job chain steps will be set by the job AgentBatchInstaller while reading the xml input file.

FEATURE AVAILABILITY STARTING FROM RELEASE 1.10

The JobScheduler Universal Agent Batch Installer is available starting starting from release 1.10.2.

 

Universal Agent Batch Installer Job Chain

 

universal_agent_installer.job_chain.xml

The job chain definitin

<job_chain  name="universal_agent_installer">
    <job_chain_node  state="CheckPreRequisite" job="CheckPreRequisite" next_state="TransferInstallationSetup" error_state="error"/>
    <job_chain_node  state="TransferInstallationSetup" job="TransferInstallationSetup" next_state="PerformInstall" error_state="error"/>
    <job_chain_node  state="PerformInstall" job="PerformInstall" next_state="success" error_state="error" on_error="suspend"/>
    <job_chain_node  state="success"/>
    <job_chain_node  state="error"/>
</job_chain>

 

The job chain for update/installation of one Agent has the nodes:

  • CheckPreRequisite
    • Checking for the correct java version >= 1.8
    • Checking whether JobScheduler Universal Agent already has been installed.
      • If so
        • Shutting down a probably running instance
        • remove the installation
  • TransferInstallaionSetup: To transfer the setup jar file to the host
    • Before the transfer starts, the preprocessor will create the instancescript.sh from the template jobscheduler_agent_instance_script.txt
    • The files, that will be transerfered are specified in globals/transfer/file_spec=.*\.(sh|gz)$

    • With this regular expression, these files will be transfered
      • The JobScheduler Universal Agent: jobscheduler_unix_universal_agent.1.11.0-SNAPSHOT.tar.gz
      • The instancescript: jobscheduler_agent_<port>.sh
      • The installation scipt: jobscheduler_universal_agent_install.sh
  • PerformInstall: To start the installation using SSH.
    • The in the step before transfered installation script will be executed
    • The installation script will install and start the JobScheduler Universal Agent
    • Finally the postprocessing commands will be executed
      • The postprocessing commands are stored in installation/postprocessing/command
      • Three commands will be executed
        • if you have more than three commands, please add the call in the job PerformInstall
      • The default commands will setup the autostart for JobScheduler Universal Agent on an Ubuntu server
        • echo ${ssh.sudo_password} | sudo -S cp ${install_path}/bin/jobscheduler_agent.sh /etc/init.d/jobscheduler
        • echo ${ssh.sudo_password} | sudo -S update-rc.d jobscheduler defaults

Parameters

All jobs have access to the following parameters. These parameters also will be used for the JobScheduler Universal Agent instance script to the options for the start.

 

ParameterDescriptionXML
agent_options.scheduler_ip_address
The ip address of the JobScheduler Universal Agents hostinstallation/agent_options/scheduler_ip_address
agent_options.scheduler_http_port
The port of the JobScheduler Universal Agentinstallation/agent_options/scheduler_http_port
agent_options.java_home
 installation/agent_options/java_home
agent_options.java_options
 installation/agent_options/java_options
agent_options.scheduler_home

If SCHEDULER_HOME is not set, the default will be set.

Default: install_path + "/jobscheduler_agent"

installation/agent_options/scheduler_home
agent_options.scheduler_user
 installation/agent_options/scheduler_user
agent_options.scheduler_log_dir
 installation/agent_options/scheduler_log_dir
agent_options.scheduler_kill_script
 installation/agent_options/scheduler_kill_script
agent_options.scheduler_pid_file_dir
 installation/agent_options/scheduler_pid_file_dir
host
The ssh ip address of the JobScheduler Universal Agents hostinstallation/ssh/host
port
The ssh port of the JobScheduler Universal Agentinstallation/ssh/port
user
The ssh user of the JobScheduler Universal Agentinstallation/ssh/user
auth_method
The ssh auth method of the JobScheduler Universal Agentinstallation/ssh/auth_method
auth_file
The ssh auth file of the JobScheduler Universal Agentinstallation/ssh/auth_file
password
The ssh password  of the JobScheduler Universal Agentinstallation/ssh/password
sudo_password
The sudo password for ssh userinstallation/ssh/sudo_password
install_path
The folder, where JobScheduler Universal Agent will be installedinstallation/install_path

 

TransferInstallationSetup

For more detailed explanation of the parameters have a look at the yade documentaiton.

Please note that it is recommended only to specify the settings file in the globals entry and use the default profile name <host>:<port>. Then the entry installation/transfer is not neccessary in the installation input file

settings
The file with the profiles, one for each JobSchedule Universal Agentinstallation/transfert/settings
profile
The name of the profile for the actual installation. Default=<host>:<port>installation/transfer/profile
file_spec
a regular expression that defines the set of files that should be transferedinstallation/transfer/file_spec
target_host
the host to which the files should be transferedinstallation/transfer/target/host
target_port
the port according to the given protocol (e.g. 21 when protocol is ftp)installation/transfer/target/port
target_protocol
the protocol with wich the target will be transered (ftp, sftp, file, local)installation/transfer/target/protocol
target_user
the user (if the protocol needs one) with wich the target will be transeredinstallation/transfer/target/user
target_password
the password (if the protocol needs one) with wich the target will be transeredinstallation/transfer/target/password
target_dir
the location where the files will be copied toinstallation/transfer/target/dir
target_ssh_auth_method
the authentication method if the protocol is ssh.installation/transfer/target/auth_method
target_ssh_auth_file
the authentication file if the protocol is ssh and auth_method is publickeyinstallation/transfer/target/auth_file
source_host
the host from where the files should be transferedinstallation/transfer/source/host
source_port
the port according to the given protocol (e.g. 21 when protocol is ftp)installation/transfer/source/port
source
the protocol with wich the source will be transered (ftp, sftp, file, local)installation/transfer/source/protocol
source_user
the user (if the protocol needs one) with wich the source will be transeredinstallation/transfer/source/user
source_password
the password (if the protocol needs one) with wich the source will be transeredinstallation/transfer/source/password
source_dir
the location from where the files will be copiedinstallation/transfer/source/dir
source_ssh_auth_method
the authentication method if the protocol is ssh.installation/transfer/source/auth_method
source_ssh_auth_file
the authentication file if the protocol is ssh and auth_method is publickeyinstallation/transfer/source/auth_file

PerformInstall

For more detailed explanation of the parameters have a look at the jitl ssh job documentaiton

ParameterDescriptionXML
command_n
n = 0...  The commands for the postprocessing. E.g. enabling the autostart of the Universal Agentinstallation/postprocessing/command
command_counter
The number of given command 

Starting the update/installation

The update/installation procedure will be started by running the job AgentBatchInstaller.

The job AgentBatchInstaller

The job AgentBatchInstaller reads the input file and creates orders for each element <installation>

The element <installation> defines the source and the target of the installation files, the port for the JobScheduler Universal Agent and the ssh connection parameters. Please note

  • For all elements you can define defaults in the globals section
  • You can use variables for the values e.g. ${ssh.host}
  • It is recommended to specify the filename of the YADE settings in the globals element and use the default profile name. When doing this, no installation/transfer elements are needed

Parameters

ParameterDescriptionExample
installation_definition_file
The xml file that contains all parameters for the batch installationc:\batch\batchinstall.xml
filter_install_host
A host name if only one order should be created that matches the given hostmyAgent
filter_install_port
A port if only one order should be created that matches the given host

4445

installation_job_chain
The name of the job chain for which the orders should be created/batch_install_universal_agent/universal_agent_installer
update

If true an order for all <instances> will be created.

If false an order for <instances> that do not have the last_run attribute will be created

true

Customizing the installation process

You can customize the installation process.

  • Changing the script part in the ssh jobs CheckPreRequisite and PerformInstall e.g. if you want to check other prerequisites or execute some commands before or after the installation.
  • Changing the installation script jobscheduler_universal_agent_install.sh
  • Changing the template for instance start script jobscheduler_agent_instance_script.txt

Checking Prerequisites

The job CheckPreRequisite checks the java version, shutdown a probably running JobScheduler Universal Agent and removes the lib folder from  the old installation.

echo .JobScheduler Universal Agent
echo .checking prerequisites ${install_path}
echo .checking Java Runtime Version >= 1.8? ......
JAVA_VER=$(java -version 2>&1 | sed 's/java version "\(.*\)\.\(.*\)\..*"/\1\2/; 1q')
if [ "$JAVA_VER" > "18" ]
then
  echo ..java version: $JAVA_VER ok
else
  echo ..wrong Java version found
  echo ..expectet version is >= 1.8
  echo ..installed version is $JAVA_VER
  exit 8
fi
echo .checking whether JobScheduler Universal Agent is already installed?
if [ -f ${install_path}/jobscheduler_agent/bin/jobscheduler_agent.sh ]
then
  echo ..JobScheduler Universal Agent already installed
  echo ..shutting down JobScheduler Universal Agent 
  ${install_path}/jobscheduler_agent/bin/jobscheduler_agent.sh stop
  echo ..removing installation from  ${install_path}
  rm -Rf ${install_path}/lib
else
  echo .. this is the first installation of JobScheduler Universal Agent 
fi
exit $?

 

PerformInstall

The job PerformInstall calls the installation script and executes the commands given in the <postprocessing> element.

echo Executing PerformInstall ${target_dir}
chmod a+x ${target_dir}/jobscheduler_universal_agent_install.sh 
${target_dir}/jobscheduler_universal_agent_install.sh -d${install_path} -p${agent_options.scheduler_http_port}
${command_0}
${command_1}
${command_2}

 

The installation script

The installation script unzip the installation file and starts the JobScheduler Universal Agent

#!/bin/sh 
# Copyright (c) 2015 SOS GmbH, Berlin, Germany.
# All rights reserved.
export TERM=dumb
  clear
  echo "+---------------------------------------------------------+"
  echo "*     JobScheduler Univeral Agent                         *"
  echo "*     SOS Software- und Organisations Service GmbH        *" 
  echo "*     Giesebrechtstr. 15                                  *"
  echo "*     10629 Berlin                                        *"
  echo "*     ----------------------------------------------      *"
  echo "*     info@sos-berlin.com                                 *"
  echo "*     http://www.sos-berlin.com                           *"
  echo "+---------------------------------------------------------+"
  echo ""
  echo ""
  echo "+---------------------------------------------------------+"
  echo "*  This will install JobScheduler Universal Agent now ... *"
  echo "+---------------------------------------------------------+"
 
# -----------------------------------------------
cd `dirname $0`
  
SETUP_WORKING_DIR=`pwd` 
SETUP_INSTALL_PATH=""
SHOW_EXTRACTED_FILESNAMES=0
SETUP_LOG_FILE="$SETUP_WORKING_DIR/jobscheduler_universal_install.log"
SETUP_ERR=0
 


# ----------------------------------------------------------------- 
log_write() {
  case "$1" in
      0  ) msg_hint="[info]   ";;
      1  ) msg_hint="[warning]";;
      2  ) msg_hint="[error]  ";;
      3  ) msg_hint="[fatal]  ";;
      *  ) msg_hint="[info]   ";;
  esac
  timestamp=`date "+%Y-%m-%d %T"`
  msg="$timestamp   $msg_hint   $2"
  echo "$msg"
  echo "$msg">>"$SETUP_LOG_FILE"
}

show_usage() {
  echo "`basename $1` will install the 'JobScheduler Universal Agent' at [install_dir]." 
  echo "install_dir=Path with the JobScheduler Universal Agent"
  echo
  echo "Usage: `basename $1` [OPTION]"
  echo
  echo "Samples:"
  echo "  `basename $1` -co  -d/home/jobscheduler  -p4445 "      
  echo 
  echo "Options:"
  echo "  -h, --help                          | Shows this usage"
  echo "  -s, --show                          | show all extracted filenames"
  echo "  -w  --install_dir                   | Path of the JobScheduler Universal Agent installation"
  echo "  -p  --port                          | Port of JobScheduler Universal Agent. Default=4445"
 
}
 
# ----------------------------------------------------------------------------usage
if [ $# -eq 0 ]
then
  show_usage $0
  exit 65
fi 



# --------------------------------------------
     

for arg in "$@"
do
  long_opt=0
  echo "`expr match $arg '\(..\)'`" 
  case "`expr match $arg '\(..\)'`" in
    "-h"        )  show_usage $0; exit 64;;
    "-d"        )  SETUP_INSTALL_PATH="`expr match $arg '-d\(.*\)'`";;
    "-p"        )  UNIVERSAL_AGENT_PORT="`expr match $arg '-p\(.*\)'`";;
    "-s"        )  SHOW_EXTRACTED_FILESNAMES="`expr match $arg '-d\(.*\)'`";;
    "--"        )  long_opt=1;;
  esac 
  
  if [ "$long_opt" -eq 1 ]
  then
  case `expr match "$arg" '\(--[^=]*\)'` in
    "--show_extracted_filenames"    )  SHOW_EXTRACTED_FILESNAMES=`expr match "$arg" '--.*=\(.*\)'`;;
    "--help"    )  show_usage $0; exit 64;;
    "--install_dir"    )  SETUP_INSTALL_PATH=`expr match "$arg" '--.*=\(.*\)'`;;
    "--port"    )  UNIVERSAL_AGENT_PORT=`expr match "$arg" '--.*=\(.*\)'`;;
  esac
  fi

done

 
# ---------------

  
check_dir(){
     SETUP_INSTALL_PATH="$1"
     echo SETUP_INSTALL_PATH="$SETUP_INSTALL_PATH"

     if [ -d "$SETUP_INSTALL_PATH" ]
     then
       if [ -r "$SETUP_INSTALL_PATH" ]
       then
         cd "$SETUP_INSTALL_PATH"
         SETUP_INSTALL_PATH=`pwd`
         chmod 700 "$SETUP_INSTALL_PATH" 2>/dev/null
         SETUP_ERR=$?
       else
         log_write 2 "$SETUP_INSTALL_PATH is not readable."
         SETUP_ERR=1
       fi
     else
       echo create $SETUP_INSTALL_PATH

       mkdir -m700 -p "$SETUP_INSTALL_PATH" 2>/dev/null
       SETUP_ERR=$?
       if [ $SETUP_ERR -eq 0 ]
       then
      
         cd  "$SETUP_INSTALL_PATH"
         SETUP_INSTALL_PATH=`pwd`
       else
         log_write 2 "$SETUP_INSTALL_PATH can not create."
         SETUP_ERR=1 
       fi
     fi
     
     
     cd  "$SETUP_WORKING_DIR"
     if [ $SETUP_ERR -ne 0 ]
     then
        log_write 2 "You haven't the necessary rights on the installation directory $SETUP_INSTALL_PATH."
     exit $SETUP_ERR
fi
}

tarx (){
  if [ -f "$1.gz" ]
  then
     gzip -fd $1.gz
  fi
  
  if [ ! -d "$2" ]
  then
      mkdir $2
  fi
 
  cd  $2
  tar $tarpar $SETUP_WORKING_DIR/$1
  cd  $SETUP_WORKING_DIR
} 

tarpar="-xf "

if [ "$SHOW_EXTRACTED_FILESNAMES" = "1" ]
then
   tarpar="-xvf "
fi


SETUP_INSTALL_PATH="$SETUP_INSTALL_PATH"

echo SETUP_INSTALL_PATH=$SETUP_INSTALL_PATH
check_dir $SETUP_INSTALL_PATH
  

# Installing
log_write 0 "Installing --> $SETUP_INSTALL_PATH"
tarx  jobscheduler_unix_universal_agent.1.11.0-SNAPSHOT.tar  $SETUP_INSTALL_PATH 

if [ -z "$UNIVERSAL_AGENT_PORT" ]
then
$UNIVERSAL_AGENT_PORT=4445
fi

mv `dirname $0`/jobscheduler_agent_$UNIVERSAL_AGENT_PORT.sh $SETUP_INSTALL_PATH/jobscheduler_agent/bin 
chmod a+x  $SETUP_INSTALL_PATH/jobscheduler_agent/bin/jobscheduler_agent_$UNIVERSAL_AGENT_PORT.sh   
log_write 0 "Start JobScheduler Universal Agent"
log_write 0 $SETUP_INSTALL_PATH/jobscheduler_agent/bin/jobscheduler_agent_$UNIVERSAL_AGENT_PORT.sh start
$SETUP_INSTALL_PATH/jobscheduler_agent/bin/jobscheduler_agent_$UNIVERSAL_AGENT_PORT.sh start 1 > /dev/null

exit $?

 

The installation input File

The installation input file contains a <installation> element for each JobScheduler Universal Agent instance.

The minimum Example for the xml configuration file:

 <?xml version="1.0" encoding="utf-8"?>
<installations lastRun="2015-11-06 15:11">
  <globals>
    <transfer>
      <settings><![CDATA[C:\Temp\batch_install\yade_agent_batch_installer_settings.ini]]></settings>
    </transfer>
  </globals>
  <installation lastRun="2015-11-06 15:11">
    <install_path><![CDATA[/home/test/temp/xxx]]></install_path>
    <agent_options>
      <scheduler_http_port>4445</scheduler_http_port>
      <scheduler_ip_address><![CDATA[192.11.0.116]]></scheduler_ip_address>
    </agent_options>
    <ssh>
      <user><![CDATA[test]]></user>
      <password><![CDATA[12345]]></password>
      <sudo_password><![CDATA[12345]]></sudo_password>
      <auth_method><![CDATA[password]]></auth_method>
      <port><![CDATA[22]]></port>
    </ssh>
    <postprocessing>
      <command><![CDATA[echo ${ssh.sudo_password} | sudo cp ${install_path}/bin/jobscheduler_agent.sh /etc/init.d/jobscheduler]]></command>
      <command><![CDATA[echo ${ssh.sudo_password} | sudo update-rc.d jobscheduler defaults]]></command>
    </postprocessing>
  </installation>
</installations>

 

 

In the global entry, you specify the values which are used for all installation items. The installation items can overwrite these values. There is one installation item for each agent. You can make use of substitution in the values of the entries.

            <command>echo ${ssh.sudo_password} | sudo cp ${install_path}/bin/jobscheduler_agent.sh /etc/init.d/jobscheduler</command>

 

 

Using the XML Editor

Then XML Editor can be used to manage the installation input file. It is also used th manage the YADE profiles for the transfer of the installation file

The XML Editor is available here: http://www.sos-berlin.com/xml-editor-downloads

After downloading the file, extrac the zip file. Then start "sos.xml.editor.exe"

Open your installtion input file an assign the schema scheduler_universal_agent_installations.xsd

Managing the YADE profiles for the file transfer

Each installation has a profile in the YADE settings file. The default profile name is <host>:<port> where <host> will be substituted with the value in installation/agent_option/scheduler_ip_address and <port> will be substituted with the value in installation/agent_option/scheduler_http_port.

  1. Please specify the name of the YADE settings file in globals/settings.
  2. Open the YADE settings file (or create a new one).
  3. Assign the schema http://www.sos-berlin.com/schema/yade/YADE_configuration_v1.0.xsd
  4. Add <ProtocolFragments> for each host you want install JobScheduler Universal Agent
  5. Add <Profile> for each host you want install JobScheduler Universal Agent.
  6. The default for the profile_id is <host>:<port> If you do not use the default, you have to specify the profile name in the installations input file.

 

Installation of the Batch Installer for the JobScheduler Universal Agent

Unzip the JobScheduler Universal Agent Batch Installer zip file.

  1. copy the content of the live folder to the live folder of a JobScheduler instance
  2. copy the content of the folder batch_install to any folder
  3. create an installation input file
  4. adjust the parameter values for the job AgentBatchInstaller
  5. set the name for settings file in the installation input file (globals/transfer/settings)
  6. create the YADE settings file
    1. Add <ProtocolFragments> for each host you want install JobScheduler Universal Agent
    2. Add a <Profile> for each host you want install JobScheduler Universal Agent.

 

  • No labels