JS7 JobScheduler

Space shortcuts

Page tree

Introduction

JS7 - Job Resources are a means to make variables available that can be shared by a number of jobs. Variables from Job Resources are exposed to:

  • Shell jobs from OS environment variables
  • JVM jobs from the JS7 - Job API

Job Resources are assigned a workflow to make variables available to all included jobs or they are assigned invidual jobs. When a Job Resource is updated then JS7 takes care that all Agents assigned to jobs that make use of the Job Resource will receive the updated copy of the Job Resource.

Typical use cases include:

  • to make global settings such as paths etc. available from environment variables for shell jobs.
  • to store encrypted secrets to Job Resources variables that can be decrypted by jobs. For details see JS7 - How to encrypt and decrypt.
  • to store configuration files to Job Resource variables. The JS7 will make the configuration file available from temporary files per job instance with releated Agents. Configuration files can be encrypted before being added to Job Resources, accordingly jobs can decrypt configuration files on-the-fly. On termination of the job instance temporary files are removed by JS7 Agents.

The JS7 offers to perform any operation on Job Resources by the JS7 - REST Web Service API, for details see the /inventory resource from the Technical Documentation of the REST Web Service API.

The Job Resource Update Script introduced in this article can be used to create and to update variables to JS7 - Job Resources.

An implementation for use with workflows is available from the JS7 - JITL SetJobResourceJob.

Job Resource Update Script

The Job Resource Update Script is provided for download and can be used to automate updates of Job Resources.

  • The script is available for Linux and MacOS® using bash, dash, ksh and zsh POSIX-compatible shells.
  • The script terminates with exit code 0 to signal success, with exit code 1 for command line argument errors and with exit code 4 for non-recoverable errors.
  • The script is intended as a baseline example for customization by JS7 users and by SOS within the scope of professional services.

Prerequisites

The Job Resource Update Script makes use of the curl utility.

The Job Resource Update Script requires the jq utility to be available from the operating system. 

jq ships with the MIT license, see https://opensource.org/licenses/MIT.

Download

The Job Resource Update Script is available from the Utilities/Deployment Management section of: JS7 - Download

Usage

Invoking the script without arguments displays the usage clause:


Usage
Usage: js7_set_job_resource.sh [Options] [Switches]

  Options:
    --url=<url>                   | required: JOC Cockpit URL
    --controller-id=<identifier>  | required: Controller ID
    --user=<account>              | required: JOC Cockpit user account
    --password=<password>         | optional: JOC Cockpit password or Client Keystore password
    --job-resource=<path>         | required: path to job resource
    --key=<identifier>            | required: name of argument in job resource
    --value=<string>              | alternate: value of argument in job resource
    --file=<path>                 | alternate: value of argument in job resource from a file
    --env-var=<identifier>        | optional: name of environment variable in job resource
    --java-home=<directory>       | optional: Java Home directory for use with encryption
    --java-lib=<directory>        | optional: Java library directory for use with encryption, default: ../lib
    --java-options=<options>      | optional: Java Options for use with encryption
    --encrypt-cert=<path>         | optional: path to Agent's certificate or public key used to encrypt value
    --cacert=<path>               | optional: path to Root CA Certificate for HTTPS connections
    --client-cert=<path>          | optional: path to Client Certificate file (PEM) for HTTPS mutual authentication
    --client-key=<path>           | optional: path to Client Private Key file (PEM) for HTTPS mutual authentication
    --client-keystore=<path>      | optional: path to Client Keystore file (PKCS12) holding Private Key/Certificate for HTTPS mutual authentication
    --timeout=<seconds>           | optional: timeout for connections to JOC Cockpit in seconds, default: 15
    --log-dir=<directory>         | optional: log directory for log output of this script

  Switches:
    -h | --help                   | displays usage
    -v | --verbose                | shows verbose log output
    -p | --password               | asks for password
    --show-logs                   | shows log output if --log-dir is used
    --make-dirs                   | creates directories if they do not exist

Options

  • --url
  • --controller-id
    • Specifies the Controller ID to which the updated job resource should be deployed.
  • --user
    • Specifies the user account for login to JOC Cockpit. Specification of the user account can be omitted if a JS7 - Identity Services is available for Client authentication certificates that are specified with the --client-cert and --client-key options.
  • --password
    • For user/password authentication the --password option can be used to specify a password and the -p switch can be used to prompt for secure input of a password.
  • --job-resource
    • Specifies the path to the job resource in the JOC Cockpit inventory.
  • --key
    • Specifies the name of an argument in the job resource specified with the --job-resource option.
    • Job Resource arguments are case-sensitive.
  • --value
    • Specifies the value of the argument in the job resource specified with the --job-resource option.
  • --file
    • Job Resources can be used to transfer files to related Agents. The option expects the path to a file that will be added the Job Resource.
    • The --file option is an alternative to use of the --value option.
  • --env-var
    • Optionally specifies the name of an environment variable in the job resource specified with the --job-resource option.
    • The environment variable will be assigned a reference to the argument specified by the --key option to hold the same value as the argument.
    • Job Resource environment variables are case-sensitive.
  • --java-home
    • Specifies the Java home directory. Java is required if Job Resource values should be encrypted.
  • --java-lib
    • Specifies the directory in which Java libraries are available if Job Resource values should be encrypted.
    • The download archive for JS7 Encryption holds the Job Resource Update Script and includes the lib sub-directory with related Java libraries. By default the lib directory will be used assuming that the Job Resource Update Script is located in the bin directory at the same file system hierarchy level.
  • --java-options
    • Specifies the Java options that are used if Job Resource values should be encrypted.
    • If more than one Java option is used then the value has to be quoted, for example --java-options="-Xms64m -Xmx128m".
  • --encrypt-cert
    • Specifies the path to a certificate file or public key file that will be used to encrypt the value of the Job Resource. This applies to values specified with the --value option and --file option. For details see JS7 - Encryption and Decryption.
    • An Agent holding the matching private key can decrypt the values of encrypted Job Resource variables, for details see JS7 - How to encrypt and decrypt.
    • The Job Resource Update Script will create a one-time symmetric key that is used to encrypt the value of the Job Resource variable. The symmetric key will be encrypted by use of the recipient's certificate/public key. The value of the Job Resource variable will be assigned the following items separated by spaces: the encrypted copy of the symmetric key, an initialization vector and encrypted value. This guarantees that the owner of the private key, typically a JS7 Agent, is the only party able to decrypt the Job Resource variable's value. Encryption is performed by the Job Resource Update Script before the Job Resource is added using the JS7 REST Web Service API.
  • --cacert
    • Specifies the path to a .pem file that holds the Root CA Certificate and optionally Intermediate CA Certificates to verify HTTPS connections to JOC Cockpit.
  • --client-cert
    • Specifies the path to a .pem file that holds the Client Certificate if HTTPS mutual authentication is used..
  • --client-key
    • Specifies the path to a .pem file that holds the Client Private Key if HTTPS mutual authentication is used..
  • --client-keystore
    • Specifies the path to a keystore that holds the private key and certificate used for HTTPS mutual authentication. The keystore is expected in PKCS12 format, typically using the .p12 or .pft extension.
    • Use of a keystore is an alternative to specifying the --client-key and --client-cert options.
    • If the keystore is protected then the --password option specifies the password.
  • --timeout
    • Specifies the timeout for connecting to JOC Cockpit. If the connection is not established and the timeout is exceeded then the script will be terminated.
    • By default the timeout is 15s.
  • --log-dir
    • If a log directory is specified then the script will log information about processing steps to a log file in this directory.
    • File names are created according to the pattern: set_job_resource.<yyyy>-<MM>-<dd>T<hh>-<mm>-<ss>.log
    • For example: set_job_resource.2022-03-19T20-50-45.log

Switches

  • -h | --help
    • Displays usage.
  • -v | --verbose
    • Displays verbose log output.
  • -p | --password
    • Asks for interactive keyboard input for the password as a secure alternative to use of the --password= option
  • --show-logs
    • Displays the log output created by the script if the --log-dir option is used.
  • --make-dirs
    • If directories are missing that are indicated with the --home, --backup-dir or --log-dir options then they will be created.

Exit Codes

  • 1: argument errors
  • 4: this exit code is returned if the JS7 REST Web Service is not reachable or reports errors

Examples

The following examples illustrate typical use cases.

Update Job Resource Variable using HTTP Connection

Example for Update of a Job Resource Variable
./js7_set_job_resource.sh \
    --url=http://joc-2-0-primary:7446 \
    --controller-id=controller \
    --user=root \
    --password=root \
    --job-resource=/ProductDemo/Variables/pdBusinessDate \
    --key=businessDate \
    --env-var=BUSINESS_DATE \
    --value=$(date +'%Y-%m-%d')

# establishes the connection to JOC Cockpit by HTTP
# updates the "businessDate" Job Resource variable from the current date in the local time zone
# receiving jobs can use the "BUSINESS_DATE" environment variable to receive the current value

Update Job Resource Variable using HTTPS Connection

Example for Update of a Job Resource Variable
./js7_set_job_resource.sh \
    --url=https://joc-2-0-primary:7443 \
    --cacert=/home/sos/jstest/certs/root-ca.pem \
    --controller-id=controller \
    --user=root \
    -p \
    --job-resource=/ProductDemo/Variables/pdBusinessDate \
    --key=businessDate \
    --env-var=BUSINESS_DATE \
    --value=$(TZ=Europe/London date +'%Y-%m-%d')

# establishes the connection to JOC Cockpit by HTTPS using the Root CA Certificate specified from the path to a .pem file
# updates the "businessDate" Job Resource variable from the current date in the London time zone
# updates the "BUSINESS_DATE" environment variable that is assigned the same date from a reference to the variable
# prompts the user for input of the password

Authenticate from Client Certificate using HTTPS Connection

Example for Update of a Job Resource Variable
./js7_set_job_resource.sh \
    --url=https://joc-2-0-primary:7443 \
    --cacert=$HOME/certs/root-ca.crt \
    --client-key=$HOME/certs/centostest-primary.key \
    --client-cert=$HOME/certs/centostest-primary.crt \
    --controller-id=controller \
    --user=root \
    --job-resource=/ProductDemo/Variables/pdBusinessDate \
    --key=businessDate \
    --value=$(TZ=Europe/London date +'%Y-%m-%d')

# updates the "businessDate" Job Resource variable from the current date in the London time zone
# establishes the connection to JOC Cockpit by HTTPS using a Client Key and Client Certificate for authentication

Update Job Resource Variable from File

Example for Update of a Job Resource Variable
./js7_set_job_resource.sh \
    --url=http://joc-2-0-primary:7446 \
    --controller-id=controller \
    --user=root \
    --password=root \
    --job-resource=/ProductDemo/Variables/pdConfigurationData \
    --key=configurationData \
    --env-var=CONFIGURATION_DATA \
    --file=./config.ini

# updates the "configurationData" Job Resource variable from the configuration file specified
# receiving jobs use the "CONFIGURATION_DATA" environment variable that specifies the path to a temporary file holding the configuration data

Update Job Resource Variable from Encrypted Value

Example for Update of a Job Resource Variable
./js7_set_job_resource.sh \
    --url=http://joc-2-0-primary:7446 \
    --controller-id=controller \
    --user=root \
    --password=root \
    --job-resource=/ProductDemo/Variables/pdBusinessSecret \
    --key=businessSecret \
    --value='secret' \
    --encrypt-cert=./foobar.crt
 
# updates the "businessSecret" Job Resource variable from the given value
# encrypts the value of the "businessSecret" variable using an Agent's certificate from the foobar.crt file

Update Job Resource Variable from Encrypted File

Example for Update of a Job Resource Variable
./js7_set_job_resource.sh \
    --url=http://joc-2-0-primary:7446 \
    --controller-id=controller \
    --user=root \
    --password=root \
    --job-resource=/ProductDemo/Variables/pdConfigurationDataSecret \
    --key=configurationDataSecret \
    --env-var=CONFIGURATION_DATA_SECRET \
    --file=./config.ini \
    --encrypt-cert./foobar.crt

# updates the "configurationDataSecret" Job Resource variable from the configuration file specified
# encrypts the content of the file using the target Agent's certificate from the foobar.crt file
# creates the additional variable "configurationDateSecret_key" and the environment variable "CONFIGURATION_DATA_SECRET_key" that both hold the encrypted symmetric key and initialization vector

Resources