Introduction

Continuous Integration / Continuous Delivery (CI/CD) is one of the supported methods for JS7 - Rollout of Scheduling Objects.

The scenarios are described in the JS7 - Git Repository Interface.
CI/CD includes to perform the following steps
- in a test environment
  - to store scheduling objects of a JOC Cockpit inventory to a local Git repository,
  - to add, to commit and to push changes to the remote Git repository.
- in a prod environment
  - to copy (cherry-pick) changes from the remote test Git repository to the remote prod Git repository,
  - to push changes from the remote prod Git repository to the local prod Git repository,
  - to update the JOC Cockpit inventory from the local Git repository,
  - to release and to deploy scheduling objects to Controllers and Agents.

Users who wish to automate the steps for rollout can use the following resources:

The JS7 - REST Web Service API allows the same operations for Git integration to be performed as offered from the JOC Cockpit GUI.
- See Technical Documentation of the REST Web Service API
The JS7 - Unix Shell Command Line Interface offers simplified access to the REST Web Service API for scripting purposes.
- The Unix Shell examples when executed might prove to be instructive for logging of REST API calls when used with the -v switch.
The JS7 - PowerShell Module offers simplified access to the REST Web Service API for scripting purposes.
- The PowerShell examples when executed might prove to be instructive for logging of REST API calls when used with the -debug option.

The following examples assume that a local Git repository is set up in the test and prod environments - see JS7 - How to set up Git Access and JS7 - How to set up a local Git Repository.

Documentation for the relevant cmdlets is provided in the PowerShell CLI 2.0 - Cmdlets - Git Repository Integration article.

FEATURE AVAILABILITY STARTING FROM RELEASE 2.3.0

Steps to perform in the TEST Environment

The example below assumes that:

a top-level folder Accounting is used in the JOC Cockpit inventory,
this folder is mapped to a sub-directory in the file system with the same name and spelling, for example:
- for Unix: /var/sos-berlin.com/js7/joc/jetty_base/resources/joc/repositories/rollout/Accounting
- for Windows: C:\ProgramData\sos-berlin.com\js7˜\joc\jetty_base\resources\joc\repositories\rollout\Accounting
This folder is created when setting up the repository, otherwise it will be created by the JOC Cockpit the first time that objects are to be stored in the repository.

Storing Scheduling Objects to JOC Cockpit Git Repository

Storing Scheduling Objects using JS7 Unix Shell CLI

The following commands are used:

Example how to store changes to a Git repository using Unix Shell CLI

# common options for connection to JS7 REST API
request_options=(--url=http://localhost:4446 --user=root --password=root --controller-id=controller)

# store items to rollout repository and exclude draft versions from use
./deploy-git.sh store-item  "${request_options[@]}" --folder=/Accounting --recursive --no-draft

Storing Scheduling Objects using JS7 PowerShell Module

The following cmdlets are used:

Example how to store changes to a Git repository using PowerShell

Import-Module JS7
Connect-JS7 -Url http://root:root@localhost:4446 -Id 'controller' | Out-Null

# Store scheduling objects to rollout repository and exclude draft versions from use
Set-JS7RepositoryItem -Folder '/Accounting' -Recursive -NoDraft

Explanation:

Line 2: A number of methods for connecting to the JOC Cockpit are available, see JS7 - How to connect to JOC Cockpit using the PowerShell Module.
Line 5: The Accounting top-level folder and any sub-folders are removed from the local repository, for details see Remove-JS7RepositoryItem. This step is not required, however, it can be useful for consistency if scheduling objects have been removed or renamed in the JOC Cockpit inventory.
Line 8: The Accounting top-level folder and any sub-folders are added to the local repository, see Set-JS7RepositoryItem.

Committing and pushing Changes to remote Git Repository

Committing and pushing Changes using JS7 Unix Shell CLI

The following OS and Git commands implement a typical sequence of steps:

Example how to commit and push changes to a remote repository using Unix Shell

# common options for connection to JS7 REST API
request_options=(--url=http://localhost:4446 --user=root --password=root --controller-id=controller)

 # store items to rollout repository: folder
./deploy-git.sh store-item   "${request_options[@]}" --folder=/TestRepo --recursive

# add items to repository
./deploy-git.sh add          "${request_options[@]}" --folder=/TestRepo

# commit changes to repository and keep commit hash
hash=(./deploy-git.sh commit "${request_options[@]}" --folder=/TestRepo --message="v.1.23.34")

# push changes to remote repository
./deploy-git.sh push         "${request_options[@]}" --folder=/TestRepo

Committing and pushing Changes using JS7 PowerShell Module

The following OS and Git commands implement a typical sequence of steps:

Example how to commit and push changes to a remote repository using PowerShell

Import-Module JS7
Connect-JS7 -Url http://root:root@localhost:4446 -Id 'controller' | Out-Null

# Add changes to the remote Git repository
Invoke-JS7GitRepositoryAdd -Folder '/Accounting' | Out-Null

# Commit changes
$response = Invoke-JS7GitRepositoryCommit -Folder '/Accounting' -Message 'changes for release v1.13'
# Parse commit hash from response
if ( $matchInfo = ( $response.stdout | Select-String '^\[.* +([0-9a-z]+)\]' ) )
{
	$commitHash = $matchInfo.Matches.Groups[1].Value
}

# Push changes
Invoke-JS7GitRepositoryPush -Folder '/Accounting' | Out-Null

Example of a CI/CD Pipeline Script for TEST Environment

The example below includes use of the above cmdlets in a pipeline script.

Download PowerShell Script: Rollout-JS7FromTest.ps1

Example CI/CD pipeline script for test environment

# --- Parameterization ---

$url = 'http://root:root@test_host:4446'
$controllerId = 'Controller'
$folder = '/Accounting'


# --- Connection ---

Import-Module JS7
Connect-JS7 -Url $url -Id $controllerId | Out-Null


# --- Step 1: Check status of releasable and deployable objects ---

$items = Get-JS7ReleasableItem -Folder $folder -NoReleased
if ( $items.count )
{
    Write-Warning "CI/CD pipeline stopped: unreleased objects found:"
    foreach( $item in $items )
    {
        Write-Warning "unreleased object: type=$($item.objectType), valid=$($item.valid), folder=$($item.folder), name=$($item.objectName)"
    }

    # optionally release items
    $items | Publish-JS7ReleasableItem
}

$items = Get-JS7DeployableItem -Folder $folder -NoDeployed
if ( $items.count )
{
    Write-Warning "CI/CD pipeline stopped: undeployed objects found:"
    foreach( $item in $items )
    {
        Write-Warning "undeployed object: type=$($item.objectType), valid=$($item.valid), folder=$($item.folder), name=$($item.objectName)"
    }

    # optionally deploy items
    $items | Publish-JS7DeployableItem -ControllerId $controllerId
}


# --- Step 2: Cleanup repository and store scheduling objects to repository ---

# Cleanup local repository by removing existing objects
Remove-JS7RepositoryItem -Folder $folder
 
# Store scheduling objects of the given folder and exclude draft versions to be used
Set-JS7RepositoryItem -Folder $folder -Recursive -NoDraft


# --- Step 3: add objects, commit and push to remote repository ---

# Add changes to local staging area
Invoke-JS7GitRepositoryAdd -Folder $folder | Out-Null

# Commit changes
$response = Invoke-JS7GitRepositoryCommit -Folder $folder -Message 'changes to accounting jobs for v12.3'

# Parse commit hash from response
if ( $matchInfo = ( $response.stdout | Select-String '^\[.* +([0-9a-z]+)\]' ) )
{
	$commitHash = $matchInfo.Matches.Groups[1].Value
}

# Push changes to remote repository
Invoke-JS7GitRepositoryPush -Folder $folder | Out-Null


# --- Connection ---

Disconnect-JS7

Steps to perform in the PROD Environment

Copying Changes from TEST to PROD Git Repository

There are a number of ways of achieving this using Git commands:

Some users might wish to cherry-pick individual commits from a test Git repository as shown in the example below.
Some users might prefer to copy the complete test repository to a prod repository using branches.
Some users might prefer to simplify rollout by use of a single repository and will therefore skip this step.

Example how to pull changes to a local Git repository

# add the test repo as a remote repository 
git remote add oldrepo git@github.com:sos-berlin/js7-demo-inventory-rollout-test.git

# get the test repo commits
git remote update

# examine the whole tree
git log --all --oneline --graph --decorate

# copy/cherry-pick the commits from the test repo into your local prod repo
git cherry-pick <commit-hash>

# check local prod repo
git log

# push changes to remote prod repo
git push origin master

# remove the reference to the test repo
git remote remove oldrepo

Explanation:

Line 11: The <commit-hash> identifies commits performed when pushing scheduling objects to the test repository. This step can be performed repeatedly to cherry-pick a number of commits.

Pulling Changes to local Git Repository

Pulling Changes using Unix Shell CLI

The following OS and Git commands are used:

Example how to pull changes to a local Git repository using Unix Shell

# common options for connection to JS7 REST API
request_options=(--url=http://localhost:4446 --user=root --password=root --controller-id=controller)

# pull changes from remote repository
./deploy-git.sh pull "${request_options[@]}" --folder=/Accounting

Pulling Changes using PowerShell

The following OS and Git commands are used:

Example how to pull changes to a local Git repository using PowerShell

Import-Module JS7
Connect-JS7 -Url http://root:root@prod_host:4446 -Id 'Controller' | Out-Null

# Pull changes from remote Git repository
Invoke-JS7GitRepositoryPull -Folder '/Accounting' | Out-Null

Updating JOC Cockpit Inventory from local Git Repository, Release and Deploy to Controller

Updating JOC Cockpit Inventory using Unix Shell CLI

The following commands are used:

Example how to update JOC Cockpit inventory from changes to a local Git repository using PowerShell

# common options for connection to JS7 REST API
request_options=(--url=http://localhost:4446 --user=root --password=root --controller-id=controller)

# update items from rollout repository: folder
./deploy-git.sh update-item "${request_options[@]}" --folder=/Accounting

# deploy objects from folder recursively and update daily plan
./deploy-workflow.sh deploy "${request_options[@]}" --folder=/Accounting --recursive --date-from=now

# release objects from folder and update daily plan
./deploy-workflow.sh release "${request_options[@]}" --folder=/Accounting --recursive --date-from=now

Updating JOC Cockpit Inventory using PowerShell

The following cmdlets are used:

Example how to update JOC Cockpit inventory from changes to a local Git repository using PowerShell

Import-Module JS7
Connect-JS7 -Url http://root:root@prod_host:4446 -Id 'controller' | Out-Null

# Update the JOC Cockpit inventory from the local repository of the given folder
Update-JS7FromRepositoryItem -Folder '/Accounting'

# Release scheduling objects of the given folder
Publish-JS7ReleasableItem -Folder '/Accounting' -Recursive -NoReleased

# Deploy scheduling objects from the given folder to the Controller
Publish-JS7DeployableItem -ControllerId 'Controller' -Folder '/Accounting' -Recursive -NoDeployed

Explanation:

Line 5: The JOC Cockpit inventory is updated from the local repository. This translates to the fact that objects with the same name are overwritten and that new objects are added. Updated objects are put to a draft status in the JOC Cockpit inventory. Find details in Update-JS7FromRepositoryItem.
Line 8: Scheduling objects that are not deployed to a Controller such as schedules and calendars are released. If such objects are included in the changes to the local repository then they are in a draft status and should be released. The -NoReleased parameter prevents scheduling objects with a non-draft status from being released, for example objects that are not stored in the local repository but which have previously been released. For details see Publish-JS7ReleasableItem.
Line 11: Scheduling objects are deployed to the given Controller. The -NoDeployed parameter prevents scheduling objects with a non-draft status from being deployed, for example objects that are not stored in the local repository but which have previously been deployed. For details see Publish-JS7DeployableItem.

Example of a CI/CD Pipeline Script for PROD Environment

The example below shows use of the cmdlets described above in a pipeline script.

Download PowerShell Script: Rollout-JS7ToProd.ps1

Example CI/CD pipeline script for prod environment

# --- Parameterization ---

$url = 'http://root:root@prod_host:4446'
$controllerId = 'Controller'
$folder = '/Accounting'

$repo = 'git@github.com:sos-berlin/js7-demo-inventory-rollout-test.git'
$commitHash = '04b07d8'


# --- Connection ---

Import-Module JS7
Connect-JS7 -Url $url -Id $controllerId | Out-Null


# --- Step 1: Cherry-pick changes from remote test repository to remote prod repository ---

# navigate to local repository folder on prod host
cd C:\ProgramData\sos-berlin.com\js7\joc\jetty_base\resources\joc\repositories\rollout\Accounting

# add the test repo as a remote repository
git remote add oldrepo $repo
 
# get the test repo commits
git remote update
 
# examine the whole tree
git log --all --oneline --graph --decorate
 
# copy/cherry-pick the commits from the test repo into your local prod repo
git cherry-pick $commitHash
 
# check local prod repo
git log
 
# push changes to remote prod repo
git push origin master
 
# remove the reference to the test repo
git remote remove oldrepo


# --- Step 2: Pull Changes to local Git Repository ---

# Pull changes
Invoke-JS7GitRepositoryPull -Folder $folder | Out-Null


# --- Step 3: Cleanup repository and store scheduling objects to repository ---

# Update the JOC Cockpit inventory from the local repository of the given folder
Update-JS7FromRepositoryItem -Folder $folder
 
# Release scheduling objects of the given folder
Publish-JS7ReleasableItem -Folder $folder -Recursive -NoReleased
 
# Deploy scheduling objects from the given folder to the Controller
Publish-JS7DeployableItem -ControllerId $controllerId -Folder $folder -Recursive -NoDeployed


# --- Connection ---

Disconnect-JS7

Further Resources

The above OS commands, Git commands and PowerShell cmdlets can be executed from JS7 workflows for the automation of a CI/CD pipeline.

Space shortcuts

Page tree

Introduction

Steps to perform in the TEST Environment

Storing Scheduling Objects to JOC Cockpit Git Repository

Storing Scheduling Objects using JS7 Unix Shell CLI

Storing Scheduling Objects using JS7 PowerShell Module

Committing and pushing Changes to remote Git Repository

Committing and pushing Changes using JS7 Unix Shell CLI

Committing and pushing Changes using JS7 PowerShell Module

Example of a CI/CD Pipeline Script for TEST Environment

Steps to perform in the PROD Environment

Copying Changes from TEST to PROD Git Repository

Pulling Changes to local Git Repository

Pulling Changes using Unix Shell CLI

Pulling Changes using PowerShell

Updating JOC Cockpit Inventory from local Git Repository, Release and Deploy to Controller

Updating JOC Cockpit Inventory using Unix Shell CLI

Updating JOC Cockpit Inventory using PowerShell

Example of a CI/CD Pipeline Script for PROD Environment

Further Resources

Space shortcuts

Page tree

JS7 - How to rollout from test to prod environments in a CI CD pipeline

Introduction

Steps to perform in the TEST Environment

Storing Scheduling Objects to JOC Cockpit Git Repository

Storing Scheduling Objects using JS7 Unix Shell CLI

Storing Scheduling Objects using JS7 PowerShell Module

Committing and pushing Changes to remote Git Repository

Committing and pushing Changes using JS7 Unix Shell CLI

Committing and pushing Changes using JS7 PowerShell Module

Example of a CI/CD Pipeline Script for TEST Environment

Steps to perform in the PROD Environment

Copying Changes from TEST to PROD Git Repository

Pulling Changes to local Git Repository

Pulling Changes using Unix Shell CLI

Pulling Changes using PowerShell

Updating JOC Cockpit Inventory from local Git Repository, Release and Deploy to Controller

Updating JOC Cockpit Inventory using Unix Shell CLI

Updating JOC Cockpit Inventory using PowerShell

Example of a CI/CD Pipeline Script for PROD Environment

Further Resources