Introduction

Continuous Integration / Continuous Delivery (CI/CD) is a one of the supported method methods for JS7 - Rollout of Scheduling Objects.

• The scenarios are described with in the JS7 - Git Repository Interface.
• CI/CD includes to perform the following steps
  
  • in a test environment
    
    • to store scheduling objects of a JOC Cockpit inventory to a local Git repository,
    • to add, to commit and to push changes to the remote Git repository.
  • in a prod environment
    • to copy (cherry-pick) changes from the remote test Git repository to the local remote prod Git repository,
    • to push changes from the local remote prod Git repository to the remote local prod Git repository,
    • to update the JOC Cockpit inventory from the local Git repository,
    • to release and to deploy scheduling objects to Controllers and Agents.

Users who wish to automate the steps for rollout can use the following resources:

• The JS7 - REST Web Service API allows the same operations for Git integration to be performed as offered from the JOC Cockpit GUI.
  • See Technical Documentation of the REST Web Service API
• The JS7 - PowerShell Module offers simplified access to the REST Web Service API for scripting purposes.
  • The PowerShell examples when executed might prove to be instructive for logging of REST API calls when used with the -debug option.

The following examples assume that a local Git repository is set up in the test and prod environments , see - see JS7 - How to set up Git Access and JS7 - How to set up a local Git Repository.

Documentation for the relevant cmdlets is provided in the  PowerShell CLI 2.0 - Set up local Cmdlets - Git Repository Integration article.

Steps to perform

...

in the TEST Environment

The below example example below assumes that that:

• a top-level folder Accounting is used in the JOC Cockpit inventory,
• this folder is mapped to a sub-directory in the file system with the same name and spelling in the file system,, for example:
  • for Unix: /var/sos-berlin.com/js7/joc/jetty_base/resources/joc/repositories/rollout/Accounting
  • for Windows: C:\ProgramData\sos-berlin.com\js7˜\joc\jetty_base\resources\joc\repositories\rollout\Accounting
• This this folder is created when setting up the repository, otherwise it will be created by the JOC Cockpit the first time that objects should are to be stored to in the repository.

Store Scheduling Objects to local Git Repository

The following cmdlets are used:

Import-Module JS7
Connect-JS7 -Url http://root:root@test-_host:4446 -Id 'Controller' | Out-Null

# Cleanup local repository by removing existing objects
Remove-JS7RepositoryItem -Folder '/Accounting'

# Store scheduling objects of the given folder and exclude draft versions to be used
Set-JS7RepositoryItem -Folder '/Accounting' -Recursive -NoDraft

Explanation:

• Line 2: A number of ways are offered how to connect to methods for connecting to the JOC Cockpit are available, see JS7 - How to connect to JOC Cockpit using the PowerShell Module.
• Line 5: The Accounting top-level folder and any sub-folders are removed from the local repository, for details see Remove-JS7RepositoryItem. This step is not required, however, it can be useful for consistency if scheduling objects that have been removed or renamed in the JOC Cockpit inventory.
• Line 8: The Accounting top-level folder and any sub-folders are added to the local repository, see Set-JS7RepositoryItem.

Commit and push Changes to remote Git Repository

The following OS and Git commands suggest implement a typical sequence of steps:

# Find the repository sub-directory managed by JOC Cockpit
cd /var/sos-berlin.com/js7/joc/jetty_base/resources/joc/repositories/rollout/Accounting

# Commit and push changes to the remote Git repository
git add .
git commit -m "changes to accounting jobs for v12.3"
git push

...

Import-Module JS7
Connect-JS7 -Url http://root:root@test_host:4446 -Id 'Controller' | Out-Null

# Add changes to the remote Git repository
Invoke-JS7GitRepositoryAdd -Folder '/Accounting' | Out-Null

# Commit changes
$reponse = Invoke-JS7GitRepositoryCommit -Folder '/Accounting' -Message 'changes for release v1.13'
# Parse commit hash from response
if ( $matchInfo = ( $response.stdout | Select-String '^\[.* +([0-9a-z]+)\]' ) )
{
	$commitHash = $matchInfo.Matches.Groups[1].Value
}

# Push changes
Invoke-JS7GitRepositoryPush -Folder '/Accounting' | Out-Null

Example of a CI/CD Pipeline Script for TEST Environment

The below example example below includes use of the above cmdlets in a pipeline script.

Download PowerShell Script: Rollout-JS7FromTest.ps1

# --- Parameterization ---

$url = 'http://root:root@localhostroot@test_host:4446'
$controllerId = 'Controller'

$folder = '/TestRepo/testMyFolder'
$directory = 'C:\ProgramData\sos-berlin.com\js7\joc\jetty_base\resources\joc\repositories\rollout\TestRepo'
Accounting'


# --- Connection ---

Import-Module JS7
Connect-JS7 -Url $url -Id $controllerId | Out-Null


# --- Step 1: Check status of releasable and deployable objects ---

$items = Get-JS7ReleasableItem -Folder $folder -NoReleased
if ( $items.count )
{
    Write-Warning "CI/CD pipeline stopped: unreleased objects found:"
    foreach( $item in $items )
    {
        Write-Warning "unreleased object: type=$($item.objectType), valid=$($item.valid), folder=$($item.folder), name=$($item.objectName)"
    }

    # optionally release items
    $items | Publish-JS7ReleasableItem
}

$items = Get-JS7DeployableItem -Folder $folder -NoDeployed
if ( $items.count )
{
    Write-Warning "CI/CD pipeline stopped: undeployed objects found:"
    foreach( $item in $items )
    {
        Write-Warning "undeployed object: type=$($item.objectType), valid=$($item.valid), folder=$($item.folder), name=$($item.objectName)"
    }

    # optionally deploy items
    $items | Publish-JS7DeployableItem -ControllerId $controllerId
}


# --- Step 2: Cleanup repository and store scheduling objects to repository ---

# Cleanup local repository by removing existing objects
Remove-JS7RepositoryItem -Folder $folder
 
# Store scheduling objects of the given folder and exclude draft versions to be used
Set-JS7RepositoryItem -Folder $folder -Recursive -NoDraft


# --- Step 3: add objects, commit and push to remote repository ---

cd $directory

git add .
git commit -m "
# Add changes to local staging area
Invoke-JS7GitRepositoryAdd -Folder $folder | Out-Null

# Commit changes
$response = Invoke-JS7GitRepositoryCommit -Folder $folder -Message 'changes to accounting jobs for v12.3"
git push'

# Parse commit hash from response
if ( $matchInfo = ( $response.stdout | Select-String '^\[.* +([0-9a-z]+)\]' ) )
{
	$commitHash = $matchInfo.Matches.Groups[1].Value
}

# Push changes to remote repository
Invoke-JS7GitRepositoryPush -Folder $folder | Out-Null


# --- Connection ---

Disconnect-JS7

Steps to perform

...

in the PROD Environment

Copy Changes from TEST to PROD Git Repository

There are a number of ways how to achieve of achieving this using Git commands:

• Some users might wish to cherry-pick individual commits from a test Git repository as shown in the example below.
• Some users might prefer to copy the complete test repository to a prod repository using branches.
• Some users might prefer to simplify rollout by use of a single repository and will therefore skip this step.
  
  

# add the test repo as a remote repository 
git remote add oldrepo git@github.com:sos-berlin/js7-demo-inventory-rollout-test.git

# get the test repo commits
git remote update

# examine the whole tree
git log --all --oneline --graph --decorate

# copy/cherry-pick the commits from the test repo into your local prod repo
git cherry-pick <commit-hash>

# check local prod repo
git log

# push changes to remote prod repo
git push origin master

# remove the reference to the test repo
git remote remove oldrepo

...

• Line 11: The <commit-hash> identifies commits performed when pushing scheduling objects to the test repository. This step can be performed repeatedly to cherry-pick a number of commits.

Pull Changes to local Git Repository

The following OS and Git commands are used:

# Find the repository sub-directory managed by JOC Cockpit
cd /var/sos-berlin.com/js7/joc/jetty_base/resources/joc/repositories/rollout/AccountingImport-Module JS7
Connect-JS7 -Url http://root:root@prod_host:4446 -Id 'Controller' | Out-Null

# Pull changes
git pull from remote Git repository
Invoke-JS7GitRepositoryPull -Folder '/Accounting' | Out-Null

Update JOC Cockpit Inventory from local Git Repository, Release and Deploy to Controller

The following cmdlets are used:

Import-Module JS7
Connect-JS7 -Url http://root:root@prod-_host:4446 -Id 'Controller' | Out-Null

# Update the JOC Cockpit inventory from the local repository of the given folder
Update-JS7FromRepositoryItem -Folder '/Accounting'

# Release scheduling objects of the given folder
Publish-JS7ReleasableItem -Folder '/Accounting' -Recursive -NoReleased

# Deploy scheduling objects from the given folder to the Controller
Publish-JS7DeployableItem -ControllerId 'Controller' -Folder '/Accounting' -Recursive -NoDeployed

Explanation:

• Line 5: The JOC Cockpit inventory is updated from the local repository. This translates to the fact that objects with the same name are overwritten and that new objects are added. Updated objects are put to a draft status in the JOC Cockpit inventory. Find details from in Update-JS7FromRepositoryItem.
• Line 8: Scheduling objects that are not deployed to a Controller such as schedules and calendars are released. Should If such objects be are included in the changes to the local repository then they are in a draft status and should be released. The -NoReleased parameter prevents scheduling objects in with a non-draft status from being released, for example objects that are not stored in the local repository but that which have previously been released. For details see Publish-JS7ReleasableItem.
• Line 11: Scheduling objects are deployed to the given Controller. The -NoDeployed parameter prevents scheduling objects in with a non-draft status from being deployed, for example objects that are not stored in the local repository but that which have previously been deployed. For details see Publish-JS7DeployableItem.

Example of a CI/CD Pipeline Script for PROD Environment

The example below example includes shows use of the cmdlets described above cmdlets in a pipeline script.

Download PowerShell Script: Rollout-JS7ToProd.ps1

# --- Parameterization ---

$url = 'http://root:root@prod_host:4446'
$controllerId = 'Controller'
$folder = '/Accounting'

$repo = 'git@github.com:sos-berlin/js7-demo-inventory-rollout-test.git'
$commitHash = '04b07d8'


# --- Connection ---

Import-Module JS7
Connect-JS7 -Url $url -Id $controllerId | Out-Null


# --- Step 1: Cherry-pick changes from remote test repository to remote prod repository ---

# navigate to local repository folder on prod host
cd C:\ProgramData\sos-berlin.com\js7\joc\jetty_base\resources\joc\repositories\rollout\Accounting

# add the test repo as a remote repository
git remote add oldrepo $repo
 
# get the test repo commits
git remote update
 
# examine the whole tree
git log --all --oneline --graph --decorate
 
# copy/cherry-pick the commits from the test repo into your local prod repo
git cherry-pick $commitHash
 
# check local prod repo
git log
 
# push changes to remote prod repo
git push origin master
 
# remove the reference to the test repo
git remote remove oldrepo


# --- Step 2: Pull Changes to local Git Repository ---

# Pull changes
Invoke-JS7GitRepositoryPull -Folder $folder | Out-Null


# --- Step 3: Cleanup repository and store scheduling objects to repository ---

# Update the JOC Cockpit inventory from the local repository of the given folder
Update-JS7FromRepositoryItem -Folder $folder
 
# Release scheduling objects of the given folder
Publish-JS7ReleasableItem -Folder $folder -Recursive -NoReleased
 
# Deploy scheduling objects from the given folder to the Controller
Publish-JS7DeployableItem -ControllerId $controllerId -Folder $folder -Recursive -NoDeployed


# --- Connection ---

Disconnect-JS7x

Further Resources

The above OS commands, Git commands and PowerShell cmdlets can be executed from JS7 workflows for the automation of a CI/CD pipeline.

...