...

• JS7 JOC Cockpit can be configured to interface with Git repositories to store and to rollout its configuration objects such as workflows and jobs.
  • JS7 supports use of Git compliant servers to manage remote repositories such as GitLab, GitHub.
  • JS7 makes use of a Git CLI Client that has to be provided by the user.
• Configuration objects can be versioned (tagged) with Git and can be rolled out to different JS7 environments via Git. This allows to create and to test workflows in a development environment and to perform rollouts to a production environment by use of a Git repository.
• A connection to a Git repository is considered an additional option to manage versions and rollouts of JS7 configuration objects.
• The Git Repository Interface is made available in two steps:

  • Display feature availability
    StartingFromRelease 2.3.0

     
  • Release 2.3.0: Make JS7 objects available from the file system to allow use of Git tools.

    • Jira
      server SOS JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 6dc67751-9d67-34cd-985b-194a8cdc9602 key JOC-1144

  • Release 2.4.0: Offer the GUI functionality from JOC Cockpit to manage JS7 objects with Git.

...

JS7 configuration objects that are independent from a specific environment (dev, test, prod) and that can be rolled out across environments include

...

Such objects can be managed with JS7 in a way that allows to re-use the objects - without any changes - across rollout environments such as dev, test, prod.

Local Objects

The following configuration objects typically hold values that are specific for an individual environment (dev, test, prod) and therefore usually are added to a separate repository. Such objects should are not be rolled out across environments.

...

• Security level LOW maps to use of a single Git account used by all users. Credentials are stored with the root account's profile in JS7 database.
• Security level MEDIUM maps to use of an individual Git account per user. Credentials are stored with each user account's profile in the JS7 database.
• Security level HIGH does not allow to store Git credentials with JOC Cockpit. Operations on the remote Git repository have to be performed with tools external to JOC Cockpit such as Git Extensions.

The following credentials are stored with a userused depending on the Git server's profileauthentication method:

• git_account
  • account that is used to connect to a Git server
• git_password
  • the user's password for Git access
• git_key
  • the user's private key for Git access
• git_access_token
  • the access token for Git access that works as an alternative to use of a private key

If repositories are used from a number of Git servers that require different sets of user credentials then separate user accounts in JOC Cockpit have to be used.

...

• The local repository for a JS7 root-level folder is created when a user performs the checkout/ or clone operation for the Git repository via JOC Cockpit.
• The local repository is persistent and is updated from the JOC Cockpit database (store operation) and from the remote Git repository (pull operation).
• The remote Git repository is updated from the local repository by use of the push operation.
• The JS7 - Cleanup Service removes any local repositories if the JS7 root-level folder no longer exists, i.e. if a root-level folder is dropped from JOC Cockpit.

...

• git-checkout
  • checks out a specific branch or tag from the Git repository to the local repository
• git-clone
  • initially clones a Git repository to a local repository
• git-add
  • adds or updates (stage) a new or modified configuration object on the filesystem file system to the local repository
• git-reset
  • unstages un-stages a modification from the local repository, the modified file in the filesystem file system remains unchanged and still contains the modification
• git-delete
  • removes an already added configuration object from the local repository
• git-commit
  • commits newly staged configuration objects to a local repository
• git-push
  • pushes changes from the local repository to the remote Git repository

...

Rollout between different JS7 instances is performed by use of the Git functionality. The JS7 REST Web Service API manages the processes for checkout and cloning.

...

The synchronization of configuration objects between the JOC Cockpit database and the local repositories are managed by the JS7 JOC REST Web Service API.

...

• ./inventory/repository/read
  • reads file tree of the local repository file system folder and updates the JOC Cockpit database
• ./inventory/repository/store
  • stores a set of configuration objects from the JOC Cockpit database to the file system folder of the local repository
• ./inventory/repository/delete
  
  • deletes a set of configuration objects from the file system folder of the local repository
• ./inventory/repository/update
  
  • adds or updates a set of configuration objects in the JOC Cockpit database with the configuration objects from the local repositories file system folder

Use Cases

Single Client - Multiple Repositories

...

Scenario

• Architecture
  • A dedicated JOC Cockpit instance is used with Controller and Agents for each rollout environment such as dev, test, prod.
  • A Git repository is set of two Git repositories are used per folder and per rollout environment (dev, test, prod), each JOC Cockpit instance is acting on a specific repository per rollout environment.
  • The Git repository is used to hold a copy of JS7 configuration objects and, depending on the rollout environment, to create versions of JS7 configuration objects.
  • :
    • one repository holding local objects such as job resources that are specific for a rollout environment,
    • one repository holding rollout objects such as workflows that are rolled out across environments.
• Users
  • All users act in the same JOC Cockpit instance of the respective rollout environment that is a single client to the Git server.
  • Changes to the Git repositories are performed by on behalf of the users' individual accounts and are added to the remote repositories.
• Rollout
  • Deployments are performed from a single source (JOC Cockpit) to each Controller and Agents per rollout environment (dev, test, prod).
  • The Git repositories are used to checkout branches and to rollout JS7 configuration objects to different Controllers and Agents, such as Rollouts are performed by copying rollout objects from a local Git repository to a remote Git repository of the target rollout environment (dev => test => prod).
  • The Git repository can be used to fallback to previous versions of configuration objects, however, only one version is available with a given JOC Cockpit instance at a given point in time.
  • After an update of configurations objects in a given JOC Cockpit instance the updated configuration objects have to be deployed anew.

Repositories marked as "env" are specific for an environment and are Repositories holding local objects are excluded from rollout. Repositories marked as "holding rollout " are environment-independent and are forwarded across scheduling objects are copied across repositories and environments.

USER1a [label="  User: 1a  ",fillcolor="lightskyblue1"]
USER1b [label="  User: 1b  ",fillcolor="lightskyblue1"]
USER2 [label="  User: 2  ",fillcolor="lightskyblue2"]
USER3 [label="  User: 3  ",fillcolor="lightskyblue3"]

JOC1 [label="  JOC Cockpit: dev  ",fillcolor="lightskyblue1"]
JOC2 [label="  JOC Cockpit: test  ",fillcolor="lightskyblue2"]
JOC3 [label="  JOC Cockpit: prod  ",fillcolor="lightskyblue3"]

FOLDER1 [label="  Folder: 1 (dev)  ",fillcolor="white"]
FOLDER2 [label="  Folder: 1 (test)  ",fillcolor="white"]
FOLDER3 [label="  Folder: 1 (prod)  ",fillcolor="white"]

DATABASE1 [shape="ellipse",label="  Database: dev  ",fillcolor="lightskyblue1"]
DATABASE2 [shape="ellipse",label="  Database: test  ",fillcolor="lightskyblue2"]
DATABASE3 [shape="ellipse",label="  Database: prod  ",fillcolor="lightskyblue3"]

# GIT [label="Git Server",fillcolor="orange"]
REPO1ENVREPO1ENV [shape="ellipse",label="Repository: 1-dev-local\njob resources etc.",fillcolor="darkolivegreen1"]
REPO1ROL [shape="ellipse",label="Repository: 1-dev-rollout\nworkflows etc.",fillcolor="darkolivegreen1"]
REPO2ENV [shape="ellipse",label="Repository: 1-test-local\njob resources etc.",fillcolor="darkolivegreen2"]
REPO2ROL [shape="ellipse",label="Repository: 1-test-rollout\nworkflows etc.",fillcolor="darkolivegreen2"]
REPO3ENV [shape="ellipse",label="Repository: 1-prod-local\njob resources etc.",fillcolor="darkolivegreen3"]
REPO3ROL [shape="ellipse",label="Repository: 1-prod-rollout\nworkflows etc.",fillcolor="darkolivegreen3"]

{ rank = same; JOC1 -> DATABASE1 }
{ rank = same; JOC2 -> DATABASE2 }
{ rank = same; JOC3 -> DATABASE3; }

JOC1 -> FOLDER1 
FOLDER1 -> REPO1ENV [label="  commit/push\npull\n  push/pull  "]
FOLDER1 -> REPO1ROL [label="  commit/push\npull\n  push/pull  "]

JOC2 -> FOLDER2
FOLDER2 -> REPO2ENV [label="  commit/push\npull\n  push/pull  "]
FOLDER2 -> REPO2ROL [label="  commit/push\npull\n  push/pull  "]

JOC3 -> FOLDER3 
FOLDER3 -> REPO3ENV  [label="  commit/push\npull\n  push/pull  "]
FOLDER3 -> REPO3ROL  [label="  commit/push\npull\n  push/pull  "]

REPO1ROL -> REPO2ROL [label="  copy  "]
REPO2ROL -> REPO3ROL [label="  copy  "]

USER1a -> JOC1;
USER1b -> JOC1;
USER2 -> JOC2;
USER3 -> JOC3;

Git Repository Operations

• management
• versioning
• rollout

...

JOC Cockpit Operations

The JOC Cockpit database holds the JS7 configuration objects:

• each JOC Cockpit instance is connected to its own database,
• each Git Repository repository is mapped to an individual JOC Cockpit instance,
• the database can be updated from a local Git branchrepository.
• each Git branch makes use of its own version of configuration objects.

Multiple Clients - Multiple Repositories

...

Scenario

• Architecture
  • A dedicated JOC Cockpit instance is used with each Controller and Agents
    • for each user in the dev
    rollout
    • environment
    .A dedicated JOC Cockpit instance is used with each Controller and Agents
    • ,
    • for all users in the test and prod
    rollout
    • environments.
  • Individual A set of two Git repositories are used per folder and per rollout environment (dev, test, prod) by JOC Cockpit instances, each instance is acting on a specific repository per rollout environment.The Git repository is used to hold a copy of JS7 configuration objects, depending on the rollout environment, to create versions of JS7 configuration objects:
    • one repository holding local objects such as job resources that are specific for a rollout environment,
    • one repository holding rollout objects such as workflows that are rolled out across environments.
• Users
  • Users act in individual JOC Cockpit instances of the dev rollout environment as multiple clients to the Git server.
  • Users act in the same JOC Cockpit instance of the test and prod rollout environment environments as a single client to the Git server.
  • Changes to the Git repositories are performed by on behalf of the users' individual accounts.
• Rollout
  • Deployments are performed from a multiple sources (JOC Cockpit) to the connected Controller and Agents per rollout environment (dev, test, prod).
  • The Git repositories are used to checkout branches and to rollout JS7 configuration objects to different Controllers and Agents, such as Rollouts are performed by copying rollout objects from a local Git repository to a remote Git repository of the target rollout environment (dev => test => prod).
  • The Git repository can be used to fallback (pull) to previous versions of configuration objects, however, only one version is available with a given JOC Cockpit instance at a given point in time.
  • After an update of configurations objects in a given JOC Cockpit instance the updated configuration objects have to be deployed anew.

Repositories marked as "env" are specific for an environment and are excluded from rollout. Repositories marked as "rollout" are environment-independent and are forwarded across scheduling environments.

Repositories holding local objects are excluded from rollout. Repositories holding rollout objects are copied across repositories and environments.

USER1a [label="  User: 1a  ",fillcolor="lightskyblue1"]
USER1b [label="  User: 1b  ",fillcolor="lightskyblue1"]
USER2a [label="  User: 2a  ",fillcolor="lightskyblue2"]
USER2b [label="  User: 2b  ",fillcolor="lightskyblue2"]
USER3 [label="  User: 3  ",fillcolor="lightskyblue3"]

JOC1a [label="JOC Cockpit: dev 1a",fillcolor="lightskyblue1"]
JOC1b [label="JOC Cockpit: dev 1b",fillcolor="lightskyblue1"]
JOC2  [label="JOC Cockpit: test",fillcolor="lightskyblue2"]
JOC3  [label="JOC Cockpit: prod",fillcolor="lightskyblue3"]

DATABASE1a [shape="ellipse",label="Database: dev 1a",fillcolor="lightskyblue1"]
DATABASE1b [shape="ellipse",label="Database: dev 1b",fillcolor="lightskyblue1"]
DATABASE2  [shape="ellipse",label="Database: test",fillcolor="lightskyblue2"]
DATABASE3  [shape="ellipse",label="Database: prod",fillcolor="lightskyblue3"]

REPO1ENV1 [shape="ellipse",label="Repository: dev-local-1a\njob resources etc.",fillcolor="darkolivegreen1"]
REPO1ENV2 

USER1a [label="User: 1a",fillcolor="lightskyblue"]
USER1b [label="User: 1b",fillcolor="lightskyblue"]
USER2a [label="User: 2a",fillcolor="lightskyblue"]
USER2b [label="User: 2b",fillcolor="lightskyblue"]
USER3 [label="User: 3",fillcolor="lightskyblue"]

JOC1a [label="JOC Cockpit: dev 1a",fillcolor="lightskyblue"]
JOC1b [label="JOC Cockpit: dev 1b",fillcolor="lightskyblue"]
JOC2  [label="JOC Cockpit: test",fillcolor="lightskyblue"]
JOC3  [label="JOC Cockpit: prod",fillcolor="lightskyblue"]

DATABASE1a [shape="ellipse",label="Database: dev 1a",fillcolor="lightskyblue"]
DATABASE1b [shape="ellipse",label="Database: dev 1b",fillcolor="lightskyblue"]
DATABASE2  [shape="ellipse",label="Database: test",fillcolor="lightskyblue"]
DATABASE3  [shape="ellipse",label="Database: prod",fillcolor="lightskyblue"]

# GIT [label="Git Server",fillcolor="orange"]
REPO1ENV1 [label="Repository: dev env 1a",fillcolor="darkolivegreen1"]
REPO1ENV2 [label="Repository: dev env 1b",fillcolor="darkolivegreen1"]
REPO1ROL  [label="Repository: dev rollout",fillcolor="darkolivegreen1"]
REPO2ENV  [label="Repository: test env",fillcolor="darkolivegreen2"]
REPO2ROL  [label="Repository: test rollout",fillcolor="darkolivegreen2"]
REPO3ENV  [label="Repository: prod env",fillcolor="darkolivegreen3"]
REPO3ROL  [label="Repository: prod rollout",fillcolor="darkolivegreen3"]

# BRANCHES1R [shape="ellipse",label="BranchesRepository: env devdev-local-1b\njob resources etc.",fillcolor="darkolivegreen1"]
#REPO1ROL BRANCHES1I [shape="ellipse",label="BranchesRepository: dev-rollout\nworkflows devetc.",fillcolor="darkolivegreen1"]
#REPO2ENV BRANCHES2R [shape="ellipse",label="Branches: env testRepository: test-local\njob resources etc.",fillcolor="darkolivegreen2"]
#REPO2ROL BRANCHES2I [shape="ellipse",label="BranchesRepository: test-rollout\nworkflows testetc.",fillcolor="darkolivegreen2"]
#REPO3ENV BRANCHES3R [shape="ellipse",label="BranchesRepository: env prodprod-local\njob resources etc.",fillcolor="darkolivegreen3"]
#REPO3ROL BRANCHES3I [shape="ellipse",label="BranchesRepository: prod-rollout\nworkflows prodeetc.",fillcolor="darkolivegreen3"]

{ rank = same; JOC1a -> DATABASE1a }
{ rank = same; JOC1b -> DATABASE1b }
{ rank = same; JOC2 -> DATABASE2 }
{ rank = same; JOC3 -> DATABASE3; }

JOC1a -> REPO1ENV1 [label="Commit/Push\nPull  commit/push/pull  "]
JOC1a -> REPO1ROL  [label="Commit/Push\nPull  commit/push/pull  "]
JOC1b -> REPO1ENV2 [label="Commit/Push\nPull  commit/push/pull  "]
JOC1b -> REPO1ROL  [label="Commit/Push\nPull  commit/push/pull  "]
JOC2  -> REPO2ENV  [label="Commit/Push\nPull  commit/push/pull  "]
JOC2  -> REPO2ROL  [label="Commit/Push\nPull  commit/push/pull  "]
JOC3  -> REPO3ENV  [label="Commit/Push\nPull  commit/push/pull  "]
JOC3  -> REPO3ROL  [label="Commit/Push\nPull  commit/push/pull  "]

REPO1ROL -> REPO2ROL [label=""  copy  "]
REPO2ROL -> REPO3ROL [label="  copy  "]

# REPO1 -> GIT
# REPO2 -> GIT
# REPO3 -> GIT

USER1a -> JOC1a
USER1b -> JOC1b
USER2a -> JOC2
USER2b -> JOC2
USER3 -> JOC3

Git Repository Operations

• management
• versioning
• rollout

...

JOC Cockpit Operations

• The JOC Cockpit database holds the configuration objects:
  • each JOC Cockpit instance is connected to its own database.
  • each Git Repository repository is connected to one or more JOC Cockpit instances, e.g. dev rollout.
  Each Git branch makes use of its own versions of configuration objects.
• Each user (client) makes use of an individual JS7 instance using JOC Cockpit and database, Controller and Agents.

...

• Use Case
  • Each environment makes use of an individual JS7 instance using JOC Cockpit and database, Controller and Agents.
  • Environment dev: 
    • A configuration object, e.g. a workflow, is created or changed by a developer
    • when finished and tested (deployed) by the developer in his environment the configuration object will be stored to the local dev repository
    • the change to the local repository will be commited and pushed to the remote dev repository
  • Environment test:
    • A test user with access to the developers env repository, pulls the current changes.
    • The current changes will be copied from the developers env repository (source) to the test users test repository (target).
    • The test user updates his JOC instance with the current changes from his local test repository.
    • The test user deploys the changed workflow to his environment and runs the desired tests.
    • When finished and approved the current changes from the test users local test repository are commited and pushed to the remote test repository
      • additionally the latest commit can be tagged with a label, e.g. "v2_approved"
  • Environment prod:
    • A production user with access to the test users test repository, pulls the current changes and checks out the tag "v2_approved".
    • The current changes will be copied from the test users test repository (source) to the production users prod repository (target).
    • The production user updates his JOC instance with the current changes from his local prod repository and checks the updated changes before deployment.
    • When checked, the current changes from the production users local prod repository are commited and pushed to the remote prod repository
    • The new change will be tagged with a label, e.g. "v2"
    • The production user deploys the changed workflow to the production environment.

Requirements

• The JOC REST Web Service API will provide web services to interact with the local filesystem of the local git repository.
  • Web Serivce APIs
    • read
    • store
    • update
    • delete
• The JOC REST Web Service API will provide web services to interact with the local git repositories.
  • The Web Serivce APIs will make use the git CLI.
    • Git Commands
      • add
      • restore
      • reset
      • commit
      • checkout
• The JOC REST Web Service API will provide web services to interact with remote git repositories.
  • The Web Serivce APIs will make use the git CLI. 
    • Git Commands
      • clone
      • push
      • pull
• The JOC REST Web Service API will provide web services to interact between local git repositories (read source/target, copy source/target) to support transfer of configuration objects between environments (dev, test, prod).
  • All 3rd party git tool cover the interaction between local (clone) git repositories and remote (origin) git repositories also.
  • Most 3rd party git tools do not cover interaction between local repositories, e.g. local dev → local test

...