Introduction

• Users might be interested in specifying jobs and dependencies from Microsoft Excel® spreadsheets.
  • In a situation when business staff is in charge of specifying jobs then Excel® is a frequently used tool.
  • As a no-code approach this allows to specify jobs and dependencies by users without detailed know-how of JS7 - Workflows.
• The solution explained from the article creates jobs and workflows from Excel® spreadsheets ready for import into JS7.
  
  • The solution makes use of a PowerShell cmdlet for Linux and Windows and can be operated for PowerShell 5.1, 6, 7.

Prerequisites

The following PowerShell modules are used for this purpose:

• ImportExcel PowerShell Module (3rd party) to read Excel® spreadsheets on Linux and Windows.
• Optionally the JS7 PowerShell Module to access the JS7 - REST Web Service API for automated import and deployment.
  • If the module is not used then users can manually import resulting workflows using the JOC Cockpit user interface.

Download

The following files are used for the solution:

• Sample Excel® spreadsheet: jobs.xlsx
• PowerShell cmdlet: New-JS7WorkflowFromExcel.ps1

Getting Started

Specify Jobs and Dependencies with Excel®

Download the jobs.xlsx Excel® spreadsheet file.

• The first worksheet offers a number of jobs and dependencies from the following columns:
  • NodeID: A unique identifier for the occurrence of a job. If the same job occurs a number of times then the Name column will hold the same job name, however, a unique value is specified for the NodeID column.
  • SuccessorID: Specifies the NodeIDs of jobs that are direct successors of the current job. If more than one job is specified then they will will be executed in parallel. NodeIDs in this column are separated by a space.
  • Instruction:  Makes use of the fixed value Job. This column is reserved for later support of further JS7 - Workflow Instructions.
  • Name: Specifies the job name.
  • Description: Optionally specifies a descriptive title for the job.
  • Agent, Subagnet Cluster: Specifies the name of a Standalone Agent that is configured in your JS7 scheduling environment. If an Agent Cluster is used then its name is specified as the Agent and the respective Subagent Cluster is specified, see JS7 - Agent Cluster.
  • Script: Specifies the job script that will be executed. Multiline input is allowed.
  • Error Handling: Specifies one of the following options for JS7 - How to apply error handling:
    • STOP: If the job fails then the order will be put to the FAILED state and will remain with the job. Failed orders require user intervention to resume, to suspend or to cancel the respective order.
    • IGNORE: Any job error is ignored and order processing continues, see JS7 - Try-Catch Instruction.
    • RETRY: The job will be retried in case of error for 3 times with a delay of 60s, see JS7 - Retry Instruction.
    • LEAVE: The order will leave the workflow with an unsuccessful History outcome, see JS7 - Finish Instruction.
  • Fail on stderr: Specifies that in case that the job will write output to the stderr channel the job will be considered to be failed, see JS7 - Job Instruction.
• Users can adjust column names, see JS7 - PowerShell Import from Excel Script - New-JS7WorkflowFromExcel.ps1.

Explanation:

• Job dependencies are designed along a Directed acyclic graph (DAG). This includes a few rules for input to the Excel® spreadsheet:
  • One or more initial jobs are required as the graph's root. Initial jobs must not be specified as successors in the SuccessorID column.
  • One final job has to be specified that does not indicate successor jobs in the SuccessorID column.
• In a simplified way this works with two columns:
  • The NodeID column specifies the unique identifier of the given job.
  • The SuccessorID column specifies one or more NodeIDs of successor jobs that are separated by a space.
    • If more than one NodeID is specified then this will fork processing of successor jobs to be performed in parallel.
    • Forking can include any number of branches in 
• x

Run the PowerShell Cmdlet

Download the PowerShell cmdlet: New-JS7WorkflowFromExcel.ps1

• Basically two arguments have to be provided:
  • -ExcelPath: Specifies the path to the Excel® spreadsheet file that holds the specification of jobs and dependencies.
  • -OutputDirectory: Specifies the directory to which workflow *.json files will be stored.

./New-JS7WorkflowFromExcel.ps1 -ExcelPath /home/sos/excel/input/jobs.xlsx -OutputDirectory /home/sos/excel/output/jobs

./New-JS7WorkflowFromExcel.ps1 -ExcelPath "C:\js7\excel\input\jobs.xlsx" -OutputDirectory "C:\js7\excel\output\jobs"

The above examples will create workflows per each worksheet included with the Excel® spreadsheet file. The name of the workflow corresponds to the worksheet.

Users who prefer to run the PowerShell cmdlet directly from the Unix Shell can add the call to the script and its parameters to an executable shell script like this:

#!/usr/bin/env pwsh

$env:SCRIPT_HOME=$(cd "$(dirname "`$0")" >/dev/null && pwd)
Set-Location -Path $env:SCRIPT_HOME

./New-JS7WorkflowFromExcel.ps1 `
    -ExcelPath "$($env:SCRIPT_HOME)/input/jobs.xlsx" `
    -OutputDirectory "$($env:SCRIPT_HOME)/output/jobs"

Examples

Examples for advanced parameterization are provided as follows.

Create Workflows for specific Worksheets and Column Names

./New-JS7WorkflowFromExcel.ps1 `
    -ExcelPath /home/sos/excel/input/jobs.xlsx `
    -WorksheetName Workflow01 `
    -WorksheetColumns @{ 'nodeid' = 'NodeID'; 'successorid' = 'SuccessorID'; 'instruction' = 'Instruction'; 'name' = 'Name'; 'description' = 'Description'; 'agent' = 'Agent'; 'subagentCluster' = 'Subagent Cluster'; 'script' = 'Script'; 'errorHandling' = 'Error Handling'; 'failOnStderr' = 'Fail on stderr' } `
    -OutputDirectory /home/sos/excel/output/jobs `
    -WorkflowName EOD-Workflow `
    -Title "EOD Processing" `
    -AgentName primaryAgent

Explanation:

• The -WorksheetName parameter is used to limit creation of workflows to the given worksheet within the Excel® spreadsheet file.
• The call to the PowerShell cmdlet specifies the -WorksheetColumns parameter that holds a hashtable for the mapping of internal names and column names in the given spreadsheet file. This allows to use different column names in a spreadsheet file that will be mapped to internal names. Any additional columns in a spreadsheet are ignored.
• The -WorkflowName parameter specifies the name of the workflow. If this parameter is omitted then the workflow name will be used from the name of the worksheet.
• The -Title parameter specifies the title that will be assigned the workflow.
• The -AgentName parameter specifies the name of the Agent that will be assigned to jobs for which the Excel® spreadsheet file does not specify a value in the related column.

Create Workflows from a number of Excel® Spreadsheet Files

Get-ChildItem *.xlsx | Foreach-Object { ./New-JS7WorkflowFromExcel.ps1 `
    -ExcelPath $_.FullName `
    -Title $_.BaseName `
    -OutputDirectory /home/sos/excel/output/jobs
}

Explanation:

• The Get-ChildItem Cmdlet will select any Excel® spreadsheet files in the current directory. Each Excel® file is pipelined to the PowerShell Cmdlet.
• This allows to create workflows from a larger number of Excel® spreadsheet files in a bulk operation.

Further Resources

• JS7 - PowerShell Import from Excel Script - New-JS7WorkflowFromExcel.ps1