JS7 JobScheduler

Space shortcuts

Page tree

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 7 Next »

Introduction

JS7 does not make use of files for log output of order logs and task logs, see JS7 - Order Logs and Task Logs. JS7 instead streams log output from Agents to the Controller and to JOC Cockpit without use of files.

A number of users prefer to have log files available for each order execution and job execution, for example to consolidate log files on a file server or to submit log files to specific tools for log analysis.

To support a situation when log output should be consolidated to files the JS7 - REST Web Service API allows access to log output of orders and jobs. The JS7 - PowerShell Module is a lightweight wrapper for the REST Web Service API that is used for the below examples with Linux and Windows. Users are free to use the JS7 REST Web Service from their preferred scripting language to provide similar functionality.

Order Logs

Order logs include the task log output of each job and instruction included in a workflow. This provides better context than use of individual task logs.

The following cmdlets are provided:

Access Order Log Object

Example how to access order log objects
$logs = Get-JS7OrderHistory | Get-JS7OrderLog

Explanation:

  • The Get-JS7OrderHistory cmdlet returns history results that can be filtered by folders, workflows, date range, order states, see cmdlet description. By default today's order executions are returned.
  • The Get-JS7OrderLog cmdlet is used in a pipeline and returns the order log object for each history entry.
  • As a result the $logs array holds the list of order log objects.

An order log object carries attributes as visible from the following console example:

Example of a order log object
PS /> $logs = Get-JS7OrderHistory | Get-JS7OrderLog
PS /> $logs[0]

controllerId : jobscheduler
historyId    : 3092
orderId      : #2022-03-06#P31960623406-cycle2
workflow     : /Examples.Windows/05_ScheduledExecution/jdwScheduledWorkflowCyclic
position     :
state        : @{severity=6; _text=SUCCESSFUL}
plannedTime  : 06.03.2022 09:12:00
startTime    : 06.03.2022 09:12:05
endTime      : 06.03.2022 09:12:32
log          : 2022-03-06 09:12:05.265+0100 [MAIN]    [OrderStarted]   id=#2022-03-06#P31960623406-cycle2, pos=0
               2022-03-06 09:12:05.377+0100 [MAIN]    [OrderProcessingStarted] id=#2022-03-06#P31960623406-cycle2, pos=0, Job=job1, Agent (url=https://apmacwin:4245,
               id=agent_001, time=2022-03-06 09:12:05.265+0100)

               2022-03-06 09:12:05.265+0100 [MAIN]    [Start] Job=job1, Agent (url=https://apmacwin:4245, id=agent_001)
               2022-03-06 09:12:05.955+0100 [STDOUT]  using workflow: jdwScheduledWorkflowCyclic
               running job1
               order scheduler for: 2022-03-06 08:12:00+0000
               job start date: 2022-03-06 08:12:05+0000
               2022-03-06 09:12:10.653+0100 [MAIN]    [End] [Success] returnCode=0

               2022-03-06 09:12:10.753+0100 [SUCCESS] [OrderProcessed] id=#2022-03-06#P31960623406-cycle2, pos=0, Job=job1, returnCode=0
               2022-03-06 09:12:10.669+0100 [DETAIL]  [OrderForked]    id=#2022-03-06#P31960623406-cycle2, pos=1
               2022-03-06 09:12:10.669+0100 [DETAIL]  [OrderStarted]   id=#2022-03-06#P31960623406-cycle2|branch1, pos=1/branch1:0
               2022-03-06 09:12:10.669+0100 [DETAIL]  [OrderStarted]   id=#2022-03-06#P31960623406-cycle2|branch2, pos=1/branch2:0
               2022-03-06 09:12:10.753+0100 [MAIN]    [OrderProcessingStarted] id=#2022-03-06#P31960623406-cycle2|branch1, pos=1/fork+branch1:0, Job=job2_1a, Agent
               (url=https://apmacwin:4245, id=agent_001, time=2022-03-06 09:12:10.669+0100)

               2022-03-06 09:12:10.669+0100 [MAIN]    [Start] Job=job2_1a, Agent (url=https://apmacwin:4245, id=agent_001)
               2022-03-06 09:12:11.306+0100 [STDOUT]  using workflow: jdwScheduledWorkflowCyclic
               running job2_1a
               order scheduler for: 2022-03-06 08:12:00+0000
               job start date: 2022-03-06 08:12:10+0000
               2022-03-06 09:12:16.098+0100 [MAIN]    [End] [Success] returnCode=0

Write Order Log to File

Example how to write order logs to a common log file
Get-JS7OrderHistory | Get-JS7OrderLog | Out-File /tmp/history/orders.log -Encoding Unicode

Explanation:

  • Reads the logs of today's orders and writes the logs to a common file.


Example how to write order logs to individual log files
Get-JS7OrderHistory -RelativeDateFrom -8h | Get-JS7OrderLog | Select-Object @{name='path'; expression={ "/tmp/history/$(Get-Date $_.startTime -f 'yyyyMMdd-hhmmss')-$([io.path]::GetFileNameWithoutExtension($_.workflow))-$($_.orderId).log"}}, @{name='value'; expression={ $_.log }} | Set-Content

Explanation:

  • Reads the logs of orders that completed within the last 8 hours and writes the log output to individual files. The log file names are created from the start time, the workflow name and Order ID.


Example how to write order logs at an ongoing basis
# execute once
$lastHistory = Get-JS7OrderHistory -RelativeDateFrom -8h | Sort-Object -Property startTime

# execute in intervals
Get-JS7OrderHistory -DateFrom $lastHistory[0].startTime | Tee-Object -Variable lastHistory | Get-JS7OrderLog | Select-Object @{name='path'; expression={ "/tmp/history/$(Get-Date $_.startTime -f 'yyyyMMdd-hhmmss')-$([io.path]::GetFileNameWithoutExtension($_.workflow))-$($_.orderId).log"}}, @{name='value'; expression={ $_.log }} | Set-Content

Explanation:

  • Provides a mechanism to subsequently retrieve previous logs. Starting from initial execution of the Get-JS7OrderHistory cmdlet the resulting $lastHistory object is used for any subsequent calls.
    Consider use of the Tee-Object cmdlet in the pipeline that updates the $lastHistory object that can be used for later executions of the same pipeline.

  • This pipeline can e.g. be executed in a cyclic job.

Automate Log File Creation



Task Logs

The following cmdlets are provided:

Access Task Log Object

Example how to access task log objects
$logs = Get-JS7TaskHistory | Get-JS7TaskLog

Explanation:

  • The Get-JS7TaskHistory cmdlet returns history results that can be filtered by folders, workflows, date range, jobs, see cmdlet description. By default today's task executions are returned.
  • The Get-JS7TaskLog cmdlet is used in a pipeline and returns the task log object for each history entry.
  • As a result the $logs array holds the list of task log objects.

A task log object carries attributes as visible from the following console example:

Example of a task log object
PS /> $logs = Get-JS7TaskHistory | Get-JS7TaskLog
PS /> $logs[0]

controllerId : jobscheduler
agentUrl     : http://apmaccs:4449
taskId       : 15247
orderId      : #2022-03-06#P31960629618-pdCyclicSimpleWorkflowTicking
workflow     : /ProductDemo/CyclicExecution/pdCyclicSimpleWorkflowTicking
position     : 0/cycle+end=1646607600000,scheme=1,i=10,next=1646564721361:2
job          : job3
criticality  : normal
exitCode     : 0
state        : @{severity=6; _text=SUCCESSFUL}
startTime    : 06.03.2022 12:05:31
endTime      : 06.03.2022 12:05:36
log          : 2022-03-06 12:05:31.388+0100 [MAIN]    [Start] Job=job3, Agent (url=http://apmaccs:4449, id=agent_002)
               2022-03-06 12:05:31.795+0100 [STDOUT]  using workflow: pdCyclicSimpleWorkflowTicking
               running job: job3
               2022-03-06 12:05:36.399+0100 [MAIN]    [End] [Success] returnCode=0

Write Task Log to File

Example how to write task logs to a common log file
Get-JS7TaskHistory | Get-JS7TaskLog | Out-File /tmp/history/tasks.log -Encoding Unicode

Explanation:

  • Reads the logs of today's tasks and writes the logs to a common file.


Example how to write task logs to individual log files
Get-JS7TaskHistory -RelativeDateFrom -8h | Get-JS7TaskLog | Select-Object @{name='path'; expression={ "/tmp/history/$(Get-Date $_.startTime -f 'yyyyMMdd-hhmmss')-$([io.path]::GetFileNameWithoutExtension($_.job)).log"}}, @{name='value'; expression={ $_.log }} | Set-Content

Explanation:

  • Reads the logs of tasks that completed within the last 8 hours and writes the log output to individual files. The log file names are created from the start time and from the job name of each task.


Example how to write task logs at an ongoing basis
# execute once
$lastHistory = Get-JS7TaskHistory -RelativeDateFrom -8h | Sort-Object -Property startTime

# execute in intervals
Get-JS7TaskHistory -DateFrom $lastHistory[0].startTime | Tee-Object -Variable lastHistory | Get-JS7TaskLog | Select-Object @{name='path'; expression={ "/tmp/history/$(Get-Date $_.startTime -f 'yyyyMMdd-hhmmss')-$([io.path]::GetFileNameWithoutExtension($_.job)).log"}}, @{name='value'; expression={ $_.log }} | Set-Content

Explanation:

  • Provides a mechanism to subsequently retrieve previous logs. Starting from initial execution of the Get-JS7TaskHistory cmdlet the resulting $lastHistory object is used for any subsequent calls.
    Consider use of the Tee-Object cmdlet in the pipeline that updates the $lastHistory object that can be used for later executions of the same pipeline.

  • This pipeline can e.g. be executed in a cyclic job.

Automate Log File Creation



  • No labels