How to start a job chain on arrival of file(s)

UNDER PROGRESS

Introduction  

A job chains can be triggered by two type of Orders 1. time based Orders and 2. File Orders. A job chain can be configured to start on arrival of a file by using File Order Source.

A File Order Source require configured by minimum to parameters 1. Directory and 2. Regex, where Directory is the file system location where JobScheduler should watch for arrival of a file and Regex is the Regular Expression which should be matched with the filename. Most simple example of File Order Source is <file_order_source directory="c:\sandbox\outgoing" regex="^test.txt$"/>

Example: job chain  FileWatching - file order source 

Following example is a Windows operation system FileWatching job chain, configured to start as soon as the file test.txt arrives in the directory c:\sandbox\outgoing. The File Order will be processed as normal order through job chain, one step to the next, until the Order reaches to end node of job chain. In this example there are two distinctive job chain end nodes, each one for success and error state of File Order processing. 

<?xml version="1.0" encoding="ISO-8859-1"?>

<job_chain  title="File watching - file order source and regex" name="FileWatching">
    <file_order_source  directory="c:\sandbox\outgoing" regex="^test.txt$"/>
    <job_chain_node  state="100" job="show_file_name" next_state="success" error_state="error"/>
    <file_order_sink  state="success" remove="yes"/>
    <file_order_sink  state="error" remove="yes"/>
</job_chain>

Example: job - show_file_name

By default JobScheduler creates few internal environment variables during the Order processing. All the JobScheduler environment variables are prefixed by SCHEDULER_PARAMS_ prefix. JobScheduler stores the trigger file's path in environment variable SCHEDULER_PARAM_SCHEDULER_FILE_PATH In the following job example JobScheduler's inbuilt environment variable SCHEDULER_PARAM_SCHEDULER_FILE_PATH will be used to print file name on the stdout. 

<?xml version="1.0" encoding="ISO-8859-1"?>

<job  order="yes" stop_on_error="no" title="show file name" name="show_file_name">
    <script  language="shell">
        <![CDATA[
			@echo %SCHEDULER_JOB_NAME% : job starting			
            @echo .
			@echo %SCHEDULER_JOB_NAME% :Start for file [ %SCHEDULER_PARAM_SCHEDULER_FILE_PATH% ].....
            @echo .
			@echo File Name :  %SCHEDULER_PARAM_SCHEDULER_FILE_PATH%			
			@echo %SCHEDULER_JOB_NAME% :End for file [ %SCHEDULER_PARAM_SCHEDULER_FILE_PATH% ] .....
        ]]>
    </script>
    <run_time />
</job>

How to configure File Order Source - job chain 

To learn how to create a job chain, refer following KB article TBD

1. Open your job chain in JOE (JobScheduler Object Editor), expand job chain's and click on Steps/Nodes.
2. From right wondow pan select File Order Source tab
3. Click on New File Order Source, this will open a dialog to create a new File Order Source configuration.
4. Click in the text filed in front of Directory parameter, configure directory which should  be watched for arrival of file e.g. c:\sandbox\outgoing
5. Click in the text field in-front of Regex parameter, configure file name or Regular Expression e.g. ^test.txt$.
6. Click Apply File Order Source button to apply your  configuration.
7. Click Save to save job chain
   
   

File Order Sink

Every File Order is associated with a file on file system, this file was used as trigger to start the job chain. JobScheduler offers two options to handle files associated with a File Order, Once a File Order reaches to the end success/error node of job chain, file associated with it either can be remove or move to a location. 

Remove File

A file_order_sink node can be configured to remove file after processing e.g.( <file_order_sink state="success" remove="yes"/> ). File associated with the File Order will be deleted from file system location configured in the directory parameter.

Move File

A file_order_sink node can be configured to move file after processing to another directory e.g. (<file_order_sink  state="success" remove="no" move_to="c:\sandbox\success"/>). File associated with the File Order will be moved from file system location configured in the directory parameter to the location configured in the move_to parameter. 

How to configure File Order Sink - job chain

Remove File

1. Open your job chain in JOE (JobScheduler Object Editor), expand job chain's and click on Steps/Nodes.
2. From right wondow pan select Nodes tab
3. Click on state success, this will enable the dialog to configure FIle Order Sink for success state
4. Select radio option File Sink
5. Select check box Remove File to enable delete file after successful processing. 
6. Click Apply Chain Node button to apply your configuration.
7. Click Save to save job chain
8. Repeat steps 3 to 7 for error state too.

Move File

1. Open your job chain in JOE (JobScheduler Object Editor), expand job chain's and click on Steps/Nodes.
2. From right wondow pan select Nodes tab
3. Click on state success, this will enable the dialog to configure FIle Order Sink for success state
4. Select radio option File Sink
5. Click in the text field in-front of Move to parameter, configure directory location where file should be move after successful processing  e.g. c:\sandbox\success.
6. Click Apply Chain Node button to apply your configuration.
7. Click Save to save job chain
8. Repeat steps 3 to 7 for error state too.

Watching multiple directories

Watching multiple files

x