| Table of Contents | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
|
Description of JobSchedulerExistsFile - check whether a file exists
Checks for the existence of a file, a directory or for specific files inside of a directory.
| No Format |
|---|
<mscgen>
|
//file_order_event
msc {
| No Format |
|---|
width1. "900";
|
JobScheduler label"JobScheduler", textcolouh1. "navy", arctextcolour"navy",
| No Format |
|---|
JobChainPoll [labeh1. "JobChain Polling", textcolour"orange", arctextcolouh1. "orange"],
OrderPoll [label"Order for Pollling", textcolouh1. "brown", arctextcolour"brown"],
FileSystem [labeh1. "File System", textcolour"maroon", arctextcolouh1. "maroon"],
JobChainProcessing [label"JobChain Processing", textcolouh1. "orange", arctextcolour"orange"],
OrderProcessing [labeh1. "Order for Processing", textcolour"brown", arctextcolouh1. "brown"];
|
JobScheduler> JobChainPoll labelh1. "(1) start order for file watching";
| No Format |
|---|
JobScheduler .. FileSystem [label "waiting for files "];
|
OrderPollh1. > FileSystem label "(2) polling";
| No Format |
|---|
FileSystemh1. > OrderPoll [label "(3) file arrived"];
OrderPollh1. > JobChainProcessing [label "(4) create order for processing"];
OrderProcessingh1. > OrderProcessing [label "(5) check for ready state"];
|
OrderProcessingh1. > JobChainProcessing label "(6) start node";
| No Format |
|---|
OrderProcessingh1. > JobChainProcessing [label "(7) start next node(s)"];
OrderProcessingh1. > JobChainProcessing [label "(8) end jobChain"];
OrderProcessingh1. > JobChainProcessing [label "(9) process file sink"];
JobChainProcessingh1. > FileSystem [label "(10) delete/move file"];
JobScheduler .. FileSystem [labelh1. "repeat waiting for files "];
OrderPoll> FileSystem [labelh1. "(11) polling"];
|
}
</mscgen>
This job is creating a result set. A result set contain the names of all files which are selected as specified by the filter criteria. The content of the result set is returned as a parameter, but can be written to a file, too.
Parameters, which are useful for creating a result-set, are:
ParameterName | Title |
|---|---|
raise_error_if_result_set_is |
|
result_list_file |
|
expected_size_of_result_set |
|
on_empty_result_set |
|
scheduler_sosfileoperations_resultset |
|
scheduler_sosfileoperations_resultsetsize |
|
An example for a job-xml file:
| Code Block |
|---|
<job order='no' >
<params>
<param name="[[#file|file]]" value="." />
<param name="[[#file_spec|file_spec]]" value="" />
<param name="[[#gracious|gracious]]" value="false" />
<param name="[[#max_file_age|max_file_age]]" value="0" />
<param name="[[#min_file_age|min_file_age]]" value="0" />
<param name="[[#max_file_size|max_file_size]]" value="-1" />
<param name="[[#min_file_size|min_file_size]]" value="-1" />
<param name="[[#skip_first_files|skip_first_files]]" value="0" />
<param name="[[#skip_last_files|skip_last_files]]" value="0" />
<param name="[[#count_files|count_files]]" value="false" />
<param name="[[#create_order|create_order]]" value="false" />
<param name="[[#create_orders_for_all_files|create_orders_for_all_files]]" value="false" />
<param name="[[#order_jobchain_name|order_jobchain_name]]" value="" />
<param name="[[#next_state|next_state]]" value="" />
<param name="[[#on_empty_result_set|on_empty_result_set]]" value="empty" />
<param name="[[#expected_size_of_result_set|expected_size_of_result_set]]" value="0" />
<param name="[[#raise_error_if_result_set_is|raise_error_if_result_set_is]]" value="0" />
<param name="[[#result_list_file|result_list_file]]" value="empty" />
</params>
<script language="java" java_class="sos.scheduler.file.JobSchedulerExistsFile" />
</job>
|
...
Parameter used by JobSchedulerExistsFile
...
...
Name
Name |
|---|
title | mandatory | default | |
|---|---|---|---|
File or Folder to watch for | true | . | |
Regular Expression for filename filtering | false |
| |
Specify error message tolerance | false | false | |
maximum age of a file | false | 0 | |
minimum age of a file | false | 0 | |
maximum size of a file | false | -1 | |
minimum size of one or multiple files | false | -1 | |
number of files to remove from the top of the result-set | false | 0 | |
number of files to remove from the bottom of the result-set | false | 0 | |
Return the size of resultset | false | false | |
Activate file-order creation | false | false | |
Create a file-order for every file in the result-list | false | false | |
The name of the jobchain which belongs to the order | false |
| |
The first node to execute in a jobchain | false |
| |
Set next node on empty result set | false | empty | |
number of expected hits in result-list | false | 0 | |
raise error on expected size of result-set | false | 0 | |
Name of the result-list file | false | empty |
Parameter
...
file
...
: File or Folder to watch for
...
File or Folder to watch for
Checked file or directory
Supports masks for substitution in the file name and directory name with format strings that are enclosed by {*} and {*} [and] . The following format strings are supported:
| Code Block |
|---|
[date: date format ]
'''date format''' must be a valid Java data format string, e.g. '''yyyyMMddHHmmss''' , '''yyyy-MM-dd.HHmmss''' etc.
|
An example:
| Code Block |
|---|
<param name="file" value="sample/hello[date:yyyyMMdd].txt" />
|
On 2050-12-31 the parameter file contains the value "sample/hello20501231.txt" .
This parameter supports substitution of job parameter names with their value if the job parameter name is enclosed by % and % .
An example: <param namh1. name="file" value"%scheduler_file_path%" />
During the job runtime the parameter file contains the value of the job parameter scheduler_file_path . Using Directory Monitoring with File Orders the job parameter scheduler_file_path contains automatically the path of the file that triggered the order.
Data-Type : SOSOptionString
The default value for this parameter is ..
This parameter is mandatory.
Parameter
...
file_spec
...
: Regular Expression for filename filtering
...
Regular Expression for filename filtering
Regular Expression for file filtering. The behaviour is CASE_INSENSITIVE.
Only effective if the parameter file is a directory.
Some remarks on regular expression, used in JobScheduler.
A regular expression is not a wildcard . To get an impression of the differences one have a look on the meaning of the wildcard .txt, which will select all filenames with the filename-extension ".txt". A regular expression to match, e.g. works the same way, this "wildcard" must look like "^.\.txt$". That looks a little bit strange but it is much more flexible and powerfull on filtering filenames than the concept of wildcards, if one want to filter more complex names or pattern.
The general syntax of an regular expression , also referred to as regex or regexp, is described here . It is different to other RegExp definitions, e.g. as for Perl.
Data-Type : SOSOptionRegExp
Parameter
...
gracious
...
: Specify error message tolerance
...
Specify error message tolerance
Enables or disables error messages that are caused by an empty result-set, which is the result of an operation, executed by the job. Therefore this parameter can control the sequence of nodes or states in a job-chain.
Valid values:
| Code Block |
|---|
'''false, 0, off, no, n, nein, none''' , '''true, 1, on, yes, y, ja, j''' and '''all''' .
|
...
For example, the setting "graciouh1. gracious=all" will suppress all errors regarding an empty result-set and will terminate a Job (standalone and inside a jobchain) as it would be without errors.
Data-Type : SOSOptionGracious
The default value for this parameter is false.
Parameter
...
max_file_age
...
: maximum age of a file
...
maximum age of a file
Specifies the maximum age of a file. If a file is older, then it is deemed not to exist, it will be not included in the result-list.
Data-Type : SOSOptionTime
The default value for this parameter is 0.
Parameter
...
min_file_age
...
: minimum age of a file
...
_minimum age of a file_Specifies the minimum age of a files. If the file(s) is newer then it is classified as non-existing, it will be not included in the result-list.
Data-Type : SOSOptionTime
The default value for this parameter is 0.
Parameter
...
max_file_size
...
: maximum size of a file
...
_maximum size of a file_Specifies the maximum size of a file in bytes: should the size of one of the files exceed this value, then it is classified as non-existing.
valid values for file size are
...
Data-Type : SOSOptionFileSize
The default value for this parameter is -1.
Parameter
...
min_file_size
...
: minimum size of one or multiple files
...
minimum size of one or multiple files
Specifies the minimum size of one or multiple files in bytes: should the size of one of the files fall below this value, then it is not included in the result list of the operation.
valid values for file size are
...
Data-Type : SOSOptionFileSize
The default value for this parameter is -1.
Parameter
...
skip_first_files
...
: number of files to remove from the top of the result-set
...
number of files to remove from the top of the result-set
The number of files are removed from the beginning of the set resulting by min_file_size , min_file_age etc. These files are excluded from further operations.
The result set is sorted according to the used filter parameters:
...
Only either skip_first_files or skip_last_files is allowed to be set at the same time.
Data-Type : SOSOptionInteger
The default value for this parameter is 0.
Parameter
...
skip
...
_last_
...
files: number of files to remove from the bottom of the result-set
...
_number of files to remove from the bottom of the result-set_The number of files are removed from the end of the set resulting by min_file_size, min_file_age etc. These files are excluded from further operations.
...
Only either skip_first_files or skip_last_files is allowed to be set at one time.
Data-Type : SOSOptionInteger
The default value for this parameter is 0.
Parameter
...
count_files
...
: Return the size of resultset
...
Return the size of resultset
If this parameter is set true " true " the number of matches is returned in the order parameter " scheduler_SOSFileOperations_file_count ".
Valid values: true, 1, on, yes, y, ja, j and false, 0, off, no, n, nein
This parameter is valid and available for order driven jobs only. JobChains, for example, are order driven jobs. In standalone jobs this parameter will be ignored without further notice.
Data-Type : SOSOptionBoolean
The default value for this parameter is false.
Parameter
...
create_order
...
: Activate file-order creation
...
Activate file-order creation
With this parameter it is possible to specify, that for all filenames in the resultlist or for the first file only (see create_orders_for_all_files ) a file-order has to be created and launched.
Valid values: true, 1, on, yes, y, ja, j and false, 0, off, no, n, nein
Data-Type : SOSOptionBoolean
The default value for this parameter is false.
Use together with parameter:
create_orders_for_all_files - Create a file-order for every file in the result-listorder_jobchain_name - next_state -
Parameter
...
create_orders_for_all_files
...
: Create a file-order for every file in the result-list
...
Create a file-order for every file in the result-list
Valid values: true, 1, on, yes, y, ja, j and false, 0, off, no, n, nein
Data-Type : SOSOptionBoolean
The default value for this parameter is false.
Use together with parameter:
create_order - Activate file-order creationorder_jobchain_name - next_state -
Parameter
...
order_jobchain_name
...
: The name of the jobchain which belongs to the order
...
The name of the jobchain which belongs to the order
The name of the jobchain which has to be launched by the order is the value of this parameter.
One must take into account, that the name of the jobchain must contain a subfolder structure if the jobchain is not in the folder "live". An example: the jobchain "Test" is located in "live/sample/FileOperations/". The value which has to be specfied is then "/sample/FileOperations/Test".
Data-Type : SOSOptionString
Use together with parameter:
create_order - Activate file-order creationnext_state -
Parameter
...
next_state
...
: The first node to execute in a jobchain
...
The first node to execute in a jobchain
The name of the node of a jobchain, with which the execution of the chain must be started, is the value of this parameter.
Data-Type : SOSOptionJobChainNode
Use together with parameter:
create_order - Activate file-order creationorder_jobchain_name -
Parameter
...
on_empty_result_set
...
: Set next node on empty result set
...
Set next node on empty result set
The next Node (Step, Job) to execute in a JobChain can be set with this parameter. The value of the parameter is a (valid) node-name of the current JobChain. In case of an empty result-set, e.g. due to non existent files, the current job will end without an errors and the JobChain will continue with the name of the node which is given as the value of this parameter.
Data-Type : SOSOptionJobChainNode
The default value for this parameter is empty.
...
.
Parameter >expected_size_of_result_
...
set: number of expected hits in result-list
...
number of expected hits in result-list
Data-Type : SOSOptionInteger
The default value for this parameter is 0.
Use together with parameter:
raise_error_if_result_set_is - raise error on expected size of result-set
Parameter
...
raise_error_if_result_set_is
...
: raise error on expected size of result-set
...
raise error on expected size of result-set
With this parameter it is possible to raise an error if the quantity of hits of the result-list is according to the value of this parameter.
An example:
Assuming, that the parameter "raise_error_if_result_set_isneis=ne" is defined and the parameter "expected_size_of_result_seh1. set=1" is specified as well. If the number of hits is not equal to "1" an error will raised.
Data-Type : SOSOptionRelOp
The default value for this parameter is 0.
Use together with parameter:
expected_size_of_result_set - number of expected hits in result-list
Parameter
...
result_list_file
...
: Name of the result-list file
...
Name of the result-list file
If the value of this parameter specifies a valid filename the result-list will be written to this file.
Data-Type : SOSOptionFileName
The default value for this parameter is empty.
...
The order parameter described below are returned by the job to the JobScheduler. JobSchedulerExistsFile
...
...
Name | title | mandatory | default |
|---|---|---|---|
file to process for a file-order | false | empty | |
pathanme of the file to process for a file-order | false | empty | |
Name of the file to process for a file-order | false | empty | |
The result of the operation as a list of items | false | empty | |
The amount of hits in the result set of the operation | false | empty | |
Return the size of the result set after a file operation | false | 0 | |
Return the size of the result set after a file operation | false | 0 |
...
Parameter
...
scheduler_file_path
...
: file to process for a file-order
...
file to process for a file-order
Using Directory Monitoring with File Orders the job parameter scheduler_file_path contains automatically the path of the file that triggered the order.
Data-Type : SOSOptionFileName
The default value for this parameter is empty.
Parameter
...
scheduler_file_parent
...
: pathanme of the file to process for a file-order
...
pathanme of the file to process for a file-order
Data-Type : SOSOptionFileName
The default value for this parameter is empty.
Parameter
...
scheduler_file_name
...
: Name of the file to process for a file-order
...
Name of the file to process for a file-order
Data-Type : SOSOptionFileName
The default value for this parameter is empty.
Parameter
...
scheduler_sosfileoperations_resultset
...
: The result of the operation as a list of items
...
The result of the operation as a list of items
Data-Type : SOSOptionstring
The default value for this parameter is empty.
Use together with parameter:
scheduler_sosfileoperations_Resultsetsize - scheduler_sosfileoperations_file_count -
Parameter
...
scheduler_sosfileoperations_resultsetsize
...
: The amount of hits in the result set of the operation
...
The amount of hits in the result set of the operation
Data-Type : SOSOptionsInteger
The default value for this parameter is empty.
Use together with parameter:
scheduler_sosfileoperations_ResultSet - scheduler_sosfileoperations_FileCount -
Parameter
...
scheduler_sosfileoperations_file_count
...
: Return the size of the result set after a file operation
...
Return the size of the result set after a file operation
Data-Type : SOSOptionInteger
The default value for this parameter is 0.
Use together with parameter:
scheduler_sosfileoperations_Resultsetsize - scheduler_sosfileoperations_ResultSet -
Parameter
...
scheduler_sosfileoperations_file_count
...
: Return the size of the result set after a file operation
...
Return the size of the result set after a file operation
...