...
This job can create file-orders. It can be specified that a file-order is created for the first file of the result set only or for all files of the result set.
Parameter
...
Definitions
Parameters Used by JobSchedulerExistsFile
Name | titleTitle | mandatoryMandatory | defaultDefault |
|---|---|---|---|
File or Folder to watch for | true | . | |
Regular Expression for filename filtering | false |
| |
Specify error message tolerance | false | false | |
maximum Maximum age of a file | false | 0 | |
minimum Minimum age of a file | false | 0 | |
maximum Maximum size of a file | false | -1 | |
minimum Minimum size of one or multiple files | false | -1 | |
number Number of files to remove from the top of the result-set | false | 0 | |
number Number of files to remove from the bottom of the result-set | false | 0 | |
Return the size of resultset | false | false | |
Activate file-order creation | false | false | |
Create a file-order for every file in the result-list | false | false | |
| create_orders_for_new_files | |||
| param_name_file_path | |||
The name of the jobchain which belongs to the order | false |
| |
The first node to execute in a jobchain | false |
| |
| merge_order_parameter | |||
Set next node on empty result set | false | empty | |
number Number of expected hits in result-list | false | 0 | |
raise Raise error on expected size of result-set | false | 0 | |
Name of the result-list file | false | empty |
| check_steady_state_of_files | |||
| steady_state_count | |||
| check_steady_state_interval |
Parameter file: File or Folder to watch for
...
The following rules apply when the result - set is empty:
GRACIOUS | Standalone - Job | Order - Job |
|---|---|---|
false, 0, off, no, n, nein, none | error log, Task error | error log, set_state error |
true, 1, on, yes, y, ja, j | no error log, Task success | no error log, set_state error |
all | no error log, Task success | no error log, set_state success |
...
maximum age of a file
Specifies the maximum age of a file. If a file is older, then it is deemed not to exist, it will be not included in the result - list.
Data-Type : SOSOptionTime
The default value for this parameter is 0.
Parameter min_file_age: minimum age of a file
...
_minimum age of a file
_Specifies the minimum age of a files. If the file(s) is newer then it is classified as non-existing, it will be not included in the result - list.
Data-Type : SOSOptionTime
The default value for this parameter is 0.
Parameter max_file_size: maximum size of a file
...
_maximum size of a file
_Specifies the maximum size of a file in bytes: should the size of one of the files exceed this value, then it is classified as non-existing.
valid values for file size are
...
Parameter skip_last_files: number of files to remove from the bottom of the result-set
...
_ number of files to remove from the bottom of the result-set
_The number of files are removed from the end of the set resulting by min_file_size, min_file_age etc. These files are excluded from further operations.
...
Create a file-order for every file in the result-list
Valid values: true, 1, on, yes, y, ja, j and false, 0, off, no, n, nein
Data-Type : SOSOptionBoolean
The default value for this parameter is false.
Use together with parameter:
create_order - Activate file-order creationorder_jobchain_name - next_state -
Parameter
...
create_orders_for_new_files: Create a file-order for every new file in the result-list
...
Create a file-order for every new file in the result-list
If this parameter is set to "true", for each new file which is in the result set, a file-order is created and started.
This parameter is in effect only if the create_orders parameter is not set or has the value "true".
example 1: create a file-order
create_orders_for_new_files=true
Valid values: true, 1, on, yes, y, ja, j and false, 0, off, no, n, nein.
DataType: SOSOptionBoolean
Default: false
Parameter param_name_file_path:
...
This parameter sets the name of the parameter that contains the name of of the transferred file. The default value is scheduler_file_path. The name should be changed from the default if it is not desired to create file_orders that have to handle a file sink.
DataType: SOSOptionString
Default: ---
Parameter order_jobchain_name: The name of the jobchain which belongs to the order
...
The name of the jobchain which belongs to the order
The name of the job chain which has to be launched by the order is the value of this parameter.
One must take into account, that the name of the jobchain must contain a subfolder structure if the jobchain is not in the folder "live". An example: the jobchain "Test" is located in "live/sample/FileOperations/". The value which has to be specfied is then "/sample/FileOperations/Test".
Data-Type : SOSOptionString
Use together with parameter:
create_order - Activate file-order creationnext_state -
Parameter next_state: The first node to execute in a jobchain
...
The first node to execute in a jobchain
The name of the node of a jobchain, with which the execution of the chain must be started, is the value of this parameter.
Data-Type : SOSOptionJobChainNode
Use together with parameter:
create_order - Activate file-order creationorder_jobchain_name -
Parameter merge_order_parameter: Merge actual order parameter into new created order
...
merge actual order parameter into new created order
This parameter specifies, that the order, which has to be created, will be extended by the parameters of the actual order.
DataType: SOSOptionBoolean
Default: false
Parameter on_empty_result_set: Set next node on empty result set
...
Set next node on empty result set
The next Node (Step, Job) to execute in a JobChain can be set with this parameter. The value of the parameter is a (valid) node-name of the current JobChain. In case of an empty result-set, e.g. due to non existent files, the current job will end without an errors and the JobChain will continue with the name of the node which is given as the value of this parameter.
Data-Type : SOSOptionJobChainNode
The default value for this parameter is empty.
Parameter expected_size_of_result_set: number of expected hits in result list
...
number of expected hits in result-list
Data-Type : SOSOptionInteger
The default value for this parameter is 0.
Use together with parameter:
raise_error_if_result_set_is - raise error on expected size of result-set
Parameter raise_error_if_result_set_is: raise error on expected size of result set
...
raise error on expected size of result-set
With this parameter it is possible to raise an error if the quantity of hits of the result list is according to the value of this parameter.
An example:
Assuming, that the parameter "raise_error_if_result_set_is=ne" is defined and the parameter "expected_size_of_result_set=1" is specified as well. If the number of hits is not equal to "1" an error will raised.
Data-Type : SOSOptionRelOp
The default value for this parameter is 0.
Use together with parameter:
expected_size_of_result_set - number of expected hits in result-list
Parameter result_list_file: Name of the result list file
...
Name of the result-list file
If the value of this parameter specifies a valid filename the result-list will be written to this file.
Data-Type : SOSOptionFileName
The default value for this parameter is empty.
Parameter check_steady_state_of_files: Check the completeness of a file (steady state)
...
Check the completeness of a file (steady state)
In some file transfer scenarios the receiver of a file has no knowledge about the time when the sender creates the file. In case of a (very) large file it can be the situation that the receiver tries to read the file but the sender has not finished writing it. If the receiver get the file at the moment the sender is still writing, as a result he will get a corrupted, incomplete file.
Setting this parameter to "true" the receiver will check the file for completeness before he starts the transfer.
At the end, this is not a very secure approach, because the receiver is checking the date of last modification and the size of the file. If both not changing between a time intervall, which is defined by the parameters ..., the file is guessed to be complete. If the sender is terminated without writing the complete file, or the network is down, or the speed of processing the file is going slow, the receiver will get a corrupted file.
A better approach for avoiding corrupt files is to use the atomic method: writing a file and after completion of writing rename the file. For more details about this method see parameter atomic_suffix or atomic_prefix.
If more than one file is to be transferred, the transactional approach is the first choice. See parameter transactional.
DataType: SOSOptionBoolean
Default: false
Parameter steady_state_count: Maximum Number of Checkpoints
...
Maximum Number of Checkpoints
The value of this option specifies the number of retries for to check the steady state of a file.
DataType: SOSOptionInteger
Default: 30
Parameter check_steady_state_interval: Temporal distance between checkpoints
...
Temporal distance between checkpoints
The value of this option defines the temporal distance in seconds between two checkpoints.
DataType: SOSOptionTime
Alias: Steady_State_Interval
Default: 1
Return Parameters from JobSchedulerExistsFile
The order parameters
The name of the jobchain which belongs to the order
The name of the jobchain which has to be launched by the order is the value of this parameter.
One must take into account, that the name of the jobchain must contain a subfolder structure if the jobchain is not in the folder "live". An example: the jobchain "Test" is located in "live/sample/FileOperations/". The value which has to be specfied is then "/sample/FileOperations/Test".
Data-Type : SOSOptionString
Use together with parameter:
create_order - Activate file-order creationnext_state -
Parameter next_state: The first node to execute in a jobchain
The first node to execute in a jobchain
The name of the node of a jobchain, with which the execution of the chain must be started, is the value of this parameter.
Data-Type : SOSOptionJobChainNode
Use together with parameter:
create_order - Activate file-order creationorder_jobchain_name -
Parameter on_empty_result_set: Set next node on empty result set
Set next node on empty result set
The next Node (Step, Job) to execute in a JobChain can be set with this parameter. The value of the parameter is a (valid) node-name of the current JobChain. In case of an empty result-set, e.g. due to non existent files, the current job will end without an errors and the JobChain will continue with the name of the node which is given as the value of this parameter.
Data-Type : SOSOptionJobChainNode
The default value for this parameter is empty.
Parameter >expected_size_of_result_set: number of expected hits in result-list
number of expected hits in result-list
Data-Type : SOSOptionInteger
The default value for this parameter is 0.
Use together with parameter:
raise_error_if_result_set_is - raise error on expected size of result-set
Parameter raise_error_if_result_set_is: raise error on expected size of result-set
raise error on expected size of result-set
With this parameter it is possible to raise an error if the quantity of hits of the result-list is according to the value of this parameter.
An example:
Assuming, that the parameter "raise_error_if_result_set_is=ne" is defined and the parameter "expected_size_of_result_set=1" is specified as well. If the number of hits is not equal to "1" an error will raised.
Data-Type : SOSOptionRelOp
The default value for this parameter is 0.
Use together with parameter:
expected_size_of_result_set - number of expected hits in result-list
Parameter result_list_file: Name of the result-list file
Name of the result-list file
If the value of this parameter specifies a valid filename the result-list will be written to this file.
Data-Type : SOSOptionFileName
The default value for this parameter is empty.
Return parameter JobSchedulerExistsFile
The order parameter described below are returned by the job to the JobScheduler. JobSchedulerExistsFile
Name | titleTitle | mandatoryMandatory | defaultDefault | ||||
|---|---|---|---|---|---|---|---|
file File to process for a file-order | false | empty | |||||
pathanme Pathname of the file to process for a file-order | false | empty | |||||
Name of the file to process for a file-order | false | empty | |||||
The result of the operation as a list of items | false | empty | |||||
The amount of hits in the result set of the operation | false | empty | Return the size of the result set after a file operation | false | 0 | ||
Return the size of the result set after a file operation | false | 0 |
...