The root element of a YADE configuration.
An application configuration file contains settings that are specific to an app.
Properties from Java property files are applied as system properties by YADE.
This parameter specifies the hostname (e.g. foo.org) or IP address (e.g. IPv4 192.168.0.1) of the (FTP, SFTP, SSH, etc.) server to which a connection has to be made.
Support for KeePass versions 1 and 2.
Note: Availability starting with YADE-464.
Referencing CredentialStore fields.
Note: Availability starting with YADE-481.
Referencing a field in the CredentialStore is effected by the following syntax:
cs://[entry_path]@entry_field
where
<Hostname> element:
Examples
Account for authentication at one of the systems involved in file transfer, e.g. an FTP or SFTP server. Usually the account corresponds to a user name.
Should the respective server system be part of a Windows domain then the syntax domain\account can be used.
Support for KeePass versions 1 and 2.
Note: Availability starting with YADE-464.
Referencing CredentialStore fields.
Note: Availability starting with YADE-481.
Referencing a field in the CredentialStore is effected by the following syntax:
cs://[entry_path]@entry_field
where
<Hostname> element:
Examples
Password for authentication at a server that is involved in file transfer, e.g. with an FTP or SFTP protocol or Proxy protocol that makes use of BasicAuthentication.
Passwords are not displayed in the YADE log files.
A drawback is that passwords are visible in YADE configuration files. In order to avoid this you could switch to using File Transfer Protocols that allow SSHAuthentication, e.g. SFTP.
Support for KeePass versions 1 and 2.
Note: Availability starting with YADE-464.
Referencing CredentialStore fields.
Note: Availability starting with YADE-481.
Referencing a field in the CredentialStore is effected by the following syntax:
cs://[entry_path]@entry_field
where
<Hostname> element:
Examples
Passive mode is often required for FTP transfers with firewalls.
Transfer mode is used only for FTP and can be either ascii or binary.
A command delimiter option for executing the pre- and postcommands
This option determines whether the individual files from the results list should be transferred individually and then should be cumulated (i.e. concatenated into a single file) to the transfer target. The name of the cumulative file is specified using the CumulativeFilename parameter.
The CumulativeFileSeparator parameter is used to specify a string that is added to the target file, between the individual files in order to allow these files to be separated later on.
Should the source files be deleted after transfer then the Move operation can be used.
This parameter specifies whether or not the content of the source files should be compressed for the target files using a zip algorithm.
In case of sending files each file will be compressed in a single zip file. The extension of the filename is configured with the parameter CompressedFileExtension. A gzip-compatible compression is used, no further software components are required.
The following parameters are ignored should CompressFiles be used:
A Rename operation renames files during transfer from a source system to the target system.
Renamimg is performed either for files in the source system or in the target system depending on the occurence of the Rename parameter. The following applies to use with a Copy operation:
Accept a valid certificate that could not be verified as being trustworthy.
Usually a certificate verification process will always verify the DNS name of the certificate presented by the server, This setting can be used to disable the verification process of the hostname.
When using this setting uses should be aware that this will reduce the credibility of certificate and therefore should be used exclusively for hosts that you trust.
JobScheduler releases starting from JS7 2.2.2
This element specifies a JS7 Job Resource.
This element specifies the name of a JS7 Job Resource.
This element specifies the environment variable name of a JS7 Job Resource.
This element specifies the variable name of a JS7 Job Resource.
The branch element for reusable transfer configuration.
Subsequent fragments are used e.g. for file transfer specifications, notifications, mail server settings, that can be used in a number of file transfer Profiles.
A fragment specifies how a file transfer is carried out whereas a Profile specifies what should be transferred. Therefore multiple profiles can use the same fragments, e.g. to transfer different files from the same source or to reuse the same transfer specification as the source and the target in different transfer operations.
This element includes the parameters for e-mail notification.
Any number of fragments can be configured and are distinguished by YADE according to their name attribute.
A fragment is referenced by a transfer profile using the value of the NotificationFragmentRef ref attribute.
This attribute identifies the current fragment. Any number of fragments can be used and are distinguished by YADE according their name.
A fragment is referenced by a transfer profile using the value of this attribute.
Fragments are a means to organize file transfer settings in a reusable way:
Any number of fragments can be configured. They are distinguished by their name attribute. References to a fragment use the value of the name attribute to identify the respective fragment.
This element includes the parameters for an FTP file transfer connection including BasicAuthentication and BasicConnection configuration items.
Any number of fragments can be configured and are distinguished by YADE according to their name attribute. Therefore a unique identifier should be specified for the name attribute.
A fragment is referenced by a transfer Profile using the value of the FTPFragmentRef and its ref attribute.
This attribute identifies the current fragment. Any number of fragments can be used and are distinguished by YADE based on their name.
A fragment is referenced via this attribute value by a transfer profile.
This element includes the parameters for an FTPS file transfer connection including BasicAuthentication and BasicConnection configuration items.
Any number of fragments can be configured and are distinguished by YADE according to their name attribute.
A fragment is referenced by a transfer profile using the value of the FTPSFragmentRef ref attribute.
This attribute identifies the current fragment. Any number of fragments can be used and are distinguished by YADE according their name.
A fragment is referenced by a transfer profile using the value of this attribute.
This element includes the parameters for an SFTP file transfer connection including BasicConnection and SSHAuthentication configuration items.
Any number of fragments can be configured and are distinguished by YADE according to their name attribute.
A fragment is referenced by a transfer profile using the value of the SFTPFragmentRef ref attribute.
This attribute identifies the current fragment. Any number of fragments can be used and are distinguished by YADE according their name.
A fragment is referenced by a transfer profile using the value of this attribute.
This element includes the parameters for a WebDAV file transfer connection including BasicAuthentication and URLConnection configuration items.
Any number of fragments can be configured and are distinguished by YADE according to their name attribute.
A fragment is referenced by a transfer profile using the value of the WebDAVFragmentRef ref attribute.
This attribute identifies the current fragment. Any number of fragments can be used and are distinguished by YADE according their name.
A fragment is referenced by a transfer profile using the value of this attribute.
This element includes the parameters for an HTTP file transfer connection including BasicAuthentication and URLConnection configuration items.
Any number of fragments can be configured and are distinguished by YADE according to their name attribute.
A fragment is referenced by a transfer profile using the value of the HTTPFragmentRef ref attribute.
This attribute identifies the current fragment. Any number of fragments can be used and are distinguished by YADE according their name.
A fragment is referenced by a transfer profile using the value of this attribute.
This element includes the parameters for an HTTPS file transfer connection including BasicAuthentication and URLConnection configuration items.
Any number of fragments can be configured and are distinguished by YADE according to their name attribute.
A fragment is referenced by a transfer profile using the value of the HTTPSFragmentRef ref attribute.
This attribute identifies the current fragment. Any number of fragments can be used and are distinguished by YADE according their name.
A fragment is referenced by a transfer profile using the value of this attribute.
This element includes the parameters for an SMB file transfer connection including SMBAuthentication and Hostname configuration items.
Any number of fragments can be configured and are distinguished by YADE according to their name attribute.
A fragment is referenced by a transfer profile using the value of the SMBFragmentRef ref attribute.
This attribute identifies the current fragment. Any number of fragments can be used and are distinguished by YADE according their name.
A fragment is referenced by a transfer profile using the value of this attribute.
The BasicAuthentication element specifies the credentials for authentication with a server, e.g. using an FTP protocol or a Proxy Protocol. Child elements include:
BasicAuthentication is available for a number of File Transfer Protocols and Proxy Protocols. The drawback with BasicAuthentication is that passwords are stored directly in configuration files. In order to avoid this you could switch to using File Transfer Protocols that allow SSHAuthentication, e.g. SFTP.
The SMBAuthentication element specifies the credentials for authentication with a file system by SMB also known as CIFS. Windows operating systems include SMB support, for Unix operating systems SMB is optionally available.
The SSHAuthentication element specifies the credentials for authentication with a server, e.g. an FTP Server or a Proxy. Child elements include:
A connection specifies parameters for a Hostname and a Port to which a connection is established.
ProxyForFTP - will use the HTTP, SOCKS4 or SOCKS5 proxy
ProxyForWebDAV - will use the HTTP proxy
Proxies can make use of different protocols, a HTTPProxy - as the name suggests - will use the HTTP protocol
HTTP proxies optionally support authentication.
ProxyForSFTP - will use the HTTP, SOCKS4 or SOCKS5 proxy
ProxyForFTPS - will use the HTTP, SOCKS4 or SOCKS5 proxy
Proxies can make use of different protocols, a SOCKS4Proxy - as the name suggests - will use the SOCKS4 protocol
SOCKS4 proxies do not support authentication.
Proxies can make use of different protocols, a SOCKS5Proxy - as the name suggests - will use the SOCKS5 protocol
SOCKS5 proxies optionally support authentication.
The use of this element specifies that for SSHAuthentication an authentication method using a Password will be applied.
As an alternative to using passwords the authentication methods:
The use of this element specifies that for SSHAuthentication an authentication method for public/private keys using an AuthenticationFile will be applied.
This is called public/private key authentication and helps to avoid the use of passwords for authentication.
As an alternative to public/private key authentication the authentication methods:
Usage only with the YADE Client on the command shell.
The use of this element specifies that for SSHAuthentication an authentication method for keyboard interactive authentication, which allows the YADE Client on the command shell to ask a password question and the user to input a response.
As an alternative to keyboard interactive authentication the authentication methods:
The value of this parameter specifies the path and name of an account's private key file used for SSHAuthentication. This parameter must be specified if the authentication method AuthenticationMethodPublickey is used.
Should the private key file be secured by a passphrase then this must be specified using the Passphrase parameter.
Authentication files are most often stored in the user home directory in a .ssh folder and have to be secured with the proper file permissions. However, any path can be specified for an authentication file with this parameter. The location of the authentication file has to be accessbible for the account that runs the YADE Client. For Unix systems file permissions 600 are required for authentication files.
Support for KeePass versions 1 and 2.
Note: Availability starting with YADE-464.
Referencing CredentialStore fields.
Note: Availability starting with YADE-481.
Referencing a field in the CredentialStore is effected by the following syntax:
cs://[entry_path]@entry_field
where
<Hostname> element:
Examples
A passphrase secures an AuthenticationFile that is used for SSHAuthentication. The passphrase is added when creating the private key authentication file.
Passphrases are not displayed in the YADE log files.
As a drawback passphrases are visible in YADE configuration files if not protected as secure strings, e.g. from a CredentialStore.
Support for KeePass versions 1 and 2.
Note: Availability starting with YADE-464.
Referencing CredentialStore fields.
Note: Availability starting with YADE-481.
Referencing a field in the CredentialStore is effected by the following syntax:
cs://[entry_path]@entry_field
where
<Hostname> element:
Examples
The FTPS protocol type, e.g. SSL, TLS. It should be considered that depending on the protocol type usually a different Port parameter has to be specified. For SSL/TLS use of port 990 is quite common.
This parameter specifies the security mode. Possible values: explicit, implicit
A URL connection specifies parameters such as a Hostname, Port, Account by which a connection is established within the URL.
The use of URLs depends on the server system, e.g. the following protocols and configuration items could be used:
http|https :// [ account [ : password ] @ ] hostname [ : port ]
If the port is empty, the default ports will be used:
A local source is considered to be located locally to the YADE Client, i.e. in the local file system.
A local target is considered to be located on the server where the YADE Client is operated, i.e. in the local file system.
Filename and path of the attachment(s) to a notification e-mail. Multiple attachments are separated by ";".
Content type of the e-mail. Possible values are "text/plain" and "text/html".
E-mail can be specified to be sent in case of events that are configured with this element:
Send a notification in case of success.
Send a notification in case of error.
Send a notification if empty files are detected.
This element references the fragment that is used for notification of a file transfer operation.
Any number of reusable MailFragment elements can be configured and are distinguished by YADE based on their name attribute.
The reference to a fragment is based on the value of the ref attribute of this element to the corresponding name attribute of the MailFragment element.
This attribute identifies the fragment that is used for notification about file transfers.
This element references the fragment that is used for mail servers in case of notification of a file transfer operation.
Any number of reusable MailServerFragment elements can be configured and are distinguished by YADE based on their name attribute.
The reference to a fragment is based on the value of the ref attribute of this element to the corresponding name attribute of the MailServerFragment element.
This attribute identifies the fragment that is used for mail servers in case of notification about file transfers.
A Profile specifies the operation and the settings that are used for a file transfer. Multiple profiles can be included in a YADE configuration.
Profiles are distinguished by YADE based on their profile_id.
When invoking YADE the respective Profile for the file transfer operation is specified.
A profile is identified by this attribute value that is specified when invoking YADE. As multiple profiles can be stored in a YADE configuration this value is a unique identifier for the settings that are grouped in a profile.
The General configuration options refer to central settings that are applied to all file transfer configuratons such as use of the YADE Background Service and logging.
Some settings from the General branch can be superseeded by Profile settings that apply to an individual file transfer configuration.
The specification of General configuration parameters is not mandatory.
A Copy operation performs the transfer of files between a source and a target host.
Two settings groups are available to specify a Copy operation:
The Move operation is similar to the Copy operation except that the source files will be removed after successful transfer.
Two settings groups are available to specify a Move operation:
This parameter specifies the source of a Copy operation.
The following frequently used settings groups are available:
This parameter specifies the source of a Move operation.
The following frequently used settings groups are available:
This element references a readable fragment that is used to copy files in a Copy operation.
Any number of reusable Fragments elements can be configured and are distinguished by YADE based on their name attribute.
A subsequent fragment reference element points to the fragment that is used as source in the Copy operation.
This element references a readable fragment that is used to retrieve file names in a GetList operation.
Any number of reusable Fragments elements can be configured and are distinguished by YADE based on their name attribute.
A subsequent fragment reference element points to the fragment that is used as source in the GetList operation.
This element references a readable fragment that is used to move files in a Move operation.
Any number of reusable Fragments elements can be configured and are distinguished by YADE based on their name attribute.
A subsequent fragment reference element points to the fragment that is used as source in the Move operation.
This element references a writeable fragment that is used to delete files in a Remove operation.
Any number of reusable Fragments elements can be configured and are distinguished by YADE based on their name attribute.
A subsequent fragment reference element points to the fragment that is used for the Remove operation.
This element references the fragment that is used for an FTP file transfer operation.
Any number of reusable FTPFragment elements can be configured and are distinguished by YADE based on their name attribute.
The reference to a fragment is based on the value of the ref attribute of this element to the corresponding name attribute of the FTPFragment element.
This attribute identifies the fragment that is used for the FTP protocol.
Specifies commands that are executed on the FTP server before transfer of files. This includes FTP commands, not shell commands.
For a reference of possible commands see List of FTP commands. Only a small subset of commands is part of the FTP standard and therefore can be assumed to be available. The availability of most commands depends on the implementation of the FTP server product.
Specifies commands that are executed on the FTP server after transfer of files. This includes FTP commands, not shell commands.
For a reference of possible commands see List of FTP commands. Only a small subset of commands is part of the FTP standard and therefore can be assumed to be available. The availability of most commands depends on the implementation of the FTP server product.
This element references the fragment that is used for an SFTP file transfer operation. Any number of reusable SFTPFragment elements can be configured and are distinguished by YADE based on their name attribute.
The reference to a fragment is based on the value of the ref attribute of this element to the corresponding name attribute of the SFTPFragment element.
This attribute identifies the fragment that is used for the SFTP protocol.
Specifies commands that are executed on the SFTP server before transfer of files. This includes SFTP commands, not shell commands.
Specifies the commands to be executed for each file on the server before transfer of files.
The following special variables are available:
Commands to be executed before the transfer of files starts.
Specifies the commands to be executed for each file on the server after the transfer and the Rename.
The following environment variables are available in the SFTPPostProcessing and LocalPostProcessing:
The following special variables are available:
Post transfer commands execution in case of the successful transfers.
Built-in command functions:
Post transfer commands execution on transfer error.
Built-in command functions:
Post transfer commands execution at transfer end independent of the transfer status.
Built-in command functions:
Specifies commands that are executed on the SFTP server after transfer of files. This includes SFTP commands, not shell commands.
Specifies commands that are executed on the server where the YADE Client is located after transfer of files. This includes shell commands.
Specifies the built-in command functions that are executed on the server where the YADE Client is located after transfer of files.
This element references the fragment that is used for an FTPS file transfer operation. Any number of reusable FTPSFragment elements can be configured and are distinguished by YADE based on their name attribute.
The reference to a fragment is based on the value of the ref attribute of this element to the corresponding name attribute of the FTPSFragment element.
This attribute identifies the fragment that is used for the FTPS protocol.
This element references the fragment that is used for an WebDAV file transfer operation. Any number of reusable WebDAVFragment elements can be configured and are distinguished by YADE based on their name attribute.
The reference to a fragment is based on the value of the ref attribute of this element to the corresponding name attribute of the WebDAVFragment element.
This attribute identifies the fragment that is used for the WebDAV protocol.
Specifies commands that are executed on the server where the YADE Client is located before transfer of files. This includes shell commands.
Specifies commands that are executed on the server where the YADE Client is located after transfer of files. This includes shell commands.
This element references the fragment that is used for an HTTP file transfer operation. Any number of reusable HTTPFragment elements can be configured and are distinguished by YADE based on their name attribute.
The reference to a fragment is based on the value of the ref attribute of this element to the corresponding name attribute of the HTTPFragment element.
This attribute identifies the fragment that is used for the HTTP protocol.
This element references the fragment that is used for an FTPS file transfer operation. Any number of reusable HTTPSFragment elements can be configured and are distinguished by YADE based on their name attribute.
The reference to a fragment is based on the value of the ref attribute of this element to the corresponding name attribute of the HTTPSFragment element.
This attribute identifies the fragment that is used for the HTTPS protocol.
This element references the fragment that is used for an SMB file transfer operation. Any number of reusable SMBFragment elements can be configured and are distinguished by YADE based on their name attribute.
The reference to a fragment is based on the value of the ref attribute of this element to the corresponding name attribute of the SMBFragment element.
This attribute identifies the fragment that is used for the SMB protocol.
Specifies commands that are executed on the server where the YADE Client is located before transfer of files. This includes shell commands.
These options apply to the handling of files on a source server. They specify e.g. the Selection of files for Copy and Move operations.
These options apply to the handling of files on a source server. They specify e.g. the Selection of files for Copy and Move operations.
These options apply to the handling of files on a source server. They specify e.g. the Selection of files for Copy and Move operations.
These options apply to the handling of files on a source server. They specify e.g. the Selection of files for Copy and Move operations.
These options apply to the handling of files on a source server. They specify e.g. the Selection of files for Copy and Move operations.
A file Selection includes specifying one of
This parameter specifies the Selection of files from a source by the following settings:
The specified files are expected to be available from the source for transfer by YADE.
This parameter is used to specify an individual file or a number of files for processing. It is an alternative to the FileSpecSelection and FileListSelection.
More than one file can be specified by using a ";" between the file names. All files will be processed in the order in which they are defined in this parameter.
This parameter accepts absolute, relative and runtime paths of files that are to be processed.
In case of a relative path:
In case of a runtime path (file to process for a file-order in a file_order_source job chain):
This parameter specifies the Selection of files from a source by the following settings:
All files with names that correspond to the regular expression are added to a results list and are transferred by YADE.
This parameter specifies a regular expression, which is used to select files from a Directory .
All files with names that correspond to the regular expression are added to a results list and are transferred by YADE.
This parameter cannot be used with the HTTPFragment or HTTPSFragment in any file transfer operation as the respective protocols would not support to select files based on wildcards.
This parameter is used to specify a directory on one of the involved servers. Directories can be adressed using a folder structure, e.g. /folder1/folder2.
For Windows systems when used with the LocalSource parameter then a Windows path including a drive letter can be specified. For all server systems the standard syntax using can be used which includes forward slashes to separate folder names.
The directory can be set absolute or relative to the working directory, where the working directory depends on the server configuration. If for example a user "test" connects to an SFTP server then this user might have /home/test/ as a working directory. This applies to servers that use the home directory as the working directory. In this case you can use relative and absolute adressing as in
This parameter specifies a regular expression that matches the path to directories to be excluded when recursively traversing a Source directory.
This parameter specifies wether files from subdirectories should be included recursively. Recursion is effective exclusively if files are found in a directory.
This parameter specifies the Selection of files from a source by the following settings:
The specified files are expected to be available from the source for transfer by YADE.
This parameter is used to specify a number of files for processing. It is an alternative to the FileSpecSelection and FilePathSelection.
This parameter specifies a file that contains records with file names in each line. The files can be specified with absolute or relative paths. Each record in this file contains the name of a file which is to be processed. All files in the file list will be processed in the sequence they appear in the file. If a file in the list does not exist then the processing will be aborted.
Future behaviour:
In case of a relative path...
Polling is the YADE capability to wait for incoming files.
This parameter specifies the time in seconds that will be waited before a new poll for files is carried out.
This parameter specifies the total length of time in minutes that a file will be polled for.
This parameter specifies the minimum number of files that have to be found during the polling period in order to cause the transfer to start. This parameter becomes effective with the PollTimeout parameter.
This parameter determines the behavior of the polling when the source folder doesn't exist. If the value is "true" and the source folder doesn't exist then the polling continues otherwise it raise an error.
When YADE runs as a job with JobScheduler and is used for polling then this parameter specifies the node of the current job chain that should be executed after the current YADE job if the timeout as specified with the PollTimeout parameter is exceeded without finding a file.
If no state is specified then an error will be raised if no files are found after timeout.
It is required that the parameter PollingServer is "false".
This parameter specifies YADE to act as a polling server.
This setting should be "false" if YADE is executed by JobScheduler as JobScheduler itself has mechanisms to start jobs repeatedly.
This parameter specifies how long the YADE polling server should run.
It is required that the parameter PollingServer is "true".
Supported formats:
This parameter specifies an unbounded duration for Polling and would cause YADE not to terminate after processing of incoming files.
This parameter is an alternative to the PollingServerDuration that specifies a limited duration.
It is required that the parameter PollingServer is "true".
This setting should not be used if YADE is executed by JobScheduler as JobScheduler itself has mechanisms to start jobs repeatedly.
Directives are options that specify the behavior with specific file characteristics:
This parameter specifies whether an error should be raised if no files can be found for transfer.
The number of files to transfer is determined by the FileSpec, FilePath or FileList parameters.
This parameter does not affect the DisableOverwriteFiles parameter if no files are found that can be overwritten.
This parameter specifies whether zero byte files should be transferred and processed by subsequent commands.
The following settings are available:
Notes
This element references a writeable fragment that is used to copy files in a Copy operation.
Any number of reusable Fragments elements can be configured and are distinguished by YADE based on their name attribute.
A subsequent fragment reference element points to the fragment that is used as target in the Copy operation.
This element references a writeable fragment that is used to move files in a Move operation.
Any number of reusable Fragments elements can be configured and are distinguished by YADE based on their name attribute.
A subsequent fragment reference element points to the fragment that is used as target in the Move operation.
This parameter specifies the target of a Copy operation.
The following frequently used settings groups are available:
This parameter specifies the target of a Move operation.
The following frequently used settings groups are available:
This parameter specifies that after transfer of each file to a target server YADE will try to read the file from the target server and compare the number of bytes of the original file on the source server and the file on the target server.
These options apply to files on a target server. They specify e.g. the Atomicity of a file transfer for Copy and Move operations.
This parameter specifies whether existing target files can be overwritten in a file transfer.
This parameter specifies wether the content of a source file should be appended to the target file, should the target file exist.
If AppendFiles is specified with the value "true" then the parameter DisableOverwriteFiles will be ignored.
This parameter causes a checksum file to be created on the target server that contains the integrity hash value. The file name is created by using the target file name and adding the hash algorithm as an extension, e.g. a target file "myFiles.tar.gz" will cause the integrity hash file "myFiles.tar.gz.md5" to be created when using MD5 with the HashAlgorithm parameter.
Transfer options specify the optional behavior of file transfer, e.g. the transactional behavior.
This parameter specifies whether target files should be created with a prefix such as "~" and have to be renamed to the target file name after the file transfer has been completed without errors.
This mechanism is useful if the target directory is monitored for incoming files by an application such as JobScheduler and if files are required to appear atomically - i.e during transfer - instead of being shown after transfer has been completed.
The value of this parameter is the temporary prefix.
This parameter specifies whether target files should be created with a suffix such as "~" and have to be renamed to the target file name after the file transfer has been completed without errors.
This mechanism is useful if the target directory is monitored for incoming files by some application and if files are required to appear atomically instead of being subsequently written to. This setting is recommended should target directories be monitored by an application or the JobScheduler.
The value of this parameter is the temporary suffix.
Process transfers simultaneously
May be used together with the MaxConcurrentTransfers parameter
Specifies the maximum number of parallel transfers.
This parameter specifies whether a transfer should be processed within a single transaction, i.e. either all objects are successfully transferred or none.
Should an error occur during a transfer operation then all transfers will be rolled back.
When specifying the value true then the following applies:
This parameter makes use of a checksum file that is available with the source of a file transfer. The name of the checksum file is assumed to be the same as the source file with an additional extension .md5. During file transfer the contents of the checksum file is compared with the checksum that is calculated from the transfer of the file.
When used with a jump host then integrity checking applies to source and jump host, not to the target of the transfer.
With this element being used a checksum file is expected on the source system and the integrity hash for the target file is calculated and compared with the respective integrity hash of the source file. If the hashes are not equal then the file transfer will be rolled back. If the checksum file on the source system is missing then this will be reported as an information but will not affect the transfer of files.
This parameter makes use of a checksum file that is available with the source of a file transfer. The name of the checksum file is assumed to be the same as the source file with an additional extension .md5. During file transfer the contents of the checksum file is compared with the checksum that is calculated from the transfer of the file.
When used with a jump host then integrity checking applies to source and jump host, not to the target of the transfer.
With this element being used a checksum file is expected on the source system and the integrity hash for the target file is calculated and compared with the respective integrity hash of the source file. If the hashes are not equal then the file transfer will be rolled back. If the checksum file on the source system is missing then this will be reported as an information but will not affect the transfer of files.
This parameter specifies the algorithm that is used to create the checksum. At the time of writing md5 only is supported. If the parameters CheckIntegrityHash or CreateIntegrityHashFile are used then the integrity hash is calculated and compared according to this parameter.
In some file transfer scenarios the recipient of a file does not know when the sender creates the file. In case of (very) large files the recipient may try to read the file before the sender has finished writing it. This can result in the recipient retrieving an incomplete file.
Including the CheckSteadyState parameter ensures that the file is checked at its reception point for completeness before starting the transfer.
Including the CheckSteadyState parameter ensures that the file is checked at its reception point for completeness before starting the transfer.
If the check of the steady state of one incoming file did not succeed successfully then no file will be transferred.
Note that this is not a very reliable approach as the recipient checks the date of last modification and the size of the file. If neither changes during a specified a period of time then the file is assumed to be complete. However, if transmission is terminated without the file being completely written, the network goes down or the processing speed of the file is slow, then the receiver will get a corrupted file.
The shorter the check interval is chosen the more likely it may happen that a file is detected as steady, although this was not yet finished writing.
A Remove operation removes files from a source host.
The following frequently used settings groups are available:
Support for a jump host.
Note: Availability starting with YADE-356.
YADE makes use of the following log4j loggers:
The debug level specifies the verbosity of the rootLogger log entries. A value between 1 and 9 can be specified. Higher values cause more detailed information to be logged.
An application configuration file contains settings that are specific to an app. YADE allows to specify configuration files individually per transfer fragment.
YADE allows to specify Java property files at a global level and individually per transfer fragment.
YADE uses the log4j framework for logging and makes use of the following loggers:
These loggers are configured by a log4j configuration file which determines that the
If the log4j configuration file is not found then a log4j base configuration is used which logs to stdout.
YADE makes use of the following log4j loggers:
This parameter specifies the location of the file to which the log output of the reportLogger will be written. Should the file not exist then it will be created. If the file already exists then all log output will be appended.
Without specifying this parameter all log output of the reportLogger will be written to
This parameter specifies the location of a log4j configuration file. Without this parameter the following configuration is used:
The interval in seconds for checking the steady state of incoming files.
The number of tries to check the steady state of incoming files.
Header of an e-mail notification.
The next state in a job chain that is used if the check of the steady state of incoming files did not succeed successfully.
This parameter is only available if YADE runs as a JobScheduler job.
Any number of reusable JumpFragment elements can be configured and are distinguished by YADE based on their name attribute.
This element identifies the JumpFragment that is applied for use of a jump host in a file transfer.
The reference to a fragment is based on the value of the ref attribute of this element to the corresponding name attribute of the JumpFragment element.
This attribute identifies the fragment that is used for a jump host.
Domain for authentication at one of the systems involved in file transfer, e.g. an SMB server.
Support for KeePass versions 1 and 2.
Note: Availability starting with YADE-464.
Referencing CredentialStore fields.
Note: Availability starting with YADE-481.
Referencing a field in the CredentialStore is effected by the following syntax:
cs://[entry_path]@entry_field
where
<Hostname> element:
Examples
If the value of this parameter specifies a valid filename then the result set of a Selection of files from a source host will be written to this file.
The result set file is located on the host that runs the YADE Client.
Number of expected hits in result-list.
This parameter has only an effect if the YADE Client is operated with JobScheduler.
This parameter specified a comparison operator. With this parameter YADE can be caused to raise an error if the number of files that are subject to a file transfer operation does not correspond to the expected number of files.
Use of this parameter requires
At run-time the specified value is compared to the effective number of files. In case of a missing match an error is raised.
Possible values of this parameter are:
Example:
This parameter is used to specify the processing if
The respective job chain node is expected to handle the situation of missing files, e.g. to repeat processing.
The respective job chain node is only reached if no errors occurred. In particular, please note that the parameter DisableErrorOnNoFilesFound has to be "true".
Check number of expected hits in result-list.
This parameter specifies the behavior for hosts that have not previously been involved in a file transfer.
This parameter specifies that YADE will try to assign the modification date of the source file to the target file.
This functionality makes use of commands that are specific for the file transfer protocol, e.g. FTP, SFTP, and is therefore not available for all protocols. In addition server side settings might restrict the use of such commands.
YADE makes use of the following log4j loggers:
This parameter activates the logging of the Apache Commons FTP Client. All output will be written by the rootLogger to stdout.
This directory on the jump host is used to store files intermediately during file transfer between source and target host.
The files in this directory are removed at the end of the transfer operation.
A jump host is an intermediate server that is used for file transfers when the YADE Client cannot connect directly to a target server, e.g. if such connections are exclusively available in a DMZ. In such cases the YADE Client can be installed on a server in the DMZ that is used as a jump host. The YADE Client would in a first step transfer files between the source server and the jump host and in a second step execute the YADE Client on the jump host to perform the transfer to the target host.
This element includes the parameters for an jump host file transfer connection including SSHAuthentication and BasicConnection configuration items.
Any number of fragments can be configured and are distinguished by YADE according to their name attribute.
A fragment is referenced by a transfer profile using the value of the JumpFragmentRef ref attribute.
A jump host is an intermediate server that is used for file transfers when the YADE Client cannot connect directly to a target server, e.g. if such connections are exclusively available in a DMZ. In such cases the YADE Client can be installed on a server in the DMZ that is used as a jump host. The YADE Client would in a first step transfer files between the source server and the jump host and in a second step execute the YADE Client on the jump host to perform the transfer to the target host.
This element includes the parameters for an jump host file transfer connection including SSHAuthentication and BasicConnection configuration items.
Any number of fragments can be configured and are distinguished by YADE according to their name attribute.
A fragment is referenced by a transfer profile using the value of the JumpFragmentRef ref attribute.
A jump host is an intermediate server that is used for file transfers when the YADE Client cannot connect directly to a target server, e.g. if such connections are exclusively available in a DMZ. In such cases the YADE Client can be installed on a server in the DMZ that is used as a jump host. The YADE Client would in a first step transfer files between the source server and the jump host and in a second step execute the YADE Client on the jump host to perform the transfer to the target host.
This element includes the parameters for an jump host file transfer connection including SSHAuthentication and BasicConnection configuration items.
Any number of fragments can be configured and are distinguished by YADE according to their name attribute.
A fragment is referenced by a transfer profile using the value of the JumpFragmentRef ref attribute.
A jump host is an intermediate server that is used for file transfers when the YADE Client cannot connect directly to a target server, e.g. if such connections are exclusively available in a DMZ. In such cases the YADE Client can be installed on a server in the DMZ that is used as a jump host. The YADE Client would in a first step transfer files between the source server and the jump host and in a second step execute the YADE Client on the jump host to perform the transfer to the target host.
This element includes the parameters for an jump host file transfer connection including SSHAuthentication and BasicConnection configuration items.
Any number of fragments can be configured and are distinguished by YADE according to their name attribute.
A fragment is referenced by a transfer profile using the value of the JumpFragmentRef ref attribute.
This attribute identifies the current fragment. Any number of fragments can be used and are distinguished by YADE based on their name.
A fragment is referenced via this attribute value by a transfer profile.
This parameter specifies the source of a Remove operation.
The following frequently used settings groups are available:
This parameter specifies the source of a GetList operation.
The following frequently used settings groups are available:
A file transfer includes roles that can be specified by use of the Client elements. The Client information is not required to perform the file transfer but is used for the transfer history.
For SFTP file transfers the built-in compression feature of the transfer protocol can be specified to use a higher level of compression.
Compression levels are based on the following constants of the zlib library:
Use of this feature depends on the cpabilities of the SFTP servers that files are transferred to or from. For details see zlib manual http://www.zlib.net/manual.html.
For SFTP file transfers the built-in compression feature of the transfer protocol is used.
Use of this feature depends on the cpabilities of the SFTP servers that files are transferred to or from. For details see zlib manual http://www.zlib.net/manual.html.
This parameter specifies the keystore type. Possible values: JKS,JCEKS,PKCS12,PKCS11,DKS
This parameter specifies the keystore file.
This parameter specifies the password of a keystore.
The type of FTPS client security (explicit, implicit).
This parameter is used to specify a string that is placed in the target file as a separator between the content of the individual files in order to be able to parse file boundaries later on.
The following special variables are available:
The CumulativeFilename parameter specifies the name of the cumulative file.
The CumulativeFileDelete parameter causes the target file into which the content of the source files is concatenated to be deleted before transfer. Otherwise the content of the source files will be appended to the target file.
This parameter specifies the e-mail queue directory. E-mails which cannot be transferred due to temporary connection errors will be stored in this directory. The JobScheduler will later retry to send these e-mails.
This parameter specifies the maximum number of files that are considered for transfer. If additional files were available then they will be ignored.
Note that it is not possible to specify which files will be ignored should the value set for this parameter be exceeded. It therefore only makes sense to use this parameter in particular situations such as when the contents of a directory will be repeatedly polled.
The maximum size of a data block is defined with this option.
Specifies the commands to be executed for each file on the server after the transfer but before a Rename.
The following special variables are available:
A URL connection specifies parameters such as a Hostname, Port, Account by which a connection is established within the URL.
The use of URLs depends on the server system, e.g. the following protocols and configuration items could be used:
http|https :// [ account [ : password ] @ ] hostname [ : port ]
If the port is empty, the default ports will be used:
This parameter specifies that the order that is to be created will inherit the parameters of the current order.
This parameter specifies the name of the node in the job chain that the job chain should be started with.
With this parameter it is possible to specify that a file order for all files in the result list of a Selection will be created and launched for execution by JobScheduler.
To create file orders only for each new file found see CreateOrderForNewFiles parameter.
With this parameter it is possible to specify that a file order for all new files in the result list of a Selection will be created and launched for execution by JobScheduler.
To create file orders for each file found see CreateOrderForAllFiles parameter.
When YADE creates an order the parameter scheduler_file_path contains the name of the transfered file.
Therefore the order is handled as a file order and will be moved to the blacklist if the file will not be removed after the new order reaches the end of the job chain.
If the order should not be handled as an file order the name of the parameter that contains the file name can be change to another value e.g. "filename".
If so the new order will not be moved to the blacklist if the file still exists after the execution of the order.
The value of this parameter is the name of the job chain that is to be launched by the order. Should the job chain not be located in the "live" folder but in some sub-folder then the folder names have to be added to the parameter value.
An example: a job chain "Test" is located in the "live/sample/FileOperations" folder. The value to be specfied is then "/sample/FileOperations/Test".
With this parameter it is possible to specify a file order to be created and launched for execution by JobScheduler. A file order is created for the first filename in the result list of a Selection.
To create a file order for all files see CreateOrderForAllFiles parameter.
To create a file order for new files see CreateOrderForNewFiles parameter.
YADE can be configured for use with JobScheduler. In this case the YADE JITL Job is used. Parameters can be specified to control the workflow of files processed by YADE.
This parameter defines the file extension to be used for the compressed file.
This parameter specifies that YADE will try to create the directory on the target server that is specified with the Directory parameter should this directory not exist. This parameter has no effect on the subfolders during a recursive transfer. Subfolders of recursive transfer are always created if they do not exist.
This functionality makes use of commands that are specific for the file transfer protocol, e.g. FTP, SFTP, and is therefore not available for all protocols. Please consider that server side settings might restrict the use of such commands.
This parameter is used to rename files. It requires use of the parameter ReplaceWith. The rename operation is performed by specifying
This parameter expects a regular expression for a filename pattern. If the expression matches the filename then the regular expression groups in the match are replaced.
Use with capturing groups
Example:
Use without capturing groups
If no "capturing groups" are specified then the entire match is replaced.
Example:
For further information see java.util.regex.Pattern
This parameter is used to rename files. It requires use of the parameter ReplaceWhat. The rename operation is performed by specifying
If a match has been found as specified by the ReplaceWhat parameter then the following replacements can be applied:
Use with capturing groups
Example:
Use without capturing groups
If no "capturing groups" are specified then the entire match is replaced.
Example:
For further information see java.util.regex.Pattern
Notification fragments are used to specifiy how notifications will be sent to recipients in case of file transfers.
A Credential Store can be implemented by different products.
At the time of writing only "KeePassX" as a Credential Store database is supported and only KeePassX as valid parameter value is permitted.
Development status: not implemented
At run-time YADE can export the file included with the attachment field of the Credential Store database to the local file system.
For example if the attached file is an SSH key and YADE wants to use the key file for file transfer operations then YADE will export the attached file to a predefined directory, i.e. $HOME/.ssh, and the key file will require specific permissions.
Development status: not implemented
At run-time YADE can export the file included with the attachment field of the Credential Store database to the local file system.
For example if the attached file is an SSH key and YADE wants to use the key file for file transfer operations then YADE will export the attached file to a predefined directory, i.e. $HOME/.ssh. To avoid any unwanted overwriting of existing files in the $HOME/.ssh folder use this parameter.
Development status: not implemented
At run-time YADE can export the file included with the attachment field of a Credential Store database to the local file system.
With the file transfer operation being completed and irrespective of the operation's status YADE will by default delete this file. In special cases, e.g. for debuging, if you want YADE not to delete the file then use this parameter.
Development status: not implemented
At run-time YADE can export the file included with the attachment field of a Credential Store database to the local file system.
To use the exported file during a file transfer operation YADE has to know the location of the attached file in the local file system. Use this parameter to specify the path of exported file in the local file system.
This parameter specifies the path to an entry inside the Credential Store database.
This parameter specifies the password that is used to authenticate with the Credential Store.
This parameter specifies the path of the key file that is used to authenticate with the CredentialStore.
Key file format:
This parameter specifies the path of the KeePass database file with the file extension .kdb or .kdbx that is used as a Credential Store database.
Notifications can be sent by e-mail or by use of the YADE Background Service. The Notifications element allows included elements to reference NotificationFragments such as a MailServerFragment or a BackgroundServiceFragment.
Notifications can be added either at Profile level or at General level:
This parameter specifies a mail server setiings.
Fragments for credential stores are a means to organize secure settings in a reusable way:
Any number of fragments can be configured. They are distinguished by their name attribute. References to a fragment use the value of the name attribute to identify the respective fragment.
Mail server fragments are used to specifiy by what mail server notifications will be sent to recipients in case of file transfers.
This element includes the parameters for access to a secure store with credentials for file transfer operations.
If you want to store secure access data, i.e. account, password, SSH key, database connection strings, in an encrypted database, then configure access to a CredentialStore accordingly.
A CredentialStore is a KeePass database that stores secure access data. Such credentials can be referenced from YADE configuration items instead of being exposed in plain text.
Any number of CredentialStores can be configured. They are distinguished by their name attribute. References to a CredentialStore use the value of the name attribute to identify the respective CredentialStore.
Use by configuration items
Credentials can be referenced from a CredentialStore by the following configuration items:
Any number of credential store fragments can be configured and are distinguished by YADE according to their name attribute.
A fragment is referenced by a transfer Profile using the value of the CredentialStoreFragmentRef ref attribute.
Support for KeePass versions 1 and 2.
Note: Availability starting with YADE-464.
Referencing CredentialStore fields.
Note: Availability starting with YADE-481.
Referencing a field in the CredentialStore is effected by the following syntax:
cs://[entry_path]@entry_field
where
<Hostname> element:
Examples
This attribute identifies the current fragment. Any number of fragments can be used and are distinguished by YADE based on their name.
A fragment is referenced via this attribute value by a transfer profile.
This element includes the parameters for e-mail servers that are used to send notifications.
Any number of fragments can be configured and are distinguished by YADE according to their name attribute.
A fragment is referenced by a transfer profile using the value of the MailServerFragmentRef ref attribute.
This attribute identifies the current fragment. Any number of fragments can be used and are distinguished by YADE according their name.
A fragment is referenced by a transfer profile using the value of this attribute.
This element references the fragment that is used to insert configuration items from a secure store into a file transfer configuration.
Any number of reusable CredentialStoreFragment elements can be configured and are distinguished by YADE based on their name attribute.
The reference to a fragment is based on the value of the ref attribute of this element to the corresponding name attribute of the CredentialStoreFragment element.
Support for a jump host.
Note: Availability starting with YADE-488.
This attribute identifies the fragment that is used for inserting secure configuration items.
A file transfer includes the role of the supplying client that is responsible for providing the source files.
The SupplyingClient information is not required to perform the file transfer but is used for the transfer history.
A file transfer includes the role of the receiving client that is responsible for accepting the source files.
The ReceivingClient information is not required to perform the file transfer but is used for the transfer history.
A Credential Store can be accessed using a key file or using a password or a combination of both.
Child elements include:
Composite keys support:
A Credential Store can be accessed using a private key or using a password or a combination of both.
A Credential Store can be accessed using a private key or using a password or a combination of both.
At run-time YADE can export the file included with the attachment field of a Credential Store database to the local file system.
This command is used to invoke the YADE Client on a jump host that is used to store files intermediately during file transfer between source and target host.
The command includes the full path to your YADE Client installation on the jump host and ends with the file name YADE.sh (Unix) or YADE.cmd (Windows).
Specifies the commands to be executed for each file on the jump server before transfer of files.
Specifies the commands to be executed for each file on the jump server after the transfer.
Commands to be executed before the transfer of files starts on the jump server.
Post transfer commands execution in case of the successful transfers.
Post transfer commands execution on transfer error.
Post transfer commands execution at transfer end independent of the transfer status.
A command delimiter option for executing the jump pre- and postcommands:
This parameter specifies a mail host setiings.
Alternative fragements can be specified as a replacement for existing file transfer fragments in case that a connection cannot be established with those fragments.
Alternative fragments can be specified from a range of protocol fragments for use with the source of a file transfer operation without modifying files.
For each fragment type depending on the file transfer protocol in use an alternative fragment type is available.
Alternative fragements can be specified as a replacement for existing file transfer fragments in case that a connection cannot be established with those fragments.
Alternative fragments can be specified from a range of protocol fragments for use with the source or target of a file transfer operation that required modifying files.
For each fragment type depending on the file transfer protocol in use an alternative fragment type is available.
Alternative fragements can be specified as a replacement for existing file transfer fragments in case that a connection cannot be established with those fragments.
Alternative fragments can be specified from a range of protocol fragments for use with the source of a file transfer operation without modifying files.
For each fragment type depending on the file transfer protocol in use an alternative fragment type is available.
Alternative fragements can be specified as a replacement for existing file transfer fragments in case that a connection cannot be established with those fragments.
Alternative fragments can be specified from a range of protocol fragments for use with the source of a file transfer operation without modifying files.
For each fragment type depending on the file transfer protocol in use an alternative fragment type is available.
Alternative fragements can be specified as a replacement for existing file transfer fragments in case that a connection cannot be established with those fragments.
Alternative fragments can be specified from a range of protocol fragments for use with the source or target of a file transfer operation that required modifying files.
For each fragment type depending on the file transfer protocol in use an alternative fragment type is available.
Alternative fragements can be specified as a replacement for existing file transfer fragments in case that a connection cannot be established with those fragments.
Alternative fragments can be specified from a range of protocol fragments for use with the source of a file transfer operation without modifying files.
For each fragment type depending on the file transfer protocol in use an alternative fragment type is available.
This element does not specify a parameter for file transfer operations but an Assertion regarding the expected performance of file transfers. Assertions specify performance indicators that can be checked by a test suite.
The ReturnCode assertion specifies the exit code that is expected as the result of a file transfer. The ReturnCode is a numeric value between 0 and 255.
For YADE a ReturnCode value 0 signals a successful transfer, other values signal a failed transfer.
This element does not specify a parameter for file transfer operation but an Assertion regarding the expected performance of file transfers. Assertions specify performance indicators that can be checked by a test suite.
The Timeout assertion specifies the max. duration of a file transfer that is expected. The Timeout is specified in seconds.
This element does not specify parameters for file transfers but assertions regarding the performance of file transfers. Assertions specify performance indicators that can be checked by a test suite.
A number of Links can be added to a Documentation that explains the use of the file transfer configuration.
A Link is specified by its text and href attribute that points to the URI that is referenced by the Link.
A number of Links can be added to a Documentation that explains the use of the file transfer configuration.
A Link is specified by its text and href attribute that points to the URI that is referenced by the Link.
A number of Links can be added to a Documentation that explains the use of the file transfer configuration.
A Description can be added to a Documentation that explains the use of the file transfer configuration.
Descriptions include plain text, i.e. no formatting is applied.
A Documentation can be added to a file transfer configuration that includes a description and a number of links.
The Documentation can be added either at Profile level or at General level:
Specifies the order in which the client should try protocol 2 authentication methods.
This allows a client to prefer one method (e.g. publickey) over another method (e.g. password).
Specifies the authentication methods that must be successfully completed for a user to be granted access.
See SSH configuration:
Sets the interval in milliseconds to send a keep-alive message.
If zero is specified, any keep-alive message must not be sent.
Use ServerAliveCountMax parameter to specify the number of keep-alive messages which may be sent without receiving any messages back from the server.
Possible values:
Combined values (with blank as separator):
Sets the number of keep-alive messages which may be sent without receiving any messages back from the server.
If this threshold is reached while keep-alive messages are being sent, the connection will be disconnected.
This value is used as the socket timeout parameter, and also as the default connection timeout.
A value of 0 (the default value) indicates no timeout.
Possible values:
Combined values (with blank as separator):
The maximum time to wait for the channel to be established.
If 0, we wait as long as needed (but at most 2000 times 10 milliseconds each).
Possible values:
Combined values (with blank as separator):
To control reconnect behavior, the YADE has two options:
Sets for each single file transfer the number of reconnection attempts if there is a connection failure.
This option specifies the wait interval between attempts.
Possible values:
Combined values (with blank as separator):
HTTP headers let the client and the server pass additional information with an HTTP request or response.
YADE allows to specify HTTP headers individually per HTTP(S) transfer fragment or HTTP(S) transfer fragment reference.
This parameter specifies a HTTP header, which is used for a HTTP(S) file transfer.
Name and value are separated by the first space, e.g.: