Pre- & Post-Processing Options
YADE can carry out both pre- and post-processing at the source and at the target parts of the transfer.
A typical post-processing operation at the source would be to move or rename a file after it has been copied to the target.
Pre- and post-processing can be triggered for:
- Transfer operations.
- Individual file transfers, so when three files are to be transferred, processing would be carried out three times.
In addition, post-processing can be triggered for:
- File Rename operations.
Pre- and post-processing can be carried out using the FTP, SFTP and local file system protocols: they are not possible with HTTP, HTTPS, SMBF or WebDAV.
Pre- and post-processing commands
The commands that can be executed as part of a pre- or post-processing operation are protocol-dependant:
- FTP commands have to be used with the FTP protocol and
- Shell commands have to be used with the SFTP protocol.
Pre- and post-processing operations
The pre- and post-processing operations available are:
- Pre-processing operations:
- CommandBeforeFile:
- commands are executed before each individual file is transferred
- CommandBeforeOperation
- commands are executed before the transfer operation is started
- CommandBeforeFile:
- Post-processing operations:
- CommandAfterFile:
- commands are executed after each file transfer has been completed
- CommandAfterOperation
- commands are executed after the transfer operation has been completed
- CommandBeforeRename
- commands are executed before each individual file is renamed
- CommandAfterFile:
Note that special variables such as $target_dir and $date are available for pre- or post-processing commands. These variables are listed under the Parameter Reference article for the relevant protocol *FragmentRef element. See the list of links provided in the Parameter reference for pre- and post-processing operations section on this page for more information.
Specifying pre- and post-processing operations
The commands that can be executed as part of a pre- or post-processing operation are protocol specific - meaning that they are specified as child elements of a protocol fragment element such as SFTPFragmentRef.
The XML hierarchy used to specify the pre- and post-processing operations around a typical Copy operation using SFTP would be:
- Profile
- Operation
- CopySource
- CopySourceFragmetRef
- SFTPFragmentRef
- SFTPPreProcessing
- CommandBeforeFile
- CommandBeforeOperation
- SFTPPostProcessing
- CommandBeforeFile
- CommandBeforeOperation
- CommandBeforeRename
- etc. (optional)
- SFTPPreProcessing
- SFTPFragmentRef
- CopySourceFragmetRef
- CopySource
- Operation
Parameter reference for pre- and post-processing operations
Detailed information about pre- and post-processing operations can be found in the relevant parameter reference article:
Variables
Variables can be used in pre- and post-processing commands.
Such variables provide the path and name of files in a transfer.
Refactored Variables
See - YADE-448Getting issue details... STATUS
- File separator (all paths) are provided as follows:
\
: if the pre- and post-processing commands are executed on a Windows machine./
: if the pre- and post-processing commands are not executed on a Windows machine.
- All directory paths provide the path without a trailing slash or backslash.
Variables:
- Directory variables:
${TargetDirFullName}
${SourceDirFullName
}- Target file variables:
${TargetFileFullName}
${TargetFileRelativeName}
${TargetFileBaseName}
${TargetFileParentFullName}
${TargetFileParentBaseName}
- Target transfer file variables:
${TargetTransferFileFullName}
${TargetTransferFileRelativeName}
${TargetTransferFileBaseName}
${TargetTransferFileParentFullName}
${TargetTransferFileParentBaseName}
- Source file variables:
${SourceFileFullName}
${SourceFileRelativeName}
${SourceFileBaseName}
${SourceFileParentFullName}
${SourceFileParentBaseName}
- Source renamed file variables:
${SourceFileRenamedFullName}
${SourceFileRenamedRelativeName}
${SourceFileRenamedBaseName}
${SourceFileRenamedParentFullName}
${SourceFileRenamedParentBaseName}
Directory | ${SourceDirFullName} | ${TargetDirFullName} | ||||
---|---|---|---|---|---|---|
The directory where files are stored on the source or target systems |
|
| ||||
Target File Name | Target Settings | ${TargetFileFullName} | ${TargetFileRelativeName} | ${TargetFileBaseName} | ${TargetFileParentFullName} | ${TargetFileParentBaseName} |
The name of a file on the target host. |
|
|
|
|
|
|
|
|
|
|
|
| |
Target Transfer File Name | Target Settings | ${TargetTransferFileFullName} | ${TargetTransferFileRelativeName} | ${TargetTransferFileBaseName} | ${TargetTransferFileParentFullName} | ${TargetTransferFileParentBaseName} |
During transfer of a file its name can temporary differ: files can be prefixed or suffixed to indicate temporary names.
of the target |
|
|
|
|
|
|
|
|
|
|
|
| |
Source File Name | Source Settings | ${SourceFileFullName} | ${SourceFileRelativeName} | ${SourceFileBaseName} | ${SourceFileParentFullName} | ${SourceFileParentBaseName} |
The name of a file on the source host.
|
|
|
|
|
| |
Source File Renamed Name | Source Settings | ${SourceFileRenamedFullName} | ${SourceFileRenamedRelativeName} | ${SourceFileRenamedBaseName} | ${SourceFileRenamedParentFullName} | ${SourceFileRenamedParentBaseName} |
The name of a file on the source host after Rename Operation. |
| No values are provided for the variables | ||||
|
|
|
|
|
|
Deprecated Variables
See - YADE-448Getting issue details... STATUS .
- File separator (all paths) are provided as follows
\
: if the pre- and post-processing commands are executed on a Windows machine./
: if the pre- and post-processing commands are not executed on a Windows machine.
- All directory paths provide the path with a trailing slash or backslash.
Variables:
- Directory variables:
${TargetDirName}
${SourceDirName}
- Target file variables:
$TargetFileName
${TargetFileName}
- Target transfer file variables:
$TargetTransferFileName
${TargetTransferFileName}
- Source file variables:
$SourceFileName
${SourceFileName}
$SourceTransferFileName
${SourceTransferFileName}
- Source renamed file variables:
$RenamedSourceFileName
${RenamedSourceFileName}
Directory | ${SourceDirName} | ${TargetDirName} | ||
---|---|---|---|---|
The directory where files are stored on the source or target systems |
|
| ||
Target File Name | $TargetFileName | ${TargetFileName} | $TargetTransferFileName | ${TargetTransferFileName} |
The name of a file on the target host (TargetFileName) without replacements being applied. During transfer of a file its name can temporarily differ (TargetTransferFileName): files can be prefixed or suffixed to indicate temporary names. Depending on the variables used absolute and relative paths are provided for files. |
|
| During transfer files can be prefixed or suffixed to indicate temporary names.
|
|
The target file name including replacements applied with the following sample settings:
Replacements are not applied to to temporary names of files during transfer. |
|
|
|
|
Source File Name | $SourceFileName | ${SourceFileName} | $SourceTransferFileName | ${SourceTransferFileName} |
The name of a file on the source host without replacements being applied. During transfer of a file the path is tripped from the file name. |
|
|
|
|
No replacements are applied to source and target file name variables. |
|
|
|
|
Source File Name after Rename Operation | $RenamedSourceFileName | ${RenamedSourceFileName} | Note | |
If no source replacement is specified then no values are provided fore the variables. | ||||
The source file name with replacements being applied that include the following sample settings:
|
|
|
|