YADE User Manual - Pre- & Post-Processing

Pre- & Post-Processing Options

YADE can carry out both pre- and post-processing at the source and at the target parts of the transfer.

A typical post-processing operation at the source would be to move or rename a file after it has been copied to the target.

Pre- and post-processing can be triggered for:

Transfer operations.
Individual file transfers, so when three files are to be transferred, processing would be carried out three times.

In addition, post-processing can be triggered for:

File Rename operations.

Pre- and post-processing can be carried out using the FTP, SFTP and local file system protocols: they are not possible with HTTP, HTTPS, SMBF or WebDAV.

Pre- and post-processing commands

The commands that can be executed as part of a pre- or post-processing operation are protocol-dependant:

FTP commands have to be used with the FTP protocol and
Shell commands have to be used with the SFTP protocol.

Pre- and post-processing operations

The pre- and post-processing operations available are:

Pre-processing operations:
- CommandBeforeFile:
  - commands are executed before each individual file is transferred
- CommandBeforeOperation
  - commands are executed before the transfer operation is started
Post-processing operations:
- CommandAfterFile:
  - commands are executed after each file transfer has been completed
- CommandAfterOperation
  - commands are executed after the transfer operation has been completed
- CommandBeforeRename
  - commands are executed before each individual file is renamed

Note that special variables such as $target_dir and $date are available for pre- or post-processing commands. These variables are listed under the Parameter Reference article for the relevant protocol *FragmentRef element. See the list of links provided in the Parameter reference for pre- and post-processing operations section on this page for more information.

Specifying pre- and post-processing operations

The commands that can be executed as part of a pre- or post-processing operation are protocol specific - meaning that they are specified as child elements of a protocol fragment element such as SFTPFragmentRef.

The XML hierarchy used to specify the pre- and post-processing operations around a typical Copy operation using SFTP would be:

Profile
- Operation
  - CopySource
    - CopySourceFragmetRef
      - SFTPFragmentRef
        SFTPPreProcessing
        CommandBeforeFile
        CommandBeforeOperation
        SFTPPostProcessing
        CommandBeforeFile
        CommandBeforeOperation
        CommandBeforeRename
        etc. (optional)

Parameter reference for pre- and post-processing operations

Detailed information about pre- and post-processing operations can be found in the relevant parameter reference article:

Variables

Variables can be used in pre- and post-processing commands.

Such variables provide the path and name of files in a transfer.

Refactored Variables

See YADE-448 - Getting issue details... STATUS

File separator (all paths) are provided as follows:
- \ : if the pre- and post-processing commands are executed on a Windows machine.
- / : if the pre- and post-processing commands are not executed on a Windows machine.
All directory paths provide the path without a trailing slash or backslash.

Variables:

Directory variables:

${TargetDirFullName}
${SourceDirFullName}

Target file variables:

${TargetFileFullName}
${TargetFileRelativeName}
${TargetFileBaseName}
${TargetFileParentFullName}
${TargetFileParentBaseName}

Target transfer file variables:

${TargetTransferFileFullName}
${TargetTransferFileRelativeName}
${TargetTransferFileBaseName}
${TargetTransferFileParentFullName}
${TargetTransferFileParentBaseName}

Source file variables:

${SourceFileFullName}
${SourceFileRelativeName}
${SourceFileBaseName}
${SourceFileParentFullName}
${SourceFileParentBaseName}

Source renamed file variables:
- ${SourceFileRenamedFullName}
- ${SourceFileRenamedRelativeName}
- ${SourceFileRenamedBaseName}
- ${SourceFileRenamedParentFullName}
- ${SourceFileRenamedParentBaseName}

Directory		${SourceDirFullName}	${TargetDirFullName}
The directory where files are stored on the source or target systems		`C:\yade`	`\home\test\yade`
Target File Name	Target Settings	${TargetFileFullName}	${TargetFileRelativeName}	${TargetFileBaseName}	${TargetFileParentFullName}	${TargetFileParentBaseName}
The name of a file on the target host.	without replacing	`\home\test\yade\test.txt` `\home\test\yade\sub_1\test_1.txt` `\home\test\yade\sub_2\test_2.txt`	`test.txt` `sub_1\test_1.txt` `sub_2\test_2.txt`	`test.txt` `test_1.txt` `test_2.txt`	`\home\test\yade` `\home\test\yade\sub_1` `\home\test\yade\sub_2`	`yade` `sub_1` `sub_2`
The name of a file on the target host.	`replacing=test` `replacement=x\y`	`\home\test\yade\x\y.txt` `\home\test\yade\sub_1\x\y_1.txt` `\home\test\yade\sub_2\x\y_2.txt`	`x\y.txt` `sub_1\x\y_1.txt` `sub_2\x\y_2.txt`	`y.txt` `y_1.txt` `y_2.txt`	`\home\test\yade\x` `\home\test\yade\sub_1\x` `\home\test\yade\sub_2\x`	`x` `x` `x`
Target Transfer File Name	Target Settings	${TargetTransferFileFullName}	${TargetTransferFileRelativeName}	${TargetTransferFileBaseName}	${TargetTransferFileParentFullName}	${TargetTransferFileParentBaseName}
During transfer of a file its name can temporary differ: files can be prefixed or suffixed to indicate temporary names. `Target Transfer File Name` is not depend of the target `replacing` setting.	not transactional without suffix or prefix	`\home\test\yade\test.txt` `\home\test\yade\sub_1\test_1.txt` `\home\test\yade\sub_2\test_2.txt`	`test.txt` `sub_1\test_1.txt` `sub_2\test_2.txt`	`test.txt` `test_1.txt` `test_2.txt`	`\home\test\yade` `\home\test\yade\sub_1` `\home\test\yade\sub_2`	`yade` `sub_1` `sub_2`
	`transactional=true` `atomic_suffix=~`	`\home\test\yade\test.txt~` `\home\test\yade\sub_1\test_1.txt~` `\home\test\yade\sub_2\test_2.txt~`	`test.txt~` `sub_1\test_1.txt~` `sub_2\test_2.txt~`	`test.txt~` `test_1.txt~` `test_2.txt~`	`\home\test\yade` `\home\test\yade\sub_1` `\home\test\yade\sub_2`	`yade` `sub_1` `sub_2`
Source File Name	Source Settings	${SourceFileFullName}	${SourceFileRelativeName}	${SourceFileBaseName}	${SourceFileParentFullName}	${SourceFileParentBaseName}
The name of a file on the source host. `Source File Name` is not depend of the source `replacing` setting.		`C:\yade\test.txt` `C:\yade\sub_1\test_1.txt` `C:\yade\sub_2\test_2.txt`	`test.txt` `sub_1\test_1.txt` `sub_2\test_2.txt`	`test.txt` `test_1.txt` `test_2.txt`	`C:\yade` `C:\yade\sub_1` `C:\yade\sub_2`	`yade` `sub_1` `sub_2`
Source File Renamed Name	Source Settings	${SourceFileRenamedFullName}	${SourceFileRenamedRelativeName}	${SourceFileRenamedBaseName}	${SourceFileRenamedParentFullName}	${SourceFileRenamedParentBaseName}
The name of a file on the source host after Rename Operation.	without replacing	No values are provided for the variables
	`replacing=test` `replacement=x\y`	`C:\yade\x\y.txt` `C:\yade\sub_1\x\y_1.txt` `C:\yade\sub_2\x\y_2.txt`	`x\y.txt` `sub_1\x\y_1.txt` `sub_2\x\y_2.txt`	`y.txt` `y_1.txt` `y_2.txt`	`C:\yade\x` `C:\yade\sub_1\x` `C:\yade\sub_2\x`	`x` `x` `x`

Deprecated Variables

See YADE-448 - Getting issue details... STATUS .

File separator (all paths) are provided as follows
- \ : if the pre- and post-processing commands are executed on a Windows machine.
- / : if the pre- and post-processing commands are not executed on a Windows machine.
All directory paths provide the path with a trailing slash or backslash.