FEATURE AVAILABILITY STARTING FROM RELEASE 2.4.0
Log files include data which is considered sensitive by a number of users. This includes:
- Hostnames, IP Addresses, Ports
- Database Names
There are situations where log files should be anonymized before forwarding to a 3rd party, to the SOS Ticketing System or to a public forum to receive community support.
JS7 allows logs to be anonymized by replacing sensitive data with placeholders:
Log Anonymizer Script
Anonymization is available from a Java class and is invoked by a shell script available at the following default locations:
-l | --log-file=<log-file>: Specifies the location of the log file(s) to be anonymized.
- This option can be specified repeatedly for a number of files. Wildcards can be specified and directories can be specified if all included files are to be anonymized.
- Plain text log files with the .log file extension and compressed log files with the .gz file extension are considered - see the JS7 - Log Rotation article.
- Anonymized log file names are prefixed with the string:
- By default anonymized log files are stored in the directory in which original log files are found.
-o | --output-dir=<directory>: Optionally specifies the output directory in which anonymized log files are to be stored. If this argument is omitted then anonymized log files are stored in their original directory.
-r | --rules-file=<rules-file>: Optionally specifies the location of a file in YAML format that holds the rules to be applied for anonymization.
-e | --export-rules=<rules-file>: Optionally specifies the location of a file in YAML format to which the built-in rules for anonymization will be exported.
Rules include specifying regular expressions for searching and related placeholders as replacements. The built-in rules cover typical configuration items such as URLs, IP addresses, Host names etc.
- Users can export the built-in rules to a file to verify available rules and expressions.
- Users can add individual rules to a file that is used when invoking the Log Anonymizer Script. It is recommended that individual files are validated as being YAML compliant.
itemspecifies an expression to
searchfor and to
searchexpression makes use of capturing groups specified by
( ... ).
replaceplaceholder specifies a number of strings that replace the content of related capturing groups.
The following examples illustrate typical use cases.
- Creates an anonymized version of the log file in the same directory with the name:
- Creates anonymized versions of the indicated log files in the given output directory:
- Creates anonymized versions of the log files indicated from the given directory. The wildcard specifies any log files carrying a name that starts with
jocwill be used.
- The wildcard includes consideration of plain text log files with the .log file extension and also consideration of compressed log files with a .gz file extension.
- Anonymized log files are stored in the output directory.
- The build-in rules are exported to a file in YAML format.
- This file can be used to adjust rules and can be applied for anonymization later on.
- Creates anonymized versions of all log files in the indicated directory and stores them in the output directory.
- A YAML file with individual rules is applied.
Running Log Anonymizer outside of JS7
Users who wish to run the Log Anonymizer outside of a JS7 installation can:
- copy a number of Java related files as available from the Controller or Agent installation,
- create a shell script to invoke the Log Anonymizer.
Java related Files required by Log Anonymizer
The following files can be found in a Controller or Agent installation for Unix or Windows.
- Note that version numbers of binary files can change over time. The examples below assume Java related files as provided with Release 2.4.0.
- Users can copy the following Java related files from the .
./lib/sos, ./lib/stdoutdirectories to the location where they want to use Log Anonymizer:
Shell Script to run Log Anonymizer
An individual script to invoke Log Anonymizer can look like this:
The directory hierarchy is assumed like this:
anonymize-logs.sh | .cmd