...

Introduction

Log files include data that are which is considered sensitive by a number of users. This includes:

• Hostnames, IP Addresses, Ports
• Accounts
• Database Names

There are situations when log files should be anonymized before forwarding to a 3rd party, to the SOS Ticketing System or to a public forum to receive community support.

JS7 offers to anonymize logs allows logs to be anonymized by replacing sensitive data with placeholders:

...

Anonymization is available from a Java class and is invoked from the a shell script available with at the following default locations:

...

• Options
  • -l | --log-file=<log-file>: Specifies the location of a log file to be anonymized.
    • This option can be specified repeatedly for a number of files. Wildcards can be specified and directories can be specified should if all included files be anonymized.
    • Plain text log files as available with the .log file extension are considered and compressed log files available with the .gz file extension , are considered - see the JS7 - Log Rotation article.
    • Anonymized log file names are prefixed with the string: anonymized- 
    • By default anonymized log files are stored to in the directory in which original log files are found.
  • -o | --output-dir=<directory>: Optionally specifies the output directory to in which anonymized log files are stored. If this argument is omitted then anonymized log files are stored to in their original directory.
  • -r | --rules-file=<rules-file>: Optionally specifies the location of a file in YAML format that holds the rules to be applied for anonymization.
  • -e | --export-rules=<rules-file>: Optionally specifies the location of a file to which the built-in rules for anonymization are will be exported.

Rules

Rules include to specify specifying regular expressions to search for searching and related placeholders as replacements. The built-in rules cover typical configuration items such as URLs, IP addresses, Host names etc. 

• Users can export the built-in rules to a file to verify available rules and expressions.
• Users can add individual rules to a file that is used when invoking the Log Anonymizer Script. It is recommended to validate individual files to be are validated as being YAML compliant.
  
  

rules:
- item: url-component
  search: ://(.*):(\d{2,5})
  replace:
  - <host>
  - <port>
- item: ip-address
  search: (([01]?\d\d?|2[0-4]\d|25[0-5])\.([01]?\d\d?|2[0-4]\d|25[0-5])\.([01]?\d\d?|2[0-4]\d|25[0-5])\.([01]?\d\d?|2[0-4]\d|25[0-5]))
  replace:
  - <ip-address>
- item: host-install
  search: host[\s]*=[\s]*(.*)
  replace:
  - <host>

...

• Creates anonymized versions of the indicated log files files indicated from the given directory. The wildcard specifies any log files carrying a name that starts with joc to will be used.
• The wildcard includes to consider consideration of plain text log files available from with the .log file extension and it includes to consider also consideration of compressed log files that which are available from with a .gz file extension.
• Anonymized log files are stored in the output directory.

...

• Creates anonymized versions of all log files in the indicated directory and stores them to in the output directory.
• A YAML file with individual rules is applied.

...