Product Knowledge Base

Space shortcuts

Page tree

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 47 Next »

Introduction

  • The JobScheduler creates a number of different logs, to provide specific information about jobs, job chains, orders, tasks and JobScheduler internal operational itself.
  • The location of the the log files can vary per installation. By default you will find the log files in the ${SCHEDULER_DATA}/logs folder where ${SCHEDULER_DATA} is the JobScheduler data directory that has been specified during installation. For Windows systems use %SCHEDULER_DATA% instead of ${SCHEDULER_DATA}.

JobScheduler log files

  1. Debug Log: scheduler.log
  2. Main Log: scheduler-<yyyy-mm-dd-hhmmss>.<scheduler-id>.log
  3. Job Log: job.<job_chain_name>,<job_name>.log
  4. Task Log: task.<job_chain_location>,<job_chain_name>,<job_name>.log
  5. Order Log: order.<job_chain_location>,<job_chain_name>,<order_id>.log
  6. Installation Log: Install_V<release_number>_<yyyymmddhhmmss>_<timestamp>.log
  7. Web Server Log: http.log
  8. Web Service Log: web_service,<web_service_name>.log

1. Debug Log: scheduler.log

The Debug Log is created

  • during JobScheduler startup. Just before startup the JobScheduler moves an existing log file scheduler.log to scheduler-old.log, and creates an empty scheduler.log file. At any time there will be only two Debug Log files, there is no archiving for Debug Log files.
  • when log rotation is performed.
  • by all types of JobScheduler instances, i.e. Standalone, Active Cluster, Passive Cluster, Agent, not applicable for new JobScheduler Universal Agent .
  • with the constant file name scheduler.log

The Debug Log 

  • contains most of the information concerning internal working of the JobScheduler. 
  • is subject to log rotation which archive existing file with suffix old e.g. scheduler-old.log and creates a new log file by the house keeping job sos/housekeeping/scheduler_rotate_log 
  • is stored in the file system.

Each line of the Debug Log states columns with the following information:

  • Column 1: Starts with "." if demo version, Day of Month
  • Column 2: Time (with ms)
  • Column 3: application name (Windows) - empty otherwise
  • Column 4: elapsed time in ms
  • Column 5: Process ID, ".", Thread ID (hex)
  • Column 6: Used memory in MB (Windows) - empty otherwise
  • Column 7: Log text

2. Main Log: scheduler-<yyyy-mm-dd-hhmiss>.<scheduler-id>.log

The Main Log is created

  • during JobScheduler startup.
  • when log rotation is performed.
  • by all types of JobScheduler instances, i.e. Standalone, Active Cluster, Passive Cluster, Agent, not applicable for new JobScheduler Universal Agent .
  • with the file name scheduler-<yyyy-mm-dd-hhmmss>.<scheduler-id>.log where
    • <yyyy-mm-dd-hhmmss> states the timestamp when the log was started.
    • <scheduler-id> states the JobScheduler instance identification that has been specified during installation. If multiple JobScheduler instances store log files to a common log folder then this is helpful to identify the log file of a specific JobScheduler instance.

The Main Log 

  • has information about processes being executed by JobScheduler and other high level information of JobScheduler operation.
  • is subject to log rotation which creates a new log file with a new timestamp by the house keeping job sos/housekeeping/scheduler_rotate_log  
  • is stored in the file system.

3. Job Log: job.<job_chain_name>,<job_name>.log

The Job Log is created

  • during execution of individual jobs. Any job start will overwrite an existing Job Log.
  • with the file name job.<job_chain_name>,<job_name>.log where
    • <job_chain_name> is the name of the job chain that the job is executed for, i.e. the <job_chain_name> part of a <job_chain_name>.job_chain.xml job chain file.
    • <job_name> is the name of the job, i.e. the <job_name> part in a <job_name>.job.xml job file.

The Job Log

  • is stored in the file system.

4. Task Log: task.<job_chain_location>,<job_chain_name>,<job_name>.log

The Task Log is created

  • during execution of individual tasks (processes) for jobs. Any job start with a subsequent task will overwrite an existing Task Log.
  • with the file name task.<job_chain_location>,<job_chain_name>,<job_name>.log where
    • <job_chain_location> is the folder name where the job chain is located.
    • <job_chain_name> is the name of the job chain that the job is executed for, i.e. the <job_chain_name> part of a <job_chain_name>.job_chain.xml job chain file.
    • <job_name> is the name of the job, i.e. the <job_name> part in a <job_name>.job.xml job file.

The Task Log

  • contains all information about the execution of job instances, e.g. parameters, output to stdout and stderr of the script or application that is executed..
  • is stored in the file system and - after execution - is saved to the database.

5. Order Log: order.<job_chain_location>,<job_chain_name>,<order_id>.log

The Order Log is created

  • during execution of an order for jobs in a job chain, i.e. when an order traverses from one node to another in a job chain.
  • with the file name order.<job_chain_location>,<job_chain_name>,<order_id>.log where
    • <job_chain_location> is the folder name where the job chain is located.
    • <job_chain_name> is the name of the job chain that the job is executed for, i.e. the <job_chain_name> part of a <job_chain_name>.job_chain.xml job chain file.
    • <order_id> is the identification of the order, i.e. the <order_id> part in a <job_chain_name>,<order_id>.order.xml order file.

The Order Log

  • contains all output of the individual Task Logs for jobs that the order passed when being executed in a job chain. This includes repeated execution of job nodes, e.g. to an order setback.
  • is stored in the file system and - after execution - is saved to the database.

6. Installation Log: Install_V<release_number>_<yyyymmddhhmmss>_<timestamp>.log

The Installation Log is created

  • when installing JobScheduler, this includes full installs and updates.
  • with the file name Install_V<release_number>_<yyyymmddhhmmss>_<timestamp>.log where 
    • <release_number> is the JobScheduler release number in the format <major>.<minor>.<bug-fix>
    • <yyyymmddhhmmss> states the timestamp when the installation was started.
    • <timestamp> states a more precise timestamp when the installation was performed.

The Installation Log

  • contains important information about JobScheduler installation. 
  • is of vital importance in case that installation problems occur on a JobScheduler full installation or update.

Log Rotation 

The JobScheduler creates the Main Log and Debug Log on start and restart. The JobScheduler also has a housekeeping job the sos/housekeeping/scheduler_rotate_log. It is considered good practice to configure scheduler_rotate_log job to run once in the day. The scheduler_rotate_log job 

  • creates a new Main Log  and Debug Log. 
  • creates a archive copy of the existing Debug Log file before opening a new Debug Log.
    • saves an existing Debug Log to the file  scheduler-old.log. 
    • This its very important to note that if the scheduler_rotate_log job is running every 24 hours, at any point there is only 48 hours of Debug Log. 

Example:

If the scheduler_rotate_log job is being scheduled for a JobScheduler  with ID "SOS-BERLIN"  to run everyday early morning at 2am, then a new Main Log  will be created on 1st of June with time stamp eg. scheduler-2015-06-01-020000.SOS-BERLIN.log. On next day, 2nd of June, the Main Log file will will be rotated to the file name scheduler-2015-06-01-020000.SOS-BERLIN.log. 

if the JobScheduler is restarted on 3rd of June at 10:25:33 AM then a new Main Log will be created with the file name scheduler-2015-06-03-102533.SOS-BERLIN.log and again in the night a new log will be created by the scheduler_rotate_log job with the file name scheduler-2015-06-03-020000.SOS-BERLIN.log.

How to Find the Latest Log Files

For analysis of problems it is vital to know the exact point in time when an error occurs. You can find the point in time of an error e.g. from the JOC GUI:

  • If the error occurred in a job chain then see the order history tab and open the corresponding Order Log. 
  • If error has occurred in a stand alone job then check the task history tab and open the corresponding Task Log.

Then identify the according Main Log and Debug Log that covers the point in time of the error.

  • The Debug Log uses a constant name scheduler.log.
    • If the housekeeping job scheduler_rotate_log has been run after the error occurred then the Debug Log has been rotated and the previous version is available with the file scheduler-old.log.
  • The Main Log can be identified from the file name with the most recent time stamp before the point in time that the error occurred.

Finding Errors in the Debug Log and Main Log

Whenever there is an error or warning in the JobScheduler then the information will be logged to the Main Log and Debug Log.

All error and warning messages are written with the [ERROR] or [WARN] tags to the log file.

Start with the Main Log, open the file with any text editor and scan it from starting for any tail tail sign of the problem. You can search for the [ERROR] or [WARN] tags in the log. 

Log Level

Central Log Level settings 

The central Log Level setting affects the granularity of the information logged by JobScheduler in Debug Log and Main Log. The settings can be changed in the settings file $SCHEDULER_DATA/config/factory.ini. The default Log level is info. The Log Level can be one of error, warn, info, debug1 to debug9.

After changing the settings in the factory.ini file, the JobScheduler must be restarted to make settings effective.

The central Log Level settings will be applied to all the JobScheduler objects, e.g. jobs, job chains, orders etc.

 

The higher Log Level will lead to large size of Main Log and Debug Log, and also will require more database storage to save JobScheduler Order Logs and Task Logs

log_level = debug9

 

Job Log Level settings

For debugging a specific job, a higher Log Level can be configured for the job.The Log Levels are the same as of the central Log Levels.

The configuration changes in a job do not require the JobScheduler to restart. After debugging the Log Level can be set back to info.

The JobScheduler Object Editor (JOE) provides a GUI to change the Log Level..Click on the job name in left panel of the JOE, switch to options tab in the right panel, change the Log Level settings to the desired Log Level. This would result in a job configuration like this:

<job order="yes"  stop_on_error="no"
     title="Simple execution of a shell script or a shell command with Log Level debug9">
    <settings >        
           <log_level ><![CDATA[debug9]]></log_level>
    </settings>
    <script language="shell">
        <![CDATA[
            echo . Hello World!!
        ]]>
    </script>
    <run_time/>
</job>

JobScheduler Universal Agent Logging

By default you will find the log files for JobScheduler Universal Agent in the ${SCHEDULER_HOME}/logs folder where ${SCHEDULER_HOME} is the root directory of your JobScheduler Universal Agent installation.

The most recent log file is called jobscheduler_agent_<agent_port>.log.

Logfiles are rotated on a daily basis, so older log files are located in the ${SCHEDULER_HOME}/logs folder suffixed with a date.

The logging is configured in ${SCHEDULER_HOME}/lib/log4.xml.

 

See also

 

 

  • No labels