Logging in python, logging levels and logging module components

Logging in Python is a standard approach to recording information about the execution of a program. The logging module is part of the Python Standard Library and provides flexible and customizable logging functionality. Here’s a basic guide on using the logging module:

Basic Logging Example:

import logging

# Configure the logging settings
logging.basicConfig(
    level=logging.INFO,  # Set the logging level (DEBUG, INFO, WARNING, ERROR, CRITICAL)
    format='%(asctime)s - %(levelname)s - %(message)s',  # Define the log message format
    filename='example.log',  # Specify the log file
    filemode='w'  # Set the mode for opening the log file ('w' for write, 'a' for append)
)

# Create a logger object
logger = logging.getLogger(__name__)

def example_function():
    logger.info('This is an informational message.')
    try:
        result = 10 / 0  # Triggering an error for demonstration purposes
    except Exception as e:
        logger.error('An error occurred: %s', e, exc_info=True)  # Log the error with traceback information

# Call the example function
example_function()

This example demonstrates the following:

1. Configuration:

   • The basicConfig method is used to configure logging settings.
   • level: Sets the logging level. Messages with a level lower than this will be ignored.
   • format: Defines the format of the log messages.
   • filename: Specifies the name of the log file.
   • filemode: Specifies the mode for opening the log file (‘w’ for write, ‘a’ for append).

2. Logger Creation:

   • The getLogger method is used to create a logger object. The argument passed to __name__ is typically the name of the current module.

3. Logging Messages:

   • The logger.info method is used to log an informational message.
   • An exception is caught, and the logger.error method is used to log an error message with traceback information (exc_info=True).

4. Log File:

   • The log messages are written to the specified log file (‘example.log’ in this case).

Logging Levels:

The logging levels, in increasing order of severity, are:

• DEBUG
• INFO
• WARNING
• ERROR
• CRITICAL

You can set the logging level to control which messages are captured.

Logging to Console:

To log messages to the console, you can omit the filename parameter in basicConfig. The messages will then be printed to the console instead of being written to a file.

logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s')

Advanced Configurations:

For more advanced configurations, you can use handlers, formatters, and filters. The logging module provides a powerful framework for customizing logging behavior.

Note:

• Always be mindful of log levels and choose an appropriate level for each log message.
• Avoid logging sensitive information.
• Consider using different loggers for different components of your application.

Customize the logging configuration based on your specific needs and the complexity of your application.

The logging module in Python provides several components that allow you to configure and use a flexible and extensible logging framework. Here are the key components of the logging module:

1. Logger:

   • A Logger is the central object in the logging module that application code directly interacts with. It is used to emit log messages.
   • Logger names form a hierarchy based on the dot-separated namespace, similar to Python package names.

   import logging
   
   logger = logging.getLogger(__name__)
   

2. Handler:

   • A Handler is responsible for sending the log records (messages) to the appropriate output. It could be a console, a file, or any other custom destination.
   • Multiple handlers can be attached to a single logger, allowing log records to be processed in different ways.

   file_handler = logging.FileHandler('example.log')
   console_handler = logging.StreamHandler()
   
   logger.addHandler(file_handler)
   logger.addHandler(console_handler)
   

3. Formatter:

   • A Formatter specifies the layout of log messages. It defines how the information in a log record is formatted when it is emitted.
   • Each handler can have its own formatter.

   formatter = logging.Formatter('%(asctime)s - %(levelname)s - %(message)s')
   file_handler.setFormatter(formatter)
   console_handler.setFormatter(formatter)
   

4. Filter:

   • A Filter allows for more fine-grained control over which log records are processed by a handler. It can be used to selectively suppress or allow log messages based on certain conditions.

   class MyFilter(logging.Filter):
       def filter(self, record):
           # Implement custom filtering logic
           return record.levelno == logging.INFO
   
   my_filter = MyFilter()
   console_handler.addFilter(my_filter)
   

5. LoggerAdapter:

   • A LoggerAdapter is a helper class that allows additional contextual information to be added to log records. It is useful when you want to include extra information in log messages without modifying the original logger.

   extra_info = {'user_id': 123, 'request_id': 'abc123'}
   logger = logging.LoggerAdapter(logger, extra_info)
   

6. LogRecord:

   • A LogRecord represents an individual log entry. It contains information such as the log message, log level, timestamp, logger name, and other details.
   • Log records are created by the logger and processed by handlers.

7. Configurator:

   • The logging.config module provides a Configurator class that allows you to configure the logging system using configuration files, dictionaries, or other sources.

   import logging.config
   
   logging.config.fileConfig('logging_config.ini')
   

These components work together to create a flexible and customizable logging system in Python. You can configure the logging framework to meet the specific requirements of your application, and the modular design allows you to reuse components across different parts of your codebase.

The logging system using a configuration file named 'logging.config'. It then retrieves a logger named 'root' and logs messages using the configured handlers.

An explanation of each part of the code:

1. Configure Logging System:

   import logging.config
   logging.config.fileConfig('logging.config')
   

   • The logging.config module is imported to access functions for configuring the logging system.
   • The fileConfig function is called with the argument 'logging.config', which is assumed to be the name of a configuration file. This file contains settings for configuring loggers, handlers, formatters, and other logging components.

2. Get Logger:

   logger = logging.getLogger('root')
   

   • The getLogger method is called to retrieve a logger named 'root'. The logger name is typically hierarchical, following the dot-separated namespace convention.

3. Log Debug Message:

   logger.debug("FileHandler message")
   

   • The debug method of the logger is used to log a debug-level message. This message will be emitted if the logger’s level is set to DEBUG or a lower level.

4. Log Info Message:

   logger.info("message for both handlers")
   

   • The info method of the logger is used to log an info-level message. This message will be emitted if the logger’s level is set to INFO or a lower level.

5. Explanation:

   • The configuration file 'logging.config' likely contains settings that define the behavior of the logging system, including details about handlers, formatters, and logging levels.
   • The code assumes that there is a configuration file with the name 'logging.config' in the current directory. Ensure that the file exists and contains valid logging configurations.

6. Handler Message:

   • The code snippet itself does not include information about the specific handlers and their configurations. The behavior of the logging system, including where the log messages are directed, is determined by the contents of the configuration file.

[loggers]
keys=root

[handlers]
keys = FileHandler,consoleHandler,rotatingFileHandler

[formatters]
keys=simpleFormatter

[logger_root]
level = DEBUG
handlers = FileHandler,consoleHandler,rotatingFileHandler

[handler_FileHandler]
class = FileHandler
level = DEBUG
formatter=simpleFormatter
args=("fileHandler.log",)

[handler_consoleHandler]
class = StreamHandler
level = INFO
formatter=simpleFormatter
args=(sys.stdout,)

[handler_rotatingFileHandler]
class = handlers.TimedRotatingFileHandler
level = INFO
formatter=simpleFormatter
args=("rotatingFileHandler.log",)
maxBytes=1024

[formatter_simpleFormatter]
format =%(message)s
datefmt=

In the actual configuration file, you would find sections for loggers, handlers, and formatters, each with specific configurations.

Make sure to check the contents of the 'logging.config' file to understand how the logging system is configured, including details about handlers, formatters, and their associated settings.

Sets up a basic logging configuration using the logging module. It configures a logger named __name__ to log messages to a file named ‘debug.log’ with various log levels. Here’s an explanation of each part of the code:

import logging

# Create a logger with the name of the current module
logger = logging.getLogger(__name__)

# Set the logging level for the logger
logger.setLevel(logging.DEBUG)

1. Logger Creation:

   • A logger object is created using logging.getLogger(__name__). The __name__ variable typically represents the name of the current module, allowing for hierarchical logging when used in a larger project.

2. Logger Level:

   • The logging level for the logger is set to DEBUG using logger.setLevel(logging.DEBUG). This means that the logger will capture log messages at the DEBUG level and above.

# Create a FileHandler to write log messages to a file
fileHandler = logging.FileHandler('debug.log')

# Set the logging level for the file handler
fileHandler.setLevel(logging.DEBUG)

# Add the FileHandler to the logger
logger.addHandler(fileHandler)

3. FileHandler:
   • A FileHandler is created to handle log messages. It writes log messages to the specified file, ‘debug.log’.
   • The logging level for the FileHandler is set to DEBUG, indicating that it will handle log messages at the DEBUG level and above.
   • The FileHandler is added to the logger using logger.addHandler(fileHandler).

# Create a formatter to define the log message format
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')

# Set the formatter for the FileHandler
fileHandler.setFormatter(formatter)

4. Formatter:
   • A Formatter is created to define the format of log messages. The format includes the timestamp, logger name, log level, and the log message itself.
   • The formatter is set for the FileHandler using fileHandler.setFormatter(formatter).

# Log messages with different log levels
logger.debug('debug message')
logger.info('info message')
logger.warning('warning message')
logger.error('error message')
logger.critical('critical message')

5. Log Messages:
   • Log messages are generated using various log levels (DEBUG, INFO, WARNING, ERROR, CRITICAL) to demonstrate different severity levels of log messages.

Sets up a logger with a FileHandler to write log messages to a file. It configures a Formatter to define the log message format and logs messages with different levels of severity. The log messages are written to the file ‘debug.log’, and the format of each log entry is specified by the formatter.