How to Configure Apache to Use a Different Log File Format

I used to stare at Apache’s default access log trying to figure out why a specific endpoint was slow, and every time I’d realize the default format just wasn’t giving me the field I needed. Once I started customizing LogFormat, debugging performance issues and feeding logs into tools like ELK or Grafana Loki got a lot easier. Apache’s mod_log_config module makes this fully customizable, and it’s one of the first things I tune on a new server.

Here’s how I approach Apache logging and how I build custom formats.

How Apache Logging Works

Apache writes two main log streams by default:

• Access log: one line per HTTP request — client IP, method, URL, status code, and more.
• Error log: diagnostic messages, warnings, and errors from Apache and any loaded modules.

Log formats are defined with LogFormat directives using format specifiers (like %h for client IP), then applied to a log file with CustomLog.

Prerequisites

• Apache installed with mod_log_config enabled (on by default in nearly every install)
• Root or sudo access
• Access to the main config or virtual host files

I check the module is active:

apache2ctl -M | grep log_config     # Debian/Ubuntu
httpd -M | grep log_config           # RHEL/CentOS

Default Log Format Reference

Apache ships with predefined formats, usually near the top of httpd.conf or apache2.conf:

LogFormat "%h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%h %l %u %t \"%r\" %>s %O" common

combined is the one I see most often in the wild and includes referer and user-agent data.

Key Format Specifiers I Reach For

The full reference lives in the official mod_log_config docs.

Step 1: Define a Custom Log Format

I add a new named format to my main config or virtual host:

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %D" custom_timed

This extends combined with %D, the request duration — I use it constantly to spot slow endpoints.

Step 2: Apply the Format to a Log File

CustomLog ${APACHE_LOG_DIR}/access.log custom_timed

Or scoped to a specific virtual host:

<VirtualHost *:80>
    ServerName example.com
    CustomLog /var/log/apache2/example.com-access.log custom_timed
    ErrorLog /var/log/apache2/example.com-error.log
</VirtualHost>

Step 3: Test and Reload

sudo apachectl configtest
sudo systemctl reload apache2

Then I generate a request and check the new log line:

curl -s http://example.com/ > /dev/null
tail -n 1 /var/log/apache2/example.com-access.log

Building a JSON Log Format

I lean on structured JSON logs when I need them to slot cleanly into ELK, Loki, or Datadog:

LogFormat "{\"time\":\"%{%Y-%m-%dT%H:%M:%S%z}t\",\"remote_ip\":\"%a\",\"method\":\"%m\",\"uri\":\"%U%q\",\"status\":%>s,\"bytes\":%B,\"referer\":\"%{Referer}i\",\"user_agent\":\"%{User-Agent}i\",\"response_time_us\":%D}" json_log

CustomLog ${APACHE_LOG_DIR}/access.json json_log

One caveat I’ve hit: this simple approach doesn’t escape embedded quotes in user-agent or referer values, which can occasionally break a strict JSON parser. For production-grade JSON logging at scale, I’d rather use mod_log_json or run logs through Fluentd/Logstash, which handle the escaping properly.

Logging Behind a Reverse Proxy or Load Balancer

If Apache sits behind Nginx, HAProxy, or a cloud load balancer, %h logs the proxy’s IP, not the real visitor. I fix that with mod_remoteip:

sudo a2enmod remoteip

RemoteIPHeader X-Forwarded-For
RemoteIPTrustedProxy 10.0.0.0/8

LogFormat "%a %l %u %t \"%r\" %>s %b" proxy_combined
CustomLog ${APACHE_LOG_DIR}/access.log proxy_combined

With mod_remoteip active, %a reflects the real client IP once the trusted proxy header gets parsed correctly.

Conditional Logging

I skip logging for specific requests, like health checks, using environment variables:

SetEnvIf Request_URI "^/healthz$" dontlog
CustomLog ${APACHE_LOG_DIR}/access.log combined env=!dontlog

Real-World Use Cases

• Performance monitoring: adding %D to catch slow requests and bottlenecks.
• Security auditing: logging %{X-Forwarded-For}i and full headers for forensic work.
• SIEM/ELK integration: emitting structured JSON for automated ingestion and alerting.
• Multi-tenant hosting: separate CustomLog directives per virtual host for per-client analysis or billing.
• Compliance: some regulatory frameworks require specific fields logged consistently.

Mistakes I’ve Made

• Changing the format after logs were already being collected, ending up with inconsistent historical data — I document format changes and rotate to a new file when I change formats now.
• Forgetting to reload Apache after editing LogFormat, so the old format kept writing.
• Logging sensitive data (full query strings with tokens) by accident, which is a compliance headache waiting to happen.
• Applying mod_remoteip without setting RemoteIPTrustedProxy, which lets any client spoof its IP via X-Forwarded-For.
• Not rotating logs and watching them grow unbounded until disk space became a problem.

Security Best Practices

• I avoid logging full authorization headers, cookies, or tokens in custom formats.
• I restrict log file permissions to only the accounts that need them (chmod 640, appropriate group ownership).
• I only trust X-Forwarded-For from known proxy IP ranges via RemoteIPTrustedProxy.
• I ship logs off-server so an attacker who compromises the box can’t erase evidence of what happened.

Performance Optimization

• Buffered logging (BufferedLogs On) cuts I/O overhead on high-traffic servers, though it slightly delays log visibility.
• I avoid overly verbose custom formats with expensive header lookups on every request unless I actually need the data.
• logrotate keeps logs from growing unbounded and hurting disk I/O:

/var/log/apache2/*.log {
    daily
    rotate 14
    compress
    delaycompress
    missingok
    notifempty
    create 640 www-data adm
    sharedscripts
    postrotate
        systemctl reload apache2 > /dev/null
    endscript
}

Troubleshooting

Log file empty or not updating I check that CustomLog sits inside the correct <VirtualHost> block and that Apache reloaded successfully.

Real client IPs show as the load balancer’s IP mod_remoteip needs configuring as above — raw %h always shows the immediate connecting peer, not the original client, behind a proxy.

JSON logs fail to parse I look for unescaped quotes in User-Agent or Referer; a dedicated JSON logging module or log-shipping pipeline handles escaping properly.

FAQs

Can I have multiple log formats active at once? Yes — multiple CustomLog directives with different formats, each writing to its own file.

Does changing the log format affect log rotation? No, logrotate operates on the file itself regardless of internal format.

Is JSON logging slower than plain text? Negligible difference for typical traffic in my experience; the format string is evaluated the same way either way.

Summary and Key Takeaways

• LogFormat and CustomLog give me full control over what gets logged and how.
• I add specifiers like %D for timing data or switch to JSON for easier pipeline integration.
• I use mod_remoteip to log real client IPs when Apache sits behind a proxy.
• I rotate logs, restrict permissions, and never log sensitive data as part of a sane logging strategy.

References

• Apache mod_log_config Documentation: https://httpd.apache.org/docs/current/mod/mod_log_config.html
• Apache Logging Guide: https://httpd.apache.org/docs/current/logs.html
• Apache mod_remoteip Documentation: https://httpd.apache.org/docs/current/mod/mod_remoteip.html