Regex for Nginx Log Parsing

The Default Combined Log Format

Nginx ships with the combined log format enabled by default. Thelog_format directive in nginx.conf looks like:

log_format combined '$remote_addr - $remote_user [$time_local] '
                    '"$request" $status $body_bytes_sent '
                    '"$http_referer" "$http_user_agent"';

# Example line:
203.0.113.42 - - [24/Oct/2026:12:34:56 +0000] "GET /api/v1/users HTTP/1.1" 200 1842 "https://ref.example/" "Mozilla/5.0 (X11; Linux x86_64) Firefox/130.0"

The Full Parsing Pattern

/^(\S+) - (\S+) \[([^\]]+)\] "(\S+) (\S+) (\S+)" (\d+) (\d+) "([^"]*)" "([^"]*)"$/

// Capture groups
// 1  remote_addr      "203.0.113.42"
// 2  remote_user      "-" (or an auth username)
// 3  time_local       "24/Oct/2026:12:34:56 +0000"
// 4  method           "GET"
// 5  path             "/api/v1/users"
// 6  protocol         "HTTP/1.1"
// 7  status           "200"
// 8  body_bytes_sent  "1842"
// 9  http_referer     "https://ref.example/"
// 10 http_user_agent  "Mozilla/5.0 ..."

Why These Patterns Work

\S+matches "any run of non-whitespace" and is the fastest way to pull a single token. We use it for IP, user, method, path, and protocol — fields that nginx guarantees contain no spaces. For quoted fields we use "([^"]*)" to grab everything between the quotes — this works because nginx escapes any embedded quote as \x22 before writing. The timestamp uses \[([^\]]+)\] because the bracket itself is the only forbidden character inside.

Language-Specific Usage

JavaScript / Node.js

import { createReadStream } from 'fs';
import { createInterface } from 'readline';

const NGINX_LOG = /^(\S+) - (\S+) \[([^\]]+)\] "(\S+) (\S+) (\S+)" (\d+) (\d+) "([^"]*)" "([^"]*)"$/;

async function parseLog(path) {
  const rl = createInterface({
    input: createReadStream(path),
    crlfDelay: Infinity,
  });

  const stats = { total: 0, errors: 0, byStatus: {} };
  for await (const line of rl) {
    const m = NGINX_LOG.exec(line);
    if (!m) continue;
    stats.total++;
    const [, ip, user, time, method, pathReq, proto, status, bytes, ref, ua] = m;
    stats.byStatus[status] = (stats.byStatus[status] || 0) + 1;
    if (status[0] === '4' || status[0] === '5') stats.errors++;
  }
  return stats;
}

parseLog('/var/log/nginx/access.log').then(console.log);

Python

import re
from collections import Counter
from datetime import datetime

NGINX_LOG = re.compile(
    r'^(\S+) - (\S+) \[([^\]]+)\] '
    r'"(\S+) (\S+) (\S+)" (\d+) (\d+) '
    r'"([^"]*)" "([^"]*)"$'
)

def parse_line(line):
    m = NGINX_LOG.match(line)
    if not m:
        return None
    ip, user, time, method, path, proto, status, bytes_, ref, ua = m.groups()
    return {
        'ip': ip,
        'time': datetime.strptime(time, '%d/%b/%Y:%H:%M:%S %z'),
        'method': method,
        'path': path,
        'status': int(status),
        'bytes': int(bytes_),
        'referer': ref if ref != '-' else None,
        'user_agent': ua,
    }

# Top 10 error paths
status_counter = Counter()
with open('/var/log/nginx/access.log') as f:
    for line in f:
        row = parse_line(line)
        if row and row['status'] >= 400:
            status_counter[row['path']] += 1
print(status_counter.most_common(10))

Go

package main

import (
    "bufio"
    "fmt"
    "os"
    "regexp"
)

var nginxRe = regexp.MustCompile(`^(\S+) - (\S+) \[([^\]]+)\] "(\S+) (\S+) (\S+)" (\d+) (\d+) "([^"]*)" "([^"]*)"$`)

func main() {
    f, _ := os.Open("/var/log/nginx/access.log")
    defer f.Close()

    sc := bufio.NewScanner(f)
    sc.Buffer(make([]byte, 64*1024), 1024*1024)

    byStatus := map[string]int{}
    for sc.Scan() {
        m := nginxRe.FindStringSubmatch(sc.Text())
        if m == nil {
            continue
        }
        status := m[7]
        byStatus[status]++
    }
    fmt.Println(byStatus)
}

awk One-Liners

# Top 10 IPs by request count
awk '{print $1}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head

# All 5xx errors
awk '$9 ~ /^5/ {print}' /var/log/nginx/access.log

# Average bytes served per status code
awk '{s[$9] += $10; c[$9]++} END {for (k in s) printf "%s %d\n", k, s[k]/c[k]}' access.log

# Top 20 404s by path
awk '$9 == "404" {print $7}' access.log | sort | uniq -c | sort -rn | head -20

# Requests from the last hour (requires GNU date)
cutoff=$(date -d '1 hour ago' '+%d/%b/%Y:%H')
awk -v c="[$cutoff" '$4 >= c {print}' access.log

Field-Specific Micro-Patterns

When you only need one field, a focused pattern is faster than the full 10-group regex:

Common Pitfalls

Malformed lines silently dropped

A URL containing an unescaped quote or newline breaks the regex and the line is skipped with no error. For auditing, log the count of non-matching lines so you notice a 1% drop rate before it masks real problems. In Python: track fail_count alongsidesuccess_count; alert if fail_count/total > 0.001.

Custom log formats breaking the pattern

If your log_format includes $request_time, $upstream_addr, or $http_x_forwarded_for, the default regex fails on every line. Always grep log_format from nginx.conffirst and extend the regex to match. For log-format sprawl, consider using nginx's escape=json option which writes NDJSON — then no regex needed.

IPv6 addresses

IPv6 addresses contain colons (2001:db8::1). The \S+ IP capture handles them correctly because colons are not whitespace. If you ever had a stricter\d+\.\d+\.\d+\.\d+ pattern, every IPv6 request is dropped — a common silent data-quality bug.

Timezone parsing

The timestamp carries a +0000 offset but many databases store naive UTC. Always parse with the offset (Python %z) and convert to UTC before storing, or you'll misalign hours on dashboards after your server is restarted in a new TZ.

Testing Your Nginx Log Regex

Use the live Regex Tester with this canonical test block:

203.0.113.42 - - [24/Oct/2026:12:34:56 +0000] "GET /api/v1/users HTTP/1.1" 200 1842 "https://ref.example/" "Mozilla/5.0 (X11; Linux x86_64) Firefox/130.0"
198.51.100.7 - alice [24/Oct/2026:12:35:01 +0000] "POST /login HTTP/2.0" 302 0 "-" "curl/8.4.0"
10.0.0.1 - - [24/Oct/2026:12:35:02 +0000] "GET /favicon.ico HTTP/1.1" 404 548 "-" "Mozilla/5.0"
2001:db8::1 - - [24/Oct/2026:12:35:03 +0000] "GET / HTTP/1.1" 200 20480 "-" "Mozilla/5.0"

Pattern:
/^(\S+) - (\S+) \[([^\]]+)\] "(\S+) (\S+) (\S+)" (\d+) (\d+) "([^"]*)" "([^"]*)"$/gm

Expected: 4 matches, each with 10 captures.

Performance Notes

A compiled nginx regex processes ~500k lines/second on a single modern core (benchmarked with Python re on a 1.2 GHz x86 core). For streaminglog analytics at >1M lines/sec, replace regex with a dedicated parser: GoAccess, nginx-amplify, or a hand-written bufio.Scanner that tokenises on whitespace and quote boundaries. Regex is best for exploratory analysis and dashboards — not for the hot path of a streaming aggregator.

Related Tools

• Regex for IP Address
• Regex for URL Matching
• Regex for Time Format (24-hour)
• Regex for Date Format
• Regex for Named Capture Groups
• Find & Replace Tool