I still remember the first time a regular expression broke my code at 2 AM. I was trying to validate an email field, my pattern looked right, and yet every single input failed. It took me an embarrassingly long time to realize I’d forgotten to escape a dot. That one mistake taught me more about regex than any tutorial ever did, and it’s part of why I decided to put together this guide the way I have.
This isn’t a dry syntax dump. I’ve built this cheat sheet the way I wish someone had handed it to me years ago — organized by what I actually reach for when I’m writing Python, full of examples I’ve tested myself, and honest about the mistakes I’ve made along the way. Whether you’re debugging a validation script, parsing log files, or just trying to remember whether \d or \D matches digits, I want this page to be the one you bookmark and keep coming back to.
Let’s get into it.
What Is a Regular Expression, Really?
A regular expression (regex) is a sequence of characters that defines a search pattern. In Python, I use the built-in re module to compile and apply these patterns against strings — for searching, matching, splitting, and replacing text.
Here’s the bare minimum I need to get started:
import re
text = "My phone number is 555-123-4567"
pattern = r"\d{3}-\d{3}-\d{4}"
match = re.search(pattern, text)
if match:
print(match.group()) # Output: 555-123-4567
I always use raw strings (the r prefix) for patterns. It saves me from fighting Python’s own escape sequences on top of regex’s escape sequences — a mistake I made constantly as a beginner.
Quick-Reference: Core Regex Syntax
I use this table almost daily. It covers the metacharacters that show up in nearly every pattern I write.
| Symbol | Meaning | Example | Matches |
|---|---|---|---|
. | Any character except newline | a.c | “abc”, “a1c” |
^ | Start of string | ^Hello | “Hello world” |
$ | End of string | world$ | “Hello world” |
* | 0 or more repetitions | ab* | “a”, “ab”, “abbb” |
+ | 1 or more repetitions | ab+ | “ab”, “abbb” |
? | 0 or 1 repetition | ab? | “a”, “ab” |
{n} | Exactly n repetitions | a{3} | “aaa” |
{n,m} | Between n and m repetitions | a{2,4} | “aa”, “aaaa” |
[] | Character class | [aeiou] | any vowel |
[^] | Negated character class | [^0-9] | any non-digit |
| | Alternation (OR) | cat|dog | “cat” or “dog” |
() | Grouping | (ab)+ | “ab”, “abab” |
\ | Escape special character | \. | literal “.” |
Predefined Character Classes
These shorthand classes save me from writing out long character ranges every time.
| Shorthand | Meaning | Equivalent |
|---|---|---|
\d | Digit | [0-9] |
\D | Non-digit | [^0-9] |
\w | Word character | [a-zA-Z0-9_] |
\W | Non-word character | [^a-zA-Z0-9_] |
\s | Whitespace | [ \t\n\r\f\v] |
\S | Non-whitespace | [^ \t\n\r\f\v] |
\b | Word boundary | position between \w and \W |
\B | Non-word boundary | opposite of \b |
\A | Start of string (multiline-safe) | — |
\Z | End of string (multiline-safe) | — |
Example:
import re
text = "Order #4521 shipped to zip 90210"
numbers = re.findall(r"\d+", text)
print(numbers) # Output: ['4521', '90210']
The Python re Module: Functions I Use Constantly
| Function | Purpose | Returns |
|---|---|---|
re.match() | Checks for a match only at the beginning of the string | Match object or None |
re.search() | Scans the entire string for the first match | Match object or None |
re.findall() | Finds all non-overlapping matches | List of strings |
re.finditer() | Finds all matches as an iterator | Iterator of Match objects |
re.sub() | Replaces matches with a string | New string |
re.subn() | Same as sub() but also returns count | Tuple (string, count) |
re.split() | Splits a string by the pattern | List of strings |
re.compile() | Pre-compiles a pattern for reuse | Pattern object |
re.fullmatch() | Matches the entire string exactly | Match object or None |
Practical Examples for Each
import re
# match() - only checks the start
print(re.match(r"Hello", "Hello world")) # Match found
print(re.match(r"world", "Hello world")) # None
# search() - checks anywhere
print(re.search(r"world", "Hello world")) # Match found
# findall() - all occurrences
print(re.findall(r"\d+", "a1 b22 c333")) # ['1', '22', '333']
# finditer() - with positions
for m in re.finditer(r"\d+", "a1 b22 c333"):
print(m.start(), m.group())
# sub() - replace
print(re.sub(r"\d+", "#", "a1 b22 c333")) # 'a# b# c#'
# split() - split on pattern
print(re.split(r"\s*,\s*", "apple, banana,cherry")) # ['apple', 'banana', 'cherry']
# compile() - reuse pattern
pattern = re.compile(r"\d+")
print(pattern.findall("a1 b22")) # ['1', '22']
# fullmatch() - entire string must match
print(re.fullmatch(r"\d+", "12345")) # Match found
print(re.fullmatch(r"\d+", "12345a")) # None
Groups and Capturing
I use groups constantly when I need to pull structured pieces out of a larger match — think dates, names, or key-value pairs.
| Syntax | Meaning |
|---|---|
(abc) | Capturing group |
(?:abc) | Non-capturing group |
(?P<name>abc) | Named capturing group |
(?P=name) | Backreference to named group |
\1, \2 | Backreference to group 1, 2 |
(?=abc) | Positive lookahead |
(?!abc) | Negative lookahead |
(?<=abc) | Positive lookbehind |
(?<!abc) | Negative lookbehind |
import re
text = "2026-07-29"
match = re.match(r"(?P<year>\d{4})-(?P<month>\d{2})-(?P<day>\d{2})", text)
print(match.group("year")) # 2026
print(match.group("month")) # 07
print(match.groupdict()) # {'year': '2026', 'month': '07', 'day': '29'}
Lookaheads and lookbehinds are the tools I reach for when I need to match something based on context without including that context in the result:
import re
# Positive lookahead: match "price" only if followed by a number
text = "price: 500, name: item"
match = re.search(r"price(?=: \d+)", text)
print(match.group() if match else "No match") # price
# Negative lookbehind: match numbers not preceded by $
text = "Cost is $50 but tax is 5"
print(re.findall(r"(?<!\$)\b\d+\b", text)) # ['5']
Regex Flags I Actually Use
| Flag | Short Form | Purpose |
|---|---|---|
re.IGNORECASE | re.I | Case-insensitive matching |
re.MULTILINE | re.M | ^ and $ match at line boundaries |
re.DOTALL | re.S | . also matches newline |
re.VERBOSE | re.X | Allows whitespace/comments in pattern for readability |
re.ASCII | re.A | \w, \d, \s match ASCII only |
import re
pattern = re.compile(r"""
(?P<area>\d{3}) # area code
-
(?P<num>\d{4}) # number
""", re.VERBOSE)
print(pattern.search("Call 555-1234").groupdict())
Common Real-World Patterns
These are the patterns I keep in a personal snippets file because I reuse them across projects.
| Use Case | Pattern | Notes |
|---|---|---|
^[\w.+-]+@[\w-]+\.[a-zA-Z]{2,}$ | Basic validation, not RFC-complete | |
| URL | https?://[^\s]+ | Good for extracting links from text |
| IPv4 address | \b(?:\d{1,3}\.){3}\d{1,3}\b | Doesn’t validate 0-255 range strictly |
| Phone number (US) | \(?\d{3}\)?[-.\s]?\d{3}[-.\s]?\d{4} | Handles common formats |
| Hex color code | #(?:[0-9a-fA-F]{3}){1,2} | Matches #fff or #ffffff |
| Date (YYYY-MM-DD) | \d{4}-\d{2}-\d{2} | ISO format |
| Whitespace trimming | ^\s+|\s+$ | Use with re.sub and empty string |
| Password strength | ^(?=.*[A-Z])(?=.*\d)(?=.*[!@#$%^&*]).{8,}$ | At least one upper, digit, symbol, 8+ chars |
import re
def is_valid_email(email):
pattern = r"^[\w.+-]+@[\w-]+\.[a-zA-Z]{2,}$"
return bool(re.match(pattern, email))
print(is_valid_email("hello@example.com")) # True
print(is_valid_email("hello@example")) # False
Best Practices I Follow
- I always precompile patterns with
re.compile()when I’ll reuse them in a loop — it noticeably improves performance. - I use raw strings (
r"...") for every pattern, no exceptions, to avoid double-escaping headaches. - I keep patterns readable with
re.VERBOSEonce they get past a certain complexity — a dense regex with no comments is a debugging nightmare six months later. - I test incrementally. I build a pattern piece by piece rather than writing the whole thing and hoping it works.
- I avoid greedy quantifiers (
.*) when I actually want the smallest possible match —.*?(lazy) has saved me from matching way more text than intended more times than I’d like to admit. - I validate, don’t just match, when dealing with user input — matching part of a string isn’t the same as confirming the whole string is valid, which is why
fullmatch()matters. - I never use regex for deeply nested or recursive structures like HTML or JSON — that’s a job for a proper parser, not a pattern matcher.
Troubleshooting Common Regex Problems
| Problem | Likely Cause | Fix |
|---|---|---|
| Pattern matches too much | Greedy quantifier (.*) | Use lazy quantifier (.*?) |
| Pattern matches nothing | Forgot to escape special characters | Escape ., (, ), [, ] with \ |
AttributeError: 'NoneType' object has no attribute 'group' | match()/search() returned None | Check for None before calling .group() |
Unicode characters not matching \w | Using re.ASCII flag unintentionally | Remove the flag or use re.UNICODE (default in Python 3) |
| Backslash errors in pattern | Not using raw string | Prefix pattern with r |
^ and $ not matching per line | Missing re.MULTILINE flag | Add re.M |
| Catastrophic backtracking (regex hangs) | Nested quantifiers like (a+)+ | Simplify the pattern, avoid nested repetition |
Security Tips
I’ve seen regex misused in ways that create real vulnerabilities, so a few things I always keep in mind:
- Watch for ReDoS (Regular Expression Denial of Service). Patterns with nested quantifiers like
(a+)+$can cause catastrophic backtracking on crafted input, freezing your application. I test patterns against long, adversarial strings before deploying them. - Never build regex patterns from unsanitized user input using
re.escape()incorrectly or string concatenation — always callre.escape()on any user-supplied text that ends up inside a pattern. - Don’t rely on regex alone for security-critical validation, like sanitizing SQL queries or HTML — use parameterized queries and dedicated sanitization libraries instead.
- Set reasonable input length limits before running regex on user-submitted text, especially for web forms.
import re
user_input = "some (user) [input]"
safe_pattern = re.escape(user_input)
print(re.search(safe_pattern, "some (user) [input] here"))
Professional Workflow: Building a Log Parser
Here’s a workflow I actually use when I need to parse server logs — a task regex is genuinely great for.
import re
log_line = '127.0.0.1 - - [29/Jul/2026:10:15:32] "GET /index. html HTTP/1.1" 200 1024'
pattern = re.compile(r"""
(?P<ip>\d{1,3}(?:\.\d{1,3}){3})\s+
-\s+-\s+
\[(?P<timestamp>[^\]]+)\]\s+
"(?P<method>\w+)\s+(?P<path>\S+)\s+HTTP/[\d.]+"\s+
(?P<status>\d{3})\s+
(?P<size>\d+)
""", re.VERBOSE)
match = pattern.match(log_line)
if match:
print(match.groupdict())
Output:
{'ip': '127.0.0.1', 'timestamp': '29/Jul/2026:10:15:32', 'method': 'GET', 'path': '/index. html', 'status': '200', 'size': '1024'}
This is the kind of pattern I’d extend across thousands of log lines using finditer() for memory-efficient processing.
Common Mistakes I See (and Have Made Myself)
- Forgetting
re.escape()for literal special characters — trying to match a literal.or$without escaping it. - Using
match()whensearch()is needed —match()only checks from the start of the string, which trips people up constantly. - Overusing
.*— leads to unintended matches spanning far more text than expected. - Not anchoring patterns — omitting
^and$when full-string validation is required. - Ignoring Unicode — assuming
\wonly matches ASCII letters when Python 3’sreis Unicode-aware by default. - Writing unreadable one-liners — a 200-character regex with no comments becomes unmaintainable fast.
- Not precompiling — recompiling the same pattern inside a loop instead of once outside it.
Working with Match Objects in Depth
Whenever re.match() or re.search() succeeds, I get back a Match object, and I lean on its methods constantly rather than just grabbing .group() and moving on.
| Method/Attribute | Purpose |
|---|---|
.group() | Returns the whole match (or a specific group) |
.groups() | Returns a tuple of all captured groups |
.groupdict() | Returns named groups as a dictionary |
.start() | Returns the starting index of the match |
.end() | Returns the ending index of the match |
.span() | Returns a tuple of (start, end) |
.string | Returns the original string that was searched |
import re
text = "Invoice #4521 dated 2026-07-29"
match = re.search(r"#(?P<invoice>\d+).*?(?P<date>\d{4}-\d{2}-\d{2})", text)
print(match.group(0)) # Full match
print(match.group("invoice")) # 4521
print(match.span("date")) # (21, 31)
print(match.string) # Original text
I use .span() a lot when I need to slice the original string around a match — for example, replacing just the matched portion while preserving everything else exactly as it was.
Substitution Techniques Beyond the Basics
re.sub() does more than swap static text — I regularly use functions and backreferences inside substitutions to do more complex transformations.
Using Backreferences in Replacement Strings
import re
text = "John Smith, Jane Doe"
# Swap first and last names
result = re.sub(r"(\w+) (\w+)", r"\2 \1", text)
print(result) # Smith John, Doe Jane
Using a Function as the Replacement
This is one of my favorite tricks — when a simple string replacement isn’t enough, I pass a function instead:
import re
text = "I have 3 apples and 12 oranges"
def double_number(match):
return str(int(match.group()) * 2)
result = re.sub(r"\d+", double_number, text)
print(result) # I have 6 apples and 24 oranges
Limiting the Number of Replacements
import re
text = "a-b-c-d-e"
result = re.sub(r"-", "_", text, count=2)
print(result) # a_b_c-d-e
Splitting Strings with Capturing Groups
Something that catches people off guard: if you use a capturing group inside the pattern passed to re.split(), the captured text is included in the result list.
import re
text = "apple123banana456cherry"
print(re.split(r"(\d+)", text))
# Output: ['apple', '123', 'banana', '456', 'cherry']
print(re.split(r"\d+", text))
# Output: ['apple', 'banana', 'cherry']
I use this deliberately when I need to know not just how a string was split, but exactly what the delimiters were.
Performance Considerations
Regex is convenient, but it isn’t free. A few performance habits I’ve picked up over the years:
- Precompile patterns used in loops. Compiling a pattern has overhead; doing it once outside a loop instead of on every iteration can make a measurable difference on large datasets.
- Avoid unnecessary backtracking. Patterns like
(a+)+bcan cause the engine to explore an exponential number of paths when the input doesn’t match — known as catastrophic backtracking. - Anchor patterns when possible. A pattern anchored with
^can often fail fast instead of scanning the whole string. - Use non-capturing groups
(?:...)when you don’t need to extract the group’s contents — it avoids the overhead of tracking that group’s boundaries. - Profile before optimizing. For most everyday scripts, regex performance is a non-issue — I only start optimizing after actually measuring a bottleneck with something like
timeit.
import re
import timeit
pattern = re.compile(r"\d+")
text = "abc123def456" * 1000
# Compiled pattern reused across many calls
duration = timeit.timeit(lambda: pattern.findall(text), number=1000)
print(duration)
Regex vs. Alternatives: When Not to Use Regex
I’ve learned the hard way that regex isn’t always the right tool:
| Task | Better Alternative |
|---|---|
| Parsing HTML/XML | BeautifulSoup or lxml |
| Parsing JSON | Python’s built-in json module |
| Parsing CSV | Python’s built-in csv module |
| Simple substring checks | str.startswith(), str.endswith(), in |
| Complex nested/recursive structures | A proper parser or grammar (e.g., pyparsing) |
| Date/time parsing | datetime.strptime() or dateutil |
Regex is fantastic for flat, linear pattern matching — but the moment structure becomes nested or recursive, a dedicated parser will save time and prevent subtle bugs.
A Step-by-Step Example: Building a Validation Function
Here’s a workflow I follow when building something like a username validator from scratch, since it shows how I layer requirements incrementally rather than writing one giant pattern up front.
import re
def is_valid_username(username):
# Requirement 1: 3-16 characters
if not re.fullmatch(r".{3,16}", username):
return False
# Requirement 2: starts with a letter
if not re.match(r"^[A-Za-z]", username):
return False
# Requirement 3: only letters, digits, underscores
if not re.fullmatch(r"[A-Za-z0-9_]+", username):
return False
return True
print(is_valid_username("john_doe22")) # True
print(is_valid_username("22john")) # False (starts with digit)
print(is_valid_username("jo")) # False (too short)
I could combine all of this into a single dense pattern, but I usually prefer breaking validation into readable steps like this — it’s far easier to debug when a specific rule fails, and far easier for someone else to maintain later.
Frequently Asked Questions
Is regex the same across all programming languages? No. Python’s re module has its own syntax quirks (like named groups using (?P<name>...) instead of (?<name>...) in some other languages). The core concepts transfer, but syntax details don’t always.
What’s the difference between match() and fullmatch()? match() only requires the pattern to match from the start of the string — it can succeed even if there’s leftover text. fullmatch() requires the entire string to match the pattern exactly.
Should I use regex or string methods for simple tasks? For simple substring checks, I use str.startswith(), str.endswith(), or in — they’re faster and more readable. I reach for regex when I need patterns, not just literal text.
Why does my pattern work in an online regex tester but not in Python? Different regex engines (PCRE, JavaScript, Python’s re) have subtly different syntax and behavior. Always test directly in Python rather than assuming compatibility.
How do I match across multiple lines? Use the re.MULTILINE flag for ^/$ to match line boundaries, or re.DOTALL if you need . to match newlines too.
Is there a faster alternative to the re module? For very performance-sensitive applications, the third-party regex module offers more features and sometimes better performance, and libraries like re2 (via bindings) avoid catastrophic backtracking entirely.
Interview Questions on Regex (with Answers)
1. What is the difference between greedy and lazy quantifiers? Greedy quantifiers (*, +, {n,m}) match as much text as possible. Lazy quantifiers (*?, +?, {n,m}?) match as little as possible while still satisfying the pattern.
2. How do you extract all email addresses from a large text block in Python? Using re.findall() with an email pattern, ideally on a precompiled pattern object for performance on large text.
3. What’s a non-capturing group and when would you use one? (?:...) groups characters for applying quantifiers or alternation without creating a backreference-accessible group — useful for performance and cleaner group indexing.
4. Explain catastrophic backtracking. It happens when a regex engine tries an exponential number of combinations to match a pattern against a string, usually due to nested or ambiguous quantifiers, causing the program to hang.
5. How would you validate that a string is a valid password with specific complexity rules? Using lookaheads to enforce multiple independent conditions, e.g., ^(?=.*[A-Z])(?=.*\d)(?=.*[!@#$%^&*]).{8,}$, combined with fullmatch().
6. What’s the difference between \d and [0-9]? Functionally the same for ASCII digits, but \d in Python 3 also matches Unicode decimal digits unless re.ASCII is set.
Printable Quick-Reference Summary
CHARACTER CLASSES
\d digit \D non-digit
\w word char \W non-word char
\s whitespace \S non-whitespace
\b word boundary
QUANTIFIERS
* 0+ + 1+
? 0 or 1 {n} exactly n
{n,m} n to m *? +? lazy versions
ANCHORS
^ start $ end
\A string start \Z string end
GROUPS
(...) capturing
(?:...) non-capturing
(?P<name>...) named group
(?=...) lookahead
(?<=...) lookbehind
FUNCTIONS
re.match() start of string only
re.search() anywhere in string
re.findall() all matches, as list
re.finditer() all matches, as iterator
re.sub() replace matches
re.split() split by pattern
re.compile() precompile for reuse
Official Documentation and Further Reading
I keep coming back to this cheat sheet myself whenever I’m knee-deep in a parsing script and need a quick reminder. Save it, bookmark it, and if a pattern ever gets the better of you at 2 AM like it did me — take a breath, break it into pieces, and test one piece at a time. That’s really the whole secret.