I didn’t really appreciate Python’s set type until I needed to deduplicate a huge list of records and compare it against another dataset for overlaps. What would have been nested loops and manual bookkeeping turned into a single line of set intersection. Since then, sets have become one of my go-to tools whenever a problem involves membership testing, deduplication, or comparing collections. This guide covers everything I’ve learned about set operations in Python — from the fundamentals to how they’re implemented under the hood.
What Is a Set in Python
A set is an unordered collection of unique, hashable elements. Python provides two built-in set types: the mutable set and the immutable frozenset.
fruits = {"apple", "banana", "cherry"}
print(fruits)
empty_set = set() # {} creates an empty dict, not a set
print(type(empty_set)) # <class 'set'>
That last line trips up beginners constantly — {} always creates a dictionary, never an empty set. I have to explicitly call set().
Creating Sets
s1 = {1, 2, 3}
s2 = set([2, 3, 4]) # from a list
s3 = set("hello") # from a string -> {'h', 'e', 'l', 'o'}
s4 = {x for x in range(10) if x % 2 == 0} # set comprehension
print(s4) # {0, 2, 4, 6, 8}
Duplicates are automatically removed:
print(set([1, 2, 2, 3, 3, 3])) # {1, 2, 3}
This single behavior is why sets are my default tool for deduplication.
Core Set Theory Operations
Python’s set operations map directly onto mathematical set theory, and each has both an operator form and an equivalent method.
Union
Combines all unique elements from both sets.
a = {1, 2, 3}
b = {3, 4, 5}
print(a | b) # {1, 2, 3, 4, 5}
print(a.union(b)) # {1, 2, 3, 4, 5}
Intersection
Elements present in both sets.
print(a & b) # {3}
print(a.intersection(b)) # {3}
Difference
Elements in the first set but not the second.
print(a - b) # {1, 2}
print(a.difference(b)) # {1, 2}
print(b - a) # {4, 5}
Symmetric Difference
Elements in either set, but not in both.
print(a ^ b) # {1, 2, 4, 5}
print(a.symmetric_difference(b)) # {1, 2, 4, 5}
Why the Operator and Method Forms Both Exist
The method forms (.union(), .intersection(), etc.) accept any iterable, not just sets:
a = {1, 2, 3}
print(a.union([4, 5], (6, 7))) # {1, 2, 3, 4, 5, 6, 7}
The operator forms (|, &, -, ^) require both operands to actually be sets (or frozensets):
# a | [4, 5] # Raises TypeError
a | set([4, 5]) # Works fine
I use the operator form when both sides are already sets, and the method form when I need to combine a set with a plain list or tuple.
Subset, Superset, and Disjoint Checks
a = {1, 2}
b = {1, 2, 3, 4}
print(a.issubset(b)) # True, a ⊆ b
print(a <= b) # True, same as issubset
print(b.issuperset(a)) # True, b ⊇ a
print(b >= a) # True
c = {5, 6}
print(a.isdisjoint(c)) # True, no common elements
print(a < b) # True, proper subset (a is subset and a != b)
print(a <= a) # True, subset includes equal sets
print(a < a) # False, not a *proper* subset of itself
In-Place Update Operations
Alongside the operations that return new sets, Python provides in-place variants that mutate the original set:
a = {1, 2, 3}
b = {3, 4, 5}
a |= b # same as a.update(b)
print(a) # {1, 2, 3, 4, 5}
a = {1, 2, 3}
a &= {2, 3, 4} # same as a.intersection_update()
print(a) # {2, 3}
a = {1, 2, 3}
a -= {2} # same as a.difference_update({2})
print(a) # {1, 3}
a = {1, 2, 3}
a ^= {2, 3, 4} # same as a.symmetric_difference_update({2, 3, 4})
print(a) # {1, 4}
These are memory-efficient when I don’t need to preserve the original set, since they modify in place rather than allocating a brand-new set object.
Adding and Removing Elements
s = {1, 2, 3}
s.add(4)
print(s) # {1, 2, 3, 4}
s.remove(4) # Raises KeyError if element doesn't exist
s.discard(10) # Does nothing if element doesn't exist — no error
print(s) # {1, 2, 3}
popped = s.pop() # Removes and returns an arbitrary element
print(popped)
s.clear()
print(s) # set()
I default to discard() over remove() whenever I’m not certain the element exists, to avoid handling KeyError explicitly.
How Sets Are Implemented Internally
Python’s set is implemented as a hash table, the same underlying data structure used for dictionaries (in fact, CPython’s set implementation shares significant code with its dict implementation). Every element must be hashable — meaning it implements __hash__() and __eq__() consistently — which is why lists and dictionaries can’t be set members, but tuples (if all their contents are also hashable) can.
s = {(1, 2), (3, 4)} # Works, tuples are hashable
# s = {[1, 2]} # Raises TypeError: unhashable type: 'list'
Because of the hash table implementation, membership testing (in), addition, and removal are all average O(1) operations — dramatically faster than the O(n) linear scan required for lists.
import time
big_list = list(range(1_000_000))
big_set = set(big_list)
start = time.perf_counter()
999_999 in big_list
print("List lookup:", time.perf_counter() - start)
start = time.perf_counter()
999_999 in big_set
print("Set lookup:", time.perf_counter() - start)
The set lookup is essentially instantaneous regardless of size, while the list lookup scales linearly with the number of elements.
frozenset: The Immutable Set
frozenset behaves exactly like set for all read operations (union, intersection, etc.) but is immutable and therefore hashable — meaning it can be used as a dictionary key or as an element of another set, which a regular mutable set cannot.
fs = frozenset([1, 2, 3])
# fs.add(4) # Raises AttributeError, frozensets are immutable
nested = {frozenset([1, 2]), frozenset([3, 4])}
print(nested)
cache = {frozenset([1, 2, 3]): "cached_result"}
print(cache[frozenset([3, 2, 1])]) # 'cached_result', order doesn't matter
I reach for frozenset specifically when I need set semantics as a dictionary key, or when I want to guarantee a collection can’t be accidentally mutated.
Real-World Use Cases
Deduplication:
emails = ["a@x.com", "b@x.com", "a@x.com", "c@x.com"]
unique_emails = set(emails)
print(unique_emails)
Finding common elements between datasets:
users_signed_up = {"alice", "bob", "carol"}
users_active_today = {"bob", "carol", "dave"}
churn_risk = users_signed_up - users_active_today
print(churn_risk) # {'alice'}
Removing duplicates while checking membership efficiently in automation scripts:
processed_ids = set()
def process(record_id):
if record_id in processed_ids:
return
processed_ids.add(record_id)
# ... perform processing
Tag/permission matching systems:
required_permissions = {"read", "write"}
user_permissions = {"read", "write", "admin"}
has_access = required_permissions.issubset(user_permissions)
print(has_access) # True
Best Practices
- Use sets whenever membership testing or deduplication is the core need — the O(1) average lookup pays off quickly compared to lists.
- Use
frozensetfor immutable, hashable set data, especially as dict keys or nested set elements. - Prefer set comprehensions over building a list first and converting:
{x for x in data}rather thanset([x for x in data]). - Remember that sets are unordered — never rely on a particular iteration order, even though modern CPython often appears consistent for a given run.
Common Mistakes
# Mistake: expecting {} to create an empty set
x = {}
print(type(x)) # dict, not set
# Mistake: trying to put unhashable types in a set
# s = {[1, 2, 3]} # TypeError
# Mistake: assuming sets preserve insertion order
s = {3, 1, 2}
print(s) # Order is not guaranteed to match insertion
Debugging Tips
- Use
type()to confirm you actually have asetand not adictwhen using{}. - Convert to
sorted(my_set)when you need predictable output for debugging or display. - Use
len(set(my_list)) != len(my_list)as a quick way to detect duplicates in a list.
FAQs
How do I create an empty set? Use set() — {} always creates an empty dictionary.
Can sets contain other sets? Not regular mutable sets, but they can contain frozenset objects, since regular sets aren’t hashable.
Are sets ordered in Python? No, sets are unordered collections. If order matters, use a list or, for order-preserving uniqueness, a dict.fromkeys() trick or list(dict.fromkeys(items)).
What’s the time complexity of set operations? Membership testing, addition, and removal are average O(1). Union and intersection are roughly O(len(s1) + len(s2)) or O(min(len(s1), len(s2))) depending on the operation.
Summary
Sets are one of Python’s most underused tools for everyday problems involving deduplication, membership testing, and comparing collections. Union, intersection, difference, and symmetric difference map cleanly onto set theory, backed by a hash table implementation that makes lookups and mutations average O(1) — a massive improvement over list-based approaches for anything involving repeated membership checks.