<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>dailyprompt Archives | Abdul Wahab Junaid</title>
	<atom:link href="https://awjunaid.com/tag/dailyprompt/feed/" rel="self" type="application/rss+xml" />
	<link>https://awjunaid.com/tag/dailyprompt/</link>
	<description>Offensive Security Researcher &#38; Quantum Cryptography Analyst</description>
	<lastBuildDate>Sat, 15 Aug 2026 11:03:45 +0000</lastBuildDate>
	<language>en-US</language>
	<sy:updatePeriod>
	hourly	</sy:updatePeriod>
	<sy:updateFrequency>
	1	</sy:updateFrequency>
	

<image>
	<url>https://i0.wp.com/awjunaid.com/wp-content/uploads/2023/06/cropped-1668274976669.jpeg?fit=32%2C32&#038;ssl=1</url>
	<title>dailyprompt Archives | Abdul Wahab Junaid</title>
	<link>https://awjunaid.com/tag/dailyprompt/</link>
	<width>32</width>
	<height>32</height>
</image> 
<site xmlns="com-wordpress:feed-additions:1">220030102</site>	<item>
		<title>What Is a Cross-Query Collision? A Deep Dive Into a Quiet but Dangerous Bug Class</title>
		<link>https://awjunaid.com/cyber-security/what-is-cross-query-collisions/</link>
					<comments>https://awjunaid.com/cyber-security/what-is-cross-query-collisions/?noamp=mobile#respond</comments>
		
		<dc:creator><![CDATA[Abdul Wahab Junaid]]></dc:creator>
		<pubDate>Mon, 12 Aug 2024 21:43:34 +0000</pubDate>
				<category><![CDATA[Cyber Security]]></category>
		<category><![CDATA[cyber security]]></category>
		<category><![CDATA[dailyprompt]]></category>
		<guid isPermaLink="false">https://awjunaid.com/?p=9091</guid>

					<description><![CDATA[<p>I first ran into the term &#8220;cross-query collision&#8221; while digging through a caching layer bug report that made&#8230;</p>
<p>The post <a href="https://awjunaid.com/cyber-security/what-is-cross-query-collisions/">What Is a Cross-Query Collision? A Deep Dive Into a Quiet but Dangerous Bug Class</a> appeared first on <a href="https://awjunaid.com">Abdul Wahab Junaid</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<p class="wp-block-paragraph">I first ran into the term &#8220;cross-query collision&#8221; while digging through a caching layer bug report that made no sense on the surface — a user was occasionally seeing <em>someone else&#8217;s</em> search results. No SQL injection, no broken authentication, nothing screaming &#8220;vulnerability&#8221; in the logs. The root cause turned out to be a collision between cache keys generated from different queries, and once I understood the mechanism, I started seeing the same pattern everywhere: hashing, caching, rate limiting, deduplication, even cryptographic constructions.</p>



<p class="wp-block-paragraph">This article explains what cross-query collisions actually are, why they happen, where they show up, and how to prevent them.</p>



<h2 class="wp-block-heading">Defining the Problem</h2>



<p class="wp-block-paragraph">A cross-query collision happens when two logically distinct queries — different inputs, different intents, different users — end up mapping to the same internal identifier, key, bucket, or cache slot. When that identifier is trusted to uniquely represent a query, the collision causes the system to treat two different things as one thing.</p>



<p class="wp-block-paragraph">This is fundamentally a <strong>hash collision problem generalized beyond cryptography</strong>. It shows up anywhere a system compresses a large or unbounded input space (queries) down into a smaller fixed space (keys, buckets, cache slots, database shards).</p>



<div class="wp-block-merpress-mermaidjs diagram-source-mermaid"><pre class="mermaid">flowchart LR
    Q1[Query A: user=alice, filter=x] --> H[Hash / Key Function]
    Q2[Query B: user=bob, filter=y] --> H
    H --> K[Same Derived Key]
    K --> C[Shared Cache Slot / Bucket]
    C --> R1[Result Meant for A]
    C --> R2[Result Meant for B]
    R1 -.->|Collision: B sees A's data| Leak[Data Leak / Logic Error]
    R2 -.-> Leak
</pre></div>



<h2 class="wp-block-heading">Where Cross-Query Collisions Actually Occur</h2>



<h3 class="wp-block-heading">1. Caching Layers</h3>



<p class="wp-block-paragraph">Caches key results by some derived value — often a hash of query parameters. If the key derivation doesn&#8217;t fully capture the distinguishing parts of the query (for example, hashing only a subset of parameters, or truncating a hash to save memory), two different queries can collide on the same cache key. The second query then receives the first query&#8217;s cached, and possibly <em>personalized</em>, result.</p>



<p class="wp-block-paragraph">This is precisely the bug class behind several real-world &#8220;wrong person&#8217;s data displayed&#8221; incidents in production systems — usually traced back to a cache key built from something like <code>hash(endpoint + user_id)</code> where the hash was truncated too aggressively, or where an important parameter (like a permission scope) was left out of the key entirely.</p>



<h3 class="wp-block-heading">2. Rainbow Tables and Password Hash Collisions</h3>



<p class="wp-block-paragraph">In authentication systems, if two different passwords hash to the same stored value (a genuine cryptographic hash collision, or more commonly a weak/truncated hash), an attacker&#8217;s query — &#8220;does this password match?&#8221; — can succeed against an account it was never meant to unlock. This is rare with modern hash functions but was a real problem with older, weaker algorithms and with home-grown truncated hash schemes.</p>



<h3 class="wp-block-heading">3. Bloom Filters and Probabilistic Data Structures</h3>



<p class="wp-block-paragraph">Bloom filters are explicitly probabilistic — they trade a controlled false-positive rate for space efficiency. A cross-query collision here means query B triggers a &#8220;possibly present&#8221; answer meant only for query A&#8217;s data, because both hashed into overlapping bit positions. This is usually an accepted trade-off, but engineers often forget the false-positive rate compounds when many distinct queries hit a shared filter.</p>



<h3 class="wp-block-heading">4. Rate Limiting and Bucketing Systems</h3>



<p class="wp-block-paragraph">If a rate limiter buckets requests by a hashed key (e.g., <code>hash(IP + endpoint) mod N</code>), two unrelated users hashing into the same bucket can end up rate-limiting each other — one user&#8217;s traffic burns another user&#8217;s quota. This isn&#8217;t a security leak in the confidentiality sense, but it&#8217;s a denial-of-service-adjacent correctness bug with real business impact.</p>



<h3 class="wp-block-heading">5. Query Deduplication in Databases and Search Engines</h3>



<p class="wp-block-paragraph">Search engines and databases sometimes deduplicate or memoize query execution plans by a signature derived from the query text or an abstract syntax tree hash. If the signature doesn&#8217;t fully capture query semantics (for instance, ignoring bind parameter values when it shouldn&#8217;t), two different queries can be treated as identical, and the wrong cached execution plan or result set gets served.</p>



<h2 class="wp-block-heading">Why This Matters for Security, Not Just Correctness</h2>



<p class="wp-block-paragraph">Cross-query collisions sit at an uncomfortable intersection: they often <em>look</em> like ordinary bugs, but their consequences frequently overlap with security failures — information disclosure, authorization bypass, and denial of service. Because they don&#8217;t trip typical vulnerability scanners (there&#8217;s no injection payload, no malformed input), they tend to be discovered late, usually by an alert user noticing something&#8217;s wrong, or during a security audit that specifically reviews key-derivation logic.</p>



<h2 class="wp-block-heading">Table: Collision Risk by System Type</h2>



<figure class="wp-block-table"><table class="has-fixed-layout"><thead><tr><th>System</th><th>Key/Bucket Source</th><th>Collision Consequence</th><th>Typical Root Cause</th></tr></thead><tbody><tr><td>HTTP response cache</td><td>Hash of URL + params</td><td>Serving wrong user&#8217;s page</td><td>Incomplete key (missing user/session scope)</td></tr><tr><td>Password store</td><td>Password hash</td><td>Auth bypass</td><td>Weak/truncated hash algorithm</td></tr><tr><td>Bloom filter</td><td>Multiple hash functions</td><td>False &#8220;present&#8221; result</td><td>Under-sized filter for data volume</td></tr><tr><td>Rate limiter</td><td>Hash of IP/user/endpoint</td><td>Shared quota exhaustion</td><td>Small bucket space, high N</td></tr><tr><td>Query planner cache</td><td>AST/text hash</td><td>Wrong result set served</td><td>Ignoring bind parameters in signature</td></tr></tbody></table></figure>



<h2 class="wp-block-heading">How to Prevent Cross-Query Collisions</h2>



<ol class="wp-block-list">
<li><strong>Include the full distinguishing context in the key</strong>, not a lossy subset. If a query&#8217;s identity depends on user ID, tenant ID, permission scope, and filter parameters, the key must reflect <em>all</em> of them, not just the ones that seemed &#8220;important&#8221; at design time.</li>



<li><strong>Use cryptographically strong, sufficiently long hash outputs</strong> for anything security-relevant — truncating a SHA-256 output to 32 bits to save cache memory reintroduces collision risk that the full hash was designed to avoid (birthday-bound collision probability scales with output length; see our companion article on the Blind Birthday Attack).</li>



<li><strong>Namespace caches per tenant/user where personalization is involved.</strong> Don&#8217;t rely on the hash function alone to separate user contexts — add an explicit prefix or partition.</li>



<li><strong>Test for false sharing under load</strong>, not just correctness under a single query. Collisions are often only observable when many concurrent distinct queries are in flight.</li>



<li><strong>Audit key-derivation code as if it were a security boundary</strong>, because functionally, it often is one.</li>
</ol>



<pre class="wp-block-code"><code># VULNERABLE: key ignores user scope
def cache_key(query_params):
    return hashlib.md5(str(query_params).encode()).hexdigest()&#91;:8]  # truncated + no scope

# BETTER: full context, strong hash, explicit namespace
def cache_key(user_id, tenant_id, query_params):
    raw = f"{tenant_id}:{user_id}:{sorted(query_params.items())}"
    return hashlib.sha256(raw.encode()).hexdigest()
</code></pre>



<h2 class="wp-block-heading">Real-World Pattern: The &#8220;Personalized Cache&#8221; Incident Class</h2>



<p class="wp-block-paragraph">A recurring incident pattern reported across multiple SaaS platforms over the years follows this shape: a CDN or application cache is configured to cache API responses for performance. The cache key is derived from the request path and query string, but the response itself is personalized based on a session cookie or auth header that <em>isn&#8217;t</em> part of the key. Two users hitting the same path with the same query parameters, but different sessions, get cross-wired — one user&#8217;s personalized response is served to another. This is technically a cross-query (or cross-request) collision in the cache key space, and it has been the root cause of several publicly disclosed data-exposure incidents in caching infrastructure.</p>



<h2 class="wp-block-heading">Comparing Mitigation Approaches</h2>



<figure class="wp-block-table"><table class="has-fixed-layout"><thead><tr><th>Approach</th><th>Pros</th><th>Cons</th></tr></thead><tbody><tr><td>Full-context key (no truncation)</td><td>Eliminates collisions in practice</td><td>Larger key storage overhead</td></tr><tr><td>Per-tenant namespacing</td><td>Simple, strong isolation</td><td>Requires architectural discipline</td></tr><tr><td>Cryptographic hash with full output</td><td>Strong collision resistance</td><td>Slightly higher compute cost</td></tr><tr><td>Explicit collision detection/logging</td><td>Catches issues in production</td><td>Reactive, not preventive</td></tr></tbody></table></figure>



<h2 class="wp-block-heading">FAQs</h2>



<p class="wp-block-paragraph"><strong>Q: Is a cross-query collision the same as a hash collision?</strong> It&#8217;s a consequence of one. A hash collision is the mathematical event; a cross-query collision is the observable system behavior when that event causes two distinct queries to be treated as identical.</p>



<p class="wp-block-paragraph"><strong>Q: Can this happen even with SHA-256, which is considered collision-resistant?</strong> Only if the output is truncated or if the key derivation omits relevant context. Full, untruncated SHA-256 collisions are computationally infeasible with current technology.</p>



<p class="wp-block-paragraph"><strong>Q: Is this only a caching problem?</strong> No — it appears anywhere queries are mapped into a smaller key space: caching, deduplication, rate limiting, sharding, and probabilistic data structures like Bloom filters.</p>



<h2 class="wp-block-heading">Summary and Recommendations</h2>



<p class="wp-block-paragraph">Cross-query collisions are a quiet failure mode that hides in the gap between &#8220;this key function is fast and small&#8221; and &#8220;this key function fully represents the query.&#8221; The fix is almost always the same: don&#8217;t truncate, don&#8217;t omit context, and treat key-derivation logic as a security-relevant design decision rather than an implementation detail.</p>



<p class="wp-block-paragraph"><strong>Further reading:</strong></p>



<ul class="wp-block-list">
<li>OWASP Top 10 — A01:2021 Broken Access Control (cache-based exposure patterns)</li>



<li>NIST SP 800-107 — Recommendation for Applications Using Approved Hash Algorithms</li>



<li>CWE-524: Use of Cache Containing Sensitive Information</li>



<li>RFC 7234 — HTTP/1.1 Caching</li>
</ul>
<p>The post <a href="https://awjunaid.com/cyber-security/what-is-cross-query-collisions/">What Is a Cross-Query Collision? A Deep Dive Into a Quiet but Dangerous Bug Class</a> appeared first on <a href="https://awjunaid.com">Abdul Wahab Junaid</a>.</p>
]]></content:encoded>
					
					<wfw:commentRss>https://awjunaid.com/cyber-security/what-is-cross-query-collisions/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
		<post-id xmlns="com-wordpress:feed-additions:1">9091</post-id>	</item>
		<item>
		<title>Double HMAC: A Defense Against Timing Attacks</title>
		<link>https://awjunaid.com/cyber-security/double-hmac-a-defense-against-timing-attacks/</link>
					<comments>https://awjunaid.com/cyber-security/double-hmac-a-defense-against-timing-attacks/?noamp=mobile#respond</comments>
		
		<dc:creator><![CDATA[Abdul Wahab Junaid]]></dc:creator>
		<pubDate>Sat, 10 Aug 2024 23:31:15 +0000</pubDate>
				<category><![CDATA[Cyber Security]]></category>
		<category><![CDATA[cyber security]]></category>
		<category><![CDATA[dailyprompt]]></category>
		<category><![CDATA[linux]]></category>
		<guid isPermaLink="false">https://awjunaid.com/?p=9083</guid>

					<description><![CDATA[<p>Timing attacks against MAC verification are one of those vulnerabilities that survive in production systems far longer than&#8230;</p>
<p>The post <a href="https://awjunaid.com/cyber-security/double-hmac-a-defense-against-timing-attacks/">Double HMAC: A Defense Against Timing Attacks</a> appeared first on <a href="https://awjunaid.com">Abdul Wahab Junaid</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<p class="wp-block-paragraph">Timing attacks against MAC verification are one of those vulnerabilities that survive in production systems far longer than they should, because the fix — &#8220;use a constant-time comparison&#8221; — sounds trivially simple but gets undermined in practice by network jitter measurement tricks, subtle implementation slips, and language runtimes that optimize away the very constant-time behavior developers thought they wrote. Double HMAC is a defense-in-depth technique that sidesteps a lot of that fragility. This article explains what it is, why it works, and when you&#8217;d actually reach for it over a plain constant-time compare.</p>



<h2 class="wp-block-heading">The Problem Double HMAC Solves</h2>



<p class="wp-block-paragraph">When a server verifies a MAC (Message Authentication Code) — for example, checking a signed webhook payload, an API request signature, or a session token — the naive approach computes the expected MAC and compares it byte-by-byte against the one supplied by the client. If that comparison exits early on the first mismatched byte, an attacker who can measure response timing precisely enough can recover the correct MAC one byte at a time, since a correct prefix takes marginally longer to reject than an incorrect first byte.</p>



<div class="wp-block-merpress-mermaidjs diagram-source-mermaid"><pre class="mermaid">sequenceDiagram
    participant Attacker
    participant Server
    Attacker->>Server: Guess byte 0 = 0x41
    Server-->>Attacker: Reject (fast - byte 0 wrong)
    Attacker->>Server: Guess byte 0 = 0x42
    Server-->>Attacker: Reject (slightly slower - byte 0 right, byte 1 checked)
    Note over Attacker,Server: Repeat per byte, position by position
    Attacker->>Server: Full MAC reconstructed byte-by-byte
</pre></div>



<p class="wp-block-paragraph">This is a well-documented risk (see CWE-208: Observable Timing Discrepancy), and it&#8217;s why security guidance universally recommends constant-time comparison for MAC and password verification. But constant-time comparison functions can still leak timing in subtle ways — compiler optimizations, CPU branch prediction, memory access patterns, or even just variability introduced by memcmp-like functions that aren&#8217;t guaranteed constant-time across all platforms and compiler versions.</p>



<h2 class="wp-block-heading">What Double HMAC Actually Is</h2>



<p class="wp-block-paragraph">The double HMAC construction, described by Coda Hale in a widely cited 2010 write-up on secure comparison, sidesteps the need for a perfectly constant-time byte comparison entirely. Instead of comparing the two MACs directly, you <strong>HMAC both values again</strong> (with a fresh, random key generated per comparison) and compare <em>those</em> results using ordinary equality:</p>



<pre class="wp-block-code"><code>mac1 = HMAC(secret_key, message)
mac2 = &lt;value received from client&gt;

compare_key = random_bytes(32)          # fresh random key, generated per verification
result1 = HMAC(compare_key, mac1)
result2 = HMAC(compare_key, mac2)

return result1 == result2               # ordinary comparison is now safe
</code></pre>



<p class="wp-block-paragraph">The insight is subtle but important: even if the underlying <code>==</code> comparison used in the final step leaks <em>some</em> timing information about where <code>result1</code> and <code>result2</code> differ, that information is now about the output of an HMAC keyed with a value the attacker cannot predict or influence per-request. Any partial match the attacker infers from timing tells them nothing about <code>mac1</code> or <code>mac2</code> — it only tells them about a byte position in an HMAC output that&#8217;s re-randomized every single comparison. There&#8217;s no way to accumulate a multi-request guessing campaign because the target the attacker would be probing changes every time.</p>



<div class="wp-block-merpress-mermaidjs diagram-source-mermaid"><pre class="mermaid">flowchart LR
    A["mac1 = HMAC(secret_key, message)"] --> C[Double HMAC Wrap]
    B["mac2 = client-supplied MAC"] --> C
    C --> D["compare_key = fresh random bytes"]
    D --> E["result1 = HMAC(compare_key, mac1)"]
    D --> F["result2 = HMAC(compare_key, mac2)"]
    E --> G{result1 == result2 ?}
    F --> G
    G -->|Any timing leak here| H[Leak reveals nothing reusable - key changes every call]
</pre></div>



<h2 class="wp-block-heading">Why This Beats a Naive Constant-Time Compare in Some Environments</h2>



<p class="wp-block-paragraph">A hand-rolled constant-time comparison function is only as good as its implementation and the guarantees of the language runtime it&#8217;s written in. Managed languages with JIT compilers, garbage collection pauses, or aggressive optimization passes can introduce timing variance that a developer never intended and can&#8217;t fully control. Double HMAC changes the <em>security argument</em> rather than depending on perfect implementation discipline: even if the final comparison isn&#8217;t perfectly constant-time, the information it could leak is cryptographically useless to the attacker because it&#8217;s tied to a single-use random key.</p>



<p class="wp-block-paragraph">That said, double HMAC isn&#8217;t a replacement for good hygiene — it&#8217;s defense in depth. You should still use your language&#8217;s built-in constant-time comparison function for the final <code>result1 == result2</code> check where available; double HMAC just means that even if that guarantee has some slippage, exploitation is blocked at the cryptographic layer instead of relying purely on the implementation layer.</p>



<h2 class="wp-block-heading">Reference Implementation Pattern</h2>



<pre class="wp-block-code"><code>import hmac
import os
import hashlib

def secure_compare_double_hmac(mac1: bytes, mac2: bytes) -&gt; bool:
    compare_key = os.urandom(32)  # fresh per call - never reused, never logged
    r1 = hmac.new(compare_key, mac1, hashlib.sha256).digest()
    r2 = hmac.new(compare_key, mac2, hashlib.sha256).digest()
    return hmac.compare_digest(r1, r2)  # still use constant-time compare as belt-and-suspenders
</code></pre>



<p class="wp-block-paragraph">Note the implementation still uses <code>hmac.compare_digest</code> for the final step — double HMAC is layered defense, not a replacement for constant-time comparison primitives that are already available and well-tested in your language.</p>



<h2 class="wp-block-heading">When You Actually Need This</h2>



<figure class="wp-block-table"><table class="has-fixed-layout"><thead><tr><th>Scenario</th><th>Plain constant-time compare</th><th>Double HMAC recommended</th></tr></thead><tbody><tr><td>Comparing MACs in a mature language with a trusted constant-time primitive (Python <code>hmac.compare_digest</code>, Go <code>subtle.ConstantTimeCompare</code>)</td><td>Sufficient</td><td>Optional extra hardening</td></tr><tr><td>Custom/embedded environment without a vetted constant-time compare</td><td>Risky</td><td>Strongly recommended</td></tr><tr><td>High-value target (payment webhooks, signed admin tokens)</td><td>Sufficient but add defense-in-depth</td><td>Recommended</td></tr><tr><td>Extremely latency-sensitive, high-throughput internal service with negligible attacker network access</td><td>Sufficient</td><td>Usually unnecessary overhead</td></tr></tbody></table></figure>



<h2 class="wp-block-heading">Comparing Approaches to Timing-Safe MAC Verification</h2>



<figure class="wp-block-table"><table class="has-fixed-layout"><thead><tr><th>Approach</th><th>Security Basis</th><th>Overhead</th><th>Implementation Risk</th></tr></thead><tbody><tr><td>Naive <code>==</code> comparison</td><td>None — vulnerable</td><td>None</td><td>High (actively exploitable)</td></tr><tr><td>Constant-time compare function</td><td>Implementation guarantees fixed-time execution</td><td>Negligible</td><td>Medium (depends on language/runtime correctness)</td></tr><tr><td>Double HMAC</td><td>Cryptographic randomization neutralizes any leak</td><td>One extra HMAC computation per side</td><td>Low</td></tr><tr><td>Response delay padding</td><td>Obscures timing via added latency</td><td>Adds real latency to every request</td><td>Low security value; easily defeated by averaging many requests</td></tr></tbody></table></figure>



<h2 class="wp-block-heading">Common Mistakes</h2>



<ul class="wp-block-list">
<li>Reusing the same <code>compare_key</code> across multiple verification calls — this destroys the security property entirely, since a fixed key turns the double HMAC into just another comparison an attacker can probe repeatedly.</li>



<li>Logging or caching the <code>compare_key</code> — it must be ephemeral and discarded immediately after use.</li>



<li>Using a weak or predictable random source for <code>compare_key</code> (never use non-cryptographic RNGs like <code>random.random()</code> in Python — always use <code>os.urandom</code> or the language&#8217;s CSPRNG).</li>



<li>Believing double HMAC removes the need for constant-time comparison altogether — it complements it, it doesn&#8217;t replace defensive coding practice at the final step.</li>
</ul>



<h2 class="wp-block-heading">FAQs</h2>



<p class="wp-block-paragraph"><strong>Q: Does double HMAC eliminate the need for a constant-time compare function?</strong> No. It&#8217;s best used together with one. Double HMAC changes what a timing leak could reveal (nothing useful, because the key rotates every call); it doesn&#8217;t guarantee the final comparison itself has zero timing variance.</p>



<p class="wp-block-paragraph"><strong>Q: Is double HMAC standardized in any RFC?</strong> It&#8217;s a well-known engineering pattern described by security practitioners (notably Coda Hale&#8217;s widely referenced writeup) rather than a formal IETF standard, but it&#8217;s built entirely from standard, well-vetted primitives (HMAC, per RFC 2104).</p>



<p class="wp-block-paragraph"><strong>Q: Does this protect against anything other than timing attacks?</strong> Its primary purpose is timing-attack resistance for comparison operations. It doesn&#8217;t address other side channels like power or cache-timing analysis, which require separate countermeasures.</p>



<h2 class="wp-block-heading">Summary and Recommendations</h2>



<p class="wp-block-paragraph">Double HMAC is a small, cheap, and elegant defense-in-depth technique: instead of trying to make a byte comparison perfectly constant-time (which is harder than it sounds across languages and runtimes), it re-keys both values with a fresh random HMAC per verification, so any residual timing leak reveals nothing an attacker can reuse. Use it alongside your language&#8217;s native constant-time comparison function, especially for high-value verification endpoints like webhook signatures and API request authentication.</p>



<p class="wp-block-paragraph"><strong>Further reading:</strong></p>



<ul class="wp-block-list">
<li>RFC 2104 — HMAC: Keyed-Hashing for Message Authentication</li>



<li>CWE-208: Observable Timing Discrepancy</li>



<li>OWASP Cheat Sheet Series — Authentication Cheat Sheet (timing-safe comparison guidance)</li>



<li>NIST SP 800-107 — Recommendation for Applications Using Approved Hash Algorithms</li>
</ul>
<p>The post <a href="https://awjunaid.com/cyber-security/double-hmac-a-defense-against-timing-attacks/">Double HMAC: A Defense Against Timing Attacks</a> appeared first on <a href="https://awjunaid.com">Abdul Wahab Junaid</a>.</p>
]]></content:encoded>
					
					<wfw:commentRss>https://awjunaid.com/cyber-security/double-hmac-a-defense-against-timing-attacks/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
		<post-id xmlns="com-wordpress:feed-additions:1">9083</post-id>	</item>
		<item>
		<title>Steganography techniques for hiding information in images</title>
		<link>https://awjunaid.com/python/steganography-techniques-for-hiding-information-in-images/</link>
					<comments>https://awjunaid.com/python/steganography-techniques-for-hiding-information-in-images/?noamp=mobile#respond</comments>
		
		<dc:creator><![CDATA[Abdul Wahab Junaid]]></dc:creator>
		<pubDate>Wed, 27 Dec 2023 23:33:55 +0000</pubDate>
				<category><![CDATA[Networking]]></category>
		<category><![CDATA[Python]]></category>
		<category><![CDATA[dailyprompt]]></category>
		<category><![CDATA[networking]]></category>
		<category><![CDATA[python]]></category>
		<guid isPermaLink="false">https://awjunaid.com/?p=8895</guid>

					<description><![CDATA[<p>Introduction to Steganography: Hiding in Plain Sight Steganography, derived from the Greek words &#8220;steganos&#8221; (covered) and &#8220;graphein&#8221; (to&#8230;</p>
<p>The post <a href="https://awjunaid.com/python/steganography-techniques-for-hiding-information-in-images/">Steganography techniques for hiding information in images</a> appeared first on <a href="https://awjunaid.com">Abdul Wahab Junaid</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<h2 class="wp-block-heading">Introduction to Steganography: Hiding in Plain Sight</h2>



<p class="wp-block-paragraph">Steganography, derived from the Greek words &#8220;steganos&#8221; (covered) and &#8220;graphein&#8221; (to write), is the art and science of concealing information within another object. Unlike its counterpart, cryptography, which scrambles information to make it unreadable, steganography hides the presence of the message itself. It&#8217;s like whispering a secret within a bustling marketplace, hoping no one notices.</p>



<h3 class="wp-block-heading">key concepts:</h3>



<p class="wp-block-paragraph"><strong>1. Hidden Data:</strong> This can be anything you want to keep secret: text, images, audio, video, or even code.</p>



<p class="wp-block-paragraph"><strong>2. Cover Object:</strong> This is the seemingly innocuous carrier of the hidden information. It could be a digital image, video file, audio track, text document, or even a physical object like a book or painting.</p>



<p class="wp-block-paragraph"><strong>3. Embedding Techniques:</strong> These are the clever ways to hide the data within the cover object. Some common techniques include:</p>



<ul class="wp-block-list">
<li><strong>Least Significant Bit (LSB) Modification:</strong> Replacing the least significant bits of pixels in an image with information bits.</li>



<li><strong>Modulation Methods:</strong> Embedding data by slightly modifying the frequencies of audio or video signals.</li>



<li><strong>Meta-data Manipulation:</strong> Hiding information within unused or custom fields of file formats.</li>



<li><strong>Cover Object Replacement:</strong> Substituting seemingly insignificant parts of the cover object with encoded data.</li>
</ul>



<p class="wp-block-paragraph"><strong>4. Extracting Techniques:</strong> These are the methods used to retrieve the hidden information from the cover object. They depend on the specific embedding technique used, and often require knowledge of the encoding process.</p>



<p class="wp-block-paragraph"><strong>5. Applications of Steganography:</strong></p>



<ul class="wp-block-list">
<li><strong>Secure Communication:</strong> Secretly exchanging messages while avoiding detection.</li>



<li><strong>Copyright Protection:</strong> Embedding copyright information within multimedia content.</li>



<li><strong>Data Redundancy:</strong> Hiding additional information within existing data for disaster recovery purposes.</li>



<li><strong>Digital Watermarking:</strong> Identifying the ownership of digital content imperceptibly.</li>
</ul>



<p class="wp-block-paragraph"><strong>6. Ethical Considerations:</strong></p>



<p class="wp-block-paragraph">Steganography is a powerful tool, but it comes with significant ethical considerations. Its use for illegal activities like terrorism or cybercrime must be condemned. It&#8217;s crucial to use steganography responsibly and within legal boundaries.</p>



<h2 class="wp-block-heading">Steganography with LSB</h2>



<div class="wp-block-kevinbatdorf-code-block-pro cbp-has-line-numbers cbp-highlight-hover" data-code-block-pro-font-family="Code-Pro-JetBrains-Mono" style="font-size:.875rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#F8F8F2;--cbp-line-number-width:calc(2 * 0.6 * .875rem);--cbp-line-highlight-color:rgba(253, 253, 237, 0.2);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)"><span style="display:flex;align-items:center;padding:10px 0px 10px 16px;margin-bottom:-2px;width:100%;text-align:left;background-color:#34362e;color:#efefe1">Python</span><span role="button" tabindex="0" data-code="#!/usr/bin/env python

from PIL import Image

def set_LSB(value, bit):
    if bit == '0':
        value = value &amp; 254
    else:
        value = value | 1
    return value

def get_LSB(value):
    if value &amp; 1 == 0:
        return '0'
    else:
        return '1'

def get_pixel_pairs(iterable):
    a = iter(iterable)
    return zip(a, a)


def extract_message(image):
 
    c_image = Image.open(image)
    pixel_list = list(c_image.getdata())
    message = &quot;&quot;

    for pix1, pix2 in get_pixel_pairs(pixel_list):
        message_byte = &quot;0b&quot;
        for p in pix1:
            message_byte += get_LSB(p)

        for p in pix2:
            message_byte += get_LSB(p)
            
        if message_byte == &quot;0b00000000&quot;:
            break

        message += chr(int(message_byte,2))

    return message
    
def hide_message(image, message, outfile):

    message += chr(0)
    c_image = Image.open(image)
    c_image = c_image.convert('RGBA')
    out = Image.new(c_image.mode, c_image.size)
    width, height = c_image.size
    pixList = list(c_image.getdata())
    newArray = []
    
    for i in range(len(message)):
        charInt = ord(message[i])
        cb = str(bin(charInt))[2:].zfill(8)
        pix1 = pixList[i*2]
        pix2 = pixList[(i*2)+1]
        newpix1 = []
        newpix2 = []

        for j in range(0,4):
            newpix1.append(set_LSB(pix1[j], cb[j]))
            newpix2.append(set_LSB(pix2[j], cb[j+4]))

        newArray.append(tuple(newpix1))
        newArray.append(tuple(newpix2))

    newArray.extend(pixList[len(message)*2:])
    
    out.putdata(newArray)
    out.save(outfile)
    return outfile   

	
if __name__ == &quot;__main__&quot;:
    print(&quot;Testing hide message in python_secrets.png with LSB ...&quot;)
    print(hide_message('python.png', 'Hidden message', 'python_secrets.png'))
    print(&quot;Hide test passed, testing message extraction ...&quot;)
    print(extract_message('python_secrets.png'))
" style="color:#F8F8F2;display:none" aria-label="Copy" class="code-block-pro-copy-button"><svg xmlns="http://www.w3.org/2000/svg" style="width:24px;height:24px" fill="none" viewBox="0 0 24 24" stroke="currentColor" stroke-width="2"><path class="with-check" stroke-linecap="round" stroke-linejoin="round" d="M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2m-6 9l2 2 4-4"></path><path class="without-check" stroke-linecap="round" stroke-linejoin="round" d="M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2"></path></svg></span><pre class="shiki monokai" style="background-color: #272822" tabindex="0"><code><span class="line"><span style="color: #88846F">#!/usr/bin/env python</span></span>
<span class="line"></span>
<span class="line"><span style="color: #F92672">from</span><span style="color: #F8F8F2"> </span><span style="color: #AE81FF">PIL</span><span style="color: #F8F8F2"> </span><span style="color: #F92672">import</span><span style="color: #F8F8F2"> Image</span></span>
<span class="line"></span>
<span class="line"><span style="color: #66D9EF; font-style: italic">def</span><span style="color: #F8F8F2"> </span><span style="color: #A6E22E">set_LSB</span><span style="color: #F8F8F2">(</span><span style="color: #FD971F; font-style: italic">value</span><span style="color: #F8F8F2">, </span><span style="color: #FD971F; font-style: italic">bit</span><span style="color: #F8F8F2">):</span></span>
<span class="line"><span style="color: #F8F8F2">    </span><span style="color: #F92672">if</span><span style="color: #F8F8F2"> bit </span><span style="color: #F92672">==</span><span style="color: #F8F8F2"> </span><span style="color: #E6DB74">&#39;0&#39;</span><span style="color: #F8F8F2">:</span></span>
<span class="line"><span style="color: #F8F8F2">        value </span><span style="color: #F92672">=</span><span style="color: #F8F8F2"> value </span><span style="color: #F92672">&amp;</span><span style="color: #F8F8F2"> </span><span style="color: #AE81FF">254</span></span>
<span class="line"><span style="color: #F8F8F2">    </span><span style="color: #F92672">else</span><span style="color: #F8F8F2">:</span></span>
<span class="line"><span style="color: #F8F8F2">        value </span><span style="color: #F92672">=</span><span style="color: #F8F8F2"> value </span><span style="color: #F92672">|</span><span style="color: #F8F8F2"> </span><span style="color: #AE81FF">1</span></span>
<span class="line"><span style="color: #F8F8F2">    </span><span style="color: #F92672">return</span><span style="color: #F8F8F2"> value</span></span>
<span class="line"></span>
<span class="line"><span style="color: #66D9EF; font-style: italic">def</span><span style="color: #F8F8F2"> </span><span style="color: #A6E22E">get_LSB</span><span style="color: #F8F8F2">(</span><span style="color: #FD971F; font-style: italic">value</span><span style="color: #F8F8F2">):</span></span>
<span class="line"><span style="color: #F8F8F2">    </span><span style="color: #F92672">if</span><span style="color: #F8F8F2"> value </span><span style="color: #F92672">&amp;</span><span style="color: #F8F8F2"> </span><span style="color: #AE81FF">1</span><span style="color: #F8F8F2"> </span><span style="color: #F92672">==</span><span style="color: #F8F8F2"> </span><span style="color: #AE81FF">0</span><span style="color: #F8F8F2">:</span></span>
<span class="line"><span style="color: #F8F8F2">        </span><span style="color: #F92672">return</span><span style="color: #F8F8F2"> </span><span style="color: #E6DB74">&#39;0&#39;</span></span>
<span class="line"><span style="color: #F8F8F2">    </span><span style="color: #F92672">else</span><span style="color: #F8F8F2">:</span></span>
<span class="line"><span style="color: #F8F8F2">        </span><span style="color: #F92672">return</span><span style="color: #F8F8F2"> </span><span style="color: #E6DB74">&#39;1&#39;</span></span>
<span class="line"></span>
<span class="line"><span style="color: #66D9EF; font-style: italic">def</span><span style="color: #F8F8F2"> </span><span style="color: #A6E22E">get_pixel_pairs</span><span style="color: #F8F8F2">(</span><span style="color: #FD971F; font-style: italic">iterable</span><span style="color: #F8F8F2">):</span></span>
<span class="line"><span style="color: #F8F8F2">    a </span><span style="color: #F92672">=</span><span style="color: #F8F8F2"> </span><span style="color: #66D9EF">iter</span><span style="color: #F8F8F2">(iterable)</span></span>
<span class="line"><span style="color: #F8F8F2">    </span><span style="color: #F92672">return</span><span style="color: #F8F8F2"> </span><span style="color: #66D9EF">zip</span><span style="color: #F8F8F2">(a, a)</span></span>
<span class="line"></span>
<span class="line"></span>
<span class="line"><span style="color: #66D9EF; font-style: italic">def</span><span style="color: #F8F8F2"> </span><span style="color: #A6E22E">extract_message</span><span style="color: #F8F8F2">(</span><span style="color: #FD971F; font-style: italic">image</span><span style="color: #F8F8F2">):</span></span>
<span class="line"><span style="color: #F8F8F2"> </span></span>
<span class="line"><span style="color: #F8F8F2">    c_image </span><span style="color: #F92672">=</span><span style="color: #F8F8F2"> Image.open(image)</span></span>
<span class="line"><span style="color: #F8F8F2">    pixel_list </span><span style="color: #F92672">=</span><span style="color: #F8F8F2"> </span><span style="color: #66D9EF; font-style: italic">list</span><span style="color: #F8F8F2">(c_image.getdata())</span></span>
<span class="line"><span style="color: #F8F8F2">    message </span><span style="color: #F92672">=</span><span style="color: #F8F8F2"> </span><span style="color: #E6DB74">&quot;&quot;</span></span>
<span class="line"></span>
<span class="line"><span style="color: #F8F8F2">    </span><span style="color: #F92672">for</span><span style="color: #F8F8F2"> pix1, pix2 </span><span style="color: #F92672">in</span><span style="color: #F8F8F2"> get_pixel_pairs(pixel_list):</span></span>
<span class="line"><span style="color: #F8F8F2">        message_byte </span><span style="color: #F92672">=</span><span style="color: #F8F8F2"> </span><span style="color: #E6DB74">&quot;0b&quot;</span></span>
<span class="line"><span style="color: #F8F8F2">        </span><span style="color: #F92672">for</span><span style="color: #F8F8F2"> p </span><span style="color: #F92672">in</span><span style="color: #F8F8F2"> pix1:</span></span>
<span class="line"><span style="color: #F8F8F2">            message_byte </span><span style="color: #F92672">+=</span><span style="color: #F8F8F2"> get_LSB(p)</span></span>
<span class="line"></span>
<span class="line"><span style="color: #F8F8F2">        </span><span style="color: #F92672">for</span><span style="color: #F8F8F2"> p </span><span style="color: #F92672">in</span><span style="color: #F8F8F2"> pix2:</span></span>
<span class="line"><span style="color: #F8F8F2">            message_byte </span><span style="color: #F92672">+=</span><span style="color: #F8F8F2"> get_LSB(p)</span></span>
<span class="line"><span style="color: #F8F8F2">            </span></span>
<span class="line"><span style="color: #F8F8F2">        </span><span style="color: #F92672">if</span><span style="color: #F8F8F2"> message_byte </span><span style="color: #F92672">==</span><span style="color: #F8F8F2"> </span><span style="color: #E6DB74">&quot;0b00000000&quot;</span><span style="color: #F8F8F2">:</span></span>
<span class="line"><span style="color: #F8F8F2">            </span><span style="color: #F92672">break</span></span>
<span class="line"></span>
<span class="line"><span style="color: #F8F8F2">        message </span><span style="color: #F92672">+=</span><span style="color: #F8F8F2"> </span><span style="color: #66D9EF">chr</span><span style="color: #F8F8F2">(</span><span style="color: #66D9EF; font-style: italic">int</span><span style="color: #F8F8F2">(message_byte,</span><span style="color: #AE81FF">2</span><span style="color: #F8F8F2">))</span></span>
<span class="line"></span>
<span class="line"><span style="color: #F8F8F2">    </span><span style="color: #F92672">return</span><span style="color: #F8F8F2"> message</span></span>
<span class="line"><span style="color: #F8F8F2">    </span></span>
<span class="line"><span style="color: #66D9EF; font-style: italic">def</span><span style="color: #F8F8F2"> </span><span style="color: #A6E22E">hide_message</span><span style="color: #F8F8F2">(</span><span style="color: #FD971F; font-style: italic">image</span><span style="color: #F8F8F2">, </span><span style="color: #FD971F; font-style: italic">message</span><span style="color: #F8F8F2">, </span><span style="color: #FD971F; font-style: italic">outfile</span><span style="color: #F8F8F2">):</span></span>
<span class="line"></span>
<span class="line"><span style="color: #F8F8F2">    message </span><span style="color: #F92672">+=</span><span style="color: #F8F8F2"> </span><span style="color: #66D9EF">chr</span><span style="color: #F8F8F2">(</span><span style="color: #AE81FF">0</span><span style="color: #F8F8F2">)</span></span>
<span class="line"><span style="color: #F8F8F2">    c_image </span><span style="color: #F92672">=</span><span style="color: #F8F8F2"> Image.open(image)</span></span>
<span class="line"><span style="color: #F8F8F2">    c_image </span><span style="color: #F92672">=</span><span style="color: #F8F8F2"> c_image.convert(</span><span style="color: #E6DB74">&#39;RGBA&#39;</span><span style="color: #F8F8F2">)</span></span>
<span class="line"><span style="color: #F8F8F2">    out </span><span style="color: #F92672">=</span><span style="color: #F8F8F2"> Image.new(c_image.mode, c_image.size)</span></span>
<span class="line"><span style="color: #F8F8F2">    width, height </span><span style="color: #F92672">=</span><span style="color: #F8F8F2"> c_image.size</span></span>
<span class="line"><span style="color: #F8F8F2">    pixList </span><span style="color: #F92672">=</span><span style="color: #F8F8F2"> </span><span style="color: #66D9EF; font-style: italic">list</span><span style="color: #F8F8F2">(c_image.getdata())</span></span>
<span class="line"><span style="color: #F8F8F2">    newArray </span><span style="color: #F92672">=</span><span style="color: #F8F8F2"> []</span></span>
<span class="line"><span style="color: #F8F8F2">    </span></span>
<span class="line"><span style="color: #F8F8F2">    </span><span style="color: #F92672">for</span><span style="color: #F8F8F2"> i </span><span style="color: #F92672">in</span><span style="color: #F8F8F2"> </span><span style="color: #66D9EF">range</span><span style="color: #F8F8F2">(</span><span style="color: #66D9EF">len</span><span style="color: #F8F8F2">(message)):</span></span>
<span class="line"><span style="color: #F8F8F2">        charInt </span><span style="color: #F92672">=</span><span style="color: #F8F8F2"> </span><span style="color: #66D9EF">ord</span><span style="color: #F8F8F2">(message[i])</span></span>
<span class="line"><span style="color: #F8F8F2">        cb </span><span style="color: #F92672">=</span><span style="color: #F8F8F2"> </span><span style="color: #66D9EF; font-style: italic">str</span><span style="color: #F8F8F2">(</span><span style="color: #66D9EF">bin</span><span style="color: #F8F8F2">(charInt))[</span><span style="color: #AE81FF">2</span><span style="color: #F8F8F2">:].zfill(</span><span style="color: #AE81FF">8</span><span style="color: #F8F8F2">)</span></span>
<span class="line"><span style="color: #F8F8F2">        pix1 </span><span style="color: #F92672">=</span><span style="color: #F8F8F2"> pixList[i</span><span style="color: #F92672">*</span><span style="color: #AE81FF">2</span><span style="color: #F8F8F2">]</span></span>
<span class="line"><span style="color: #F8F8F2">        pix2 </span><span style="color: #F92672">=</span><span style="color: #F8F8F2"> pixList[(i</span><span style="color: #F92672">*</span><span style="color: #AE81FF">2</span><span style="color: #F8F8F2">)</span><span style="color: #F92672">+</span><span style="color: #AE81FF">1</span><span style="color: #F8F8F2">]</span></span>
<span class="line"><span style="color: #F8F8F2">        newpix1 </span><span style="color: #F92672">=</span><span style="color: #F8F8F2"> []</span></span>
<span class="line"><span style="color: #F8F8F2">        newpix2 </span><span style="color: #F92672">=</span><span style="color: #F8F8F2"> []</span></span>
<span class="line"></span>
<span class="line"><span style="color: #F8F8F2">        </span><span style="color: #F92672">for</span><span style="color: #F8F8F2"> j </span><span style="color: #F92672">in</span><span style="color: #F8F8F2"> </span><span style="color: #66D9EF">range</span><span style="color: #F8F8F2">(</span><span style="color: #AE81FF">0</span><span style="color: #F8F8F2">,</span><span style="color: #AE81FF">4</span><span style="color: #F8F8F2">):</span></span>
<span class="line"><span style="color: #F8F8F2">            newpix1.append(set_LSB(pix1[j], cb[j]))</span></span>
<span class="line"><span style="color: #F8F8F2">            newpix2.append(set_LSB(pix2[j], cb[j</span><span style="color: #F92672">+</span><span style="color: #AE81FF">4</span><span style="color: #F8F8F2">]))</span></span>
<span class="line"></span>
<span class="line"><span style="color: #F8F8F2">        newArray.append(</span><span style="color: #66D9EF; font-style: italic">tuple</span><span style="color: #F8F8F2">(newpix1))</span></span>
<span class="line"><span style="color: #F8F8F2">        newArray.append(</span><span style="color: #66D9EF; font-style: italic">tuple</span><span style="color: #F8F8F2">(newpix2))</span></span>
<span class="line"></span>
<span class="line"><span style="color: #F8F8F2">    newArray.extend(pixList[</span><span style="color: #66D9EF">len</span><span style="color: #F8F8F2">(message)</span><span style="color: #F92672">*</span><span style="color: #AE81FF">2</span><span style="color: #F8F8F2">:])</span></span>
<span class="line"><span style="color: #F8F8F2">    </span></span>
<span class="line"><span style="color: #F8F8F2">    out.putdata(newArray)</span></span>
<span class="line"><span style="color: #F8F8F2">    out.save(outfile)</span></span>
<span class="line"><span style="color: #F8F8F2">    </span><span style="color: #F92672">return</span><span style="color: #F8F8F2"> outfile   </span></span>
<span class="line"></span>
<span class="line"><span style="color: #F8F8F2">	</span></span>
<span class="line"><span style="color: #F92672">if</span><span style="color: #F8F8F2"> __name__ </span><span style="color: #F92672">==</span><span style="color: #F8F8F2"> </span><span style="color: #E6DB74">&quot;__main__&quot;</span><span style="color: #F8F8F2">:</span></span>
<span class="line"><span style="color: #F8F8F2">    </span><span style="color: #66D9EF">print</span><span style="color: #F8F8F2">(</span><span style="color: #E6DB74">&quot;Testing hide message in python_secrets.png with LSB ...&quot;</span><span style="color: #F8F8F2">)</span></span>
<span class="line"><span style="color: #F8F8F2">    </span><span style="color: #66D9EF">print</span><span style="color: #F8F8F2">(hide_message(</span><span style="color: #E6DB74">&#39;python.png&#39;</span><span style="color: #F8F8F2">, </span><span style="color: #E6DB74">&#39;Hidden message&#39;</span><span style="color: #F8F8F2">, </span><span style="color: #E6DB74">&#39;python_secrets.png&#39;</span><span style="color: #F8F8F2">))</span></span>
<span class="line"><span style="color: #F8F8F2">    </span><span style="color: #66D9EF">print</span><span style="color: #F8F8F2">(</span><span style="color: #E6DB74">&quot;Hide test passed, testing message extraction ...&quot;</span><span style="color: #F8F8F2">)</span></span>
<span class="line"><span style="color: #F8F8F2">    </span><span style="color: #66D9EF">print</span><span style="color: #F8F8F2">(extract_message(</span><span style="color: #E6DB74">&#39;python_secrets.png&#39;</span><span style="color: #F8F8F2">))</span></span>
<span class="line"></span></code></pre></div>



<div class="wp-block-jetpack-markdown"><h3>Explanation of the functions and the main code:</h3>
<ol>
<li>
<p><strong><code>set_LSB(value, bit)</code> function:</strong></p>
<ul>
<li>This function takes a pixel value (<code>value</code>) and a bit (<code>bit</code>) as parameters.</li>
<li>It modifies the least significant bit of the pixel value based on the provided bit (0 or 1) and returns the updated value.</li>
</ul>
</li>
<li>
<p><strong><code>get_LSB(value)</code> function:</strong></p>
<ul>
<li>This function takes a pixel value (<code>value</code>) as a parameter.</li>
<li>It extracts the least significant bit from the pixel value and returns it as a string (‘0’ or ‘1’).</li>
</ul>
</li>
<li>
<p><strong><code>get_pixel_pairs(iterable)</code> function:</strong></p>
<ul>
<li>This function takes an iterable and returns an iterator that generates pairs of elements from the iterable.</li>
<li>It’s used to iterate over pairs of pixels in the image.</li>
</ul>
</li>
<li>
<p><strong><code>extract_message(image)</code> function:</strong></p>
<ul>
<li>This function extracts a hidden message from an image that was previously created using LSB steganography.</li>
<li>It opens the image, retrieves the pixel data, and iterates over pairs of pixels.</li>
<li>For each pair, it extracts the LSBs from each pixel’s color channel to reconstruct the hidden message.</li>
<li>The message is terminated when ‘00000000’ (null terminator) is encountered.</li>
</ul>
</li>
<li>
<p><strong><code>hide_message(image, message, outfile)</code> function:</strong></p>
<ul>
<li>This function hides a message in an image using LSB steganography and saves the result to an output file.</li>
<li>It opens the original image, converts it to RGBA mode, and creates a new image for the output.</li>
<li>It converts each character of the message to its binary representation and replaces the LSBs of pairs of pixels with the message bits.</li>
<li>The resulting image is saved to the specified output file.</li>
</ul>
</li>
<li>
<p><strong>Main Code:</strong></p>
<ul>
<li>The script tests the <code>hide_message</code> function by hiding the message “Hidden message” in the ‘python.png’ image and saving the result as ‘python_secrets.png’.</li>
<li>It then tests the <code>extract_message</code> function by attempting to extract the hidden message from ‘python_secrets.png’.</li>
</ul>
</li>
</ol>
</div>



<h2 class="wp-block-heading">Steganography with Stegano</h2>



<div class="wp-block-jetpack-markdown"><p>Stegano is a versatile Python library for <strong>image steganography</strong>, offering features to hide and reveal secret messages within image files. Here’s a breakdown of its functionalities and a brief guide to get you started:</p>
<p><strong>Stegano Capabilities:</strong></p>
<ul>
<li><strong>Text Hiding:</strong> Embed plaintext messages within images using LSB (Least Significant Bit) steganography.</li>
<li><strong>File Hiding:</strong> Conceal files of any format within images.</li>
<li><strong>Password Protection:</strong> Encrypt hidden information with a password for added security.</li>
<li><strong>Multiple Image Formats:</strong> Supports various image formats like PNG, JPEG, BMP, and more.</li>
<li><strong>Command-Line Interface:</strong> Easy to use through command-line commands for encryption and decryption.</li>
</ul>
<p><strong>Using Stegano:</strong></p>
<ol>
<li>
<p><strong>Installation:</strong> Ensure you have Python and Stegano installed: <code>pip install stegano</code></p>
</li>
<li>
<p><strong>Hiding a Message:</strong></p>
</li>
</ol>
<pre><code class="language-python"># Hide text message &quot;This is a secret!&quot; in image &quot;photo.jpg&quot; with password &quot;mypassword&quot;
stegano -e -p mypassword -m &quot;This is a secret!&quot; photo.jpg secret_photo.jpg
</code></pre>
<ol start="3">
<li><strong>Revealing the Message:</strong></li>
</ol>
<pre><code class="language-python"># Extract hidden message from &quot;secret_photo.jpg&quot; using password &quot;mypassword&quot;
stegano -d -p mypassword secret_photo.jpg
</code></pre>
<ol start="4">
<li><strong>Hiding a File:</strong></li>
</ol>
<pre><code class="language-python"># Hide file &quot;secret_file.txt&quot; in image &quot;cover.jpg&quot; with no password
stegano -ef cover.jpg secret_file.txt stego_image.jpg
</code></pre>
<ol start="5">
<li><strong>Extracting a File:</strong></li>
</ol>
<pre><code class="language-python"># Extract hidden file from &quot;stego_image.jpg&quot;
stegano -df stego_image.jpg extracted_file.txt
</code></pre>
<p><strong>Additional Notes:</strong></p>
<ul>
<li>Image quality may slightly degrade after embedding data, depending on the size of the hidden information.</li>
<li>Stegano works best with larger, complex images containing more redundant data for hiding bits.</li>
<li>Consider password complexity and security practices when using password protection.</li>
<li>Remember, steganography is not a foolproof security measure. Advanced steganalysis techniques may be able to detect hidden messages.</li>
</ul>
</div>



<h2 class="wp-block-heading">Steganography with stepic</h2>



<p class="wp-block-paragraph"><strong>Stepic Crypto Features:</strong></p>



<ul class="wp-block-list">
<li><strong>Text Hiding:</strong>&nbsp;Embed text messages within images using LSB (Least Significant Bit) steganography.</li>



<li><strong>Image Hiding:</strong>&nbsp;Conceal smaller images within larger ones.</li>



<li><strong>Noise Addition:</strong>&nbsp;Optionally add noise to the cover image to further mask the hidden data.</li>



<li><strong>Multiple Image Formats:</strong>&nbsp;Supports formats like PNG, JPEG, BMP, and more.</li>



<li><strong>API Integration:</strong>&nbsp;Designed for integration into Python applications and projects.</li>
</ul>



<div class="wp-block-kevinbatdorf-code-block-pro cbp-has-line-numbers cbp-highlight-hover" data-code-block-pro-font-family="Code-Pro-JetBrains-Mono" style="font-size:.875rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#F8F8F2;--cbp-line-number-width:calc(1 * 0.6 * .875rem);--cbp-line-highlight-color:rgba(253, 253, 237, 0.2);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)"><span style="display:flex;align-items:center;padding:10px 0px 10px 16px;margin-bottom:-2px;width:100%;text-align:left;background-color:#34362e;color:#efefe1">Python</span><span role="button" tabindex="0" data-code="from PIL import Image
import stepic

image = Image.open(&quot;python.png&quot;)
image2 = stepic.encode(image, 'This is the hidden text'.encode(&quot;utf8&quot;))
image2.save('python_secrets.png','PNG')
image2 = Image.open('python_secrets.png')
data = stepic.decode(image2) 
print(&quot;Decoded data: &quot; + data)" style="color:#F8F8F2;display:none" aria-label="Copy" class="code-block-pro-copy-button"><svg xmlns="http://www.w3.org/2000/svg" style="width:24px;height:24px" fill="none" viewBox="0 0 24 24" stroke="currentColor" stroke-width="2"><path class="with-check" stroke-linecap="round" stroke-linejoin="round" d="M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2m-6 9l2 2 4-4"></path><path class="without-check" stroke-linecap="round" stroke-linejoin="round" d="M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2"></path></svg></span><pre class="shiki monokai" style="background-color: #272822" tabindex="0"><code><span class="line"><span style="color: #F92672">from</span><span style="color: #F8F8F2"> </span><span style="color: #AE81FF">PIL</span><span style="color: #F8F8F2"> </span><span style="color: #F92672">import</span><span style="color: #F8F8F2"> Image</span></span>
<span class="line"><span style="color: #F92672">import</span><span style="color: #F8F8F2"> stepic</span></span>
<span class="line"></span>
<span class="line"><span style="color: #F8F8F2">image </span><span style="color: #F92672">=</span><span style="color: #F8F8F2"> Image.open(</span><span style="color: #E6DB74">&quot;python.png&quot;</span><span style="color: #F8F8F2">)</span></span>
<span class="line"><span style="color: #F8F8F2">image2 </span><span style="color: #F92672">=</span><span style="color: #F8F8F2"> stepic.encode(image, </span><span style="color: #E6DB74">&#39;This is the hidden text&#39;</span><span style="color: #F8F8F2">.encode(</span><span style="color: #E6DB74">&quot;utf8&quot;</span><span style="color: #F8F8F2">))</span></span>
<span class="line"><span style="color: #F8F8F2">image2.save(</span><span style="color: #E6DB74">&#39;python_secrets.png&#39;</span><span style="color: #F8F8F2">,</span><span style="color: #E6DB74">&#39;PNG&#39;</span><span style="color: #F8F8F2">)</span></span>
<span class="line"><span style="color: #F8F8F2">image2 </span><span style="color: #F92672">=</span><span style="color: #F8F8F2"> Image.open(</span><span style="color: #E6DB74">&#39;python_secrets.png&#39;</span><span style="color: #F8F8F2">)</span></span>
<span class="line"><span style="color: #F8F8F2">data </span><span style="color: #F92672">=</span><span style="color: #F8F8F2"> stepic.decode(image2) </span></span>
<span class="line"><span style="color: #66D9EF">print</span><span style="color: #F8F8F2">(</span><span style="color: #E6DB74">&quot;Decoded data: &quot;</span><span style="color: #F8F8F2"> </span><span style="color: #F92672">+</span><span style="color: #F8F8F2"> data)</span></span></code></pre></div>



<div class="wp-block-kevinbatdorf-code-block-pro cbp-has-line-numbers cbp-highlight-hover" data-code-block-pro-font-family="Code-Pro-JetBrains-Mono" style="font-size:.875rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#F8F8F2;--cbp-line-number-width:calc(3 * 0.6 * .875rem);--cbp-line-highlight-color:rgba(253, 253, 237, 0.2);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)"><span style="display:flex;align-items:center;padding:10px 0px 10px 16px;margin-bottom:-2px;width:100%;text-align:left;background-color:#34362e;color:#efefe1">Python</span><span role="button" tabindex="0" data-code="# stepic - Python image steganography

from PIL import Image

def _validate_image(image):
    if image.mode not in ('RGB', 'RGBA', 'CMYK'):
        raise ValueError('Unsupported pixel format: '
                         'image must be RGB, RGBA, or CMYK')
    if image.format == 'JPEG':
        raise ValueError('JPEG format incompatible with steganography')


def encode_imdata(imdata, data):
    '''given a sequence of pixels, returns an iterator of pixels with
    encoded data'''

    datalen = len(data)
    if datalen == 0:
        raise ValueError('data is empty')
    if datalen * 3 &gt; len(imdata):
        raise ValueError('data is too large for image')

    imdata = iter(imdata)

    for i in range(datalen):
        pixels = [value &amp; ~1 for value in
                  imdata.__next__()[:3] + imdata.__next__()[:3] + imdata.__next__()[:3]]
        byte = data[i]
        for j in range(7, -1, -1):
            pixels[j] |= byte &amp; 1
            byte &gt;&gt;= 1
        if i == datalen - 1:
            pixels[-1] |= 1
        pixels = tuple(pixels)
        yield pixels[0:3]
        yield pixels[3:6]
        yield pixels[6:9]


def encode_inplace(image, data):
    '''hides data in an image'''

    _validate_image(image)

    w = image.size[0]
    (x, y) = (0, 0)
    for pixel in encode_imdata(image.getdata(), data):
        image.putpixel((x, y), pixel)
        if x == w - 1:
            x = 0
            y += 1
        else:
            x += 1


def encode(image, data):
    '''generates an image with hidden data, starting with an existing
    image and arbitrary data'''

    image = image.copy()
    encode_inplace(image, data)
    
    # Save image
    image.save('python-secret.png')
    
    return image


def decode_imdata(imdata):
    '''Given a sequence of pixels, returns an iterator of characters
    encoded in the image'''

    imdata = iter(imdata)
    while True:
        pixels = list(imdata.__next__()[:3] + imdata.__next__()[:3] + imdata.__next__()[:3])
        byte = 0
        for c in range(7):
            byte |= pixels[c] &amp; 1
            byte <<= 1
        byte |= pixels[7] &amp; 1
        yield chr(byte)
        if pixels[-1] &amp; 1:
            break


def decode(image):
    '''extracts data from an image'''

    _validate_image(image)

    return ''.join(decode_imdata(image.getdata()))


if __name__ == &quot;__main__&quot;:
    img = Image.open('python.png')
    #encrypt message in image
    encode(img,'this is a secret message')
    #decrypt messagte in image
    img = Image.open('python-secret.png')
    text_decoded = decode(img)
    print(text_decoded)" style="color:#F8F8F2;display:none" aria-label="Copy" class="code-block-pro-copy-button"><svg xmlns="http://www.w3.org/2000/svg" style="width:24px;height:24px" fill="none" viewBox="0 0 24 24" stroke="currentColor" stroke-width="2"><path class="with-check" stroke-linecap="round" stroke-linejoin="round" d="M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2m-6 9l2 2 4-4"></path><path class="without-check" stroke-linecap="round" stroke-linejoin="round" d="M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2"></path></svg></span><pre class="shiki monokai" style="background-color: #272822" tabindex="0"><code><span class="line"><span style="color: #88846F"># stepic - Python image steganography</span></span>
<span class="line"></span>
<span class="line"><span style="color: #F92672">from</span><span style="color: #F8F8F2"> </span><span style="color: #AE81FF">PIL</span><span style="color: #F8F8F2"> </span><span style="color: #F92672">import</span><span style="color: #F8F8F2"> Image</span></span>
<span class="line"></span>
<span class="line"><span style="color: #66D9EF; font-style: italic">def</span><span style="color: #F8F8F2"> </span><span style="color: #A6E22E">_validate_image</span><span style="color: #F8F8F2">(</span><span style="color: #FD971F; font-style: italic">image</span><span style="color: #F8F8F2">):</span></span>
<span class="line"><span style="color: #F8F8F2">    </span><span style="color: #F92672">if</span><span style="color: #F8F8F2"> image.mode </span><span style="color: #F92672">not</span><span style="color: #F8F8F2"> </span><span style="color: #F92672">in</span><span style="color: #F8F8F2"> (</span><span style="color: #E6DB74">&#39;RGB&#39;</span><span style="color: #F8F8F2">, </span><span style="color: #E6DB74">&#39;RGBA&#39;</span><span style="color: #F8F8F2">, </span><span style="color: #E6DB74">&#39;CMYK&#39;</span><span style="color: #F8F8F2">):</span></span>
<span class="line"><span style="color: #F8F8F2">        </span><span style="color: #F92672">raise</span><span style="color: #F8F8F2"> </span><span style="color: #66D9EF; font-style: italic">ValueError</span><span style="color: #F8F8F2">(</span><span style="color: #E6DB74">&#39;Unsupported pixel format: &#39;</span></span>
<span class="line"><span style="color: #F8F8F2">                         </span><span style="color: #E6DB74">&#39;image must be RGB, RGBA, or CMYK&#39;</span><span style="color: #F8F8F2">)</span></span>
<span class="line"><span style="color: #F8F8F2">    </span><span style="color: #F92672">if</span><span style="color: #F8F8F2"> image.format </span><span style="color: #F92672">==</span><span style="color: #F8F8F2"> </span><span style="color: #E6DB74">&#39;JPEG&#39;</span><span style="color: #F8F8F2">:</span></span>
<span class="line"><span style="color: #F8F8F2">        </span><span style="color: #F92672">raise</span><span style="color: #F8F8F2"> </span><span style="color: #66D9EF; font-style: italic">ValueError</span><span style="color: #F8F8F2">(</span><span style="color: #E6DB74">&#39;JPEG format incompatible with steganography&#39;</span><span style="color: #F8F8F2">)</span></span>
<span class="line"></span>
<span class="line"></span>
<span class="line"><span style="color: #66D9EF; font-style: italic">def</span><span style="color: #F8F8F2"> </span><span style="color: #A6E22E">encode_imdata</span><span style="color: #F8F8F2">(</span><span style="color: #FD971F; font-style: italic">imdata</span><span style="color: #F8F8F2">, </span><span style="color: #FD971F; font-style: italic">data</span><span style="color: #F8F8F2">):</span></span>
<span class="line"><span style="color: #F8F8F2">    </span><span style="color: #E6DB74">&#39;&#39;&#39;given a sequence of pixels, returns an iterator of pixels with</span></span>
<span class="line"><span style="color: #E6DB74">    encoded data&#39;&#39;&#39;</span></span>
<span class="line"></span>
<span class="line"><span style="color: #F8F8F2">    datalen </span><span style="color: #F92672">=</span><span style="color: #F8F8F2"> </span><span style="color: #66D9EF">len</span><span style="color: #F8F8F2">(data)</span></span>
<span class="line"><span style="color: #F8F8F2">    </span><span style="color: #F92672">if</span><span style="color: #F8F8F2"> datalen </span><span style="color: #F92672">==</span><span style="color: #F8F8F2"> </span><span style="color: #AE81FF">0</span><span style="color: #F8F8F2">:</span></span>
<span class="line"><span style="color: #F8F8F2">        </span><span style="color: #F92672">raise</span><span style="color: #F8F8F2"> </span><span style="color: #66D9EF; font-style: italic">ValueError</span><span style="color: #F8F8F2">(</span><span style="color: #E6DB74">&#39;data is empty&#39;</span><span style="color: #F8F8F2">)</span></span>
<span class="line"><span style="color: #F8F8F2">    </span><span style="color: #F92672">if</span><span style="color: #F8F8F2"> datalen </span><span style="color: #F92672">*</span><span style="color: #F8F8F2"> </span><span style="color: #AE81FF">3</span><span style="color: #F8F8F2"> </span><span style="color: #F92672">&gt;</span><span style="color: #F8F8F2"> </span><span style="color: #66D9EF">len</span><span style="color: #F8F8F2">(imdata):</span></span>
<span class="line"><span style="color: #F8F8F2">        </span><span style="color: #F92672">raise</span><span style="color: #F8F8F2"> </span><span style="color: #66D9EF; font-style: italic">ValueError</span><span style="color: #F8F8F2">(</span><span style="color: #E6DB74">&#39;data is too large for image&#39;</span><span style="color: #F8F8F2">)</span></span>
<span class="line"></span>
<span class="line"><span style="color: #F8F8F2">    imdata </span><span style="color: #F92672">=</span><span style="color: #F8F8F2"> </span><span style="color: #66D9EF">iter</span><span style="color: #F8F8F2">(imdata)</span></span>
<span class="line"></span>
<span class="line"><span style="color: #F8F8F2">    </span><span style="color: #F92672">for</span><span style="color: #F8F8F2"> i </span><span style="color: #F92672">in</span><span style="color: #F8F8F2"> </span><span style="color: #66D9EF">range</span><span style="color: #F8F8F2">(datalen):</span></span>
<span class="line"><span style="color: #F8F8F2">        pixels </span><span style="color: #F92672">=</span><span style="color: #F8F8F2"> [value </span><span style="color: #F92672">&amp;</span><span style="color: #F8F8F2"> </span><span style="color: #F92672">~</span><span style="color: #AE81FF">1</span><span style="color: #F8F8F2"> </span><span style="color: #F92672">for</span><span style="color: #F8F8F2"> value </span><span style="color: #F92672">in</span></span>
<span class="line"><span style="color: #F8F8F2">                  imdata.</span><span style="color: #66D9EF">__next__</span><span style="color: #F8F8F2">()[:</span><span style="color: #AE81FF">3</span><span style="color: #F8F8F2">] </span><span style="color: #F92672">+</span><span style="color: #F8F8F2"> imdata.</span><span style="color: #66D9EF">__next__</span><span style="color: #F8F8F2">()[:</span><span style="color: #AE81FF">3</span><span style="color: #F8F8F2">] </span><span style="color: #F92672">+</span><span style="color: #F8F8F2"> imdata.</span><span style="color: #66D9EF">__next__</span><span style="color: #F8F8F2">()[:</span><span style="color: #AE81FF">3</span><span style="color: #F8F8F2">]]</span></span>
<span class="line"><span style="color: #F8F8F2">        byte </span><span style="color: #F92672">=</span><span style="color: #F8F8F2"> data[i]</span></span>
<span class="line"><span style="color: #F8F8F2">        </span><span style="color: #F92672">for</span><span style="color: #F8F8F2"> j </span><span style="color: #F92672">in</span><span style="color: #F8F8F2"> </span><span style="color: #66D9EF">range</span><span style="color: #F8F8F2">(</span><span style="color: #AE81FF">7</span><span style="color: #F8F8F2">, </span><span style="color: #F92672">-</span><span style="color: #AE81FF">1</span><span style="color: #F8F8F2">, </span><span style="color: #F92672">-</span><span style="color: #AE81FF">1</span><span style="color: #F8F8F2">):</span></span>
<span class="line"><span style="color: #F8F8F2">            pixels[j] </span><span style="color: #F92672">|=</span><span style="color: #F8F8F2"> byte </span><span style="color: #F92672">&amp;</span><span style="color: #F8F8F2"> </span><span style="color: #AE81FF">1</span></span>
<span class="line"><span style="color: #F8F8F2">            byte </span><span style="color: #F92672">&gt;&gt;=</span><span style="color: #F8F8F2"> </span><span style="color: #AE81FF">1</span></span>
<span class="line"><span style="color: #F8F8F2">        </span><span style="color: #F92672">if</span><span style="color: #F8F8F2"> i </span><span style="color: #F92672">==</span><span style="color: #F8F8F2"> datalen </span><span style="color: #F92672">-</span><span style="color: #F8F8F2"> </span><span style="color: #AE81FF">1</span><span style="color: #F8F8F2">:</span></span>
<span class="line"><span style="color: #F8F8F2">            pixels[</span><span style="color: #F92672">-</span><span style="color: #AE81FF">1</span><span style="color: #F8F8F2">] </span><span style="color: #F92672">|=</span><span style="color: #F8F8F2"> </span><span style="color: #AE81FF">1</span></span>
<span class="line"><span style="color: #F8F8F2">        pixels </span><span style="color: #F92672">=</span><span style="color: #F8F8F2"> </span><span style="color: #66D9EF; font-style: italic">tuple</span><span style="color: #F8F8F2">(pixels)</span></span>
<span class="line"><span style="color: #F8F8F2">        </span><span style="color: #F92672">yield</span><span style="color: #F8F8F2"> pixels[</span><span style="color: #AE81FF">0</span><span style="color: #F8F8F2">:</span><span style="color: #AE81FF">3</span><span style="color: #F8F8F2">]</span></span>
<span class="line"><span style="color: #F8F8F2">        </span><span style="color: #F92672">yield</span><span style="color: #F8F8F2"> pixels[</span><span style="color: #AE81FF">3</span><span style="color: #F8F8F2">:</span><span style="color: #AE81FF">6</span><span style="color: #F8F8F2">]</span></span>
<span class="line"><span style="color: #F8F8F2">        </span><span style="color: #F92672">yield</span><span style="color: #F8F8F2"> pixels[</span><span style="color: #AE81FF">6</span><span style="color: #F8F8F2">:</span><span style="color: #AE81FF">9</span><span style="color: #F8F8F2">]</span></span>
<span class="line"></span>
<span class="line"></span>
<span class="line"><span style="color: #66D9EF; font-style: italic">def</span><span style="color: #F8F8F2"> </span><span style="color: #A6E22E">encode_inplace</span><span style="color: #F8F8F2">(</span><span style="color: #FD971F; font-style: italic">image</span><span style="color: #F8F8F2">, </span><span style="color: #FD971F; font-style: italic">data</span><span style="color: #F8F8F2">):</span></span>
<span class="line"><span style="color: #F8F8F2">    </span><span style="color: #E6DB74">&#39;&#39;&#39;hides data in an image&#39;&#39;&#39;</span></span>
<span class="line"></span>
<span class="line"><span style="color: #F8F8F2">    _validate_image(image)</span></span>
<span class="line"></span>
<span class="line"><span style="color: #F8F8F2">    w </span><span style="color: #F92672">=</span><span style="color: #F8F8F2"> image.size[</span><span style="color: #AE81FF">0</span><span style="color: #F8F8F2">]</span></span>
<span class="line"><span style="color: #F8F8F2">    (x, y) </span><span style="color: #F92672">=</span><span style="color: #F8F8F2"> (</span><span style="color: #AE81FF">0</span><span style="color: #F8F8F2">, </span><span style="color: #AE81FF">0</span><span style="color: #F8F8F2">)</span></span>
<span class="line"><span style="color: #F8F8F2">    </span><span style="color: #F92672">for</span><span style="color: #F8F8F2"> pixel </span><span style="color: #F92672">in</span><span style="color: #F8F8F2"> encode_imdata(image.getdata(), data):</span></span>
<span class="line"><span style="color: #F8F8F2">        image.putpixel((x, y), pixel)</span></span>
<span class="line"><span style="color: #F8F8F2">        </span><span style="color: #F92672">if</span><span style="color: #F8F8F2"> x </span><span style="color: #F92672">==</span><span style="color: #F8F8F2"> w </span><span style="color: #F92672">-</span><span style="color: #F8F8F2"> </span><span style="color: #AE81FF">1</span><span style="color: #F8F8F2">:</span></span>
<span class="line"><span style="color: #F8F8F2">            x </span><span style="color: #F92672">=</span><span style="color: #F8F8F2"> </span><span style="color: #AE81FF">0</span></span>
<span class="line"><span style="color: #F8F8F2">            y </span><span style="color: #F92672">+=</span><span style="color: #F8F8F2"> </span><span style="color: #AE81FF">1</span></span>
<span class="line"><span style="color: #F8F8F2">        </span><span style="color: #F92672">else</span><span style="color: #F8F8F2">:</span></span>
<span class="line"><span style="color: #F8F8F2">            x </span><span style="color: #F92672">+=</span><span style="color: #F8F8F2"> </span><span style="color: #AE81FF">1</span></span>
<span class="line"></span>
<span class="line"></span>
<span class="line"><span style="color: #66D9EF; font-style: italic">def</span><span style="color: #F8F8F2"> </span><span style="color: #A6E22E">encode</span><span style="color: #F8F8F2">(</span><span style="color: #FD971F; font-style: italic">image</span><span style="color: #F8F8F2">, </span><span style="color: #FD971F; font-style: italic">data</span><span style="color: #F8F8F2">):</span></span>
<span class="line"><span style="color: #F8F8F2">    </span><span style="color: #E6DB74">&#39;&#39;&#39;generates an image with hidden data, starting with an existing</span></span>
<span class="line"><span style="color: #E6DB74">    image and arbitrary data&#39;&#39;&#39;</span></span>
<span class="line"></span>
<span class="line"><span style="color: #F8F8F2">    image </span><span style="color: #F92672">=</span><span style="color: #F8F8F2"> image.copy()</span></span>
<span class="line"><span style="color: #F8F8F2">    encode_inplace(image, data)</span></span>
<span class="line"><span style="color: #F8F8F2">    </span></span>
<span class="line"><span style="color: #F8F8F2">    </span><span style="color: #88846F"># Save image</span></span>
<span class="line"><span style="color: #F8F8F2">    image.save(</span><span style="color: #E6DB74">&#39;python-secret.png&#39;</span><span style="color: #F8F8F2">)</span></span>
<span class="line"><span style="color: #F8F8F2">    </span></span>
<span class="line"><span style="color: #F8F8F2">    </span><span style="color: #F92672">return</span><span style="color: #F8F8F2"> image</span></span>
<span class="line"></span>
<span class="line"></span>
<span class="line"><span style="color: #66D9EF; font-style: italic">def</span><span style="color: #F8F8F2"> </span><span style="color: #A6E22E">decode_imdata</span><span style="color: #F8F8F2">(</span><span style="color: #FD971F; font-style: italic">imdata</span><span style="color: #F8F8F2">):</span></span>
<span class="line"><span style="color: #F8F8F2">    </span><span style="color: #E6DB74">&#39;&#39;&#39;Given a sequence of pixels, returns an iterator of characters</span></span>
<span class="line"><span style="color: #E6DB74">    encoded in the image&#39;&#39;&#39;</span></span>
<span class="line"></span>
<span class="line"><span style="color: #F8F8F2">    imdata </span><span style="color: #F92672">=</span><span style="color: #F8F8F2"> </span><span style="color: #66D9EF">iter</span><span style="color: #F8F8F2">(imdata)</span></span>
<span class="line"><span style="color: #F8F8F2">    </span><span style="color: #F92672">while</span><span style="color: #F8F8F2"> </span><span style="color: #AE81FF">True</span><span style="color: #F8F8F2">:</span></span>
<span class="line"><span style="color: #F8F8F2">        pixels </span><span style="color: #F92672">=</span><span style="color: #F8F8F2"> </span><span style="color: #66D9EF; font-style: italic">list</span><span style="color: #F8F8F2">(imdata.</span><span style="color: #66D9EF">__next__</span><span style="color: #F8F8F2">()[:</span><span style="color: #AE81FF">3</span><span style="color: #F8F8F2">] </span><span style="color: #F92672">+</span><span style="color: #F8F8F2"> imdata.</span><span style="color: #66D9EF">__next__</span><span style="color: #F8F8F2">()[:</span><span style="color: #AE81FF">3</span><span style="color: #F8F8F2">] </span><span style="color: #F92672">+</span><span style="color: #F8F8F2"> imdata.</span><span style="color: #66D9EF">__next__</span><span style="color: #F8F8F2">()[:</span><span style="color: #AE81FF">3</span><span style="color: #F8F8F2">])</span></span>
<span class="line"><span style="color: #F8F8F2">        byte </span><span style="color: #F92672">=</span><span style="color: #F8F8F2"> </span><span style="color: #AE81FF">0</span></span>
<span class="line"><span style="color: #F8F8F2">        </span><span style="color: #F92672">for</span><span style="color: #F8F8F2"> c </span><span style="color: #F92672">in</span><span style="color: #F8F8F2"> </span><span style="color: #66D9EF">range</span><span style="color: #F8F8F2">(</span><span style="color: #AE81FF">7</span><span style="color: #F8F8F2">):</span></span>
<span class="line"><span style="color: #F8F8F2">            byte </span><span style="color: #F92672">|=</span><span style="color: #F8F8F2"> pixels[c] </span><span style="color: #F92672">&amp;</span><span style="color: #F8F8F2"> </span><span style="color: #AE81FF">1</span></span>
<span class="line"><span style="color: #F8F8F2">            byte </span><span style="color: #F92672">&lt;&lt;=</span><span style="color: #F8F8F2"> </span><span style="color: #AE81FF">1</span></span>
<span class="line"><span style="color: #F8F8F2">        byte </span><span style="color: #F92672">|=</span><span style="color: #F8F8F2"> pixels[</span><span style="color: #AE81FF">7</span><span style="color: #F8F8F2">] </span><span style="color: #F92672">&amp;</span><span style="color: #F8F8F2"> </span><span style="color: #AE81FF">1</span></span>
<span class="line"><span style="color: #F8F8F2">        </span><span style="color: #F92672">yield</span><span style="color: #F8F8F2"> </span><span style="color: #66D9EF">chr</span><span style="color: #F8F8F2">(byte)</span></span>
<span class="line"><span style="color: #F8F8F2">        </span><span style="color: #F92672">if</span><span style="color: #F8F8F2"> pixels[</span><span style="color: #F92672">-</span><span style="color: #AE81FF">1</span><span style="color: #F8F8F2">] </span><span style="color: #F92672">&amp;</span><span style="color: #F8F8F2"> </span><span style="color: #AE81FF">1</span><span style="color: #F8F8F2">:</span></span>
<span class="line"><span style="color: #F8F8F2">            </span><span style="color: #F92672">break</span></span>
<span class="line"></span>
<span class="line"></span>
<span class="line"><span style="color: #66D9EF; font-style: italic">def</span><span style="color: #F8F8F2"> </span><span style="color: #A6E22E">decode</span><span style="color: #F8F8F2">(</span><span style="color: #FD971F; font-style: italic">image</span><span style="color: #F8F8F2">):</span></span>
<span class="line"><span style="color: #F8F8F2">    </span><span style="color: #E6DB74">&#39;&#39;&#39;extracts data from an image&#39;&#39;&#39;</span></span>
<span class="line"></span>
<span class="line"><span style="color: #F8F8F2">    _validate_image(image)</span></span>
<span class="line"></span>
<span class="line"><span style="color: #F8F8F2">    </span><span style="color: #F92672">return</span><span style="color: #F8F8F2"> </span><span style="color: #E6DB74">&#39;&#39;</span><span style="color: #F8F8F2">.join(decode_imdata(image.getdata()))</span></span>
<span class="line"></span>
<span class="line"></span>
<span class="line"><span style="color: #F92672">if</span><span style="color: #F8F8F2"> __name__ </span><span style="color: #F92672">==</span><span style="color: #F8F8F2"> </span><span style="color: #E6DB74">&quot;__main__&quot;</span><span style="color: #F8F8F2">:</span></span>
<span class="line"><span style="color: #F8F8F2">    img </span><span style="color: #F92672">=</span><span style="color: #F8F8F2"> Image.open(</span><span style="color: #E6DB74">&#39;python.png&#39;</span><span style="color: #F8F8F2">)</span></span>
<span class="line"><span style="color: #F8F8F2">    </span><span style="color: #88846F">#encrypt message in image</span></span>
<span class="line"><span style="color: #F8F8F2">    encode(img,</span><span style="color: #E6DB74">&#39;this is a secret message&#39;</span><span style="color: #F8F8F2">)</span></span>
<span class="line"><span style="color: #F8F8F2">    </span><span style="color: #88846F">#decrypt messagte in image</span></span>
<span class="line"><span style="color: #F8F8F2">    img </span><span style="color: #F92672">=</span><span style="color: #F8F8F2"> Image.open(</span><span style="color: #E6DB74">&#39;python-secret.png&#39;</span><span style="color: #F8F8F2">)</span></span>
<span class="line"><span style="color: #F8F8F2">    text_decoded </span><span style="color: #F92672">=</span><span style="color: #F8F8F2"> decode(img)</span></span>
<span class="line"><span style="color: #F8F8F2">    </span><span style="color: #66D9EF">print</span><span style="color: #F8F8F2">(text_decoded)</span></span></code></pre></div>



<div class="wp-block-jetpack-markdown"><ol>
<li>
<p><strong><code>_validate_image</code> function:</strong></p>
<ul>
<li>Validates that the image has a supported pixel format (<code>RGB</code>, <code>RGBA</code>, or <code>CMYK</code>).</li>
<li>Raises an exception if the image format is <code>JPEG</code> (incompatible with steganography).</li>
</ul>
</li>
<li>
<p><strong><code>encode_imdata</code> function:</strong></p>
<ul>
<li>Takes a sequence of pixels (<code>imdata</code>) and the data to be encoded.</li>
<li>Iterates over the pixel values, modifying the least significant bit (LSB) of each color channel to encode the data.</li>
<li>Yields the modified pixel values.</li>
</ul>
</li>
<li>
<p><strong><code>encode_inplace</code> function:</strong></p>
<ul>
<li>Takes an image and the data to be encoded.</li>
<li>Calls <code>_validate_image</code> to ensure the image is in a supported format.</li>
<li>Iterates over the image pixels, calling <code>encode_imdata</code> to modify the pixel values in place.</li>
</ul>
</li>
<li>
<p><strong><code>encode</code> function:</strong></p>
<ul>
<li>Takes an image and the data to be encoded.</li>
<li>Creates a copy of the image and calls <code>encode_inplace</code> on the copy.</li>
<li>Saves the resulting image with the hidden message as ‘python-secret.png’.</li>
<li>Returns the modified image.</li>
</ul>
</li>
<li>
<p><strong><code>decode_imdata</code> function:</strong></p>
<ul>
<li>Takes a sequence of pixels (<code>imdata</code>) and decodes characters from the LSB of the pixel values.</li>
<li>Yields the decoded characters until the LSB of the last pixel indicates the end of the message.</li>
</ul>
</li>
<li>
<p><strong><code>decode</code> function:</strong></p>
<ul>
<li>Takes an image and decodes the hidden message using <code>decode_imdata</code>.</li>
</ul>
</li>
<li>
<p><strong>Main Code:</strong></p>
<ul>
<li>Opens the image ‘python.png’.</li>
<li>Calls <code>encode</code> to hide the message ‘this is a secret message’ in the image.</li>
<li>Opens the encoded image ‘python-secret.png’.</li>
<li>Calls <code>decode</code> to extract and print the hidden message.</li>
</ul>
</li>
</ol>
</div>
<p>The post <a href="https://awjunaid.com/python/steganography-techniques-for-hiding-information-in-images/">Steganography techniques for hiding information in images</a> appeared first on <a href="https://awjunaid.com">Abdul Wahab Junaid</a>.</p>
]]></content:encoded>
					
					<wfw:commentRss>https://awjunaid.com/python/steganography-techniques-for-hiding-information-in-images/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
		<post-id xmlns="com-wordpress:feed-additions:1">8895</post-id>	</item>
		<item>
		<title>Explain the concept of a CPU burst in the context of CPU scheduling</title>
		<link>https://awjunaid.com/operating-system/explain-the-concept-of-a-cpu-burst-in-the-context-of-cpu-scheduling/</link>
					<comments>https://awjunaid.com/operating-system/explain-the-concept-of-a-cpu-burst-in-the-context-of-cpu-scheduling/?noamp=mobile#respond</comments>
		
		<dc:creator><![CDATA[Abdul Wahab Junaid]]></dc:creator>
		<pubDate>Wed, 20 Dec 2023 16:57:12 +0000</pubDate>
				<category><![CDATA[Operating System]]></category>
		<category><![CDATA[dailyprompt]]></category>
		<category><![CDATA[operating system]]></category>
		<guid isPermaLink="false">https://awjunaid.com/?p=8539</guid>

					<description><![CDATA[<p>If you&#8217;ve ever read an operating systems textbook diagram showing a process alternating between little blocks labeled &#8220;CPU&#8221;&#8230;</p>
<p>The post <a href="https://awjunaid.com/operating-system/explain-the-concept-of-a-cpu-burst-in-the-context-of-cpu-scheduling/">Explain the concept of a CPU burst in the context of CPU scheduling</a> appeared first on <a href="https://awjunaid.com">Abdul Wahab Junaid</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<p class="wp-block-paragraph">If you&#8217;ve ever read an operating systems textbook diagram showing a process alternating between little blocks labeled &#8220;CPU&#8221; and &#8220;I/O,&#8221; you&#8217;ve already seen the concept of a CPU burst without necessarily having the vocabulary for it. It&#8217;s one of the most foundational ideas in scheduling theory — practically every scheduling algorithm, from Shortest Job First to CFS, is implicitly reasoning about CPU bursts, even when the term itself doesn&#8217;t appear in the code. This article explains what a CPU burst actually is, how it&#8217;s measured and predicted, and why it matters for real scheduler design.</p>



<h2 class="wp-block-heading">Definition</h2>



<p class="wp-block-paragraph">A <strong>CPU burst</strong> is a single, uninterrupted stretch of time during which a process is actively executing instructions on the CPU, before it either blocks (waiting for I/O, a lock, a timer, or another event) or finishes. Program execution, viewed over time, is a repeating pattern:</p>



<pre class="wp-block-code"><code>CPU burst → I/O burst → CPU burst → I/O burst → ... → CPU burst → terminate
</code></pre>



<p class="wp-block-paragraph">This alternation is called the <strong>CPU-I/O burst cycle</strong>, and it&#8217;s the fundamental behavioral model that CPU scheduling theory is built on. A process spends its life alternating between wanting the CPU and waiting on something else.</p>



<h2 class="wp-block-heading">Burst Patterns: CPU-Bound vs. I/O-Bound</h2>



<p class="wp-block-paragraph">Processes fall on a spectrum based on the shape of their burst pattern:</p>



<ul class="wp-block-list">
<li><strong>CPU-bound processes</strong> have long CPU bursts and infrequent, short (or no) I/O bursts. Examples: video encoding, scientific simulation, cryptographic hashing, compiling code, machine learning training.</li>



<li><strong>I/O-bound processes</strong> have short CPU bursts and frequent, often long I/O bursts. Examples: text editors waiting on keystrokes, web servers waiting on network I/O, database systems waiting on disk reads, GUI applications waiting on user interaction.</li>
</ul>



<p class="wp-block-paragraph">This distinction matters enormously for scheduling because a good scheduler behaves very differently depending on which type of process it&#8217;s dealing with. If a scheduler treats every process&#8217;s next CPU burst as unpredictable, it can&#8217;t optimize; if it can reasonably predict burst length or behavior class, it can make much better decisions.</p>



<h2 class="wp-block-heading">Measuring and Visualizing CPU Bursts</h2>



<p class="wp-block-paragraph">Classic OS textbook studies (some dating back to research in the 1960s-70s on early timesharing systems) measured burst-length distributions across real workloads and found a consistent pattern: <strong>most CPU bursts are short, and a small number are very long</strong>, producing a distribution with a large peak near zero and a long tail — often modeled approximately as an exponential or hyperexponential distribution.</p>



<pre class="wp-block-code"><code>Frequency
   │
   │██
   │████
   │██████
   │████████
   │██████████▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁
   └───────────────────────────────────────────▶ Burst length
     (many short bursts)      (few very long bursts)
</code></pre>



<p class="wp-block-paragraph">This shape is exactly why algorithms like <strong>Shortest Job First (SJF)</strong> and <strong>Shortest Remaining Time First (SRTF)</strong> work so well in theory — since most bursts are short, prioritizing shorter predicted bursts lets the scheduler clear a large number of processes quickly, minimizing average waiting time, while the rare long burst gets deferred but doesn&#8217;t disproportionately hurt overall average wait time metrics.</p>



<h2 class="wp-block-heading">Why Scheduling Algorithms Care About Burst Length</h2>



<h3 class="wp-block-heading">Shortest Job First (SJF) / Shortest Remaining Time First (SRTF)</h3>



<p class="wp-block-paragraph">These algorithms explicitly prioritize processes with the shortest predicted next CPU burst. This provably minimizes average waiting time among all non-preemptive scheduling algorithms, given accurate burst predictions — but real systems don&#8217;t know the future length of a burst in advance, so SJF requires <em>estimating</em> it.</p>



<h3 class="wp-block-heading">Burst Prediction: Exponential Averaging</h3>



<p class="wp-block-paragraph">Since the actual length of the next CPU burst isn&#8217;t known ahead of time, real implementations of SJF-like scheduling use <strong>exponential averaging</strong> to predict it from history:</p>



<pre class="wp-block-code"><code>τ(n+1) = α * t(n) + (1 - α) * τ(n)
</code></pre>



<p class="wp-block-paragraph">Where:</p>



<ul class="wp-block-list">
<li><code>t(n)</code> = the actual length of the most recent CPU burst</li>



<li><code>τ(n)</code> = the previously predicted length</li>



<li><code>τ(n+1)</code> = the new prediction for the next burst</li>



<li><code>α</code> (0 ≤ α ≤ 1) = a weighting factor controlling how much recent history matters vs. long-term history</li>
</ul>



<p class="wp-block-paragraph">With <code>α = 0.5</code>, the predictor gives equal weight to the most recent burst and the previous running estimate — a simple, effective way to adapt to a process&#8217;s recent behavior (e.g., a process that recently started doing longer computation phases) without being thrown off entirely by a single anomalous burst.</p>



<h3 class="wp-block-heading">Priority and Multilevel Feedback Queues</h3>



<p class="wp-block-paragraph">As covered in the aging-focused article, MLFQ scheduling explicitly uses observed CPU burst behavior to reclassify processes: a process that uses its <em>entire</em> time slice without blocking is inferred to be CPU-bound and gets demoted to a lower-priority queue with a longer time slice (batched for throughput); a process that blocks quickly (short CPU burst before an I/O wait) is inferred to be I/O-bound/interactive and stays at high priority for responsiveness. This is, in effect, using burst length as a live signal for scheduling decisions without needing an explicit prediction formula.</p>



<h3 class="wp-block-heading">Linux CFS and Burst Behavior</h3>



<p class="wp-block-paragraph">CFS doesn&#8217;t explicitly predict burst lengths, but its <strong>sleeper fairness</strong> mechanism achieves a similar practical effect: a process with short CPU bursts and frequent blocking accumulates vruntime slowly (since it&#8217;s rarely running), so it naturally sits near the front of the scheduling queue and gets picked quickly whenever it becomes runnable again — the emergent behavior favors I/O-bound processes without any explicit burst-length bookkeeping.</p>



<h2 class="wp-block-heading">Real-World Example: Observing Burst Behavior</h2>



<p class="wp-block-paragraph">You can observe burst-like behavior directly using Linux tracing tools.</p>



<pre class="wp-block-code"><code># Trace scheduling switches for a specific process
perf sched record -p &lt;pid&gt; -- sleep 5
perf sched timehist

# See voluntary vs involuntary context switches — a proxy for burst pattern
cat /proc/&lt;pid&gt;/status | grep ctxt_switches
</code></pre>



<p class="wp-block-paragraph">A process with a high ratio of <strong>voluntary</strong> context switches (it blocked on its own, e.g., waiting for I/O) relative to <strong>involuntary</strong> ones (it was preempted mid-burst by the scheduler) is exhibiting classic I/O-bound, short-burst behavior. A process dominated by involuntary switches is CPU-bound with long bursts that keep getting cut off by the scheduler&#8217;s fairness/preemption rules.</p>



<h2 class="wp-block-heading">Diagram: CPU-I/O Burst Cycle for a Sample Process</h2>



<pre class="wp-block-code"><code>Time ─────────────────────────────────────────────────▶

Process P:
&#91;CPU: 5ms]──&#91;I/O: 40ms wait]──&#91;CPU: 3ms]──&#91;I/O: 60ms]──&#91;CPU: 8ms]──terminate

           ▲                              ▲
       short burst                  short burst
       (I/O-bound behavior — classic interactive/database pattern)
</code></pre>



<p class="wp-block-paragraph">Compare with a CPU-bound process:</p>



<pre class="wp-block-code"><code>Process Q:
&#91;CPU: 800ms]──&#91;I/O: 2ms]──&#91;CPU: 750ms]──&#91;I/O: 1ms]──&#91;CPU: 900ms]──terminate

       ▲
   long, dominant burst
   (CPU-bound behavior — classic batch/compute pattern)
</code></pre>



<h2 class="wp-block-heading">Practical Implications Across Systems</h2>



<h3 class="wp-block-heading">Servers and Batch Processing</h3>



<p class="wp-block-paragraph">Understanding that most workloads are dominated by short bursts (interactive requests, small transactions) with occasional long-burst outliers (large batch jobs, report generation) informs decisions like separating &#8220;interactive&#8221; and &#8220;batch&#8221; worker pools, or setting different nice values/cgroup weights for different job classes so long bursts don&#8217;t degrade the responsiveness of short-burst request-handling processes.</p>



<h3 class="wp-block-heading">Databases</h3>



<p class="wp-block-paragraph">Query engines often explicitly separate short OLTP-style queries (short CPU bursts, frequent small I/O) from long-running OLAP/analytical queries (long CPU bursts, large sequential I/O), using separate resource pools or priority classes — a direct real-world application of burst-pattern awareness at the application layer, complementing OS-level scheduling.</p>



<h3 class="wp-block-heading">Mobile and Android</h3>



<p class="wp-block-paragraph">Frame rendering on Android/iOS is a tightly time-boxed CPU burst — a UI thread has roughly 16.6ms (for 60fps) to complete its work for a frame. Schedulers on these platforms (Android&#8217;s EAS-augmented CFS, XNU&#8217;s QoS classes) are tuned to recognize and prioritize these specific short, latency-critical bursts over longer background bursts, since missing the burst&#8217;s deadline causes visible jank.</p>



<h2 class="wp-block-heading">Troubleshooting Using Burst Awareness</h2>



<p class="wp-block-paragraph"><strong>Symptom: Interactive application still feels slow despite low overall CPU usage.</strong> Investigate whether its CPU bursts are being delayed rather than lengthened — use <code>perf sched latency</code> to check wait time between becoming runnable and actually running, not just total CPU time consumed.</p>



<p class="wp-block-paragraph"><strong>Symptom: Batch job runs slower than expected despite having the CPU &#8220;mostly to itself.&#8221;</strong> Check for excessive involuntary context switches (<code>/proc/&lt;pid&gt;/status</code>), which would indicate its long CPU bursts are being fragmented by preemption from other processes more often than expected — possibly due to overly aggressive preemption granularity settings.</p>



<p class="wp-block-paragraph"><strong>Symptom: A workload&#8217;s performance is inconsistent between runs.</strong> Burst-length variability itself might be the issue — some workloads (e.g., garbage-collected languages, JIT-compiled runtimes) have bursty CPU demand with occasional very long bursts (e.g., a GC pause), which can interact poorly with prediction-based schedulers or fixed time-slice policies; profiling tools like <code>perf record</code>/<code>perf report</code>, or language-specific profilers, can reveal these patterns.</p>



<h2 class="wp-block-heading">Best Practices</h2>



<ul class="wp-block-list">
<li>When designing multi-tenant systems, classify and separate workloads by burst pattern (short/interactive vs. long/batch) rather than assuming a single scheduling policy suits everyone.</li>



<li>Use exponential-averaging-style prediction (or your platform&#8217;s equivalent adaptive heuristics) when building custom schedulers or resource managers, rather than static assumptions about job length.</li>



<li>Profile actual burst behavior (via <code>perf sched</code>, context-switch counters, or application-level tracing) before tuning scheduler parameters — intuitions about whether a workload is &#8220;CPU-bound&#8221; or &#8220;I/O-bound&#8221; are often wrong until measured.</li>



<li>For latency-critical, deadline-bound bursts (UI rendering, real-time audio), don&#8217;t rely on general-purpose burst prediction — use explicit deadline-aware scheduling (<code>SCHED_DEADLINE</code> on Linux, or platform-specific real-time APIs).</li>
</ul>



<h2 class="wp-block-heading">Summary</h2>



<p class="wp-block-paragraph">A CPU burst is simply a contiguous stretch of CPU execution between waits, but this humble concept underlies nearly every classic CPU scheduling algorithm: Shortest Job First&#8217;s entire premise is prioritizing short predicted bursts, multilevel feedback queues use observed burst behavior to classify processes as interactive or batch, and even modern fairness-first schedulers like Linux&#8217;s CFS achieve favorable treatment of short-burst (I/O-bound) processes as an emergent property of their fairness math. Understanding whether a workload is dominated by short or long bursts — and measuring it rather than assuming it — remains one of the most practically useful diagnostic lenses for both scheduler designers and application developers optimizing system performance.</p>



<h2 class="wp-block-heading">FAQs</h2>



<p class="wp-block-paragraph"><strong>Is a CPU burst the same as a time slice/quantum?</strong> No. A time slice (quantum) is an artificial limit the <em>scheduler</em> imposes on how long a process may run before being preempted. A CPU burst is a <em>natural</em> property of the process itself — how long it would run if left uninterrupted before it needs to wait on something. A single CPU burst can span multiple time slices if the process keeps getting preempted and resumed before it actually needs to block.</p>



<p class="wp-block-paragraph"><strong>Why do most CPU bursts tend to be short in real workloads?</strong> Empirically, most programs interact frequently with the outside world (memory access patterns aside) — reading input, writing output, calling library/system functions that eventually touch I/O or synchronization — so uninterrupted pure computation stretches tend to be short, with occasional longer bursts during dense computational sections.</p>



<p class="wp-block-paragraph"><strong>Does burst length prediction still matter given modern hardware and schedulers?</strong> The explicit exponential-averaging formula is mostly a teaching tool today, since most production schedulers (CFS, Windows&#8217; scheduler) don&#8217;t use SJF-style explicit prediction. But the underlying principle — infer and adapt to a process&#8217;s actual behavior pattern rather than treating all processes identically — remains deeply embedded in modern scheduler design, just implemented differently.</p>



<p class="wp-block-paragraph"><strong>How does burst behavior relate to context-switch overhead?</strong> Very short CPU bursts combined with very frequent switching can make context-switch overhead (cache/TLB flushing, register save/restore) a significant fraction of total CPU time, which is why schedulers impose a minimum granularity floor even for latency-sensitive workloads.</p>



<h2 class="wp-block-heading">References</h2>



<ul class="wp-block-list">
<li>Silberschatz, Galvin, Gagne — &#8220;Operating System Concepts,&#8221; CPU Scheduling chapter (CPU-I/O Burst Cycle, SJF, exponential averaging formula)</li>



<li>Tanenbaum — &#8220;Modern Operating Systems,&#8221; process/thread scheduling chapters</li>



<li><code>man perf-sched</code> — Linux performance analysis of scheduler behavior</li>



<li>Silberschatz et al., historical references to early timesharing burst-distribution studies</li>
</ul>
<p>The post <a href="https://awjunaid.com/operating-system/explain-the-concept-of-a-cpu-burst-in-the-context-of-cpu-scheduling/">Explain the concept of a CPU burst in the context of CPU scheduling</a> appeared first on <a href="https://awjunaid.com">Abdul Wahab Junaid</a>.</p>
]]></content:encoded>
					
					<wfw:commentRss>https://awjunaid.com/operating-system/explain-the-concept-of-a-cpu-burst-in-the-context-of-cpu-scheduling/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
		<post-id xmlns="com-wordpress:feed-additions:1">8539</post-id>	</item>
		<item>
		<title>How can administrators identify and manage zombie processes in UNIX</title>
		<link>https://awjunaid.com/operating-system/how-can-administrators-identify-and-manage-zombie-processes-in-unix/</link>
					<comments>https://awjunaid.com/operating-system/how-can-administrators-identify-and-manage-zombie-processes-in-unix/?noamp=mobile#respond</comments>
		
		<dc:creator><![CDATA[Abdul Wahab Junaid]]></dc:creator>
		<pubDate>Wed, 20 Dec 2023 15:59:12 +0000</pubDate>
				<category><![CDATA[Operating System]]></category>
		<category><![CDATA[dailyprompt]]></category>
		<category><![CDATA[operating system]]></category>
		<guid isPermaLink="false">https://awjunaid.com/?p=8476</guid>

					<description><![CDATA[<p>If you administer UNIX or Linux servers long enough, you&#8217;ll eventually get an alert about high process counts,&#8230;</p>
<p>The post <a href="https://awjunaid.com/operating-system/how-can-administrators-identify-and-manage-zombie-processes-in-unix/">How can administrators identify and manage zombie processes in UNIX</a> appeared first on <a href="https://awjunaid.com">Abdul Wahab Junaid</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<p class="wp-block-paragraph">If you administer UNIX or Linux servers long enough, you&#8217;ll eventually get an alert about high process counts, or you&#8217;ll just be poking around a slow system and notice a handful of processes marked with a <code>Z</code> state. Knowing how to correctly identify a zombie process, understand why it&#8217;s there, and clean it up — or more precisely, get its parent to clean it up — is a genuinely useful piece of practical sysadmin knowledge. I&#8217;ll walk through the identification tools, the diagnostic process, and the actual remediation steps.</p>



<h2 class="wp-block-heading">What You&#8217;re Looking For</h2>



<p class="wp-block-paragraph">A zombie process (sometimes shown as &#8220;defunct&#8221; in process listings) is a process that has already terminated but whose exit status hasn&#8217;t been collected by its parent via <code>wait()</code>. It shows up in process listings but does essentially nothing — it&#8217;s not consuming CPU, and its memory has already been released back to the system. The only thing it&#8217;s &#8220;using&#8221; is a slot in the kernel&#8217;s process table.</p>



<h2 class="wp-block-heading">Identifying Zombie Processes</h2>



<h3 class="wp-block-heading">Using <code>ps</code></h3>



<p class="wp-block-paragraph">The most common and direct way to spot zombies is the <code>ps</code> command, checking the process state column:</p>



<pre class="wp-block-code"><code>ps aux | grep 'Z'
</code></pre>



<p class="wp-block-paragraph">Or more precisely, filtering by the actual state field to avoid false positives from matching a &#8220;Z&#8221; somewhere else in the output:</p>



<pre class="wp-block-code"><code>ps -eo pid,ppid,stat,cmd | awk '$3 ~ /Z/'
</code></pre>



<p class="wp-block-paragraph">In <code>ps</code> output, the <code>STAT</code> column shows <code>Z</code> for zombie processes, and the command name is typically shown as <code>&lt;defunct&gt;</code>:</p>



<pre class="wp-block-code"><code>  PID  PPID STAT CMD
 4821  4790 Z    &#91;worker] &lt;defunct&gt;
</code></pre>



<h3 class="wp-block-heading">Using <code>top</code> or <code>htop</code></h3>



<p class="wp-block-paragraph">Both <code>top</code> and <code>htop</code> show a live count of processes in each state, including zombies, right in the summary header:</p>



<pre class="wp-block-code"><code>Tasks: 215 total,   1 running, 212 sleeping,   0 stopped,   2 zombie
</code></pre>



<p class="wp-block-paragraph">This is often the fastest way to notice a zombie accumulation trend at a glance, especially if you have <code>top</code> running in a monitoring dashboard or check it periodically during troubleshooting.</p>



<h3 class="wp-block-heading">Using <code>/proc</code></h3>



<p class="wp-block-paragraph">Since Linux exposes process information through the <code>/proc</code> filesystem, you can also inspect a specific process&#8217;s state directly:</p>



<pre class="wp-block-code"><code>cat /proc/4821/status | grep State
</code></pre>



<pre class="wp-block-code"><code>State:  Z (zombie)
</code></pre>



<p class="wp-block-paragraph">This approach is useful for scripting automated checks, since you can iterate over <code>/proc/[pid]/status</code> for every PID without needing to parse <code>ps</code> output.</p>



<h3 class="wp-block-heading">Counting Zombies System-Wide</h3>



<p class="wp-block-paragraph">For a quick health check or monitoring script, counting zombies across the whole system is straightforward:</p>



<pre class="wp-block-code"><code>ps -eo stat | grep -c '^Z'
</code></pre>



<p class="wp-block-paragraph">This is the kind of one-liner worth wiring into a monitoring system (Nagios, Zabbix, Prometheus node exporter custom metrics, or a simple cron-based alert) so you get notified if the zombie count trends upward over time rather than discovering the problem only after the process table fills up.</p>



<h2 class="wp-block-heading">Diagnosing the Root Cause</h2>



<p class="wp-block-paragraph">Finding zombies is the easy part; the real diagnostic work is identifying which <strong>parent process</strong> is failing to reap its children, because killing individual zombies isn&#8217;t possible — they&#8217;re already dead, so signals have no effect on them. You have to address the parent.</p>



<h3 class="wp-block-heading">Step 1: Identify the Parent PID (PPID)</h3>



<pre class="wp-block-code"><code>ps -eo pid,ppid,stat,cmd | awk '$3 ~ /Z/ {print $2}'
</code></pre>



<p class="wp-block-paragraph">This gives you the PPID of each zombie, which tells you exactly which running process is responsible for cleaning it up.</p>



<h3 class="wp-block-heading">Step 2: Investigate the Parent Process</h3>



<pre class="wp-block-code"><code>ps -p &lt;PPID&gt; -o pid,cmd,etime
</code></pre>



<p class="wp-block-paragraph">Look at what that parent process actually is. Common offenders include:</p>



<ul class="wp-block-list">
<li>Custom application code with a bug in its <code>fork</code>/<code>wait</code> logic</li>



<li>Shell scripts that background many jobs (<code>&amp;</code>) without ever calling <code>wait</code></li>



<li>Buggy or outdated daemons that spawn worker subprocesses without proper <code>SIGCHLD</code> handling</li>



<li>Containerized applications running directly as PID 1 without a proper init process</li>
</ul>



<h3 class="wp-block-heading">Step 3: Check Whether the Parent Is Still Actively Running</h3>



<p class="wp-block-paragraph">If the parent process is still alive and simply hasn&#8217;t gotten around to reaping (for instance, it&#8217;s busy handling other work and will eventually call <code>wait()</code>), the zombie is often transient and will clear itself out shortly. If the parent has effectively hung, or has a genuine bug that prevents it from ever calling <code>wait()</code>, the zombie will persist indefinitely until you intervene.</p>



<h2 class="wp-block-heading">Managing and Resolving Zombie Processes</h2>



<h3 class="wp-block-heading">Option 1: Wait for the Parent to Reap Naturally</h3>



<p class="wp-block-paragraph">If zombies are transient (appearing and disappearing quickly as the system churns through normal fork/exec/wait cycles), no action is needed — this is completely normal.</p>



<h3 class="wp-block-heading">Option 2: Send a Signal to the Parent</h3>



<p class="wp-block-paragraph">If the parent process is buggy and never calls <code>wait()</code>, you can often trigger cleanup indirectly by sending it a <code>SIGCHLD</code> signal manually, which — if the parent has a properly implemented signal handler that&#8217;s just not being triggered for some reason — can prompt it to reap pending children:</p>



<pre class="wp-block-code"><code>kill -SIGCHLD &lt;PPID&gt;
</code></pre>



<p class="wp-block-paragraph">This doesn&#8217;t always work if the bug is deeper than a missed signal delivery, but it&#8217;s a reasonable first, non-disruptive step.</p>



<h3 class="wp-block-heading">Option 3: Restart the Parent Process</h3>



<p class="wp-block-paragraph">If the parent process has a genuine bug and won&#8217;t reap its zombie children, restarting it is often the most reliable fix. When the parent terminates, its zombie children (which are still technically &#8220;children&#8221; of that process, even in zombie state) get re-parented to <code>init</code> (PID 1) or a designated subreaper like systemd, both of which are specifically designed to promptly reap orphaned zombies.</p>



<pre class="wp-block-code"><code>systemctl restart &lt;service-name&gt;
</code></pre>



<p class="wp-block-paragraph">Or, for a process not managed by systemd:</p>



<pre class="wp-block-code"><code>kill &lt;PPID&gt;          # graceful termination first
# if that fails to clear things up:
kill -9 &lt;PPID&gt;        # forceful termination as a last resort
</code></pre>



<p class="wp-block-paragraph">Be cautious with this — killing the parent affects everything else that parent is responsible for, not just the zombie cleanup, so this should be a deliberate decision, not a reflexive one.</p>



<h3 class="wp-block-heading">Option 4: Reboot as a Last Resort</h3>



<p class="wp-block-paragraph">In genuinely severe cases — for instance, if <code>init</code> or the primary subreaper itself has a bug (which is rare but not unheard of) — a full reboot clears every zombie because the entire process table is reset. This should be a last resort after other diagnostic and remediation steps have failed, and it&#8217;s worth filing a bug report against whatever software caused the issue if you end up here.</p>



<h2 class="wp-block-heading">Building Long-Term Monitoring</h2>



<p class="wp-block-paragraph">Rather than treating zombie identification as a one-off troubleshooting exercise, it&#8217;s worth setting up ongoing monitoring:</p>



<ul class="wp-block-list">
<li><strong>Prometheus node exporter</strong> can expose process state counts as metrics, letting you graph zombie counts over time and set alerting thresholds.</li>



<li><strong>Nagios/Icinga checks</strong> can run a simple <code>ps</code>-based script on a schedule and alert if the zombie count exceeds a defined threshold.</li>



<li><strong>Log aggregation</strong> — correlating zombie accumulation with deploys or specific application versions can help pinpoint exactly which code change introduced a reaping bug.</li>
</ul>



<h2 class="wp-block-heading">A Sample Diagnostic Script</h2>



<p class="wp-block-paragraph">Here&#8217;s a simple bash script that identifies zombies and their parents in one pass, useful as a starting point for a custom monitoring check:</p>



<pre class="wp-block-code"><code>#!/bin/bash
echo "Zombie processes found:"
ps -eo pid,ppid,stat,cmd | awk '$3 ~ /Z/ {print}'

echo ""
echo "Unique parent processes responsible:"
ps -eo pid,ppid,stat,cmd | awk '$3 ~ /Z/ {print $2}' | sort -u | while read ppid; do
    ps -p "$ppid" -o pid,cmd,etime 2&gt;/dev/null
done
</code></pre>



<h2 class="wp-block-heading">Best Practices for Administrators</h2>



<ul class="wp-block-list">
<li>Build zombie-count monitoring into your standard server health checks rather than relying on manual discovery.</li>



<li>Always resolve zombies by addressing the parent process, never by attempting to signal the zombie directly (which has no effect).</li>



<li>Favor restarting a misbehaving parent process over rebooting the whole system whenever possible.</li>



<li>For containerized deployments, verify that a proper init process (<code>tini</code>, <code>--init</code>, <code>dumb-init</code>) is in place, since PID-1 zombie accumulation is one of the most common real-world causes in modern infrastructure.</li>



<li>Track down and fix the underlying application bug rather than treating restarts as a permanent solution — recurring zombie accumulation is a sign of a genuine defect in the parent process&#8217;s code.</li>
</ul>



<h2 class="wp-block-heading">Advanced Identification Techniques</h2>



<p class="wp-block-paragraph">Beyond the basic <code>ps</code>/<code>top</code> approach, administrators managing large fleets of servers benefit from more systematic identification techniques.</p>



<h3 class="wp-block-heading">Using systemtap or eBPF for Deep Diagnostics</h3>



<p class="wp-block-paragraph">On modern Linux systems, tools built on eBPF (extended Berkeley Packet Filter) can trace <code>fork()</code>, <code>exit()</code>, and <code>wait()</code> system calls in real time across the whole system, giving administrators visibility not just into <em>that</em> zombies exist, but into the exact sequence of events leading up to them — which is invaluable when the root cause isn&#8217;t obvious from a static snapshot. Tools like <code>bpftrace</code> make this accessible without needing to write custom kernel modules:</p>



<pre class="wp-block-code"><code>sudo bpftrace -e 'tracepoint:sched:sched_process_exit { printf("%s (pid %d) exited\n", comm, pid); }'
</code></pre>



<p class="wp-block-paragraph">Running this alongside a <code>SIGCHLD</code>/<code>wait()</code> trace can reveal timing gaps between when a child exits and when (or if) its parent actually calls <code>wait()</code>, which is far more precise than inferring the problem indirectly from <code>ps</code> snapshots taken minutes apart.</p>



<h3 class="wp-block-heading">Correlating Zombies With Application Logs</h3>



<p class="wp-block-paragraph">When a specific service is repeatedly implicated in zombie accumulation, correlating the timestamps of zombie appearances with that service&#8217;s own application logs can reveal the pattern — for instance, discovering that zombies specifically accumulate during a particular batch job or under a particular request pattern, narrowing the investigation considerably before you even need to read the source code.</p>



<h3 class="wp-block-heading">Fleet-Wide Monitoring Dashboards</h3>



<p class="wp-block-paragraph">For organizations running many servers, aggregating zombie counts across the fleet into a single dashboard (via Prometheus, Grafana, Datadog, or similar) turns an individual-server troubleshooting task into a fleet-health signal — a sudden spike in average zombie count across many hosts simultaneously, following a deploy, is a strong, fast signal that a new release introduced a reaping regression, often faster than waiting for the process table exhaustion symptoms to show up on any individual host.</p>



<h2 class="wp-block-heading">Common Mistakes Administrators Make When Handling Zombies</h2>



<p class="wp-block-paragraph"><strong>Attempting <code>kill -9</code> repeatedly on the zombie PID itself</strong> — a very common first instinct that simply doesn&#8217;t work, since the zombie has no running code left to signal; time spent here is time not spent looking at the parent process.</p>



<p class="wp-block-paragraph"><strong>Restarting the entire server reflexively</strong> — while a reboot does clear zombies, it&#8217;s a disproportionate response for what&#8217;s usually a fixable issue at the parent-process level, and it doesn&#8217;t address the underlying bug, which will simply recur.</p>



<p class="wp-block-paragraph"><strong>Ignoring a small, stable zombie count indefinitely</strong> — while a handful of transient zombies is normal, &#8220;stable at a low number&#8221; can sometimes mask a slow leak that hasn&#8217;t yet become obviously visible; trending the count over weeks, not just checking it once, is a more reliable signal.</p>



<p class="wp-block-paragraph"><strong>Not distinguishing between different parent processes responsible for different zombies</strong> — treating &#8220;we have 40 zombies&#8221; as one problem when it might actually be five separate bugs across five different services, each responsible for roughly eight zombies, leads to wasted investigation time chasing a single root cause that doesn&#8217;t exist.</p>



<h2 class="wp-block-heading">Working With Container-Specific Tooling</h2>



<p class="wp-block-paragraph">In containerized environments, standard host-level <code>ps</code> may not show you the full picture if you&#8217;re checking from outside the container&#8217;s PID namespace. Instead:</p>



<pre class="wp-block-code"><code># Check zombies inside a specific running container
docker exec &lt;container_id&gt; ps -eo pid,ppid,stat,cmd | awk '$3 ~ /Z/'

# Or, from the host, using the container's PID namespace directly
nsenter -t &lt;container_pid&gt; -p ps -eo pid,ppid,stat,cmd
</code></pre>



<p class="wp-block-paragraph">If zombies are found and the container&#8217;s PID 1 process is the application itself (rather than a proper init), the fix isn&#8217;t really a &#8220;management&#8221; action at all — it requires rebuilding the container image to include a proper init process like <code>tini</code>, since there&#8217;s no clean way to retrofit correct reaping behavior onto a running container without restarting it with the corrected image.</p>



<h2 class="wp-block-heading">Documenting Incidents for Future Reference</h2>



<p class="wp-block-paragraph">A practice worth building into any team&#8217;s operational runbooks is documenting each zombie-related incident once it&#8217;s resolved — which parent process was responsible, what the root cause turned out to be (a missing <code>SIGCHLD</code> handler, a missing <code>wait()</code> call in a specific code path, a container missing a proper init process), and what the fix was. Over time, this kind of documentation becomes genuinely valuable, both because zombie-related bugs have a tendency to recur in similar forms across different services written by different teams, and because a new on-call engineer encountering their first zombie alert benefits enormously from being able to search prior incidents rather than re-deriving the entire diagnostic process from scratch under time pressure during an active incident.</p>



<h2 class="wp-block-heading">Summary</h2>



<p class="wp-block-paragraph">Identifying zombie processes in UNIX is straightforward with <code>ps</code>, <code>top</code>, <code>htop</code>, or direct inspection of <code>/proc</code>, all of which expose the <code>Z</code> (zombie) process state. The harder and more important part is diagnosing which parent process is failing to call <code>wait()</code> on its terminated children, since zombies themselves can&#8217;t be directly killed or manipulated — remediation always goes through the parent, whether that means waiting for natural cleanup, signaling the parent, restarting it, or, in rare severe cases, rebooting the system entirely. Long-term, the right move is to combine ongoing monitoring with fixing the underlying application bugs that cause zombies to accumulate in the first place.</p>



<h2 class="wp-block-heading">FAQs</h2>



<p class="wp-block-paragraph"><strong>Can I kill a zombie process directly?</strong> No — zombies are already terminated and have no running code to signal. Any <code>kill</code> command targeted at a zombie&#8217;s own PID has no effect; you have to act on the parent process instead.</p>



<p class="wp-block-paragraph"><strong>Why does <code>kill -9</code> not remove a zombie?</strong> Because <code>kill -9</code> sends <code>SIGKILL</code>, which the kernel delivers to a running process to terminate it — but a zombie isn&#8217;t running at all, so there&#8217;s nothing for the signal to act upon.</p>



<p class="wp-block-paragraph"><strong>Will restarting the parent process affect other unrelated processes?</strong> It can, if the parent manages other work besides the zombie&#8217;s task — always check what else depends on that parent process before restarting it in a production environment.</p>



<p class="wp-block-paragraph"><strong>Is a high zombie count always a sign of a bug?</strong> A small, fluctuating number of transient zombies is normal. A persistently growing count, especially one tied to a specific parent process, is a strong sign of a reaping bug that needs to be fixed in that application&#8217;s code.</p>



<p class="wp-block-paragraph"><strong>Do container orchestrators like Kubernetes handle this automatically?</strong> Kubernetes itself doesn&#8217;t automatically fix zombie accumulation inside a container — that&#8217;s still the responsibility of whatever process runs as PID 1 inside the container. Using a proper init process inside your container image is the standard remedy.</p>



<h2 class="wp-block-heading">References</h2>



<ul class="wp-block-list">
<li>Linux man-pages — <code>ps(1)</code>, <code>top(1)</code>, <code>proc(5)</code>, <code>wait(2)</code></li>



<li>Stevens &amp; Rago — <em>Advanced Programming in the UNIX Environment</em>, Process Control chapter</li>



<li>systemd documentation — process supervision and orphan reaping</li>



<li>Docker documentation — &#8220;Using the &#8211;init flag&#8221;</li>



<li>Prometheus documentation — node_exporter process metrics</li>
</ul>
<p>The post <a href="https://awjunaid.com/operating-system/how-can-administrators-identify-and-manage-zombie-processes-in-unix/">How can administrators identify and manage zombie processes in UNIX</a> appeared first on <a href="https://awjunaid.com">Abdul Wahab Junaid</a>.</p>
]]></content:encoded>
					
					<wfw:commentRss>https://awjunaid.com/operating-system/how-can-administrators-identify-and-manage-zombie-processes-in-unix/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
		<post-id xmlns="com-wordpress:feed-additions:1">8476</post-id>	</item>
		<item>
		<title>How does the parent process handle the exit status of a child process in UNIX</title>
		<link>https://awjunaid.com/operating-system/how-does-the-parent-process-handle-the-exit-status-of-a-child-process-in-unix/</link>
					<comments>https://awjunaid.com/operating-system/how-does-the-parent-process-handle-the-exit-status-of-a-child-process-in-unix/?noamp=mobile#respond</comments>
		
		<dc:creator><![CDATA[Abdul Wahab Junaid]]></dc:creator>
		<pubDate>Sun, 03 Dec 2023 13:44:05 +0000</pubDate>
				<category><![CDATA[Operating System]]></category>
		<category><![CDATA[dailyprompt]]></category>
		<category><![CDATA[operating system]]></category>
		<guid isPermaLink="false">https://awjunaid.com/?p=8471</guid>

					<description><![CDATA[<p>Every time you run a command in a shell and the shell tells you it succeeded or failed,&#8230;</p>
<p>The post <a href="https://awjunaid.com/operating-system/how-does-the-parent-process-handle-the-exit-status-of-a-child-process-in-unix/">How does the parent process handle the exit status of a child process in UNIX</a> appeared first on <a href="https://awjunaid.com">Abdul Wahab Junaid</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<p class="wp-block-paragraph">Every time you run a command in a shell and the shell tells you it succeeded or failed, or a build script decides whether to continue based on whether the previous step &#8220;worked,&#8221; there&#8217;s a specific UNIX mechanism making that possible: the parent process retrieving the exit status of its child. It&#8217;s a small piece of machinery, but it&#8217;s foundational to how UNIX process control, shell scripting, and even service supervision all work. I want to go through exactly how this works, from the system call level up to shell-visible behavior.</p>



<h2 class="wp-block-heading">The Basic Model: fork, exec, exit, wait</h2>



<p class="wp-block-paragraph">UNIX process creation follows a well-established pattern. A process calls <code>fork()</code> to create a nearly identical copy of itself (the child), the child typically calls one of the <code>exec()</code> family of functions to replace its memory image with a new program, and eventually the child terminates by calling <code>exit()</code> (or being killed by a signal). The parent, at some point, calls <code>wait()</code> or <code>waitpid()</code> to retrieve information about how the child ended.</p>



<pre class="wp-block-code"><code>Parent Process
    |
    fork() ---------------&gt; Child Process
    |                            |
    |                          exec("/bin/ls")
    |                            |
    |                          ... runs ...
    |                            |
    |                          exit(0)
    |                            |
    wait() &lt;-------------- (kernel holds exit status until collected)
    |
   retrieves exit status
</code></pre>



<h2 class="wp-block-heading">What Happens When a Child Exits</h2>



<p class="wp-block-paragraph">When a child process calls <code>exit(status)</code> (or returns from <code>main()</code>, which implicitly calls <code>exit()</code> with the return value), the kernel doesn&#8217;t immediately remove all traces of that process. Instead, it:</p>



<ol class="wp-block-list">
<li>Releases the process&#8217;s memory, open file descriptors, and most other resources back to the system.</li>



<li>Converts the process into a zombie — a minimal kernel-level record containing the process ID, the exit status, and some resource usage statistics (CPU time consumed, etc.).</li>



<li>Sends a <code>SIGCHLD</code> signal to the parent process, notifying it that a child has changed state (exited, or in some cases stopped/continued if job control signals are being tracked).</li>



<li>Waits for the parent to call <code>wait()</code> or <code>waitpid()</code> to retrieve that information, at which point the zombie&#8217;s table entry is finally removed entirely.</li>
</ol>



<p class="wp-block-paragraph">This design exists because the kernel can&#8217;t assume in advance whether the parent cares about the exit status — so it holds onto that minimal record rather than discarding potentially important information.</p>



<h2 class="wp-block-heading">The wait() and waitpid() System Calls</h2>



<p class="wp-block-paragraph">The <code>wait()</code> system call is the simplest form — it blocks the calling process until any one of its children terminates, then returns that child&#8217;s PID and stores its exit status in the provided integer pointer.</p>



<pre class="wp-block-code"><code>#include &lt;sys/wait.h&gt;

pid_t pid = fork();
if (pid == 0) {
    // child
    exit(42);
} else {
    int status;
    pid_t child_pid = wait(&amp;status);
    // status now encodes how child_pid exited
}
</code></pre>



<p class="wp-block-paragraph"><code>waitpid()</code> is a more flexible version that lets you wait on a <em>specific</em> child PID, or use flags like <code>WNOHANG</code> to avoid blocking if no child has exited yet:</p>



<pre class="wp-block-code"><code>pid_t result = waitpid(pid, &amp;status, WNOHANG);
if (result == 0) {
    // child hasn't exited yet, keep doing other work
} else if (result == pid) {
    // child has exited, status is populated
}
</code></pre>



<h2 class="wp-block-heading">Decoding the Exit Status</h2>



<p class="wp-block-paragraph">The <code>status</code> value returned by <code>wait()</code>/<code>waitpid()</code> isn&#8217;t a plain integer you can read directly — it&#8217;s a packed bitfield that encodes multiple pieces of information: whether the process exited normally or was killed by a signal, the actual exit code, or the signal number that killed it. POSIX defines a set of macros specifically for decoding this safely:</p>



<ul class="wp-block-list">
<li><code>WIFEXITED(status)</code> — true if the child terminated normally, via <code>exit()</code> or returning from <code>main()</code>.</li>



<li><code>WEXITSTATUS(status)</code> — if <code>WIFEXITED</code> is true, extracts the actual exit code (0-255) the child passed to <code>exit()</code>.</li>



<li><code>WIFSIGNALED(status)</code> — true if the child was terminated by an unhandled signal (like <code>SIGSEGV</code> or <code>SIGKILL</code>).</li>



<li><code>WTERMSIG(status)</code> — if <code>WIFSIGNALED</code> is true, extracts which signal caused the termination.</li>



<li><code>WIFSTOPPED(status)</code> / <code>WSTOPSIG(status)</code> — relevant for job control, when a child has been stopped (not terminated) by a signal like <code>SIGSTOP</code>.</li>
</ul>



<pre class="wp-block-code"><code>if (WIFEXITED(status)) {
    printf("Child exited normally with code %d\n", WEXITSTATUS(status));
} else if (WIFSIGNALED(status)) {
    printf("Child was killed by signal %d\n", WTERMSIG(status));
}
</code></pre>



<h2 class="wp-block-heading">Why Exit Codes Matter</h2>



<p class="wp-block-paragraph">By UNIX convention, an exit code of <code>0</code> means success, and any non-zero value indicates some kind of failure or specific error condition, with the exact meaning of non-zero codes defined by whatever program is being run. This convention is what makes shell scripting and process chaining work:</p>



<pre class="wp-block-code"><code>if command1; then
    echo "command1 succeeded"
else
    echo "command1 failed with exit code $?"
fi
</code></pre>



<p class="wp-block-paragraph">The <code>$?</code> shell variable holds the exit code of the most recently completed foreground command, which the shell itself obtains via the same <code>wait()</code>/<code>waitpid()</code> mechanism under the hood — the shell is, itself, just another parent process managing child processes.</p>



<p class="wp-block-paragraph">Chained operators like <code>&amp;&amp;</code> and <code>||</code> in shell scripts also rely directly on this: <code>command1 &amp;&amp; command2</code> only runs <code>command2</code> if <code>command1</code>&#8216;s exit status was 0.</p>



<h2 class="wp-block-heading">Handling SIGCHLD Asynchronously</h2>



<p class="wp-block-paragraph">For long-running programs (servers, supervisors, shells with job control) that don&#8217;t want to block on <code>wait()</code> while other work needs to continue, the standard approach is to install a signal handler for <code>SIGCHLD</code> and call <code>waitpid()</code> with <code>WNOHANG</code> inside that handler, looping until there are no more terminated children to reap:</p>



<pre class="wp-block-code"><code>void sigchld_handler(int sig) {
    int status;
    pid_t pid;
    while ((pid = waitpid(-1, &amp;status, WNOHANG)) &gt; 0) {
        if (WIFEXITED(status)) {
            log_message("Child %d exited with code %d", pid, WEXITSTATUS(status));
        }
    }
}

int main() {
    signal(SIGCHLD, sigchld_handler);
    // continue with other work; children are reaped asynchronously
}
</code></pre>



<p class="wp-block-paragraph">This pattern is exactly what process supervisors like <code>systemd</code>, <code>supervisord</code>, and shells with job control use internally to track multiple background jobs simultaneously without blocking.</p>



<h2 class="wp-block-heading">What Happens If the Parent Never Retrieves the Exit Status</h2>



<p class="wp-block-paragraph">If the parent never calls <code>wait()</code>/<code>waitpid()</code>, the child remains in the zombie state indefinitely, occupying a slot in the kernel&#8217;s process table. This is the exact mechanism behind zombie process accumulation, which I cover in more depth in companion articles on preventing and managing zombies. If the parent itself terminates before reaping its children, those children (including any that are already zombies) get re-parented to <code>init</code> (PID 1) or a designated subreaper, which is specifically designed to reap orphaned processes promptly.</p>



<h2 class="wp-block-heading">Real-World Example: How a Shell Handles This</h2>



<p class="wp-block-paragraph">When you run <code>ls | grep foo</code> in bash, the shell forks two child processes (one for <code>ls</code>, one for <code>grep</code>), connects them with a pipe, and then waits for both to complete, tracking each one&#8217;s exit status separately. The overall exit status reported for the pipeline (<code>$?</code>) is, by default, the exit status of the <em>last</em> command in the pipeline, though bash&#8217;s <code>pipefail</code> option can change this behavior to report failure if <em>any</em> command in the pipeline fails — a subtlety that trips up a lot of shell scripters until they hit a bug caused by an early pipeline command failing silently.</p>



<h2 class="wp-block-heading">How Different Languages Expose This</h2>



<ul class="wp-block-list">
<li><strong>C/C++</strong> — direct access via <code>wait()</code>/<code>waitpid()</code> and the <code>WIF*</code>/<code>W*</code> macros, as shown above.</li>



<li><strong>Python</strong> — the <code>subprocess</code> module exposes this through <code>Popen.returncode</code> after calling <code>.wait()</code> or <code>.communicate()</code>, abstracting away the raw bitfield decoding.</li>



<li><strong>Node.js</strong> — the <code>child_process</code> module&#8217;s <code>'exit'</code> event handler receives both <code>code</code> and <code>signal</code> parameters separately, mirroring the <code>WIFEXITED</code>/<code>WIFSIGNALED</code> distinction.</li>



<li><strong>Go</strong> — <code>exec.Cmd.Wait()</code> returns an error that can be inspected via <code>ExitError.ExitCode()</code> to get the numeric exit status.</li>
</ul>



<h2 class="wp-block-heading">Best Practices</h2>



<ul class="wp-block-list">
<li>Always check exit status rather than assuming a command succeeded, particularly in scripts that chain multiple operations together.</li>



<li>Use <code>WIFEXITED</code>/<code>WIFSIGNALED</code> macros (or their language-level equivalents) rather than trying to interpret the raw status value directly, since the bit-packing format isn&#8217;t something application code should depend on.</li>



<li>In long-running processes managing multiple children, use <code>SIGCHLD</code> handling with <code>WNOHANG</code> rather than blocking <code>wait()</code> calls, to keep the process responsive.</li>



<li>Be deliberate about <code>pipefail</code>-style settings in shell scripts if pipeline failures matter to your logic.</li>



<li>Log both the exit code and any relevant context when a child process fails, to make debugging production issues far easier.</li>
</ul>



<h2 class="wp-block-heading">Exit Status Propagation in Process Chains</h2>



<p class="wp-block-paragraph">One subtlety worth understanding in depth is how exit status behaves across more complex process relationships than a simple single parent-child pair.</p>



<h3 class="wp-block-heading">Pipelines</h3>



<p class="wp-block-paragraph">As mentioned, a pipeline like <code>cmd1 | cmd2 | cmd3</code> forks a separate process for each command, and the shell tracks each one&#8217;s exit status independently. By default, only the last command&#8217;s exit status becomes <code>$?</code>, which means a failure earlier in the pipeline can go completely unnoticed unless you explicitly check for it:</p>



<pre class="wp-block-code"><code>false | true
echo $?   # prints 0, even though "false" failed, because "true" (the last command) succeeded
</code></pre>



<p class="wp-block-paragraph">Bash&#8217;s <code>PIPESTATUS</code> array gives you access to every command&#8217;s individual exit status in the most recently executed pipeline:</p>



<pre class="wp-block-code"><code>false | true
echo "${PIPESTATUS&#91;0]} ${PIPESTATUS&#91;1]}"   # prints "1 0"
</code></pre>



<p class="wp-block-paragraph">And the <code>set -o pipefail</code> option changes the overall pipeline exit status to reflect the <em>first</em> non-zero exit code among all commands in the pipeline, which is a much safer default for scripts where any stage failing should be treated as an overall failure.</p>



<h3 class="wp-block-heading">Subshells and Command Substitution</h3>



<p class="wp-block-paragraph">When you use command substitution (<code>$(command)</code>), the exit status of the substituted command is available via <code>$?</code> immediately after the substitution completes, following the same wait/exit-status mechanics as any other child process, just wrapped in slightly different shell syntax.</p>



<h3 class="wp-block-heading">Process Groups and Job Control</h3>



<p class="wp-block-paragraph">In interactive shells, background jobs (<code>command &amp;</code>) are tracked as separate jobs, each with their own PID, and the shell&#8217;s built-in <code>jobs</code> and <code>wait</code> commands let you query or block on specific jobs by job number rather than raw PID, which is more convenient for interactive use. Internally, this is still built on the same <code>waitpid()</code> mechanism, just with a friendlier interface layered on top by the shell itself.</p>



<h2 class="wp-block-heading">Common Pitfalls Around Exit Status Handling</h2>



<p class="wp-block-paragraph"><strong>Forgetting that <code>$?</code> only reflects the most recent command</strong> — if you run several commands and then check <code>$?</code>, you&#8217;re only getting the last one&#8217;s status; anything you needed to check from earlier commands must be captured immediately after each one runs, before the next command overwrites <code>$?</code>.</p>



<p class="wp-block-paragraph"><strong>Assuming a non-zero exit code always means &#8220;the same kind of failure&#8221;</strong> — different programs use different non-zero codes to mean different things (a common convention reserves specific ranges for specific error categories), so treating &#8220;any non-zero&#8221; as a single generic failure case can lose useful diagnostic information that a more careful script could have surfaced.</p>



<p class="wp-block-paragraph"><strong>Not handling the case where a child is killed by a signal versus exiting normally with a non-zero code</strong> — these are meaningfully different situations (a deliberate error exit versus a crash or external termination), and code that only checks <code>WEXITSTATUS</code> without first checking <code>WIFEXITED</code> can silently misinterpret a signal-based termination as if it were a normal exit with a garbage status value.</p>



<p class="wp-block-paragraph"><strong>Race conditions in signal-handler-based reaping</strong> — as covered in the companion article on zombie process circumstances, a <code>SIGCHLD</code> handler that doesn&#8217;t loop with <code>WNOHANG</code> can miss additional children that exited around the same time, since <code>SIGCHLD</code> delivery isn&#8217;t guaranteed to happen once per child if multiple children exit in quick succession.</p>



<h2 class="wp-block-heading">Exit Status in Process Supervision Software</h2>



<p class="wp-block-paragraph">Production process supervisors (systemd, supervisord, PM2 for Node.js applications, and similar tools) build fairly sophisticated logic on top of this basic exit-status mechanism — for example, distinguishing between a clean shutdown (exit code 0), a crash (non-zero exit code or signal termination), and using that distinction to decide whether to automatically restart the service, how long to wait before restarting (often with exponential backoff), and whether to alert an operator after repeated failures. Understanding the underlying <code>wait()</code>/exit-status mechanics makes it much easier to reason about why a supervisor is behaving a certain way — for instance, why it considers a service &#8220;flapping&#8221; and stops trying to restart it after a certain number of rapid failures.</p>



<h2 class="wp-block-heading">Exit Status Conventions Across Common Tools</h2>



<p class="wp-block-paragraph">Beyond the general 0-for-success convention, many widely used command-line tools follow more specific, documented exit code schemes that scripts can rely on for finer-grained decision making rather than treating every non-zero result identically. <code>grep</code>, for instance, returns 0 if a match was found, 1 if no match was found (which isn&#8217;t necessarily an &#8220;error&#8221; in the everyday sense, just a negative result), and 2 if an actual error occurred, such as being given a nonexistent file to search — a script that treats exit codes 1 and 2 identically would fail to distinguish &#8220;nothing matched&#8221; from &#8220;something actually went wrong.&#8221; Similarly, <code>curl</code> has a well-documented range of exit codes indicating specific failure categories (connection failures, timeout, SSL errors, and so on), and many well-designed command-line tools follow the broader convention, common across UNIX utilities, of reserving specific codes above 128 to indicate the process was terminated by a signal (the convention being 128 plus the signal number), letting a careful script distinguish a deliberate non-zero exit from an external termination even without directly inspecting <code>WIFSIGNALED</code>. Knowing that these conventions exist — and checking a tool&#8217;s documentation for its specific exit code meanings rather than assuming a generic pass/fail — is a genuinely useful habit for anyone writing production shell scripts or automation that needs to react intelligently to failure.</p>



<h2 class="wp-block-heading">Summary</h2>



<p class="wp-block-paragraph">When a child process terminates in UNIX, the kernel preserves its exit status in a minimal zombie record until the parent process retrieves it via <code>wait()</code> or <code>waitpid()</code>. That exit status is a packed value decoded with macros like <code>WIFEXITED</code> and <code>WEXITSTATUS</code>, distinguishing between normal termination with an exit code and termination by an unhandled signal. This mechanism underpins everything from simple shell exit-code checks (<code>$?</code>) to sophisticated process supervisors that asynchronously reap many children via <code>SIGCHLD</code> handlers — and neglecting it entirely is precisely what causes zombie processes to accumulate.</p>



<h2 class="wp-block-heading">FAQs</h2>



<p class="wp-block-paragraph"><strong>What&#8217;s the difference between <code>wait()</code> and <code>waitpid()</code>?</strong> <code>wait()</code> blocks until any child terminates and only works with a single child at a time in a simple way; <code>waitpid()</code> lets you target a specific child PID, use non-blocking flags like <code>WNOHANG</code>, and offers finer control overall.</p>



<p class="wp-block-paragraph"><strong>Can a parent retrieve a child&#8217;s exit status more than once?</strong> No — once <code>wait()</code>/<code>waitpid()</code> successfully retrieves a child&#8217;s status, the kernel removes the zombie entry, and that information is gone; there&#8217;s no way to query it again afterward.</p>



<p class="wp-block-paragraph"><strong>What exit status does a process killed by <code>kill -9</code> produce?</strong> <code>WIFEXITED</code> will be false and <code>WIFSIGNALED</code> will be true, with <code>WTERMSIG</code> returning the signal number for <code>SIGKILL</code> (9), since the process didn&#8217;t terminate via a normal <code>exit()</code> call.</p>



<p class="wp-block-paragraph"><strong>Why does <code>$?</code> in bash sometimes show a number greater than 255?</strong> It typically doesn&#8217;t — exit codes are conventionally limited to the 0–255 range because that&#8217;s what fits in the lower byte of the status value; values you might see above that are usually the result of specific shell or signal-related encoding conventions, not a literal larger exit code.</p>



<p class="wp-block-paragraph"><strong>Does this exit-status mechanism exist on Windows too?</strong> Windows has its own analogous concept — process exit codes retrievable via <code>GetExitCodeProcess()</code> — but the underlying mechanics (zombie states, <code>SIGCHLD</code>, <code>wait()</code>) are specific to the UNIX process model and don&#8217;t map directly onto how Windows manages process lifecycles.</p>



<h2 class="wp-block-heading">References</h2>



<ul class="wp-block-list">
<li>POSIX.1-2017 — <code>wait()</code>, <code>waitpid()</code>, <code>exit()</code> specification</li>



<li>Linux man-pages — <code>wait(2)</code>, <code>wait(3type)</code>, <code>signal(7)</code></li>



<li>Stevens &amp; Rago — <em>Advanced Programming in the UNIX Environment</em>, Process Control chapter</li>



<li>Bash Reference Manual — &#8220;Exit Status&#8221; and &#8220;The Set Builtin&#8221; (<code>pipefail</code>)</li>



<li>Python documentation — <code>subprocess</code> module</li>
</ul>
<p>The post <a href="https://awjunaid.com/operating-system/how-does-the-parent-process-handle-the-exit-status-of-a-child-process-in-unix/">How does the parent process handle the exit status of a child process in UNIX</a> appeared first on <a href="https://awjunaid.com">Abdul Wahab Junaid</a>.</p>
]]></content:encoded>
					
					<wfw:commentRss>https://awjunaid.com/operating-system/how-does-the-parent-process-handle-the-exit-status-of-a-child-process-in-unix/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
		<post-id xmlns="com-wordpress:feed-additions:1">8471</post-id>	</item>
		<item>
		<title>Explain the circumstances under which a process becomes a zombie in UNIX</title>
		<link>https://awjunaid.com/operating-system/explain-the-circumstances-under-which-a-process-becomes-a-zombie-in-unix/</link>
					<comments>https://awjunaid.com/operating-system/explain-the-circumstances-under-which-a-process-becomes-a-zombie-in-unix/?noamp=mobile#respond</comments>
		
		<dc:creator><![CDATA[Abdul Wahab Junaid]]></dc:creator>
		<pubDate>Sun, 03 Dec 2023 13:39:47 +0000</pubDate>
				<category><![CDATA[Operating System]]></category>
		<category><![CDATA[dailyprompt]]></category>
		<category><![CDATA[operating system]]></category>
		<guid isPermaLink="false">https://awjunaid.com/?p=8468</guid>

					<description><![CDATA[<p>The term &#8220;zombie process&#8221; sounds almost like a joke the first time you hear it, but it describes&#8230;</p>
<p>The post <a href="https://awjunaid.com/operating-system/explain-the-circumstances-under-which-a-process-becomes-a-zombie-in-unix/">Explain the circumstances under which a process becomes a zombie in UNIX</a> appeared first on <a href="https://awjunaid.com">Abdul Wahab Junaid</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<p class="wp-block-paragraph">The term &#8220;zombie process&#8221; sounds almost like a joke the first time you hear it, but it describes a very real and specific state in the UNIX process lifecycle. I want to lay out exactly, step by step, the precise circumstances under which a process ends up as a zombie, why the kernel allows this state to exist at all, and walk through concrete scenarios that trigger it in practice.</p>



<h2 class="wp-block-heading">The UNIX Process Lifecycle, Briefly</h2>



<p class="wp-block-paragraph">To understand when a process becomes a zombie, it helps to see the full lifecycle a process moves through:</p>



<pre class="wp-block-code"><code>   fork()
     |
     v
  &#91;Running/Ready] &lt;----&gt; &#91;Sleeping/Waiting]
     |
     | exit() called, or terminated by signal
     v
  &#91;Zombie]  &lt;-- process has terminated but exit status not yet collected
     |
     | parent calls wait()/waitpid()
     v
  &#91;Removed from process table]
</code></pre>



<p class="wp-block-paragraph">A process is a zombie specifically during the window between when it terminates and when its parent retrieves its exit status. Outside of that window, it&#8217;s either actively running/waiting, or it no longer exists in the process table at all.</p>



<h2 class="wp-block-heading">The Precise Circumstance: Termination Without Parental Acknowledgment</h2>



<p class="wp-block-paragraph">A process becomes a zombie under exactly one core circumstance: <strong>it has finished executing (via a normal <code>exit()</code> call or termination by an unhandled signal), and its parent process has not yet called <code>wait()</code> or <code>waitpid()</code> to retrieve its exit status.</strong></p>



<p class="wp-block-paragraph">This is a deliberate kernel design decision, not a bug or an unintended side effect. When a process terminates, the kernel:</p>



<ol class="wp-block-list">
<li>Deallocates almost all of the process&#8217;s resources — its memory pages, most open file descriptors, and so on.</li>



<li>Retains a minimal record: the process ID, parent process ID, exit status (or the signal that killed it), and some resource accounting information like total CPU time used.</li>



<li>Sends <code>SIGCHLD</code> to the parent to notify it that a child has changed state.</li>



<li>Leaves that minimal record in the process table, marked with the zombie state (<code>Z</code> in <code>ps</code> output), until the parent explicitly retrieves it.</li>
</ol>



<p class="wp-block-paragraph">The reasoning behind this design is that the kernel cannot know in advance whether the parent process cares about how the child exited. Many programs genuinely do care — a shell needs the exit code to decide whether to run the next command in a script, a build system needs to know if a compilation step failed, a process supervisor needs to know if a worker crashed so it can restart it. Rather than silently discarding that information, UNIX holds it until it&#8217;s explicitly collected.</p>



<h2 class="wp-block-heading">Specific Scenarios That Produce Zombies</h2>



<h3 class="wp-block-heading">Scenario 1: The Parent Simply Never Calls wait()</h3>



<p class="wp-block-paragraph">The most direct cause. A programmer forks a child process, the child does its work and exits, but the parent&#8217;s code path never includes a call to <code>wait()</code> or <code>waitpid()</code>, and no <code>SIGCHLD</code> handler is registered either. The child sits as a zombie for as long as the parent process continues running without ever collecting it.</p>



<pre class="wp-block-code"><code>pid_t pid = fork();
if (pid == 0) {
    exit(0);  // child finishes immediately
}
// parent never calls wait() — zombie persists
sleep(3600);  // parent stays alive for an hour, zombie exists the whole time
</code></pre>



<h3 class="wp-block-heading">Scenario 2: The Parent Is Busy and Delays Reaping</h3>



<p class="wp-block-paragraph">A parent process might have registered a <code>SIGCHLD</code> handler correctly, but if that parent is itself blocked on something else (a long I/O operation, waiting on a different lock), the child remains a zombie until the parent gets around to processing the signal and calling <code>wait()</code>. This kind of zombie is typically transient — it clears up once the parent&#8217;s current operation completes — but under sustained load, with many children exiting faster than the parent processes signals, this can create a temporary but visible buildup.</p>



<h3 class="wp-block-heading">Scenario 3: A Buggy or Non-Standard SIGCHLD Handler</h3>



<p class="wp-block-paragraph">If a <code>SIGCHLD</code> handler is registered but implemented incorrectly — for example, calling <code>wait()</code> without a loop, so that only one of several simultaneously-exited children gets reaped per signal delivery — zombies can accumulate because <code>SIGCHLD</code> signals aren&#8217;t queued the way some other signals are; multiple children exiting in quick succession can result in only a single <code>SIGCHLD</code> delivery, and a handler that doesn&#8217;t loop with <code>WNOHANG</code> until no more children are pending will miss some.</p>



<pre class="wp-block-code"><code>// BUGGY: only reaps one child per signal, even if several exited
void handler(int sig) {
    int status;
    wait(&amp;status);  // should loop with waitpid(-1, &amp;status, WNOHANG) instead
}
</code></pre>



<h3 class="wp-block-heading">Scenario 4: Shell Scripts Backgrounding Jobs Without wait</h3>



<p class="wp-block-paragraph">In shell scripting, launching background jobs with <code>&amp;</code> and never calling the <code>wait</code> builtin can leave zombie entries under the shell process, particularly in long-running scripts or service wrapper scripts that spawn many short-lived background tasks over their lifetime.</p>



<pre class="wp-block-code"><code>for i in {1..100}; do
    some_short_command &amp;
done
# without "wait" here, zombies can accumulate under this shell process
</code></pre>



<h3 class="wp-block-heading">Scenario 5: PID 1 Inside Containers Without Proper Init</h3>



<p class="wp-block-paragraph">This is one of the most common real-world causes in modern infrastructure. When an application is run directly as PID 1 inside a container (as is common with naive Dockerfiles), and that application spawns subprocesses without implementing proper <code>SIGCHLD</code> handling — because it was never designed to run as an init process — any orphaned or zombie processes that would normally be re-parented to a proper init system and reaped have nowhere to go, since PID 1 itself is the misbehaving application. This is specifically why tools like <code>tini</code>, <code>dumb-init</code>, and Docker&#8217;s <code>--init</code> flag exist: they act as a correct, minimal init process specifically to handle this reaping responsibility.</p>



<h3 class="wp-block-heading">Scenario 6: The Original Parent Terminates Before Reaping</h3>



<p class="wp-block-paragraph">If a parent process exits (whether normally or due to a crash) before it has reaped a zombie child, that zombie doesn&#8217;t just disappear — it gets re-parented to <code>init</code> (PID 1) or a designated subreaper process. This re-parenting is handled automatically by the kernel, and <code>init</code>/systemd is specifically designed to promptly reap any process re-parented to it, so this scenario is usually self-correcting quickly, unless the new parent (<code>init</code> itself) has some unusual issue, which is rare.</p>



<h2 class="wp-block-heading">Why Zombies Aren&#8217;t (Usually) Dangerous by Themselves</h2>



<p class="wp-block-paragraph">It&#8217;s worth being precise here: a small number of transient zombies is completely normal and expected behavior in any UNIX system that&#8217;s actively spawning and terminating processes — the zombie state exists for a brief moment between exit and reaping essentially always. The real problem is <em>accumulation</em> — zombies that persist and keep growing in number because something is systematically preventing reaping from happening. Since each process table entry (even a minimal zombie one) counts against the system&#8217;s finite process ID space, unbounded accumulation can eventually exhaust <code>pid_max</code> and prevent new processes — including critical system processes — from being created at all.</p>



<h2 class="wp-block-heading">Distinguishing Zombie Circumstances From Orphan Circumstances</h2>



<p class="wp-block-paragraph">It&#8217;s easy to blur zombies and orphans together, but the triggering circumstances are different:</p>



<ul class="wp-block-list">
<li>A process becomes an <strong>orphan</strong> when its parent terminates <em>before</em> the child does — the child is still running, just now under a new parent (<code>init</code>/systemd).</li>



<li>A process becomes a <strong>zombie</strong> when the child terminates <em>before</em> its parent has retrieved its exit status — regardless of whether the parent is still running or not.</li>
</ul>



<p class="wp-block-paragraph">A process can pass through being an orphan on its way to eventually being reaped correctly by its new parent, without ever becoming a zombie. Conversely, a process can become a zombie under its <em>original</em> parent, long before that parent ever terminates.</p>



<h2 class="wp-block-heading">Diagnosing Which Circumstance Applies</h2>



<p class="wp-block-paragraph">When you find zombies in production, the practical diagnostic path is:</p>



<ol class="wp-block-list">
<li>Identify the zombie&#8217;s parent PID with <code>ps -eo pid,ppid,stat,cmd | awk '$3 ~ /Z/'</code>.</li>



<li>Check whether that parent process is still alive and, if so, what it&#8217;s doing (<code>ps -p &lt;PPID></code>).</li>



<li>If the parent is alive but the zombie persists, it&#8217;s likely Scenario 1, 2, or 3 above — a code-level reaping bug or delay.</li>



<li>If the zombie&#8217;s parent PID is <code>1</code> (or the container&#8217;s designated subreaper), it&#8217;s likely Scenario 6 that self-corrected, and the zombie should clear shortly if <code>init</code>/systemd is functioning normally.</li>



<li>In containers, check whether the application is running directly as PID 1 without a proper init wrapper, pointing to Scenario 5.</li>
</ol>



<h2 class="wp-block-heading">Best Practices to Prevent These Circumstances</h2>



<ul class="wp-block-list">
<li>Always pair <code>fork()</code> with proper <code>wait()</code>/<code>waitpid()</code> handling, using <code>WNOHANG</code> in a loop within <code>SIGCHLD</code> handlers to avoid missing multiple simultaneous child exits.</li>



<li>Use the double-fork pattern for daemons that shouldn&#8217;t be tracked by their spawning process directly.</li>



<li>Run containerized applications under a proper init process (<code>tini</code>, <code>--init</code>) rather than directly as PID 1.</li>



<li>Use <code>wait</code> in shell scripts that background multiple jobs.</li>



<li>Monitor zombie counts over time to catch accumulation trends before they become critical.</li>
</ul>



<h2 class="wp-block-heading">A Closer Look at Why SIGCHLD Doesn&#8217;t Queue</h2>



<p class="wp-block-paragraph">Scenario 3 above — a buggy handler that only reaps one child per signal — deserves a deeper explanation, because it trips up even experienced developers who assume signals behave like a queue of discrete notifications, one per event. On UNIX systems, standard signals (as opposed to real-time signals, which do queue) are not guaranteed to be delivered once per occurrence. If a process is already handling a <code>SIGCHLD</code> delivery, or if the signal is temporarily blocked, and additional children exit during that window, the kernel does not queue up multiple pending <code>SIGCHLD</code> deliveries — it simply notes that at least one is pending, and the handler will only be invoked once when signals are unblocked, even if three children exited in that window.</p>



<p class="wp-block-paragraph">This is precisely why every correctly written <code>SIGCHLD</code> handler needs to loop:</p>



<pre class="wp-block-code"><code>void sigchld_handler(int sig) {
    int status;
    pid_t pid;
    // Keep reaping until no more children are immediately available
    while ((pid = waitpid(-1, &amp;status, WNOHANG)) &gt; 0) {
        // process each reaped child here
    }
}
</code></pre>



<p class="wp-block-paragraph">Without the <code>while</code> loop — using a single <code>wait()</code> or <code>waitpid()</code> call instead — the handler will reap exactly one child per invocation, and any additional children that exited during the same signal-coalescing window will be missed entirely, remaining as zombies until some <em>future</em> <code>SIGCHLD</code> delivery happens to trigger the handler again (which might not happen for a while, depending on how often children exit afterward).</p>



<h2 class="wp-block-heading">The Role of pid_max and Process Table Exhaustion</h2>



<p class="wp-block-paragraph">Understanding the circumstances that create zombies matters practically because of what happens if they&#8217;re allowed to accumulate without bound. Linux exposes a tunable kernel parameter, <code>/proc/sys/kernel/pid_max</code>, which defines the upper limit on process (and thread) IDs the system can allocate at once. Every zombie, despite consuming almost no active resources, still occupies one process table slot and counts against this limit. On a default configuration, <code>pid_max</code> is often set to a value like 32768 or higher on modern 64-bit systems, but a runaway zombie-generating bug in a busy service can plausibly reach that number surprisingly quickly under sustained load, at which point the system simply cannot create any new process at all — not just for the buggy application, but system-wide, including for critical administrative tasks like opening a new SSH session to investigate the problem, which is precisely the kind of compounding failure that turns a minor bug into a serious incident.</p>



<pre class="wp-block-code"><code>cat /proc/sys/kernel/pid_max
# 4194304   (example value on many modern systems)
</code></pre>



<h2 class="wp-block-heading">Zombies in Multi-Threaded Programs</h2>



<p class="wp-block-paragraph">It&#8217;s worth clarifying a related nuance: the zombie state, as classically defined, applies to <em>processes</em>, not individual threads within a process. When a thread within a multi-threaded process terminates (as opposed to the whole process), it doesn&#8217;t become a zombie in the traditional sense — thread cleanup is handled differently, typically requiring a <code>pthread_join()</code> call (the threading analog of <code>wait()</code>) to release thread-specific resources, but a &#8220;zombie thread&#8221; isn&#8217;t tracked in the system-wide process table the way a zombie process is, and won&#8217;t show up in <code>ps</code> output as a distinct zombie entry the way an unreaped child process does. Confusing these two — process-level <code>wait()</code>/zombie semantics versus thread-level <code>pthread_join()</code>/thread cleanup — is a common source of confusion for developers working across both models.</p>



<h2 class="wp-block-heading">A Comparative Look at How Different init Systems Handle Reaping</h2>



<p class="wp-block-paragraph">It&#8217;s worth knowing that not every init system has historically handled orphan and zombie reaping with equal reliability, which has practical implications for which circumstances actually resolve themselves quickly versus which ones linger. Older SysV-style init systems generally did reap orphaned children correctly, since this responsibility has always been a defining part of what it means to be PID 1 on a UNIX system, but some minimal or purpose-built init replacements used in constrained embedded environments have historically had bugs or incomplete implementations of this responsibility, occasionally leading to exactly the kind of &#8220;zombie&#8217;s parent has exited, but reaping still doesn&#8217;t happen promptly&#8221; scenario that shouldn&#8217;t normally occur. Modern systemd, by contrast, has been extensively tested specifically around this responsibility, given how central process supervision is to its overall design philosophy, and is generally considered highly reliable for prompt reaping of anything re-parented to it, whether at the true PID 1 level on a full Linux system or as a subreaper within a systemd user session or container context.</p>



<h2 class="wp-block-heading">Summary</h2>



<p class="wp-block-paragraph">A process becomes a zombie in UNIX under one precise circumstance: it has terminated, but its parent hasn&#8217;t yet retrieved its exit status via <code>wait()</code> or <code>waitpid()</code>. This can happen for several concrete reasons in practice — a parent that never calls <code>wait()</code> at all, a parent that&#8217;s delayed in processing <code>SIGCHLD</code>, a buggy signal handler that doesn&#8217;t loop through all pending exited children, shell scripts that background jobs without waiting, or containerized applications running as PID 1 without proper init handling. A small number of transient zombies is entirely normal; the real risk lies in sustained accumulation caused by one of these underlying issues going unaddressed.</p>



<h2 class="wp-block-heading">FAQs</h2>



<p class="wp-block-paragraph"><strong>Is it possible for a process to skip the zombie state entirely?</strong> No — every terminating process technically passes through the zombie state, even if only for a fraction of a second before being reaped; it&#8217;s an unavoidable part of the standard UNIX process termination sequence.</p>



<p class="wp-block-paragraph"><strong>Can a process become a zombie if it&#8217;s killed by <code>SIGKILL</code>?</strong> Yes — regardless of whether a process terminates normally via <code>exit()</code> or is killed by an unhandled signal like <code>SIGKILL</code>, it still becomes a zombie until its parent retrieves the exit status (in this case, information about which signal killed it).</p>



<p class="wp-block-paragraph"><strong>Does the zombie state consume system resources beyond the process table entry?</strong> No — memory, file descriptors, and other resources are released at termination; the zombie retains only a minimal kernel-level record.</p>



<p class="wp-block-paragraph"><strong>If a program forks but never calls exec(), can the child still become a zombie?</strong> Yes — whether or not <code>exec()</code> is called is irrelevant to the zombie mechanism; what matters is only whether the child has terminated and whether the parent has collected its exit status.</p>



<p class="wp-block-paragraph"><strong>Are zombies specific to Linux, or all UNIX-like systems?</strong> The zombie process concept is part of the general UNIX process model defined by POSIX, so it applies broadly across Linux, BSD variants, macOS, and other UNIX-like systems, not just Linux specifically.</p>



<h2 class="wp-block-heading">References</h2>



<ul class="wp-block-list">
<li>POSIX.1-2017 — process termination and <code>wait()</code>/<code>waitpid()</code> specification</li>



<li>Linux man-pages — <code>wait(2)</code>, <code>signal(7)</code>, <code>proc(5)</code></li>



<li>Stevens &amp; Rago — <em>Advanced Programming in the UNIX Environment</em>, Process Control chapter</li>



<li>Docker documentation — &#8220;Using the &#8211;init flag&#8221;</li>



<li>Bach, Maurice J. — <em>The Design of the UNIX Operating System</em></li>
</ul>
<p>The post <a href="https://awjunaid.com/operating-system/explain-the-circumstances-under-which-a-process-becomes-a-zombie-in-unix/">Explain the circumstances under which a process becomes a zombie in UNIX</a> appeared first on <a href="https://awjunaid.com">Abdul Wahab Junaid</a>.</p>
]]></content:encoded>
					
					<wfw:commentRss>https://awjunaid.com/operating-system/explain-the-circumstances-under-which-a-process-becomes-a-zombie-in-unix/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
		<post-id xmlns="com-wordpress:feed-additions:1">8468</post-id>	</item>
		<item>
		<title>Define zombie and orphan processes in the UNIX operating system</title>
		<link>https://awjunaid.com/operating-system/define-zombie-and-orphan-processes-in-the-unix-operating-system/</link>
					<comments>https://awjunaid.com/operating-system/define-zombie-and-orphan-processes-in-the-unix-operating-system/?noamp=mobile#respond</comments>
		
		<dc:creator><![CDATA[Abdul Wahab Junaid]]></dc:creator>
		<pubDate>Sun, 03 Dec 2023 13:34:33 +0000</pubDate>
				<category><![CDATA[Operating System]]></category>
		<category><![CDATA[dailyprompt]]></category>
		<category><![CDATA[operating system]]></category>
		<guid isPermaLink="false">https://awjunaid.com/?p=8465</guid>

					<description><![CDATA[<p>These two terms — zombie and orphan — get used interchangeably by people who are new to UNIX&#8230;</p>
<p>The post <a href="https://awjunaid.com/operating-system/define-zombie-and-orphan-processes-in-the-unix-operating-system/">Define zombie and orphan processes in the UNIX operating system</a> appeared first on <a href="https://awjunaid.com">Abdul Wahab Junaid</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<p class="wp-block-paragraph">These two terms — zombie and orphan — get used interchangeably by people who are new to UNIX process management, and I get why: both sound like something has gone wrong with a process&#8217;s relationship to its parent. But they describe genuinely different states with different causes, different implications, and different remediation paths. I want to define each one precisely, show exactly how they arise, and clear up the confusion between them once and for all.</p>



<h2 class="wp-block-heading">The UNIX Process Family Tree</h2>



<p class="wp-block-paragraph">Every process in UNIX, except the very first one (<code>init</code>, PID 1, or <code>systemd</code> on modern Linux systems), has a parent process — the process that called <code>fork()</code> to create it. This creates a tree structure, visible with a command like <code>pstree</code>:</p>



<pre class="wp-block-code"><code>systemd(1)---sshd(842)---bash(1021)---python3(1980)---worker(1985)
                                              |
                                            worker(1986)
</code></pre>



<p class="wp-block-paragraph">Understanding zombie and orphan states requires understanding this parent-child relationship, because both terms describe something about how a child process relates to its parent at a specific point in time.</p>



<h2 class="wp-block-heading">Defining a Zombie Process</h2>



<p class="wp-block-paragraph">A <strong>zombie process</strong> is a process that has finished executing — it has called <code>exit()</code>, or been terminated by an unhandled signal — but whose entry remains in the kernel&#8217;s process table because its parent has not yet retrieved its exit status via <code>wait()</code> or <code>waitpid()</code>.</p>



<p class="wp-block-paragraph">Key characteristics of a zombie:</p>



<ul class="wp-block-list">
<li>It is <strong>not actually running</strong> — it consumes no CPU time and its memory has already been released back to the system.</li>



<li>It retains only a <strong>minimal kernel record</strong>: PID, parent PID, exit status, and some resource usage statistics.</li>



<li>It shows up in <code>ps</code> output with a state of <code>Z</code> and is often labeled <code>&lt;defunct></code>.</li>



<li>It <strong>cannot be killed</strong> with any signal, including <code>SIGKILL</code>, because it has no running code left to terminate.</li>



<li>It is removed from the process table <strong>only</strong> when its parent calls <code>wait()</code>/<code>waitpid()</code> — an action commonly called &#8220;reaping.&#8221;</li>
</ul>



<pre class="wp-block-code"><code>$ ps -eo pid,ppid,stat,cmd | grep Z
 4821  4790 Z    &#91;worker] &lt;defunct&gt;
</code></pre>



<p class="wp-block-paragraph">A zombie exists as a deliberate design choice in UNIX: the kernel preserves a terminated child&#8217;s exit status specifically because the parent might need it, and it can&#8217;t know in advance whether the parent cares. The zombie state is the holding pattern between &#8220;child has finished&#8221; and &#8220;parent has acknowledged that the child finished.&#8221;</p>



<h2 class="wp-block-heading">Defining an Orphan Process</h2>



<p class="wp-block-paragraph">An <strong>orphan process</strong> is a process whose parent has terminated while the child is still running. Unlike a zombie, an orphan process is fully alive and active — it&#8217;s just missing its original parent.</p>



<p class="wp-block-paragraph">Key characteristics of an orphan:</p>



<ul class="wp-block-list">
<li>It is <strong>actively running</strong>, using CPU and memory normally, just like any other process.</li>



<li>It gets <strong>automatically re-parented</strong> by the kernel — typically to <code>init</code> (PID 1) or, on modern Linux systems, to whichever process has been designated as a subreaper (often the nearest ancestor process group leader or <code>systemd</code> in user session scopes).</li>



<li>This re-parenting happens <strong>immediately and automatically</strong>; there&#8217;s no window where the orphan has no parent at all.</li>



<li>Once re-parented, the new parent (<code>init</code>/systemd) is specifically designed to reap the orphan promptly once it eventually does terminate, which means orphans rarely, if ever, become long-lived zombies themselves.</li>
</ul>



<pre class="wp-block-code"><code>$ ps -eo pid,ppid,cmd | grep worker
 1985     1 &#91;worker]     # PPID is 1, meaning it was re-parented to init
</code></pre>



<p class="wp-block-paragraph">Orphans are a completely normal and common occurrence — background daemons and services are deliberately designed to become orphans (via the double-fork technique) specifically so they can keep running independently of whatever process originally launched them, without being tied to that launching process&#8217;s lifetime.</p>



<h2 class="wp-block-heading">The Core Distinction</h2>



<figure class="wp-block-table"><table class="has-fixed-layout"><thead><tr><th>Aspect</th><th>Zombie Process</th><th>Orphan Process</th></tr></thead><tbody><tr><td>Is it running?</td><td>No — already terminated</td><td>Yes — fully active</td></tr><tr><td>What triggers the state?</td><td>Child exits before parent calls <code>wait()</code></td><td>Parent exits before child does</td></tr><tr><td>CPU/memory usage</td><td>None — resources already released</td><td>Normal, same as any running process</td></tr><tr><td>Can it be killed with a signal?</td><td>No — it&#8217;s already dead</td><td>Yes — it&#8217;s a normal running process</td></tr><tr><td>How does it resolve?</td><td>Original parent calls <code>wait()</code>, or parent terminates and <code>init</code>/systemd reaps it</td><td>Kernel automatically re-parents it to <code>init</code>/systemd</td></tr><tr><td>Is it usually a problem?</td><td>Only if it accumulates persistently</td><td>No — often an intentional pattern for daemons</td></tr></tbody></table></figure>



<h2 class="wp-block-heading">Visualizing the Difference</h2>



<pre class="wp-block-code"><code>ZOMBIE PROCESS SCENARIO
Parent (still running) ---- fork() ----&gt; Child
Parent: ... busy doing other things ...
Child: exit() -----&gt; &#91;ZOMBIE - waiting to be reaped]
Parent: (eventually) wait() -----&gt; zombie is removed


ORPHAN PROCESS SCENARIO
Parent ---- fork() ----&gt; Child (still running)
Parent: exit() -----&gt; &#91;Parent terminates]
Child: (kernel re-parents child) -----&gt; now child of init/systemd
Child: continues running normally, will be reaped promptly by init/systemd when it eventually exits
</code></pre>



<h2 class="wp-block-heading">Can a Process Be Both?</h2>



<p class="wp-block-paragraph">Interestingly, yes — but not simultaneously in the way people sometimes assume. A process can become an orphan first (its original parent terminates while it&#8217;s still running), get re-parented to <code>init</code>/systemd, and <em>later</em>, when it eventually exits, briefly pass through the zombie state until <code>init</code>/systemd reaps it — which it does very promptly, since reaping orphaned children is specifically part of <code>init</code>&#8216;s/systemd&#8217;s job. So while a process can experience both states across its lifetime, it&#8217;s not accurate to say a single process is &#8220;a zombie orphan&#8221; at one moment — these are two different transitions that can happen sequentially.</p>



<h2 class="wp-block-heading">Why Orphans Are (Usually) Intentional and Zombies Are (Usually) Accidental</h2>



<p class="wp-block-paragraph">This is probably the most important practical distinction. Orphaning is frequently done <strong>on purpose</strong> — the classic double-fork daemonization technique deliberately creates an orphan so that a long-running background service isn&#8217;t tied to the lifetime of whatever shell or process launched it initially:</p>



<pre class="wp-block-code"><code>Original process
   |
   fork()
   |
Child --- fork() ---&gt; Grandchild (the actual daemon)
   |
  exit()   &lt;- child exits immediately, orphaning the grandchild
             (grandchild is now re-parented to init/systemd and runs independently)
</code></pre>



<p class="wp-block-paragraph">Zombie accumulation, on the other hand, is almost always <strong>accidental</strong> — the result of a programming bug where a parent process fails to call <code>wait()</code>/<code>waitpid()</code> for children it should be tracking. There&#8217;s no standard, intentional pattern where you <em>want</em> zombies to persist; the zombie state is only ever meant to be transient.</p>



<h2 class="wp-block-heading">Real-World Examples</h2>



<p class="wp-block-paragraph"><strong>Orphan example:</strong> A web server process forks a worker to handle a long file conversion job, then the parent web server is restarted as part of a deploy. The worker keeps running (it&#8217;s now an orphan, re-parented to <code>init</code>), finishes the conversion, and exits normally — briefly becoming a zombie until <code>init</code> reaps it, all without any manual intervention needed.</p>



<p class="wp-block-paragraph"><strong>Zombie example:</strong> A custom job scheduler forks a subprocess for every scheduled task but has a bug where its <code>SIGCHLD</code> handler only reaps one child per signal instead of looping with <code>WNOHANG</code>. Under heavy load, with many tasks completing in quick succession, zombies accumulate because signal delivery doesn&#8217;t queue multiple simultaneous <code>SIGCHLD</code> events — this requires a code fix, not just waiting it out.</p>



<h2 class="wp-block-heading">How to Check for Each in Practice</h2>



<pre class="wp-block-code"><code># Find zombies
ps -eo pid,ppid,stat,cmd | awk '$3 ~ /Z/'

# Find processes re-parented to init (PPID 1) — a sign they were orphaned
ps -eo pid,ppid,cmd | awk '$2 == 1'
</code></pre>



<p class="wp-block-paragraph">Note that finding processes with PPID 1 doesn&#8217;t necessarily mean they were all orphaned — some processes are intentionally started directly under <code>init</code>/systemd as services — so this check is more of an indicator than a definitive diagnostic on its own.</p>



<h2 class="wp-block-heading">Best Practices</h2>



<ul class="wp-block-list">
<li>Understand that orphaning is often intentional (daemonization) while zombie accumulation is almost always a bug that needs fixing.</li>



<li>Never try to &#8220;kill&#8221; a zombie directly — always address the parent process instead.</li>



<li>Use the double-fork pattern deliberately when you want a background process to survive independently of its launcher.</li>



<li>Monitor zombie counts, not orphan counts, as the meaningful health signal — orphans re-parented to <code>init</code>/systemd are expected and self-managing.</li>



<li>Ensure proper <code>SIGCHLD</code> handling in any code that forks multiple children, to avoid accidental zombie accumulation.</li>
</ul>



<h2 class="wp-block-heading">The Historical Origin of the Terminology</h2>



<p class="wp-block-paragraph">The term &#8220;zombie&#8221; for this process state has been in continuous use in UNIX documentation and folklore since at least the early Berkeley UNIX (BSD) manuals of the 1980s, and it&#8217;s stuck around precisely because it&#8217;s such an apt metaphor — a process that has technically &#8220;died&#8221; (terminated) but continues to exist in a limited, non-functional form (occupying a process table slot) until something (the parent calling <code>wait()</code>) allows it to be properly &#8220;buried&#8221; (removed from the table). &#8220;Orphan,&#8221; similarly, borrows directly from the everyday meaning of a child whose parent is no longer present, though unlike the grim implications of a human orphan, a UNIX orphan process is usually in a perfectly fine, actively running state — it&#8217;s arguably the less concerning of the two despite the more emotionally loaded name.</p>



<h2 class="wp-block-heading">Subreapers: A More Precise Look at Modern Re-Parenting</h2>



<p class="wp-block-paragraph">While it&#8217;s common shorthand to say orphans get re-parented to &#8220;init&#8221; or &#8220;PID 1,&#8221; modern Linux actually offers more flexibility here through the <code>prctl(PR_SET_CHILD_SUBREAPER)</code> mechanism, introduced specifically to address a gap in container and process-supervision use cases. A process can mark itself as a &#8220;subreaper,&#8221; meaning that any of its descendants that would otherwise be orphaned and re-parented all the way up to PID 1 instead get re-parented to the nearest ancestor marked as a subreaper. This is exactly the mechanism that tools like <code>tini</code>, <code>systemd</code> (within user sessions and containers), and various container runtimes use to correctly reap orphaned processes within a container&#8217;s own process namespace, without needing every container to somehow re-parent orphans to the <em>host&#8217;s</em> actual PID 1, which would be both impractical and would break the process isolation containers are meant to provide in the first place.</p>



<pre class="wp-block-code"><code>Without subreaper:
Container's PID 1 (app) --- fork() ---&gt; worker
Container's PID 1 (app) exits --- worker becomes orphan, re-parented to actual host init (if visible) or left stranded

With a proper subreaper (e.g., tini as container PID 1):
tini (PID 1, subreaper) --- fork() ---&gt; app --- fork() ---&gt; worker
app exits --- worker becomes orphan, correctly re-parented to tini, which reaps it promptly
</code></pre>



<p class="wp-block-paragraph">This distinction matters in practice because it&#8217;s exactly why &#8220;just running as PID 1 in a container&#8221; isn&#8217;t automatically equivalent to having correct init-level orphan and zombie handling — the re-parenting target matters, and a naive application binary run directly as PID 1 typically hasn&#8217;t implemented subreaper logic or proper <code>SIGCHLD</code> handling at all, since it was never designed with that responsibility in mind.</p>



<h2 class="wp-block-heading">Observing the Full Lifecycle in a Hands-On Example</h2>



<p class="wp-block-paragraph">To make the distinction between these two states completely concrete, here&#8217;s a small demonstration using shell commands you can run yourself on a Linux system.</p>



<pre class="wp-block-code"><code># Terminal 1: launch a background process, then immediately exit the launching shell
(sleep 60 &amp;) 
# the sleep process is now an orphan almost immediately, since the subshell
# that forked it exits right after backgrounding it

# Check its parent — should now show PPID of 1 (or your session's subreaper)
ps -eo pid,ppid,cmd | grep sleep
</code></pre>



<p class="wp-block-paragraph">And to observe a zombie directly:</p>



<pre class="wp-block-code"><code># A simple C program that forks and never waits, with a long-lived parent
cat &lt;&lt;'EOF' &gt; zombie_demo.c
#include &lt;stdio.h&gt;
#include &lt;unistd.h&gt;
#include &lt;sys/wait.h&gt;
int main() {
    pid_t pid = fork();
    if (pid == 0) {
        _exit(0);  // child exits almost immediately
    } else {
        sleep(30); // parent stays alive, never calls wait()
    }
    return 0;
}
EOF
gcc -o zombie_demo zombie_demo.c
./zombie_demo &amp;
sleep 1
ps -eo pid,ppid,stat,cmd | grep defunct
</code></pre>



<p class="wp-block-paragraph">Running this should show the child process in state <code>Z</code> for the roughly 30-second window before the parent exits and the zombie gets cleaned up via re-parenting to <code>init</code>/systemd.</p>



<h2 class="wp-block-heading">Common Points of Confusion Worth Clearing Up Explicitly</h2>



<p class="wp-block-paragraph">Because these terms come up so often in interview questions and documentation without always being explained carefully, it&#8217;s worth directly addressing a few misconceptions I see repeated frequently. First, an orphan is not automatically a problem to be fixed — unlike a zombie, there&#8217;s no &#8220;reaping bug&#8221; implied by a process being an orphan; it&#8217;s simply a normal, expected state that resolves itself through automatic re-parenting. Second, a zombie is not &#8220;still running in the background&#8221; in any meaningful sense, despite sometimes being described casually that way — it has fully terminated, and referring to it as &#8220;running&#8221; causes real confusion when someone then wonders why <code>top</code> shows 0% CPU for a process they&#8217;ve been told is still executing. Third, neither state indicates data loss or corruption by itself — a zombie&#8217;s minimal record actually preserves useful information (the exit status) rather than losing it, and an orphan&#8217;s re-parenting doesn&#8217;t affect the orphaned process&#8217;s own internal state or the work it&#8217;s doing at all, only its position in the process tree.</p>



<h2 class="wp-block-heading">Why This Distinction Comes Up So Often in Technical Interviews</h2>



<p class="wp-block-paragraph">If you&#8217;re studying these concepts for a systems programming or DevOps interview, it&#8217;s worth knowing why this particular pair of definitions is such a popular question: it tests whether a candidate actually understands the UNIX process lifecycle at a mechanical level, rather than just having memorized &#8220;zombie bad, orphan also sounds bad.&#8221; A strong answer distinguishes not just the definitions but the <em>practical implications</em> — that zombies typically signal a code-level bug worth fixing, while orphans are frequently the deliberate, correct outcome of a well-known daemonization pattern. Interviewers often follow up by asking how you&#8217;d actually diagnose and resolve a zombie accumulation issue on a live system, which is exactly the kind of hands-on <code>ps</code>/<code>waitpid()</code>/parent-process reasoning covered in the companion article on identifying and managing zombie processes.</p>



<h2 class="wp-block-heading">Summary</h2>



<p class="wp-block-paragraph">A zombie process is a terminated process still occupying a process table slot because its parent hasn&#8217;t retrieved its exit status; it&#8217;s dead, consumes no active resources, and can only be cleared by the parent reaping it. An orphan process is a still-running process whose original parent has terminated, automatically re-parented by the kernel to <code>init</code> or a subreaper, which will then reap it normally when it eventually exits. Zombies are typically the result of a bug and are only ever meant to be transient; orphans are frequently an intentional and standard pattern used to run background daemons independently of the process that launched them.</p>



<h2 class="wp-block-heading">FAQs</h2>



<p class="wp-block-paragraph"><strong>Is an orphan process dangerous?</strong> No — orphans are normal and expected, especially for background daemons using the double-fork technique. The kernel handles re-parenting automatically and reliably.</p>



<p class="wp-block-paragraph"><strong>Can zombies be prevented entirely?</strong> Yes, with disciplined use of <code>wait()</code>/<code>waitpid()</code>, proper <code>SIGCHLD</code> handling, and (in containers) a correct init process — zombie accumulation is always addressable through correct process management code.</p>



<p class="wp-block-paragraph"><strong>Why does init/systemd reap orphans so reliably?</strong> Because <code>init</code>/systemd is specifically designed as the ultimate ancestor of every process on the system, and part of its designated responsibility is to promptly call <code>wait()</code> on any process re-parented to it.</p>



<p class="wp-block-paragraph"><strong>What UNIX command shows the parent-child relationship most clearly?</strong> <code>pstree</code> gives the clearest visual hierarchy of parent-child relationships across the whole system, while <code>ps -eo pid,ppid,stat,cmd</code> gives a more detailed tabular view including process state.</p>



<p class="wp-block-paragraph"><strong>Do zombies and orphans exist on macOS as well as Linux?</strong> Yes — macOS is built on a UNIX-derived kernel (Darwin/XNU) and follows the same POSIX process model, so both concepts apply there in the same way they do on Linux and other UNIX variants.</p>



<h2 class="wp-block-heading">References</h2>



<ul class="wp-block-list">
<li>POSIX.1-2017 — process lifecycle, <code>wait()</code>, <code>fork()</code> specification</li>



<li>Linux man-pages — <code>wait(2)</code>, <code>proc(5)</code>, <code>pstree(1)</code></li>



<li>Stevens &amp; Rago — <em>Advanced Programming in the UNIX Environment</em>, Process Control chapter</li>



<li>Bach, Maurice J. — <em>The Design of the UNIX Operating System</em></li>



<li>systemd documentation — process supervision and subreaper behavior</li>
</ul>
<p>The post <a href="https://awjunaid.com/operating-system/define-zombie-and-orphan-processes-in-the-unix-operating-system/">Define zombie and orphan processes in the UNIX operating system</a> appeared first on <a href="https://awjunaid.com">Abdul Wahab Junaid</a>.</p>
]]></content:encoded>
					
					<wfw:commentRss>https://awjunaid.com/operating-system/define-zombie-and-orphan-processes-in-the-unix-operating-system/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
		<post-id xmlns="com-wordpress:feed-additions:1">8465</post-id>	</item>
		<item>
		<title>Describe the role of the Chrome Task Manager in monitoring and managing processes</title>
		<link>https://awjunaid.com/operating-system/describe-the-role-of-the-chrome-task-manager-in-monitoring-and-managing-processes/</link>
					<comments>https://awjunaid.com/operating-system/describe-the-role-of-the-chrome-task-manager-in-monitoring-and-managing-processes/?noamp=mobile#respond</comments>
		
		<dc:creator><![CDATA[Abdul Wahab Junaid]]></dc:creator>
		<pubDate>Sun, 03 Dec 2023 13:32:25 +0000</pubDate>
				<category><![CDATA[Operating System]]></category>
		<category><![CDATA[dailyprompt]]></category>
		<category><![CDATA[operating system]]></category>
		<guid isPermaLink="false">https://awjunaid.com/?p=8462</guid>

					<description><![CDATA[<p>Most people know about the Windows Task Manager or Activity Monitor on macOS, but far fewer people know&#8230;</p>
<p>The post <a href="https://awjunaid.com/operating-system/describe-the-role-of-the-chrome-task-manager-in-monitoring-and-managing-processes/">Describe the role of the Chrome Task Manager in monitoring and managing processes</a> appeared first on <a href="https://awjunaid.com">Abdul Wahab Junaid</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<p class="wp-block-paragraph">Most people know about the Windows Task Manager or Activity Monitor on macOS, but far fewer people know that Chrome has its own dedicated task manager, tucked away in the browser&#8217;s menus, purpose-built for exactly one job: showing you what every tab, extension, and background process inside Chrome is actually doing to your system&#8217;s CPU and memory. I want to explain why this tool exists at all, how it works under the hood, and how to actually use it to diagnose a sluggish browser.</p>



<h2 class="wp-block-heading">Why Chrome Needs Its Own Task Manager</h2>



<p class="wp-block-paragraph">To understand why the Chrome Task Manager exists, you have to understand Chrome&#8217;s underlying process architecture, which is fundamentally different from older browsers. Early browsers (and still some today) run as a single monolithic process — every tab, every extension, every plugin all sharing one process&#8217;s memory space. This design has a serious flaw: if any one tab crashes, or if a memory leak occurs anywhere, it can take down the entire browser, every tab included.</p>



<p class="wp-block-paragraph">Chrome, since its original 2008 launch, adopted a <strong>multi-process architecture</strong> instead. Each tab (in many cases), each extension, and various browser subsystems run as separate OS-level processes, coordinated by a central browser process. This is directly analogous to how an operating system manages independent processes — Chrome effectively behaves like its own miniature operating system, which is precisely why the terminology and monitoring tools mirror OS-level concepts so closely.</p>



<p class="wp-block-paragraph">Because of this architecture, your regular OS-level Task Manager (Windows) or Activity Monitor (macOS) shows you a pile of generic-looking <code>chrome.exe</code> or <code>Google Chrome Helper</code> entries, without telling you <em>which tab or extension</em> each process actually belongs to. That&#8217;s the exact gap the Chrome Task Manager fills.</p>



<h2 class="wp-block-heading">How to Open the Chrome Task Manager</h2>



<p class="wp-block-paragraph">You can open it in a few ways:</p>



<ul class="wp-block-list">
<li><strong>Menu path:</strong> Click the three-dot menu (top right) → More Tools → Task Manager</li>



<li><strong>Keyboard shortcut:</strong> <code>Shift+Esc</code> (works on Windows, Linux, and ChromeOS; on macOS, it&#8217;s accessible via the Window menu)</li>



<li><strong>Right-click on empty tab bar space:</strong> Some versions of Chrome expose it directly from this context menu</li>
</ul>



<h2 class="wp-block-heading">What the Chrome Task Manager Shows</h2>



<p class="wp-block-paragraph">Once open, it presents a table very similar in spirit to the OS-level task managers, but scoped specifically to Chrome&#8217;s internal processes:</p>



<pre class="wp-block-code"><code>Task                          Memory Footprint   CPU   Network   Process ID
Tab: Gmail                     180 MB              2%    0 KB/s    4821
Tab: YouTube                   310 MB              15%   45 KB/s   4903
Extension: Ad Blocker Plus      45 MB              0%    0 KB/s    4955
GPU Process                     95 MB              5%    0 KB/s    4790
Subframe: doubleclick.net        20 MB              1%    2 KB/s    5012
Browser                         120 MB              1%    0 KB/s    4780
</code></pre>



<p class="wp-block-paragraph">Key columns and what they mean:</p>



<ul class="wp-block-list">
<li><strong>Task</strong> — identifies exactly which tab, extension, subframe, or internal Chrome subsystem this row represents.</li>



<li><strong>Memory Footprint</strong> — how much physical memory that specific process is currently using; this is one of the most commonly used columns for diagnosing &#8220;why is my computer running out of RAM.&#8221;</li>



<li><strong>CPU</strong> — the percentage of CPU currently being consumed, invaluable for finding a runaway tab (a common culprit: an ad-heavy page with a poorly optimized JavaScript animation loop, or a cryptocurrency-mining script embedded maliciously in a compromised site).</li>



<li><strong>Network</strong> — live network throughput for that specific process, useful for spotting a tab that&#8217;s unexpectedly still transferring data in the background.</li>



<li><strong>Process ID</strong> — the actual OS-level PID, which is the bridge between Chrome&#8217;s internal task manager and your operating system&#8217;s own process tools (<code>ps</code>, Task Manager, Activity Monitor) if you need to correlate or take further action at the OS level.</li>
</ul>



<p class="wp-block-paragraph">You can right-click the column header to add additional columns, including a <strong>Process ID</strong>, <strong>GPU Memory</strong>, <strong>SQLite Memory</strong>, and <strong>JavaScript Memory</strong> breakdown for deeper diagnostics.</p>



<h2 class="wp-block-heading">Chrome&#8217;s Process Model in More Depth</h2>



<p class="wp-block-paragraph">Chrome&#8217;s architecture typically breaks down into these process categories:</p>



<ul class="wp-block-list">
<li><strong>Browser process</strong> — the main coordinating process, handling the UI, address bar, bookmarks, and overall browser chrome (the window frame, not to be confused with tab content).</li>



<li><strong>Renderer processes</strong> — each responsible for parsing HTML, executing JavaScript, and rendering the actual content of one or more tabs. By default, Chrome uses a &#8220;site isolation&#8221; model where different sites (even different tabs of the same site, depending on settings and version) get separate renderer processes specifically as a security boundary, preventing one compromised page from reading another page&#8217;s data through a shared process memory space.</li>



<li><strong>GPU process</strong> — handles hardware-accelerated graphics rendering, shared across tabs since spinning up a separate GPU process per tab would be wasteful.</li>



<li><strong>Extension processes</strong> — many extensions run in their own isolated processes, both for stability (a buggy extension won&#8217;t crash your tabs) and security.</li>



<li><strong>Utility processes</strong> — handle specific isolated tasks like audio processing, network service functionality, or PDF rendering.</li>
</ul>



<p class="wp-block-paragraph">This is conceptually very similar to how an operating system isolates processes from each other using virtual memory and separate address spaces — Chrome deliberately borrowed this OS-level design pattern specifically to gain the same benefits: fault isolation (one crashing tab doesn&#8217;t take down the whole browser) and security isolation (a compromised renderer process has a much harder time accessing data outside its own sandbox).</p>



<h2 class="wp-block-heading">Practical Use Cases for the Chrome Task Manager</h2>



<h3 class="wp-block-heading">Diagnosing a Sluggish Browser</h3>



<p class="wp-block-paragraph">If your whole system feels slow and you suspect Chrome is the culprit, sorting the Task Manager by CPU or Memory Footprint quickly reveals which specific tab or extension is responsible, rather than you having to guess and close tabs one at a time.</p>



<h3 class="wp-block-heading">Identifying Memory Leaks in Web Apps</h3>



<p class="wp-block-paragraph">Web developers testing their own applications can watch the Memory Footprint and JavaScript Memory columns over time while using their app; a steadily climbing number that never plateaus, even during idle periods, is a strong signal of a memory leak in the page&#8217;s JavaScript — often caused by event listeners or timers that are never cleaned up.</p>



<h3 class="wp-block-heading">Spotting Malicious or Poorly Behaved Extensions</h3>



<p class="wp-block-paragraph">An extension that shows unexpectedly high CPU or network usage, especially when you&#8217;re not actively interacting with any tab, is worth investigating further — this is a legitimate, lightweight way to catch a misbehaving or potentially malicious browser extension before resorting to a full uninstall-and-reinstall cycle to identify the culprit.</p>



<h3 class="wp-block-heading">Ending a Frozen Tab Without Restarting the Whole Browser</h3>



<p class="wp-block-paragraph">Because each tab typically runs as its own OS process, you can select the specific frozen or runaway task in the Task Manager and click &#8220;End process,&#8221; which kills just that tab&#8217;s renderer process — your other tabs, extensions, and the browser itself remain completely unaffected. This is a direct practical benefit of Chrome&#8217;s multi-process architecture: an isolated failure doesn&#8217;t need an isolated fix that&#8217;s disruptive to everything else.</p>



<h2 class="wp-block-heading">Comparing the Chrome Task Manager to OS-Level Tools</h2>



<figure class="wp-block-table"><table class="has-fixed-layout"><thead><tr><th>Aspect</th><th>Chrome Task Manager</th><th>OS Task Manager / Activity Monitor / <code>ps</code></th></tr></thead><tbody><tr><td>Scope</td><td>Only Chrome&#8217;s internal processes</td><td>Every process on the entire system</td></tr><tr><td>Granularity</td><td>Per-tab, per-extension, per-subsystem detail</td><td>Often shows generic <code>chrome.exe</code>/<code>Google Chrome Helper</code> entries without per-tab detail</td></tr><tr><td>Network detail</td><td>Per-task network throughput within Chrome</td><td>System-wide network usage, not attributed to specific browser tabs</td></tr><tr><td>Ending a task</td><td>Kills just that tab/extension&#8217;s process</td><td>Can kill any process on the system, including non-browser software</td></tr><tr><td>Use case</td><td>Diagnosing browser-specific slowdowns</td><td>General system-wide process and resource monitoring</td></tr></tbody></table></figure>



<p class="wp-block-paragraph">They&#8217;re genuinely complementary tools — the OS-level tool tells you Chrome overall is using a lot of RAM; Chrome&#8217;s own task manager tells you <em>which tab</em> is responsible for that RAM usage.</p>



<h2 class="wp-block-heading">Chrome Task Manager on Different Platforms</h2>



<p class="wp-block-paragraph">The Task Manager is available on <strong>Windows</strong>, <strong>macOS</strong>, and <strong>Linux</strong> builds of Chrome, accessible through the same menu path, though the keyboard shortcut differs slightly (macOS uses the Window menu since <code>Shift+Esc</code> is reserved differently there in some configurations). On <strong>ChromeOS</strong>, since the entire operating system is essentially built around the Chrome browser and web technologies, the Task Manager takes on even greater importance as a primary system monitoring tool, similar in role to a desktop OS&#8217;s built-in task manager. On <strong>Android</strong> and <strong>iOS</strong>, Chrome&#8217;s mobile versions don&#8217;t expose an equivalent Task Manager UI, largely because those platforms manage app and tab memory very differently under stricter, OS-enforced resource constraints, and mobile Chrome doesn&#8217;t offer the same level of granular per-tab process control to end users.</p>



<h2 class="wp-block-heading">Best Practices for Using It Effectively</h2>



<ul class="wp-block-list">
<li>Sort by Memory Footprint or CPU regularly if you keep large numbers of tabs open, to catch problem tabs before they degrade overall system performance.</li>



<li>Add the Process ID column if you need to correlate a specific Chrome task with what your OS-level tools are reporting.</li>



<li>Use &#8220;End process&#8221; selectively on a misbehaving tab rather than restarting the entire browser, preserving your other open tabs and their state.</li>



<li>Periodically review extension resource usage, especially after installing new extensions, to catch anything unexpectedly resource-hungry early.</li>



<li>Combine Chrome&#8217;s Task Manager with <code>chrome://process-internals</code> or <code>chrome://tracing</code> for deeper, more technical diagnostics if you&#8217;re a developer investigating a specific performance issue.</li>
</ul>



<h2 class="wp-block-heading">Deeper Diagnostic Views Beyond the Basic Task Manager</h2>



<p class="wp-block-paragraph">For users and developers who need more than the standard Task Manager window offers, Chrome exposes several internal diagnostic pages, accessible by typing a special URL directly into the address bar, that go considerably deeper than the Task Manager&#8217;s summary table.</p>



<p class="wp-block-paragraph"><strong><code>chrome://process-internals</code></strong> — provides a detailed, hierarchical view of every process Chrome has spawned, including which specific site or origin each renderer process is hosting, which is especially useful for understanding exactly how Site Isolation is partitioning your open tabs behind the scenes, something the standard Task Manager only hints at through its task labels.</p>



<p class="wp-block-paragraph"><strong><code>chrome://tracing</code></strong> (or the newer <code>chrome://tracing</code> successor tooling integrated into DevTools&#8217; Performance panel) — captures extremely detailed, timestamped traces of everything happening across Chrome&#8217;s processes, down to individual rendering frames, garbage collection pauses, and network events, intended primarily for engineers debugging deep performance issues in Chrome itself or in a specific web application running inside it.</p>



<p class="wp-block-paragraph"><strong><code>chrome://memory-redirect</code></strong> and DevTools&#8217; Memory panel — offer heap snapshots and allocation timelines scoped to a specific tab&#8217;s JavaScript execution, letting a web developer pinpoint exactly which JavaScript objects are accumulating and never being garbage collected, going well beyond the single &#8220;Memory Footprint&#8221; number the Task Manager shows for that tab as a whole.</p>



<p class="wp-block-paragraph"><strong><code>chrome://discards</code></strong> — shows Chrome&#8217;s own internal tab-discarding decisions, relevant because on systems under memory pressure, Chrome will proactively &#8220;discard&#8221; (unload) background tabs to free memory, and this page reveals which tabs are eligible for discarding and why, which helps explain a tab that unexpectedly reloads from scratch when you switch back to it after leaving it in the background for a while.</p>



<h2 class="wp-block-heading">The Task Manager&#8217;s Role in Understanding Site Isolation</h2>



<p class="wp-block-paragraph">Site Isolation, a major security feature rolled out across Chrome starting around 2018, deliberately increased the number of separate renderer processes Chrome creates, specifically to ensure that even different iframes or subframes from different origins within the same tab run in fully separate OS processes, closing a category of side-channel and cross-origin data-leak vulnerabilities (including those related to speculative execution attacks like Spectre) that a shared-process model couldn&#8217;t fully protect against. One visible side effect of this security improvement is that the Chrome Task Manager itself often shows more granular entries than users might expect — a single tab embedding ads or widgets from several different third-party domains might show up as several separate &#8220;Subframe&#8221; entries in the Task Manager, each representing an isolated process for that specific embedded content&#8217;s origin. Understanding this connection between the security architecture and what you see in the Task Manager helps explain why a single visually simple webpage can sometimes correspond to five or six separate process rows.</p>



<h2 class="wp-block-heading">Comparing Chrome&#8217;s Approach to Other Browsers</h2>



<p class="wp-block-paragraph">It&#8217;s worth noting that Chrome&#8217;s dedicated Task Manager approach, and the underlying multi-process architecture that motivates it, has been influential across the browser industry. Microsoft Edge, since its 2020 rebuild on the Chromium engine, inherited essentially the same multi-process architecture and offers its own nearly identical Browser Task Manager (accessible via <code>Shift+Esc</code> there too, or through its own menu). Firefox, while architected differently under the hood (using a project called &#8220;Electrolysis&#8221; and later &#8220;Fission&#8221; to introduce multi-process and site-isolation capabilities of its own), also offers an &#8220;about:processes&#8221; page providing broadly comparable per-tab resource visibility, though the terminology and level of detail differ somewhat from Chrome&#8217;s presentation. Safari, by contrast, has historically exposed considerably less granular per-tab process information to end users directly, though Activity Monitor on macOS can still show separate <code>com.apple.WebKit.WebContent</code> process entries corresponding to Safari&#8217;s own internal multi-process design.</p>



<h2 class="wp-block-heading">Practical Troubleshooting Workflow Using the Task Manager</h2>



<p class="wp-block-paragraph">When a user reports &#8220;Chrome is using all my RAM&#8221; or &#8220;my fan is spinning constantly with Chrome open,&#8221; a systematic troubleshooting approach using the Task Manager looks roughly like this:</p>



<ol class="wp-block-list">
<li>Open the Task Manager (<code>Shift+Esc</code>) and sort by Memory Footprint, noting the top few consumers.</li>



<li>Sort by CPU separately, since the memory and CPU culprits aren&#8217;t always the same tab — a tab can be memory-heavy but idle, or CPU-heavy with modest memory use (common with tabs playing video or running continuous animations).</li>



<li>For any suspiciously high-usage tab, check whether it corresponds to a page you actually still need open, or a forgotten background tab.</li>



<li>For extensions showing unexpected usage, especially when no tabs are actively being used, consider temporarily disabling them one at a time (via <code>chrome://extensions</code>) to isolate which one is responsible, since the Task Manager identifies the resource usage but not necessarily the specific <em>cause</em> within a complex extension&#8217;s internal logic.</li>



<li>If a specific renderer process seems to be consuming CPU indefinitely with no clear cause, use &#8220;End process&#8221; to kill just that tab rather than restarting the entire browser, preserving your other work.</li>
</ol>



<h2 class="wp-block-heading">Summary</h2>



<p class="wp-block-paragraph">The Chrome Task Manager exists because Chrome&#8217;s multi-process architecture — separate processes for the browser shell, each tab&#8217;s renderer, the GPU, and extensions — means that generic OS-level task managers can&#8217;t show you which specific tab or extension is responsible for high CPU, memory, or network usage. It provides per-task visibility mirroring OS-level process monitoring concepts, lets you end individual misbehaving tabs without restarting the whole browser thanks to Chrome&#8217;s process isolation design, and serves as a genuinely useful diagnostic tool for everyday users troubleshooting a sluggish browser as well as developers hunting down memory leaks or performance issues in their own web applications.</p>



<h2 class="wp-block-heading">FAQs</h2>



<p class="wp-block-paragraph"><strong>Is the Chrome Task Manager the same as my operating system&#8217;s Task Manager?</strong> No — it&#8217;s a Chrome-specific tool that only shows Chrome&#8217;s internal tabs, extensions, and subsystems, whereas your OS task manager shows every process running on the whole computer.</p>



<p class="wp-block-paragraph"><strong>Does ending a task in Chrome&#8217;s Task Manager close the tab?</strong> It kills the underlying process, which typically causes the tab to show a &#8220;crashed&#8221; or &#8220;aw, snap&#8221; page rather than cleanly closing — you can still close the tab normally afterward.</p>



<p class="wp-block-paragraph"><strong>Why do I see multiple Chrome processes in my OS task manager but they&#8217;re not all listed separately in Chrome&#8217;s own Task Manager?</strong> Chrome&#8217;s Task Manager already breaks things down at a similar granularity, but some underlying OS processes (like certain sandboxed helper processes) may not each get their own dedicated top-level row if they&#8217;re tightly coupled to another listed task.</p>



<p class="wp-block-paragraph"><strong>Can extensions hide their resource usage from the Chrome Task Manager?</strong> Generally no — extensions run within Chrome&#8217;s process model and their resource consumption is visible through the Task Manager, though how clearly a specific extension&#8217;s activity is attributed can depend on how it&#8217;s implemented (e.g., background service workers versus content scripts).</p>



<p class="wp-block-paragraph"><strong>Is there a command-line or scriptable equivalent to Chrome&#8217;s Task Manager?</strong> Yes — <code>chrome://process-internals</code> provides a more detailed, page-based view of Chrome&#8217;s process architecture, and Chrome also supports remote debugging protocols that developers can use to programmatically inspect process and performance metrics.</p>



<h2 class="wp-block-heading">References</h2>



<ul class="wp-block-list">
<li>Google Chrome Help — &#8220;Manage extensions, tabs &amp; apps with Task Manager&#8221;</li>



<li>Chromium Project documentation — &#8220;Process Models&#8221; and multi-process architecture design docs</li>



<li>Chromium Project — Site Isolation design documentation</li>



<li>Google Developers — Chrome DevTools and <code>chrome://tracing</code> documentation</li>
</ul>
<p>The post <a href="https://awjunaid.com/operating-system/describe-the-role-of-the-chrome-task-manager-in-monitoring-and-managing-processes/">Describe the role of the Chrome Task Manager in monitoring and managing processes</a> appeared first on <a href="https://awjunaid.com">Abdul Wahab Junaid</a>.</p>
]]></content:encoded>
					
					<wfw:commentRss>https://awjunaid.com/operating-system/describe-the-role-of-the-chrome-task-manager-in-monitoring-and-managing-processes/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
		<post-id xmlns="com-wordpress:feed-additions:1">8462</post-id>	</item>
		<item>
		<title>How does the multiprocess model in Chrome improve security compared to single-process browsers</title>
		<link>https://awjunaid.com/operating-system/how-does-the-multiprocess-model-in-chrome-improve-security-compared-to-single-process-browsers/</link>
					<comments>https://awjunaid.com/operating-system/how-does-the-multiprocess-model-in-chrome-improve-security-compared-to-single-process-browsers/?noamp=mobile#respond</comments>
		
		<dc:creator><![CDATA[Abdul Wahab Junaid]]></dc:creator>
		<pubDate>Sun, 03 Dec 2023 13:28:57 +0000</pubDate>
				<category><![CDATA[Operating System]]></category>
		<category><![CDATA[dailyprompt]]></category>
		<category><![CDATA[operating system]]></category>
		<guid isPermaLink="false">https://awjunaid.com/?p=8459</guid>

					<description><![CDATA[<p>When Google Chrome launched in 2008, it introduced something that seems obvious in hindsight but was genuinely radical&#8230;</p>
<p>The post <a href="https://awjunaid.com/operating-system/how-does-the-multiprocess-model-in-chrome-improve-security-compared-to-single-process-browsers/">How does the multiprocess model in Chrome improve security compared to single-process browsers</a> appeared first on <a href="https://awjunaid.com">Abdul Wahab Junaid</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<p class="wp-block-paragraph">When Google Chrome launched in 2008, it introduced something that seems obvious in hindsight but was genuinely radical at the time: instead of running as one giant program, the browser split itself into many smaller, isolated processes. Before Chrome, browsers like early Firefox and Internet Explorer ran nearly everything — the UI, the rendering engine, plugins, and every open tab — inside a single process and often a single thread. That design choice had serious consequences for security, and Chrome&#8217;s multiprocess architecture was built specifically to address them.</p>



<p class="wp-block-paragraph">This article walks through exactly why the multiprocess model matters for security, how it works under the hood, and how it compares to the single-process browsers that came before it.</p>



<h2 class="wp-block-heading">The Problem With Single-Process Browsers</h2>



<p class="wp-block-paragraph">Think about what a browser actually does. It parses untrusted HTML from random websites. It executes untrusted JavaScript. It decodes images, videos, fonts, and other media formats, many of which have historically been riddled with parsing bugs. It renders complex layouts using engines written in C++, a language where a single buffer overflow or use-after-free bug can hand an attacker control of the entire process.</p>



<p class="wp-block-paragraph">In a single-process browser, all of this happens inside one address space. If an attacker found a memory corruption bug in the rendering engine while parsing a malicious webpage, they didn&#8217;t just crash a tab — they gained code execution privileges equivalent to the entire browser. From there, an attacker could:</p>



<ul class="wp-block-list">
<li>Read the memory of every other open tab, including banking sessions, email, and saved passwords</li>



<li>Access the browser&#8217;s own process memory, including cookies, saved credentials, and autofill data</li>



<li>In many cases, pivot into the operating system itself, since the browser process typically ran with the full privileges of the logged-in user</li>
</ul>



<p class="wp-block-paragraph">This is the core problem: a single vulnerability anywhere in the browser&#8217;s attack surface (HTML parser, JS engine, image decoder, font renderer, plugin) was enough to compromise everything the browser touched. Security researchers call this a &#8220;confused deputy&#8221; problem at scale — the browser process is trusted with everything, so compromising it compromises everything.</p>



<h2 class="wp-block-heading">Chrome&#8217;s Answer: Process Isolation</h2>



<p class="wp-block-paragraph">Chrome&#8217;s designers, drawing heavily on operating-system-level security principles, decided to treat the browser less like a single application and more like a mini operating system with regular processes. The idea was simple: don&#8217;t let one compromised piece of code have access to everything else.</p>



<p class="wp-block-paragraph">In the multiprocess model, Chrome splits its work across several process types:</p>



<ol class="wp-block-list">
<li><strong>The Browser Process</strong> — the privileged process that manages the address bar, bookmarks, network requests, and the disk, and talks to the operating system directly.</li>



<li><strong>Renderer Processes</strong> — one per tab (or per site, in modern Chrome), responsible for parsing HTML/CSS, running JavaScript, and rendering the page.</li>



<li><strong>GPU Process</strong> — handles graphics acceleration separately from rendering logic.</li>



<li><strong>Plugin/Utility Processes</strong> — isolate third-party code and helper tasks like PDF viewing or audio decoding.</li>
</ol>



<p class="wp-block-paragraph">Each renderer process runs inside a <strong>sandbox</strong>, a restricted execution environment with drastically reduced privileges. A sandboxed renderer typically cannot write to disk, cannot open arbitrary network sockets, and cannot make most system calls directly. If it needs to do something privileged — like saving a downloaded file — it has to ask the browser process to do it on its behalf, through a tightly controlled inter-process communication (IPC) channel.</p>



<h2 class="wp-block-heading">Why This Matters for Security Specifically</h2>



<h3 class="wp-block-heading">1. Fault and exploit containment</h3>



<p class="wp-block-paragraph">If an attacker manages to exploit a bug in the rendering engine of a single tab, that exploit only gains control of a low-privilege, sandboxed renderer process. It cannot directly read the memory of other tabs, cannot access the file system, and cannot make network connections outside of what the sandbox permits. The attacker essentially has to find a <em>second</em> vulnerability — a sandbox escape — to do anything meaningful with the initial exploit. This &#8220;defense in depth&#8221; approach dramatically raises the cost of a successful attack. A single bug is no longer enough.</p>



<h3 class="wp-block-heading">2. Site Isolation</h3>



<p class="wp-block-paragraph">Modern Chrome extends this idea further with a feature called <strong>Site Isolation</strong>, enabled by default since Chrome 67 in the wake of the Spectre and Meltdown hardware vulnerabilities. Under Site Isolation, each renderer process is dedicated to content from a single site (defined by scheme + registrable domain), even if that means an iframe from a different origin gets its own process, separate from the page that embeds it. This closes a subtle but serious gap: even if a malicious site&#8217;s renderer process could somehow read arbitrary memory within its own process (as Spectre-style speculative execution attacks allow), it would only be able to read data belonging to <em>that same site</em>, not sensitive data from a different origin loaded in an adjacent iframe.</p>



<h3 class="wp-block-heading">3. Reduced attack surface for the privileged process</h3>



<p class="wp-block-paragraph">The browser process, which does have real system privileges, is deliberately kept as small and simple as possible in terms of code that handles untrusted input. Complex, bug-prone tasks like HTML parsing and JavaScript execution are pushed into the sandboxed renderers. This follows the principle of least privilege: give each component only the access it strictly needs, and keep the most powerful component doing the least amount of risky work.</p>



<h3 class="wp-block-heading">4. Independent crash and compromise domains</h3>



<p class="wp-block-paragraph">Because tabs are isolated, a compromised or crashed renderer cannot corrupt the memory of the browser process or other tabs. This isn&#8217;t just a stability win (discussed more in a companion article) — it&#8217;s a security win too, since memory corruption in one process can&#8217;t cascade into another process&#8217;s address space. Address space separation, enforced by the OS&#8217;s virtual memory system, is a hard boundary that malicious code cannot casually cross.</p>



<h2 class="wp-block-heading">How Chrome&#8217;s Sandbox Actually Works (Platform Specifics)</h2>



<p class="wp-block-paragraph">Chrome&#8217;s sandboxing implementation is platform-dependent, since it has to interface with each OS&#8217;s own security primitives:</p>



<ul class="wp-block-list">
<li><strong>Windows</strong>: Chrome uses restricted tokens, job objects, and integrity levels to strip renderer processes of privileges. Renderers run at &#8220;Low&#8221; integrity level, meaning they cannot write to most parts of the file system or registry even if they tried, because Windows&#8217; mandatory integrity control blocks it at the kernel level.</li>



<li><strong>Linux</strong>: Chrome uses a combination of <code>seccomp-bpf</code> (which filters which system calls a process is allowed to make) and namespaces/chroot-like techniques to restrict renderer processes.</li>



<li><strong>macOS</strong>: Chrome uses the Seatbelt sandbox (<code>sandbox_init</code> / Sandbox.kext or the modern Sandbox.framework) to apply similar restrictions.</li>



<li><strong>Android</strong>: Chrome leverages Android&#8217;s own application-level sandboxing, running renderers as isolated processes with minimal Android permissions.</li>
</ul>



<p class="wp-block-paragraph">In every case, the underlying principle is the same: the OS kernel itself enforces the boundary, not just application logic. This matters because it means even a fully exploited renderer process, with arbitrary code execution, is still bound by kernel-enforced restrictions it cannot bypass through software tricks alone.</p>



<h2 class="wp-block-heading">Comparison: Single-Process vs. Multiprocess Security Model</h2>



<figure class="wp-block-table"><table class="has-fixed-layout"><thead><tr><th>Aspect</th><th>Single-Process Browser</th><th>Chrome&#8217;s Multiprocess Model</th></tr></thead><tbody><tr><td>Blast radius of a rendering bug</td><td>Entire browser, all tabs, all data</td><td>One sandboxed renderer</td></tr><tr><td>Privilege of compromised component</td><td>Full user privileges</td><td>Heavily restricted sandbox</td></tr><tr><td>Cross-tab data exposure</td><td>Trivial once compromised</td><td>Blocked by OS process isolation</td></tr><tr><td>Effort required for full compromise</td><td>One bug</td><td>Chained exploits (renderer bug + sandbox escape)</td></tr><tr><td>Site Isolation against Spectre-like attacks</td><td>Not applicable/not possible</td><td>Enabled by default</td></tr></tbody></table></figure>



<h2 class="wp-block-heading">Real-World Impact</h2>



<p class="wp-block-paragraph">This architecture has repeatedly proven its worth. Security researchers and bug bounty hunters targeting Chrome typically need to chain <em>multiple</em> vulnerabilities together — a renderer exploit plus a sandbox escape, sometimes plus a kernel exploit — to achieve full system compromise. Compare this to older single-process browsers or unsandboxed rendering engines, where a single memory corruption bug in the layout engine was often sufient for full compromise. Chrome&#8217;s bug bounty program (and events like Pwn2Own) consistently show that reliable, complete exploit chains against Chrome are rare, expensive, and technically demanding, precisely because of this layered defense.</p>



<h2 class="wp-block-heading">Best Practices That Follow From This Architecture</h2>



<p class="wp-block-paragraph">For everyday users and IT administrators, understanding this model translates into some practical guidance:</p>



<ul class="wp-block-list">
<li><strong>Keep Chrome updated.</strong> Security patches often close the gap for renderer bugs or sandbox escapes; an out-of-date Chrome loses the benefit of ongoing hardening.</li>



<li><strong>Leave Site Isolation enabled.</strong> Some enterprise policies disable it for memory savings on constrained devices; this trades security for performance and should be a deliberate decision.</li>



<li><strong>Be cautious with extensions.</strong> Extensions can run with elevated privileges relative to web content and are a common way attackers try to bypass the renderer sandbox entirely — install only from trusted sources.</li>



<li><strong>Understand that sandboxing is layered, not absolute.</strong> Sandbox escapes do happen, rarely, which is why defense in depth (multiple independent barriers) matters more than any single barrier being perfect.</li>
</ul>



<h2 class="wp-block-heading">Troubleshooting Note</h2>



<p class="wp-block-paragraph">Users sometimes notice <code>chrome.exe *32</code> or multiple <code>chrome</code> processes in Task Manager / Activity Monitor / <code>ps aux</code> and worry it indicates malware or a memory leak. In most cases, this is completely expected — it&#8217;s the multiprocess model at work, with each tab, extension, and the GPU compositor running as separate OS processes. You can inspect this yourself via Chrome&#8217;s built-in Task Manager (<code>Shift+Esc</code> on Windows/Linux, or via the three-dot menu → More Tools → Task Manager), which shows memory and CPU usage per process, distinct from the OS-level task manager.</p>



<h2 class="wp-block-heading">FAQs</h2>



<p class="wp-block-paragraph"><strong>Q: Does every tab really get its own process?</strong> A: Generally yes, though Chrome uses heuristics to sometimes group same-site tabs together for efficiency, especially on memory-constrained devices. With Site Isolation, cross-site iframes also get dedicated processes.</p>



<p class="wp-block-paragraph"><strong>Q: Does the multiprocess model make Chrome slower than single-process browsers?</strong> A: It increases baseline memory usage since each process has its own overhead, but it does not meaningfully increase CPU cost, and the parallelism it enables (crash isolation, independent rendering) generally improves perceived responsiveness.</p>



<p class="wp-block-paragraph"><strong>Q: Can malware still escape Chrome&#8217;s sandbox?</strong> A: In rare cases, yes — sandbox escape vulnerabilities are discovered periodically and patched quickly. No sandbox is a perfect guarantee, but it substantially raises the bar for attackers.</p>



<p class="wp-block-paragraph"><strong>Q: Is this the same as Site Isolation?</strong> A: No. The multiprocess model is the general architecture of separating browser functions into processes. Site Isolation is a specific, more granular security feature built on top of that architecture, ensuring each site (not just each tab) gets its own process.</p>



<p class="wp-block-paragraph"><strong>Q: Do other browsers use this model too?</strong> A: Yes. Firefox adopted a similar architecture (called Electrolysis, or e10s, and later Fission for full site isolation). Microsoft Edge, being Chromium-based, inherits Chrome&#8217;s multiprocess model directly. Safari also uses a multiprocess architecture (WebKit&#8217;s WebContent process).</p>



<h2 class="wp-block-heading">Summary</h2>



<p class="wp-block-paragraph">Chrome&#8217;s multiprocess model fundamentally changed browser security by treating each tab, plugin, and browser subsystem as an isolated, sandboxed process rather than trusting one monolithic program with everything. This containment strategy means a single rendering bug no longer equals full compromise — attackers must chain multiple, harder-to-find vulnerabilities together. Combined with Site Isolation, kernel-enforced sandboxing, and the principle of least privilege, this architecture is one of the primary reasons modern Chrome (and its Chromium-based descendants) is significantly harder to exploit than the single-process browsers of the pre-2008 era.</p>



<h2 class="wp-block-heading">References</h2>



<ul class="wp-block-list">
<li>Google Chromium Project — Multi-process Architecture: https://www.chromium.org/developers/design-documents/multi-process-architecture/</li>



<li>Google Chromium Project — Site Isolation: https://www.chromium.org/Home/chromium-security/site-isolation/</li>



<li>Google Chromium Project — Sandbox design documents: https://chromium.googlesource.com/chromium/src/+/HEAD/docs/design/sandbox.md</li>



<li>Chrome Security Team blog: https://blog.chromium.org/search/label/security</li>
</ul>
<p>The post <a href="https://awjunaid.com/operating-system/how-does-the-multiprocess-model-in-chrome-improve-security-compared-to-single-process-browsers/">How does the multiprocess model in Chrome improve security compared to single-process browsers</a> appeared first on <a href="https://awjunaid.com">Abdul Wahab Junaid</a>.</p>
]]></content:encoded>
					
					<wfw:commentRss>https://awjunaid.com/operating-system/how-does-the-multiprocess-model-in-chrome-improve-security-compared-to-single-process-browsers/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
		<post-id xmlns="com-wordpress:feed-additions:1">8459</post-id>	</item>
	</channel>
</rss>
