Before you can build anything from source, you have to get it onto your system and unpacked in a sane, verifiable way. It sounds trivial, and most of the time it is, but I’ve been burned enough times by corrupted archives, mismatched checksums, and confusing nested directory structures that I’ve developed a pretty disciplined routine around this step. This guide is that routine, laid out in full.
Where to Get Source Packages
There are a few common sources I pull from, each with different trust and verification considerations.
Official project websites and release pages — the gold standard, usually offering signed tarballs and published checksums alongside the download.
Git repositories — for bleeding-edge or development versions, cloned directly rather than downloaded as an archive.
Distribution source package repositories — Debian’s apt-get source, Fedora’s dnf download --source, giving you the exact source plus any distro-applied patches, useful when you want to stay close to what your distribution actually ships.
Mirrors — many large projects (like the Linux kernel itself, or GNU software) use a network of mirrors; picking a nearby, trustworthy mirror speeds up downloads considerably.
Downloading with wget
I covered wget in depth elsewhere, but here’s the core pattern for grabbing a source tarball:
wget https://example.org/releases/software-2.1.0.tar.gz
I always download into a dedicated working directory rather than cluttering my home directory root:
mkdir -p ~/src
cd ~/src
wget https://example.org/releases/software-2.1.0.tar.gz
For resuming an interrupted download of a large tarball:
wget -c https://example.org/releases/software-2.1.0.tar.gz
Downloading with curl
curl is equally common for this, and I use it interchangeably with wget depending on habit and what’s already installed:
curl -LO https://example.org/releases/software-2.1.0.tar.gz
-L follows redirects (curl doesn’t follow them by default, unlike wget), and -O saves using the remote filename.
Verifying Integrity Before You Extract Anything
This is the step people skip most often, and it’s the one that’s saved me the most grief. Before extracting anything, I verify it wasn’t corrupted in transit and, ideally, that it’s actually what the project published.
Checksum Verification
Most projects publish SHA256 checksums alongside their downloads:
wget https://example.org/releases/software-2.1.0.tar.gz.sha256
sha256sum -c software-2.1.0.tar.gz.sha256
Or, if you just have the raw hash value published on a webpage rather than a .sha256 file:
sha256sum software-2.1.0.tar.gz
Then compare that output manually against the published value.
GPG Signature Verification
For projects that cryptographically sign releases (common for security-sensitive software and core GNU/Linux infrastructure projects):
wget https://example.org/releases/software-2.1.0.tar.gz.asc
gpg --verify software-2.1.0.tar.gz.asc software-2.1.0.tar.gz
If you don’t have the signer’s public key yet, you’ll need to import it first, typically from a keyserver:
gpg --keyserver keyserver.ubuntu.com --recv-keys <KEY_ID>
I treat a failed signature verification as a hard stop — I don’t proceed with extraction or building until I understand why it failed, since it could indicate a tampered download, a mirror serving stale content, or simply an outdated local key.
Understanding Archive Formats
Source packages come in a handful of common formats, and knowing the difference matters for picking the right extraction command.
.tar— a plain, uncompressed archive (tape archive format), just bundling files together without compression..tar.gz/.tgz— gzip-compressed tar archive, the most common format historically..tar.bz2/.tbz2— bzip2-compressed, generally smaller than gzip but slower to compress/decompress..tar.xz— xz-compressed, typically the smallest of the common options and increasingly the default for modern projects (the Linux kernel itself ships primarily as.tar.xznow)..tar.zst— zstd-compressed, gaining popularity for its excellent balance of compression ratio and speed..zip— common cross-platform format, especially for projects also targeting Windows users.
Extracting tar Archives
The tar command handles all the compressed tar variants through consistent flags, auto-detecting compression in modern versions:
tar -xf software-2.1.0.tar.gz
The -x flag means extract, and -f specifies the file to operate on. Modern GNU tar auto-detects the compression format from the file itself, so you rarely need to specify it explicitly, but being explicit doesn’t hurt and makes scripts clearer:
tar -xzf software-2.1.0.tar.gz # gzip
tar -xjf software-2.1.0.tar.bz2 # bzip2
tar -xJf software-2.1.0.tar.xz # xz
tar --zstd -xf software-2.1.0.tar.zst # zstd
I add -v (verbose) when I want to see every file being extracted, which is useful for a first look at an unfamiliar archive’s structure, though I mostly skip it for archives I already trust and know the layout of:
tar -xvzf software-2.1.0.tar.gz
Extracting to a Specific Directory
mkdir -p /opt/src/software-2.1.0
tar -xzf software-2.1.0.tar.gz -C /opt/src/software-2.1.0 --strip-components=1
-C sets the extraction target directory, and --strip-components=1 removes the top-level directory the archive normally contains, useful when you already have a specific directory prepared and don’t want an unnecessary extra nesting level.
Listing Contents Without Extracting
Before extracting an unfamiliar archive, I often peek inside first to check what I’m actually about to unpack:
tar -tzf software-2.1.0.tar.gz | head -20
-t lists contents rather than extracting them. I do this specifically to check whether the archive is well-formed (a single top-level directory containing everything, the conventional and expected structure) versus a poorly packaged archive that dumps loose files directly into the current directory, which would make a mess if extracted carelessly.
Extracting zip Archives
unzip software-2.1.0.zip
To preview contents first:
unzip -l software-2.1.0.zip
To extract into a specific directory:
unzip software-2.1.0.zip -d /opt/src/
If unzip isn’t installed:
sudo apt install unzip # Debian/Ubuntu
sudo dnf install unzip # RHEL/Fedora
Cloning from Git for Development Versions
When I need the absolute latest, unreleased code, or I want to track a specific branch or tag:
git clone https://github.com/example/software.git
cd software
git checkout v2.1.0 # specific tagged release
For a lighter clone when I don’t need full history, useful for large repositories:
git clone --depth=1 https://github.com/example/software.git
If a project uses git submodules for its dependencies:
git submodule update --init --recursive
I’ve lost time before to a build failing mysteriously simply because I forgot this step and a submodule directory was sitting there empty.
Preparing the Source Tree for Building
Once extracted, before running ./configure or cmake, I do a quick walk through the source tree to orient myself:
cd software-2.1.0
ls -la
cat README* INSTALL* 2>/dev/null | less
Reading the README and INSTALL files first, every time, has saved me from countless avoidable mistakes — dependency lists, non-standard build instructions, required environment variables, and platform-specific caveats are usually all documented right there.
Checking for a configure Script vs Needing Autoreconf
Official release tarballs almost always ship a pre-generated configure script. Git checkouts frequently do not, since that script is itself generated output that’s normally excluded from version control:
ls configure 2>/dev/null || echo "No configure script present, may need autoreconf"
If missing, and the project uses Autotools:
sudo apt install autoconf automake libtool
autoreconf -i
Organizing a Source Directory Structure
I keep a fairly disciplined layout on machines where I regularly build from source, which makes cleanup and tracking far easier over time:
mkdir -p ~/src/downloads # raw archives, kept for reference/re-extraction
mkdir -p ~/src/build # extracted, actively worked-on source trees
mkdir -p ~/src/installed.log # simple text log of what I've installed and where
A minimal example of what goes into that log after each install:
echo "software-2.1.0 installed to /usr/local via checkinstall on $(date)" >> ~/src/installed.log
It’s a small habit, but months later when I’m trying to remember what’s built from source versus what came from the package manager, it pays for itself many times over.
Handling Patches Before Building
Sometimes you need to apply a patch to source before building — a security fix not yet released, a local customization, or a fix for a bug specific to your environment:
cd software-2.1.0
patch -p1 < ../fix-compile-error.patch
-p1 strips the first path component from the patch file’s paths, which is the near-universal convention for patches generated against a project’s root directory.
For patches distributed as .diff files generated by git diff:
git apply ../my-change.patch
Cleaning Up After Yourself
Source directories and build artifacts accumulate fast. Once software is built and installed (and ideally tracked via checkinstall, stow, or a package build as covered in my source-building guide), I don’t usually need the extracted source tree anymore, though I do keep the original archive around in case I need to rebuild or reference it later:
cd ~/src/build
rm -rf software-2.1.0/
# but keep ~/src/downloads/software-2.1.0.tar.gz
Troubleshooting Common Extraction Problems
“gzip: stdin: not in gzip format” — the file extension claims gzip compression but the actual content doesn’t match; the download may be corrupted or incomplete, or the URL served an HTML error page instead of the actual file (check with file software-2.1.0.tar.gz to confirm the actual type).
Checksum mismatch after download — retry the download entirely rather than assuming a partial re-download will fix it; corrupted downloads are sometimes caused by a flaky connection dropping bytes silently.
Archive extracts hundreds of loose files directly into the current directory — this means the tarball wasn’t packaged with a proper top-level directory; always run tar -tzf first to check before extracting into a shared or important directory, and extract into a dedicated empty subdirectory as a safety habit.
“tar: Unrecognized archive format” — double check the actual file type rather than trusting the extension:
file software-2.1.0.tar.gz
If it reports something unexpected (like Zip archive data despite the .tar.gz name), the file was likely renamed incorrectly somewhere along the way, or the download itself pulled the wrong content.
Security Considerations
Never extract an untrusted archive as root, and be cautious of archives containing files with unusual permissions, absolute paths, or ../ path traversal sequences designed to write outside the intended extraction directory (a known historical class of archive-based vulnerability, often called “tar bombs” or path traversal exploits). Modern tar implementations refuse absolute paths and parent-directory traversal by default, but I still extract unfamiliar archives inside a dedicated, disposable directory as a matter of habit:
mkdir -p /tmp/extract-test
tar -xzf suspicious-package.tar.gz -C /tmp/extract-test
Compatibility Across Distributions
tar, wget, curl, unzip, and gpg are available as standard packages across essentially every Linux distribution, though some minimal container base images (like scratch or slim Docker images) may not include them by default and need explicit installation. The core tar command’s flag behavior is remarkably consistent across GNU/Linux systems since nearly all of them use GNU tar specifically; BSD-derived systems (including macOS’s default tar) have subtly different flag behavior in some edge cases, which is worth knowing if you’re writing scripts meant to be portable across both.
Working with Compressed Single Files (Non-Archive)
Not everything you download is a tar archive — sometimes you’re dealing with a single compressed file, like a .gz compressed log file or a standalone .xz compressed binary:
gunzip file.txt.gz # decompresses in place, replacing the .gz
gunzip -k file.txt.gz # -k keeps the original .gz file around too
xz -d file.bin.xz
zstd -d file.dat.zst
I use -k/--keep fairly often specifically to avoid accidentally destroying a downloaded compressed file I might want to re-verify or re-extract later, since the default behavior for most of these single-file compression tools is to replace the compressed file with the decompressed version, deleting the original.
Choosing Between Different Compression Formats When You Have a Choice
When a project offers multiple archive formats for the same release (common for larger, well-established projects), I generally prefer .tar.xz or .tar.zst over .tar.gz when available, purely for the smaller download size and, in zstd’s case, notably faster decompression — meaningful when you’re regularly re-extracting the same large archive during iterative build testing. The practical difference matters less for small archives, but for genuinely large source trees (some projects’ full source exceeds several hundred megabytes), the choice of compression format has a real, measurable effect on both download time and extraction time.
Handling Redirects and Content-Disposition Filenames
Some download URLs redirect through a series of intermediate hosts before landing on the actual file, and some servers specify the intended filename via an HTTP header (Content-Disposition) rather than relying on the URL path itself. Both wget and curl handle this reasonably well by default, but I’ve occasionally needed to be explicit:
curl -LOJ https://example.org/download?id=12345
The -J flag tells curl to honor the server-suggested filename from the Content-Disposition header rather than deriving a (often nonsensical) filename from the URL’s query string itself, which matters for download links that don’t have a clean, predictable filename baked into the URL path.
Summary
Downloading and unpacking software correctly is really about discipline more than complexity: fetch from a source you trust, verify what you downloaded actually matches what was published, understand the archive format you’re dealing with before blindly extracting it, and keep your source trees organized enough that future-you can reconstruct what came from where. Get this foundational step right, and everything downstream — configuring, compiling, installing — goes far more smoothly, because you’re not troubleshooting a corrupted or mismatched source tree on top of everything else.
References
- GNU tar manual:
man 1 tar - GNU Wget manual:
man 1 wget man 1 gpgfor signature verification- Debian New Maintainers’ Guide, source package handling
- Individual project release pages, typically the authoritative source for checksums and signatures