Why sed's s/old/new/g Fails With File Paths — Delimiters, Capture Groups, and the GNU vs BSD sed Difference

sed's find-and-replace syntax (s/old/new/g) looks simple, but the forward slash delimiter creates a problem the moment your pattern or replacement contains a forward slash — like a file path — and the solution (changing the delimiter to any other character) is known by most sed users but forgotten at exactly the moment it's most needed

The previous articles on this site covered basic find and replace, regex patterns for data cleaning, spreadsheet substitution, capture groups and multi-cursor editing, catastrophic backtracking, and why regex fails on structured formats. This article addresses find and replace in command-line tools — specifically sed, awk, and grep, the delimiter problem, and the specific patterns that look like they should work but produce unexpected results.

sed's s command: the complete syntax

sed (stream editor) processes text line by line and applies editing commands. The substitution command is the most used:

Basic syntax: s/pattern/replacement/flags

Common flags:

• g — global: replace all occurrences on the line (not just the first)
• i — case-insensitive matching
• I — case-insensitive (some sed versions)
• 1, 2, N — replace only the Nth occurrence
• p — print the line if substitution was made (used with -n to only print changed lines)

The delimiter character: the / after s is the delimiter — it separates the command letter from the pattern, pattern from replacement, and replacement from flags. The delimiter can be any character that doesn't appear in the pattern or replacement:

# Standard delimiter
sed 's/old/new/g' file.txt

# When pattern or replacement contains /
# This fails:
sed 's/usr/local/usr/share/g' file.txt  # Ambiguous delimiters

# Use a different delimiter (any character works):
sed 's|/usr/local|/usr/share|g' file.txt  # Uses | as delimiter
sed 's#/usr/local#/usr/share#g' file.txt  # Uses # as delimiter
sed 's,/usr/local,/usr/share,g' file.txt  # Uses , as delimiter

The ampersand (&) in replacement: the matched string reference

In sed's replacement string, & refers to the entire matched pattern. This enables wrapping, prefixing, or surrounding matched content without knowing what it is:

# Wrap all numbers in square brackets
echo "The value is 42 and also 100" | sed 's/[0-9][0-9]*/[&]/g'
# Output: The value is [42] and also [100]

# Add quotes around all words
sed 's/[A-Za-z]*/\"&\"/g' file.txt

# Add a prefix to all matching lines
sed 's/^error/[ERROR] &/' logfile.txt
# Prefixes each line starting with "error" with "[ERROR] " while keeping "error" in place

Capture groups in sed: the \1 reference

sed supports capture groups (subexpressions enclosed in \(...\) in basic regex, or (...) with ERE flag -E):

# Swap first and last name (basic regex)
echo "Smith, John" | sed 's/\([A-Za-z]*\), \([A-Za-z]*\)/\2 \1/'
# Output: John Smith

# Same with extended regex (-E flag):
echo "Smith, John" | sed -E 's/([A-Za-z]+), ([A-Za-z]+)/\2 \1/'
# Output: John Smith

The GNU sed vs BSD sed difference: macOS uses BSD sed; Linux typically uses GNU sed. They have subtle differences:

• -i for in-place editing requires a backup extension on BSD sed: sed -i '' 's/old/new/g' file (note the empty string '')
• GNU sed: sed -i 's/old/new/g' file (no backup extension required)

This is the most common cross-platform sed surprise for scripts written on Linux that are run on macOS or vice versa.

The g flag and line-level vs file-level scope

sed processes files one line at a time — the g flag means "replace all occurrences on this line," not "replace all occurrences in the file." A pattern that spans multiple lines requires special handling:

# This only replaces the first occurrence per line
sed 's/foo/bar/' file.txt

# This replaces ALL occurrences per line
sed 's/foo/bar/g' file.txt

# For multi-line matching in sed, you need N command or other techniques:
sed 'N; s/foo\nbar/replacement/g' file.txt  # Pairs consecutive lines

For file-wide replacements, sed works correctly when the pattern is contained within a single line — which covers the vast majority of practical use cases.

awk's gsub and sub: a cleaner alternative for complex replacements

awk (Aho, Weinberger, Kernighan) is a more powerful text processing tool that includes substitution functions:

sub(pattern, replacement): replaces the first occurrence in the current field or record. gsub(pattern, replacement): replaces all occurrences (the g stands for "global").

# Replace in the entire line ($0)
awk '{gsub(/old/, "new"); print}' file.txt

# Replace only in the second field
awk '{sub(/old/, "new", $2); print}' file.txt

# The & reference works the same as in sed
awk '{gsub(/[0-9]+/, "[&]"); print}' file.txt

awk's advantage over sed for complex replacements: awk allows conditional replacements (replace only on lines matching another condition), field-specific replacements (replace in column 3 but not column 5), and more complex logic around the substitution.

grep for finding before replacing: the preview step

A critical workflow for any significant find-and-replace operation is previewing the matches before making changes:

# See all lines that will be affected
grep 'pattern' file.txt

# Count the matches
grep -c 'pattern' file.txt

# See the specific matches highlighted
grep --color 'pattern' file.txt

# Only show the matched portion (not the whole line)
grep -o 'pattern' file.txt

The -n flag shows line numbers — useful for verifying you're matching the intended lines before running sed:

grep -n 'pattern' file.txt  # Shows: "23:matched content on line 23"

The workflow: grep first to understand what will be changed, then run sed in preview mode (sed -n 's/old/new/gp'), then apply with sed -i.

How to use the Find & Replace tool on sadiqbd.com

1. Test before applying to files: use the tool to verify your regex pattern produces the expected replacements before running sed or grep on actual files — especially important for patterns with special characters
2. For delimiter testing: if your pattern contains /, test the replacement in the tool to confirm the logic, then use an alternative delimiter (|, #, ,) when writing the actual sed command
3. Capture group preview: test capture group syntax (\1, \2) in the tool with sample input to verify the group references produce the expected output before applying to production data

Frequently Asked Questions

What's the difference between sed 's/foo/bar/' and sed 's/foo/bar/g'? Without g, sed replaces only the first occurrence on each line. With g, it replaces all occurrences on each line. For most practical use cases (replacing variable names, normalising data), g is what you want — not having g is a common source of confusion when a replacement appears to only partially work. A specific case where you'd omit g: when a pattern might accidentally match more than intended and you only want the first occurrence changed — for example, adding a prefix to a line (s/^/PREFIX_/) where g is irrelevant because ^ matches only once, or transforming only the first word of each line.

Is the Find & Replace tool free? Yes — completely free, no sign-up required.

Try the Find & Replace tool free at sadiqbd.com — search and replace text with regex, case options, and live preview.