search replace in multiple files useing grep xargs sed

On many occasions we need to search and replace some text across multiple files on server. We use combination of grep, sed and xargs to achieve it mostly.

We will explain commands with examples:

Search and replace URLs in PHP files

We came across a poorly coded PHP site which was using many 4 different versions for domain names:

1. http://example.com – without www and http
2. http://www.example.com – with www and http
3. https://example.com – without www and https
4. https://www.example.com – with www and https

Final destination was: https://www.example.com (with https and www). We setup necessary redirects on nginx side to be safe but we also decided to replace variant 1 to 3 with final variant no. 4.

Commands

Search files with matching URLs

grep -lr --include=*.php "http://example.com" /var/www/example.com/htdocs
grep -lr --include=*.php "http://www.example.com" /var/www/example.com/htdocs
grep -lr --include=*.php "https://example.com" /var/www/example.com/htdocs
grep -lr --include=*.php "https://www.example.com" /var/www/example.com/htdocs

Replace files with new URL

For variant 1 to 3… Following will replace file content without backup!

grep -lr --include=*.php "http://example.com" /var/www/example.com/htdocs | xargs sed -i -e 's/http:\/\/example.com/https:\/\/www.example.com/g'
grep -lr --include=*.php "http://www.example.com" /var/www/example.com/htdocs | xargs sed -i -e 's/http:\/\/www.example.com/https:\/\/www.example.com/g'
grep -lr --include=*.php "https://example.com" /var/www/example.com/htdocs | xargs sed -i -e 's/https:\/\/www.example.com/https:\/\/www.example.com/g'

Pass an extension next to -i to create backup. For example:

grep -lr --include=*.php "http://example.com" /var/www/example.com/htdocs | xargs sed -i .bak -e 's/http:\/\/example.com/https:\/\/www.example.com/g'

Changing git-remote in multiple repos

We recently shifted our git repos to another server. Below commands came in handy!

Find all config files with old remote:

grep -lr --include=config "@rtcamp.com" .

Replace with new remote:

grep -lr --include=config "@rtcamp.com" . | xargs sed -i -e 's/@rtcamp.com/@ac.rtcamp.com/g'

You may want to search again to make sure if all files with old remotes are replaced.

Problems with xargs? Try parallel

On Ubuntu:

apt-get install moreutils

On Mac:

brew install parallel

Just replace word xargs with parallel in commands.