That's what the man sed.

D'oh!!!!

  2016 July 5    Tech   Computer

Introduction


Recently I had some HTML files where each one had a typo.  I had copy-pasted the same line into at least a dozen of them.  Then I noticed the typo.  After all that, no way I was going to go back and fix each one by hand.

I tried using the sed command. 

It went very wrong. 

Let's see what happened, and then we'll see how to do it right.


A Quick Note

This is a reader-supported website.  There are lots of articles and reviews on here (see the Site Map.)  When you use the links to purchase your stuff, it helps me keep this site on-line.  Thank you.

In This Article


What Is sed?

Attempt #1

The Method That Worked

Working On A Specific List Of Files

Conclusion




What Is sed?


sed stands for Stream EDitor.  It is a Linux program for handling text in various ways.  For example, it can add or remove blank spaces in a file, replace one text string with another in a file, or even count the number of characters or lines in a file.

It can also do these things in multiple files.  That's what makes sed really useful.  It's a whole lot more time-efficient than sitting there making all the changes manually with something like gedit, emacs, or vi.



Attempt #1


So I needed to replace a text string across several different files in a directory.  I figured it would have something to do with "sed", a Linux command, but I wasn't sure how.  (Okay, it was probably going to use the s/regexp/replacement/ function, as mentioned in the man page for sed.  But I can never seem to make sense of those manual pages.)

LQ had a post where someone described the exact problem, and someone else posted a solution.  Don't actually do this, but here's what it recommended:

TMPFILE=/tmp/tmp.$$

for f in *.txt; do
  sed 's/OLD_STRING/NEW_STRING' $f > $TMPFILE
  exit 1 # DEBUG
  mv $TMPFILE $f
fi


You may have noticed the "fi" is incorrect.  I think the person meant to write "done", not "fi".  Also, "NEW_STRING" should be followed by a forward slash before the single quote.  So, I corrected those errors.  I made it into a bash script, set it to executable, and went ahead and ran the script... except that I commented out the entire "exit 1" line. 

That ran the script in live mode, which means "no do-overs".

Bad idea.

Whatever else this script actually did, it blanked out all the files that it worked on.  Made them into empty files.  And like a dummy, I didn't notice that until after I went ahead and uploaded all the files to my web directory.  You know, overwriting what was there...

So, are you wondering why the "What's New Archive" is now missing over a year's worth of entries?  Welp, that's the reason.  And wouldn't you know it, I seem to have forgotten to make backups of them.  (Yeah, I know;  even the guy who posted that script said to make backups.)




The Method That Worked


It would have helped to figure this out BEFORE I wiped out a heap of important files, but as they say... ah, nevermind what they say.  Let's see the correct way to use sed to replace text across multiple files.

This time I know it's the right way, because I tested it on a bunch of dummy files.  You know, like someone who just erased over a year's worth of important files and says "Wow, I'll make sure not to do that again..."

Here ya go:

     sed -i 's/wrong.text/right-text/' test*.htm

And that's it.  It's a one-liner, so you can even run that directly on the command line.  No need to invoke 'for' loops, string variables, or any of that.  No need to invoke the 'find' or 'grep' command. 

If for some reason you need to run that as a script, you would put the usual

     #!/bin/bash

in the beginning of the script, save it to the directory where the target files are, run chmod +x on it, and so on.

If the target filename conforms to the wildcard pattern test*.htm and it contains the text string wrong.text, it will be replaced with right-text.

Thus, the script will replace that text string in files named "test01.htm", "test02.htm", "testwhatever.htm", etc.

Will this work on your particular system?  It depends on what version of sed you have, I guess, because different ones seem to use slightly different rules.  But it should work, generally.


    



Working On A Specific List Of Files


You don't have to use wildcards or even just one filename to choose the target files.  If you have a list of, say, three files you want to change, you could just list the filenames after the sed command.  Like this:

     sed -i 's/wrong.text/right-text/' test01.txt test02.txt test03.txt

This would be a good idea where a wildcard expression (such as test*.txt) might affect some of the wrong files.




Changing Every Instance Of A Word


If you run sed with the parameters I've shown, it will replace only the first instance of the word on each line.

Let's suppose the files have some lines with two or more instances of wrong.text per line.  You'll need to add a g at the end of the delimiters, like this:

     sed -i 's/wrong.text/right-text/g' test01.txt test02.txt test03.txt



Conclusion


Don't run scripts you got from Internet experts without first testing them on junk files.  (Duh.  A little "Note To Self", that.)

sed was always kind of mysterious to me.  Now, not as much.  sed probably does about 10 million other things, but now I can use it to replace text strings across multiple files.  And now, hopefully so can you.

If this article was helpful to you, please help me keep this reader-supported website on-line by shopping for your gear through any of the links on here.  Electronics, computers, shoes, just about anything.  Your help is greatly appreciated.

Thanks for visiting this page!





    


Contact me:

3 p o.t o . 1 2 0 s t u d i o.. c o m


This won't directly copy and paste.  Please manually type it into your mail program.
No spaces between letters.





Home Page


Site Map


What's New!




Disclaimer

Copyright 2016






 



Back to Top of Page