How to insert strings containing slashes with sed?

Question

I have a Visual Studio project, which is developed locally. Code files have to be deployed to a remote server. The only problem are the URLs they contain, which are hard-coded.

The project contains URLS such as ?page=one. For the link to be valid on the server, it must be /page/one .

I've decided to replace all URLS in my code files with sed before deployment, but I'm stuck on slashes.

I know this is a not a pretty solution, but it's simple and would save me a lot of time. The total number of strings I have to replace is fewer than 10. Total number of files which have to be checked is ~30.

Example describing my situation is below:

Command I'm using:

sed -f replace.txt < a.txt > b.txt

replace.txt which contains all the strings:

s/?page=one&/pageone/g
s/?page=two&/pagetwo/g
s/?page=three&/pagethree/g

a.txt:

?page=one&
?page=two&
?page=three&

Content of b.txt after I run my sed command:

pageone
pagetwo
pagethree

What I want b.txt to contain:

/page/one
/page/two
/page/three

Answer 1

The easiest way would be to use a different delimiter in your search/replace lines, e.g.:

s:?page=one&:pageone:g

You can use any character as a delimiter that's not part of either string. Or, you could escape it with a backslash:

s/\//foo/

Which would replace / with foo. You'd want to use the escaped backslash in cases where you don't know what characters might occur in the replacement strings (if they are shell variables, for example).

Answer 2

The s command can use any character as a delimiter; whatever character comes after the s is used. I was brought up to use a #. Like so:

s#?page=one&#/page/one#g

Answer 3

A very useful but lesser-known fact about sed is that the familiar s/foo/bar/ command can use any punctuation, not only slashes. A common alternative is s@foo@bar@, from which it becomes obvious how to solve your problem.

Answer 4

add \ before special characters:

s/\?page=one&/page\/one\//g

etc.

Answer 5

In a system I am developing, the string to be replaced by sed is input text from a user which is stored in a variable and passed to sed.

As noted earlier on this post, if the string contained within the sed command block contains the actual delimiter used by sed - then sed terminates on syntax error. Consider the following example:

This works:

$ VALUE=12345
$ echo "MyVar=%DEF_VALUE%" | sed -e s/%DEF_VALUE%/${VALUE}/g
MyVar=12345

This breaks:

$ VALUE=12345/6
$ echo "MyVar=%DEF_VALUE%" | sed -e s/%DEF_VALUE%/${VALUE}/g
sed: -e expression #1, char 21: unknown option to `s'

Replacing the default delimiter is not a robust solution in my case as I did not want to limit the user from entering specific characters used by sed as the delimiter (e.g. "/").

However, escaping any occurrences of the delimiter in the input string would solve the problem. Consider the below solution of systematically escaping the delimiter character in the input string before having it parsed by sed. Such escaping can be implemented as a replacement using sed itself, this replacement is safe even if the input string contains the delimiter - this is since the input string is not part of the sed command block:

$ VALUE=$(echo ${VALUE} | sed -e "s#/#\\\/#g")
$ echo "MyVar=%DEF_VALUE%" | sed -e s/%DEF_VALUE%/${VALUE}/g
MyVar=12345/6

I have converted this to a function to be used by various scripts:

escapeForwardSlashes() {

     # Validate parameters
     if [ -z "$1" ]
     then
             echo -e "Error - no parameter specified!"
             return 1
     fi

     # Perform replacement
     echo ${1} | sed -e "s#/#\\\/#g"
     return 0
}

Answer 6

this line should work for your 3 examples:

sed -r 's#\?(page)=([^&]*)&#/\1/\2#g' a.txt

• I used -r to save some escaping .
• the line should be generic for your one, two three case. you don't have to do the sub 3 times

test with your example (a.txt):

kent$  echo "?page=one&
?page=two&
?page=three&"|sed -r 's#\?(page)=([^&]*)&#/\1/\2#g'
/page/one
/page/two
/page/three

Answer 7

please see this article http://netjunky.net/sed-replace-path-with-slash-separators/

Just using | instead of /

Answer 8

replace.txt should be

s/?page=/\/page\//g
s/&//g

Answer 9

Great answer from Anonymous. \ solved my problem when I tried to escape quotes in HTML strings.

So if you use sed to return some HTML templates (on a server), use double backslash instead of single:

var htmlTemplate = "<div style=\\"color:green;\\"></div>";

Answer 10

sed is the stream editor, in that you can use | (pipe) to send standard streams (STDIN and STDOUT specifically) through sed and alter them programmatically on the fly, making it a handy tool in the Unix philosophy tradition; but can edit files directly, too, using the -i parameter mentioned below.
Consider the following:

sed -i -e 's/few/asd/g' hello.txt

s/ is used to substitute the found expression few with asd:

The few, the brave.

---

The asd, the brave.

/g stands for "global", meaning to do this for the whole line. If you leave off the /g (with s/few/asd/, there always needs to be three slashes no matter what) and few appears twice on the same line, only the first few is changed to asd:

The few men, the few women, the brave.

---

The asd men, the few women, the brave.

This is useful in some circumstances, like altering special characters at the beginnings of lines (for instance, replacing the greater-than symbols some people use to quote previous material in email threads with a horizontal tab while leaving a quoted algebraic inequality later in the line untouched), but in your example where you specify that anywhere few occurs it should be replaced, make sure you have that /g.

The following two options (flags) are combined into one, -ie:

-i option is used to edit in place on the file hello.txt.

-e option indicates the expression/command to run, in this case s/.

Note: It's important that you use -i -e to search/replace. If you do -ie, you create a backup of every file with the letter 'e' appended.

Answer 11

A simplier alternative is using AWK as on this answer:

awk '$0="prefix"$0' file > new_file