1

Let's say I have some text:

<hello> <world> <:how> are <you>

Now I want to HTML-encode it so that the <>s don't mess things up. But <:how> is special because it has a : in it, so I don't want to touch that.

I can replace it using a regex with something like {{how}}, then do the HTML encoding, and then replace it back.

But what if {{something}} already appears somewhere in the code? Then {{something}} would get converted to <:something> when it should have been left as is.

I've encountered this problem a few times in the past, and still haven't found a good way to approach it. Do people just choose something really obscure to replace with and hope that it doesn't exist elsewhere, or is there a proper way to do this?

mpen
  • 272,448
  • 266
  • 850
  • 1,236

2 Answers2

2

Using regexps for HTML parsing is bad. But let's consider you just fix a small piece of your own code.

Consider this regexp: <(?!:): it matches any < which is not followed by a :, but the colon is not included in the match, so you can just use a replace string of &lt;.

Find out where in your favorite text editor is the checkbox 'use regexps'. (In vi, it's implicitly there, checked.) The expression above only works if your editor supports decent regexp syntax; most do now.

But your original approach is also workable. If it's impractical to enumerate several complex exclusion patterns in a regexp, you can replace these patterns with some strings temporarily. Just make these strings fairy unique. I bet that if you replaced <:with LESS=THAN=AND=COLON, there's about zero chance that you clashed with something, or forgot what does this string stand for. Yes, these temp strings are an eyesore: this makes the chance that you forget to replace them back pretty slim.

Community
  • 1
  • 1
9000
  • 39,899
  • 9
  • 66
  • 104
  • Well I'm doing this in Python, and was just using the standard `cgi.escape` to do the HTML-escaping, but I suppose I could write my own escape function that skips over `<:blah>`s. But that's missing the point. The format might be completely different, might not involve HTML at all. The question is how to I safely hide chunks of text from find-and-replaces (in the general sense)? Also, FYI, I'm not parsing HTML with regexes. – mpen Sep 06 '11 at 23:18
  • I see. But I failed not to mention such a great post once it's vaguely related :) If you're not doing this in a text editor, the simpler it is. Use a regexp to find a (simple) pattern that you could need to replace, check if it's ok (with any means) and replace it if you like it. See API of `re`. There's no easy way to do this with just `str.replace()`, though. – 9000 Sep 06 '11 at 23:24
  • So in short, conditionally replace only the elements I want, rather than all of them. I could do that with a regex callback more easily. Fair enough. Still won't work if I want to run 3rd party transformations on the text though. – mpen Sep 06 '11 at 23:56
1

You could implement an escaping mechanism based on some characters that wouldn't survive the encoding process. For instance, if you are html-encoding your input, you know that after that you won't have any < or > characters, because they are replaced by html entities. So you can use as an escape code a string made of < or >. If you're going to display the final code in a browser you could use something like <!-- TOKEN --> as escape code, since it wouldn't affect the visualization.

Your encoding process could look like this:

  • input string:
    • <hello> {{world}} <:how> are <you>
  • replace <xxx> with &lt;xxx&gt; where xxx doesn't start with :
    • &lt;hello&gt; {{world}} <:how> are &lt;you&gt;
  • replace <:xxx> with {{<!-- TOKEN -->xxx}}
    • &lt;hello&gt; {{world}} {{<!-- TOKEN -->how}} are &lt;you&gt;

Displayed in browser, {{world}} and {{how}} would look the same but they would preserve decoding informations. Indeed, the corresponding decoding process would be:

  • input string:
    • &lt;hello&gt; {{world}} {{<!-- TOKEN -->how}} are &lt;you&gt;
  • replace {{<!-- TOKEN -->xxx}} with <:xxx>
    • &lt;hello&gt; {{world}} <:how> are &lt;you&gt;
  • replace &lt;xxx&gt; with <xxx>
    • <hello> {{world}} <:how> are <you>

Like I said, since the characters you are basing your escape code on can't appear by themselves in the encoded text, having an input like {{<!-- TOKEN -->how}} wouldn't break the encoding/decoding process, because it would be encoded like {{&lt;!-- TOKEN --&gt;how}} and so reversed correctly.

etuardu
  • 5,066
  • 3
  • 46
  • 58