How to input a regex in string.replace?

Question

I need some help on declaring a regex. My inputs are like the following:

this is a paragraph with<[1> in between</[1> and then there are cases ... where the<[99> number ranges from 1-100</[99>. 
and there are many other lines in the txt files
with<[3> such tags </[3>

The required output is:

this is a paragraph with in between and then there are cases ... where the number ranges from 1-100. 
and there are many other lines in the txt files
with such tags

I've tried this:

#!/usr/bin/python
import os, sys, re, glob
for infile in glob.glob(os.path.join(os.getcwd(), '*.txt')):
    for line in reader: 
        line2 = line.replace('<[1> ', '')
        line = line2.replace('</[1> ', '')
        line2 = line.replace('<[1>', '')
        line = line2.replace('</[1>', '')
        
        print line

I've also tried this (but it seems like I'm using the wrong regex syntax):

        line2 = line.replace('<[*> ', '')
        line = line2.replace('</[*> ', '')
        line2 = line.replace('<[*>', '')
        line = line2.replace('</[*>', '')

I dont want to hard-code the replace from 1 to 99.

ridgerunner · Accepted Answer · 2021-08-29T21:13:47.347

836

This tested snippet should do it:

import re
line = re.sub(r"</?\[\d+>", "", line)

Edit: Here's a commented version explaining how it works:

line = re.sub(r"""
  (?x) # Use free-spacing mode.
  <    # Match a literal '<'
  /?   # Optionally match a '/'
  \[   # Match a literal '['
  \d+  # Match one or more digits
  >    # Match a literal '>'
  """, "", line)

Regexes are fun! But I would strongly recommend spending an hour or two studying the basics. For starters, you need to learn which characters are special: "metacharacters" which need to be escaped (i.e. with a backslash placed in front - and the rules are different inside and outside character classes.) There is an excellent online tutorial at: www.regular-expressions.info. The time you spend there will pay for itself many times over. Happy regexing!

edited Aug 29 '21 at 21:13

answered Apr 14 '11 at 04:12

ridgerunner

33,777
5
57
69

18

Also don't neglect The Book on Regular Expressions - _Mastering Regular Expressions_, by [Jeffrey Friedl](http://regex.info/) – pcurry May 14 '13 at 05:05
3

Another good reference sees https://www.w3schools.com/python/python_regex.asp – Carson May 15 '20 at 11:23
2

The commented version mentions `(?x)` free-spacing mode, but that is not in the snippet. Is that a default or something? – RufusVS Sep 24 '20 at 21:38
2

@RufusVS - The '(?x)' inside the regex text tells the regex engine compiler that this regex is written in free-spacing mode. You could alternatively add the: 're.VERBOSE' compilation flag to the function call. – ridgerunner Aug 29 '21 at 21:21
691 votes and a bounty for that? You'd be lucky to get a single vote nowadays. And Rufus already knew it meant free-spacing mode, he just didn't know what that was - and you have still not explained it to him. – MikeM Aug 29 '21 at 22:09
4

@ridgerunner Actually, my point was, you have the expression `(?x)` present in your commented version of the pattern, but NOT in the uncommented version above it that you called `tested snippet`. (Edit: I googled it and found out the free-spacing expression is only needed because you have comments and spaces in the explanatory version, not needed in the `tested snippet` version. Got it now.) – RufusVS Aug 30 '21 at 03:27
Every answer on this entire site for a regex question should be written this way with comments explaining each part – Jacob Myer Dec 16 '21 at 15:11
If you reuse the pattern often, [compile](https://docs.python.org/3/library/re.html#re.compile) it first. For example, `import re; mypattern=re.compile(r'a'); mypattern.sub('o', 'faa') # 'foo'`. Instead of `'o'` you can also provide a [function](https://docs.python.org/3/library/re.html#re.sub:~:text=If%20repl%20is%20a%20function%2C%20it,argument%2C%20and%20returns%20the%20replacement%20string), for example `lambda matchobject: matchobject.group(0).upper() # 'fAA'` – Nils Lindemann Sep 15 '22 at 12:14

score 59 · Answer 2 · answered Apr 14 '11 at 04:00

59

str.replace() does fixed replacements. Use re.sub() instead.

answered Apr 14 '11 at 04:00

Ignacio Vazquez-Abrams

776,304
153
1,341
1,358

4

Also worth noting that your pattern should look something like "{0-1}\d{1-2}>" or whatever variant of regexp notation python uses. – Apr 14 '11 at 04:05
7

What does fixed replacements mean? – avi Jul 03 '15 at 11:35
@avi Probably he meant fixed word replacement rather partial word locating through regex. – Gunay Anach Jul 11 '17 at 08:48
2

fixed (literal, constant) strings – vstepaniuk Jul 31 '19 at 11:54

Lorenzo Persichetti · Answer 3 · 2013-06-27T08:33:56.577

I would go like this (regex explained in comments):

import re

# If you need to use the regex more than once it is suggested to compile it.
pattern = re.compile(r"</{0,}\[\d+>")

# <\/{0,}\[\d+>
# 
# Match the character “<” literally «<»
# Match the character “/” literally «\/{0,}»
#    Between zero and unlimited times, as many times as possible, giving back as needed (greedy) «{0,}»
# Match the character “[” literally «\[»
# Match a single digit 0..9 «\d+»
#    Between one and unlimited times, as many times as possible, giving back as needed (greedy) «+»
# Match the character “>” literally «>»

subject = """this is a paragraph with<[1> in between</[1> and then there are cases ... where the<[99> number ranges from 1-100</[99>. 
and there are many other lines in the txt files
with<[3> such tags </[3>"""

result = pattern.sub("", subject)

print(result)

If you want to learn more about regex I recomend to read Regular Expressions Cookbook by Jan Goyvaerts and Steven Levithan.

From the [python docs](https://docs.python.org/2/howto/regex.html#repeating-things): `{0,}` is the same as `*`, `{1,}` is equivalent to `+`, and `{0,1}` is the same as `?`. It’s better to use `*`, `+`, or `?` when you can, simply because they’re shorter and easier to read. — winklerrr, Aug 17 '17 at 14:07

score 17 · Answer 4 · answered May 14 '13 at 16:13

The easiest way

import re

txt='this is a paragraph with<[1> in between</[1> and then there are cases ... where the<[99> number ranges from 1-100</[99>.  and there are many other lines in the txt files with<[3> such tags </[3>'

out = re.sub("(<[^>]+>)", '', txt)
print out

score 14 · Answer 5 · answered May 16 '13 at 10:42

14

replace method of string objects does not accept regular expressions but only fixed strings (see documentation: http://docs.python.org/2/library/stdtypes.html#str.replace).

You have to use re module:

import re
newline= re.sub("<\/?\[[0-9]+>", "", line)

answered May 16 '13 at 10:42

Zac

2,180
2
23
36

6

You should use `\d+` instead of `[0-9]+` – winklerrr Aug 17 '17 at 14:05

score 5 · Answer 6 · answered Apr 14 '11 at 04:06

don't have to use regular expression (for your sample string)

>>> s
'this is a paragraph with<[1> in between</[1> and then there are cases ... where the<[99> number ranges from 1-100</[99>. \nand there are many other lines in the txt files\nwith<[3> such tags </[3>\n'

>>> for w in s.split(">"):
...   if "<" in w:
...      print w.split("<")[0]
...
this is a paragraph with
 in between
 and then there are cases ... where the
 number ranges from 1-100
.
and there are many other lines in the txt files
with
 such tags

Abena Saulka · Answer 7 · 2019-01-24T21:22:54.903

4

import os, sys, re, glob

pattern = re.compile(r"\<\[\d\>")
replacementStringMatchesPattern = "<[1>"

for infile in glob.glob(os.path.join(os.getcwd(), '*.txt')):
   for line in reader: 
      retline =  pattern.sub(replacementStringMatchesPattern, "", line)         
      sys.stdout.write(retline)
      print (retline)

edited Jan 24 '19 at 21:22

answered Jan 24 '19 at 21:15

Abena Saulka

41
4

How to input a regex in string.replace?

7 Answers7

Linked

Related