Why this pattern only match the first and the last

Question

haystack:

<h2 >a&nbsp; &middot;&nbsp;&middot;&nbsp;&middot;
</h2>
<div class="indent">
aaaa
</div>
<h2 >b&nbsp; &middot;&nbsp;&middot;&nbsp;&middot;
</h2>
<div class="indent">
bbbb
</div>

pattern I used:

#<h2[^>]*>(a|b)(?!</h2>)[\s\S]*</h2><div class="indent">((?!</div>)[\s\S]+)</div>#

this pattern only matches the first h2 content(e.g. a  · · ·) and the content in last div(e.g. bbbb)

but I whan it to match all content in the h2 and div to make an one to one map(e.g. a  · · ·=>aaaa,b  · · ·=>bbbb), how do I do this?

I'm not trying to parse it :), i'm making a spider... what else should i use to make one? — dotslashlu, Jun 14 '12 at 22:35
It should match none, as there is a newline after every ``... or did you alter your 'haystack'? — Wrikken, Jun 14 '12 at 22:40
@Daedalus: Regex is a perfectly good tool for parsing HTML, if you don't need to *fully* parse HTML, and your needs are simple enough. — Jonathan Hall, Jun 14 '12 at 23:19

score 1 · Accepted Answer · edited May 23 '17 at 12:03

[\s\S]* and [\s\S]+ are greedy, meaning they will match as many characters as possible. Try changing them to [\s\S]*? and [\s\S]+?.

With your current regex, if you were to put your [\s\S]* into a capturing group you would see that it matches the following:

&nbsp; &middot;&nbsp;&middot;&nbsp;&middot;
</h2>
<div class="indent">
aaaa
</div>
<h2 >b&nbsp; &middot;&nbsp;&middot;&nbsp;&middot;

Adding the ? at the end makes this lazy, so instead of matching as much as possible it will match as few characters as possible, so it will stop at the first </h2> like you want. The same reasoning applies to the [\s\S]+ later in your regex.

It also looks like this should fail on your sample string because you have </h2><div... in the middle of your regex, but in your sample text there is always a newline between the closing </h2> and the <div>, you should probably change this section to </h2>\s*<div.... End result:

#<h2[^>]*>(a|b)(?!</h2>)[\s\S]*?</h2>\s*<div class="indent">((?!</div>)[\s\S]+?)</div>#

But don't parse HTML with regex!

it works! thank you. And thanks for your reminding, but is crawling in the range of 'parse'? If I don't use regex to make a crawler then what should i use? — dotslashlu, Jun 14 '12 at 22:47
@Wasabi You should use an HTML parser that someone else has already written and done correctly. — Andrew Clark, Jun 14 '12 at 22:50

Why this pattern only match the first and the last

1 Answers1