How can I debug a regular expression in Python?

Question

Is there a way to debug a regular expression in Python? And I'm not referring to the process of trying and trying till they work :)

Here is how regexes can be debugged in Perl:


use re 'debug';

my $str = "GET http://some-site.com HTTP/1.1";
if($str =~/get\s+(\S+)/i) {
    print "MATCH:$1\n";
}

The code above produces the following output on my computer when ran:


Compiling REx "get\s+(\S+)"
Final program:
   1: EXACTF  (3)
   3: PLUS (5)
   4:   SPACE (0)
   5: OPEN1 (7)
   7:   PLUS (9)
   8:     NSPACE (0)
   9: CLOSE1 (11)
  11: END (0)
stclass EXACTF  minlen 5
Matching REx "get\s+(\S+)" against "GET http://some-site.com HTTP/1.1"
Matching stclass EXACTF  against "GET http://some-site.com HTTP/1.1" (33 chars)
   0           |  1:EXACTF (3)
   3        |  3:PLUS(5)
                                  SPACE can match 1 times out of 2147483647...
   4       |  5:  OPEN1(7)
   4       |  7:  PLUS(9)
                                    NSPACE can match 20 times out of 2147483647...
  24       |  9:    CLOSE1(11)
  24       | 11:    END(0)
Match successful!
MATCH:http://some-site.com
Freeing REx: "get\s+(\S+)"

Without sample code, all we can do is guess what your problem is. — S.Lott, Mar 03 '09 at 13:30
I don't have a regex problem right now. I only want to know how to perform the debugging. — Geo, Mar 03 '09 at 14:10
Maybe what the comments above imply is "To debug python regular expressions, post your SO to stackoverflow.com so poor souls who've gone before you can help" — qneill, Aug 16 '18 at 15:49
Perl has more powerful tool Regexp::Debugger - https://youtu.be/zcSFIUiMgAs?t=158 — Alexey Shrub, Apr 03 '19 at 12:17

score 21 · Accepted Answer · edited May 23 '17 at 12:10

21


>>> p = re.compile('.*', re.DEBUG)
max_repeat 0 65535
  any None
>>>

regex '|' operator vs separate runs for each sub-expression

edited May 23 '17 at 12:10

Community

1
1

answered Mar 03 '09 at 13:21

Mykola Kharechko

3,004
3
30
39

14

That's only half of the answer, it shows what the regexp compiles to, but doesn't show how it's executed against a given string. If anyone knows the other half, please share! – Nickolay Dec 30 '13 at 02:58
1

It looks like `re.search('look for: ".*"', 'look for: "this"', re.DEBUG)` gives a bunch more info. – user2561747 Oct 27 '20 at 21:21

score 4 · Answer 2 · answered Nov 26 '14 at 08:08

4

https://www.debuggex.com is also pretty good. It's an online Python (and a couple more languages) debugger, which has a pretty neat visualization of what does and what doesn't match. A pretty good resource if you need to draft a regexp quickly.

answered Nov 26 '14 at 08:08

Nikita R.

6,687
2
49
59

score 1 · Answer 3 · answered Mar 03 '09 at 13:16

1

Why don't you use some regEx tool (i usually use Regulator) and test the regex-expression there and when you are satisfied, just copy/paste it into your code.

answered Mar 03 '09 at 13:16

sabiland

2,426
1
23
23

Because using a regex tool won't tell me why my regex isn't working. – Geo Mar 03 '09 at 13:18
@Geo - what exactly do you mean by "isn't working". Isn't working at all, isn't matching the things you want to match or ... ? – Rook Mar 03 '09 at 13:24
1

At the risk of stating the obvious, a regex tool can't tell you why it isn't giving you the right matches. A regex is going to do exactly what you tell it, and the best any tool can do is step you through so that you can figure out yourself which bit is wrong. – Noldorin Mar 03 '09 at 13:27
@Noldorin - in which case I'd reccommend a book, "Learning ..." by O'Reilly, wonderful for this kinda stuff. – Rook Mar 03 '09 at 13:30
@Idigas: Not quite sure what you mean. There's a "Mastering Regular Expressions" book by O'Reilly... are you suggesting the OP reads this to understand RegEx better? – Noldorin Mar 03 '09 at 13:46
@Noldorin - yes, it's a nice book, very user friendly. Once he grasps the basics he'll have a easier time on. – Rook Mar 03 '09 at 17:46
@Noldorin - "Mastering" is also a nice book, but learning is better for starters (it's not just basics) – Rook Mar 03 '09 at 17:47

score 0 · Answer 4 · edited May 23 '17 at 10:29

0

Not sure about doing such a thing directly in Python, but I could definitely suggest using a RegEx editor tool. That's likely to be your best bet anyway. Personally, I've used The Regulator and found it to very helpful. Some others are listed in this SO thread.

edited May 23 '17 at 10:29

Community

1
1

answered Mar 03 '09 at 13:17

Noldorin

139,101
56
254
298

score 0 · Answer 5 · answered Mar 03 '09 at 13:23

0

Similar to the already mentioned, there is also Regexbuddy

answered Mar 03 '09 at 13:23

Rook

57,389
47
162
240

score 0 · Answer 6 · answered Mar 03 '09 at 14:00

I quite often use RegexPal for quick checks (an online regular expression prototyper). It has a lot of the common expressions listed along with a simple expression. Very handy when you don't have a dedicated tool and just need a quick way to work out a somple regex.

score 0 · Answer 7 · answered Mar 04 '09 at 01:44

0

What RegexBuddy has that the other tools don't have is a built-in debugger that shows you the entire matching process of both successful and failed match attempts. The other tools only show the final result (which RegexBuddy can show too).

answered Mar 04 '09 at 01:44

Jan Goyvaerts

20,743
7
59
69

How can I debug a regular expression in Python?

7 Answers7

Linked

Related