data mismatch with spark regexp_extract

Asked Oct 21 '19 at 10:32

Active Oct 21 '19 at 10:32

Viewed 33 times

Sample Data:

https://www.test.com/document/101?ref=stringA&qid=e9f7a92b
https://www.test.com/document/102?ref=stringB

Regex: ref=([a-zA-Z0-9]+\.?)/?

Match1 is stringA and stringB

Doing the same in pyspark

spark.sql("select regexp_extract(url, 'ref=([a-zA-Z0-9]+\.?)/?', 1) c1 from my_table").show()

Output:

+---------------+
|c1             |
+---------------+
|stringA&       |
|stringB&       |
+---------------+

asked Oct 21 '19 at 10:32

Dev

It seems you need to double the backslash, `'ref=([a-zA-Z0-9]+\\.?)/?'` – Wiktor Stribiżew Oct 21 '19 at 10:33
are you sure you don't need to "\\" before `.` ? – Code Maniac Oct 21 '19 at 10:33
it worked with "\\\" before `.` – Dev Oct 21 '19 at 10:36

0 Answers0