NLP/wikiextractor
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

multiline tag match fix

Browse Source 
need to add re.DOTALL so that multiline tag definitions are also matched
This commit is contained in:
Roman Prokofyev 2015-06-12 14:23:14 +02:00
parent 625d4b69b3
commit 70ee947a8b
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
WikiExtractor.py
View File

@ -215,7 +215,7 @@ comment = re.compile(r'<!--.*?-->', re.DOTALL)
# Match ignored tags # Match ignored tags
ignored_tag_patterns = [] ignored_tag_patterns = []
def ignoreTag(tag): def ignoreTag(tag):
left = re.compile(r'<%s\b.*?>' % tag, re.IGNORECASE) # both <ref> and <reference> left = re.compile(r'<%s\b.*?>' % tag, re.IGNORECASE | re.DOTALL) # both <ref> and <reference>
right = re.compile(r'</\s*%s>' % tag, re.IGNORECASE) right = re.compile(r'</\s*%s>' % tag, re.IGNORECASE)
ignored_tag_patterns.append((left, right)) ignored_tag_patterns.append((left, right))