wikiextractor

NLP/wikiextractor

Fork 0

Commit Graph

Select branches

Hide Pull Requests

gh-pages

master

#100

#102

#103

#105

#108

#110

#110

#113

#119

#131

#134

#137

#138

#138

#141

#141

#143

#147

#147

#148

#148

#149

#149

#152

#166

#174

#175

#176

#180

#182

#182

#183

#186

#188

#188

#190

#190

#194

#194

#201

#201

#202

#202

#205

#211

#214

#218

#228

#230

#230

#232

#234

#234

#238

#238

#24

#241

#244

#249

#249

#25

#250

#250

#256

#257

#257

#259

#259

#261

#261

#263

#263

#265

#265

#27

#272

#276

#276

#289

#289

#29

#294

#294

#296

#296

#297

#297

#307

#307

#31

#313

#313

#37

#45

#45

#46

#48

#56

#59

#59

#64

#64

#66

#67

#68

#68

#7

#7

#71

#71

#81

#81

#82

#86

#86

#88

#88

#9

#9

#91

#91

#94

#97

#97

#99

#99

v3.0

v3.0.4

v3.0.6

v3.0.7

6b39739e2f

Merge ab8988ebfa into 8f1b434a80 Santhosh Thottingal 2023-05-05 08:17:23 +0000
ab8988ebfa

Make the regex python 3.11 compatible Santhosh Thottingal 2023-05-05 11:59:50 +0530
46865c9025

Merge 684a4097b2 into 8f1b434a80 Wayne Workman 2023-03-31 02:28:44 +0000
684a4097b2 Add argument to preserve unicode in json output. wayne.workman2012@gmail.com 2023-03-30 21:13:51 -0500
80f75be02d

Merge 1f07147628 into 8f1b434a80 Alex Lewis 2023-03-10 20:30:10 -0800
b4309a60f6

Merge daed1d21f0 into 8f1b434a80 Joonkyo 2023-03-10 20:30:08 -0800
b4977bdf27

Merge fe2cf8d9c6 into 8f1b434a80 prokotg 2023-02-27 23:48:31 +0800
128e05bb1d

Merge 9e22d54bd0 into 8f1b434a80 Woojin718 2023-02-13 15:02:32 -0800
f787010aca

Merge 4326419cf6 into 8f1b434a80 tuxiaohui001 2023-02-09 10:52:45 +0900
6d2cde17ca

Merge cf43b2bbc2 into 8f1b434a80 Hande Gözükan 2023-02-09 10:46:33 +0900
f26e0ba26d

Merge 112e7021e7 into 8f1b434a80 Kelvin Lee 2023-02-08 11:49:48 -0600
2fb90ae88f

Merge d833f6bb4d into 8f1b434a80 Jorge 2023-02-05 12:04:52 -1000
b2aa7c5d9e

Merge eabef5cf7b into 8f1b434a80 GaaH 2023-02-02 19:19:10 +0900
ee7171b853

Merge ffe96f7c87 into 8f1b434a80 Judith Jeong 2023-01-25 16:35:25 -0500
917d156ac2

Merge a1f03bd14b into 8f1b434a80 Jorge 2023-01-25 16:35:25 -0500
8f1b434a80 Fixed handling of templates. master v3.0.7 attardi 2023-01-24 19:02:27 +0100
112e7021e7

remove 1 redundant line, although it doesn't affect the function overall Kelvin Lee 2022-11-03 21:29:23 +0800
4326419cf6 修改readme信息 tuxiaohui001 2022-10-26 13:55:32 +0800
704a73db8b 修改readme信息 tuxiaohui001 2022-10-26 11:49:39 +0800
cf43b2bbc2 remove pdb Hande Gözükan 2022-09-21 14:19:40 +0200
9e22d54bd0 error_replacement Woojin718 2022-06-24 16:42:58 +0900
f0ca16c3e9 Merge branch 'master' of https://github.com/attardi/wikiextractor Giuseppe Attardi 2022-03-07 10:40:46 +0100
05b5cc7e51 Restored template expansion and HTML preserving flag. Giuseppe Attardi 2022-03-07 10:31:01 +0100
5047501dab Fixed pickling of file wrapper Benjamin Rotendahl 2021-09-15 10:40:11 +0200
b7024c1245 Renamed "to json" flag Benjamin Rotendahl 2021-09-15 10:38:40 +0200
eabef5cf7b add encoding parameter in load_templates Gaëtan Caillaut 2021-11-05 14:29:47 +0100
97620b2efd

Merge 09d3cb58e0 into f4f9d2d008 mohab 2021-10-14 21:42:48 -0500
c0ef39764d

Merge 133c50ada4 into f4f9d2d008 ujiuji1259 2021-10-14 21:42:48 -0500
694eb068d5

Merge b4eac55da6 into f4f9d2d008 Kapil Kukreja 2021-10-14 21:42:48 -0500
f4f9d2d008

Update WikiExtractor.py v3.0.6 Giuseppe Attardi 2021-10-14 14:29:26 +0200
ae7898a92e

Update README.md Giuseppe Attardi 2021-10-14 13:46:00 +0200
1053fe2030

Update WikiExtractor.py Giuseppe Attardi 2021-10-14 13:41:27 +0200
0242d58c26

Update README.md Giuseppe Attardi 2021-10-14 10:49:46 +0200
c423d2d0a6

Update README.md HjalmarrSv 2021-10-14 00:31:29 +0200
a1f03bd14b The required version is python 3.5 Jorge 2021-07-05 23:06:11 +0200
ee5fd8cc6a Requires 3.6 since it's not working with 3.7 nor 3.8 Jorge 2021-07-05 00:41:56 +0200
ffe96f7c87 Fix several lines to make working in Windows Judith Jeong 2021-06-27 18:45:19 +0900
b4eac55da6

Corrected the logic to avoid redirect pages Kapil Kukreja 2021-06-25 14:35:41 +0530
d833f6bb4d Fix extractPage version Jorge 2021-06-06 17:19:26 +0200
133c50ada4 rm log Shogo Ujiie 2021-05-04 19:43:16 +0900
6099f32c13 fix missing articles in wikiextractor Shogo Ujiie 2021-05-03 20:21:25 +0900
daed1d21f0

Update WikiExtractor.py Joonkyo 2021-03-18 16:29:42 +0900
881f3e4252

Merge pull request #241 from JonasTriki/fix-compressed-file-write Giuseppe Attardi 2021-03-18 01:14:59 +0100
09d3cb58e0 extractor summary list definition is simplified mohab 2021-03-17 19:55:06 +0200
40d0a37865 divided up the text into summary, and contnt for NLP processing mohab 2021-03-16 18:40:01 +0200
de7b86ca91

updated import in WIkiExtractor.py vakhokoto 2021-01-17 13:58:06 +0400
df435cad4d Explicitly use utf-8 when encoding data Jonas Triki 2020-12-29 21:06:49 +0100
0458f0c1fd Add minor fix to OutputSplitter.write() Jonas Triki 2020-12-29 15:00:55 +0100
6490f5361d Added back option --json. attardi 2020-12-17 13:17:34 +0100
fe2cf8d9c6 handle sys.stdout in reduce_process Tomasz Grzegorzek 2020-12-16 22:00:35 +0100
95ddfaa451 Fix to urlbase. attardi 2020-12-16 18:10:28 +0100
1c57c06596 Fix to urllib.parse.quote. attardi 2020-12-16 17:13:58 +0100
ad92a6abf2 Open output files after forking Tomasz Grzegorzek 2020-12-15 12:25:05 +0100
1f07147628 Import NextFile and OutputSplitter from WikiExtractor AceLewis 2020-12-06 00:24:20 +0100
ca5c40f68b Use param escape_doc. attardi 2020-12-05 20:37:12 +0100
3150f604e9 Use bz2.open. v3.0.4 attardi 2020-12-05 20:13:46 +0100
a2e078f3be Better handling of encoding. attardi 2020-12-05 19:27:01 +0100
0933664d70 Fully working on python 3. attardi 2020-12-05 11:21:46 +0100
5b4302bca0 Merge branch 'master' of https://github.com/attardi/wikiextractor attardi 2020-12-05 09:08:38 +0100
9b5c1cb238 Fix to load_templates. attardi 2020-12-05 09:07:43 +0100
d2732b1477

Update README.md Giuseppe Attardi 2020-12-04 19:12:01 +0100
2ba214ab99 Fix NameError #225. attardi 2020-12-04 18:43:22 +0100
8ef37c87e2 Fix to script invocation. attardi 2020-12-04 11:31:15 +0100
3179a4c393 Import default_template. attardi 2020-12-04 09:52:05 +0100
b69de65e7b fix: len(templates) missing Junseong Kim 2020-12-01 22:41:53 +0900
ca92712777 fix: missing templates attribute Junseong Kim 2020-12-01 22:31:53 +0900
1ac2474a58 fix: define_template import missing Junseong Kim 2020-12-01 22:25:14 +0900
bd59a7e987

Merge 0b715dc388 into 87549a91a6 Cemre 2020-11-29 19:40:57 +0000
87549a91a6

Create python-publish.yml v3.0 Giuseppe Attardi 2020-11-27 16:48:03 +0100
ba80db7163 Fixed print function in extractPage.py AceLewis 2020-11-26 20:45:45 +0100
0b715dc388

change write modes Cemre 2020-11-02 00:58:33 +0300
fa2dcf341f

open files in wb mode Cemre 2020-11-02 00:56:25 +0300
6a91345e48

wrap dictionary keys in a list stmt to avoid change Cemre 2020-11-02 00:53:40 +0300
a3b7faf521

encode all output Cemre 2020-11-02 00:52:16 +0300
ddbf5c1208

x.sort() to sorted(x) Cemre 2020-11-02 00:50:33 +0300
92a34bfd0c

change out to BytesIO from StringIO Cemre 2020-11-02 00:47:37 +0300
3e9d2cd999

change options.urlbase to urlbase Cemre 2020-11-02 00:41:47 +0300
f2d0bb8afc

set acceptedNamespaces to empty list if not given Cemre 2020-11-02 00:36:58 +0300
497a4c54de

change xrange to range Cemre 2020-11-02 00:33:01 +0300
ab4c69bb8c

replace %c%c statement with a .format Cemre 2020-11-02 00:31:26 +0300
1cac27416b

remove some decodes Cemre 2020-11-02 00:24:42 +0300
6b69c1cdd1

remove dot from extract import Cemre 2020-11-01 23:57:13 +0300
3eeffcb470 Put back the missing scripts. attardi 2020-10-05 12:07:41 +0200
1682e9d214 Updated readme with correct information Rogério Pontes 2020-08-14 12:02:28 +0200
74b6f353bf Created PyPI package Rogério Pontes 2020-08-14 11:47:15 +0200
da817825be Updated gitignore with wikipedia test files Rogério Pontes 2020-08-13 22:12:34 +0200
0025711b2e Fixed file read mode of compressed wiki files Rogério Pontes 2020-08-12 16:28:49 +0200
834e384cbe

Merge pull request #214 from AndyTheFactory/patch-1 Giuseppe Attardi 2020-07-24 18:27:18 +0200
0577d75e71

Update README.md Andy ThePHPFactory 2020-07-24 00:32:51 +0300
6beefe96df

Update extract.sh Giuseppe Attardi 2020-07-22 18:29:37 +0200
6675c69fcc Created PyPi release. attardi 2020-07-22 18:25:35 +0200
b13d447d93 Removed scripts directory. attardi 2020-07-22 15:03:20 +0200
62bdbe6106 Upgrade to Python 3.3+. attardi 2020-07-22 14:12:37 +0200
6408a430fc

Merge pull request #183 from albertvillanova/fix-encoding Giuseppe Attardi 2020-07-22 12:18:17 +0200
08985cab49

Merge pull request #205 from TDesjardins/patch-1 Giuseppe Attardi 2020-07-22 12:14:36 +0200
79f76061e3

Update README.md Giuseppe Attardi 2020-07-22 11:39:15 +0200
6e9cac2357

Update README.md Giuseppe Attardi 2020-07-22 11:38:18 +0200
f8282ab410

Create LICENSE Giuseppe Attardi 2020-07-22 11:34:16 +0200
5a788d2fa8 change cgi to html chihironuko 2020-07-17 15:21:52 +0900
fbeef46bef

Update cirrus-extract.py HjalmarrSv 2020-05-19 22:06:41 +0200