Commit Graph

45 Commits

Author SHA1 Message Date
rugantio
96d3423b8d Merge branch 'master' of https://github.com/rugantio/fbcrawl 2019-02-18 02:14:01 +01:00
rugantio
b3d12c4e6b refactoring comments spider 2019-02-18 02:12:52 +01:00
Rugantio Costa
98642cffd8
Update README.md 2019-02-05 04:49:57 +01:00
rugantio
bdeae9f4b5 parse_page refactoring complete 2019-02-05 03:48:00 +01:00
rugantio
71f80356dc fixed attribute parsing 2019-02-04 20:27:26 +01:00
rugantio
811f4e396d fixed attribute parsing 2019-02-04 20:25:54 +01:00
rugantio
d28d214993 fixed recursion on pages 2019-02-04 19:27:44 +01:00
rugantio
dafd01c8bd fixed recursion on pages 2019-02-04 19:26:00 +01:00
rugantio
918cd9ce64 added new features, simplified presentation 2019-01-31 07:28:08 +01:00
rugantio
a9982865d9 improved support for languages en, es, fr, it, pt 2019-01-31 06:54:31 +01:00
rugantio
fb32a4213e added experimental support for languages en, es, fr, it, pt 2019-01-30 20:34:25 +01:00
rugantio
9de51e0ce8 steady recursion implemented 2019-01-30 17:30:18 +01:00
rugantio
b24fc61dbb steady recursion implemented 2019-01-30 17:21:43 +01:00
rugantio
eaaa2a32e3 cleaning up 2019-01-29 22:27:55 +01:00
rugantio
17430a06a9 fixed datetime parser 2019-01-29 22:26:01 +01:00
rugantio
b06883dc3c fixed gitignore 2019-01-29 22:08:37 +01:00
rugantio
8d343b057c gitignore added 2019-01-29 21:57:48 +01:00
rugantio
b8d8444b3f fix xpath in comment crawler 2019-01-26 02:52:26 +01:00
rugantio
80a10f176f fix xpath in comment crawler 2019-01-26 02:51:18 +01:00
Rugantio Costa
0c0f3129cd
Update README.md 2018-12-27 02:20:46 +01:00
Rugantio Costa
e95c70c844
Update README.md 2018-12-27 01:47:29 +01:00
rugantio
c04509499f changed user-agent and fixed date parsers in items.py 2018-12-13 05:33:09 +01:00
rugantio
30c04c2fca changed user-agent and fixed date parsers in items.py 2018-12-13 05:31:18 +01:00
rugantio
168bc2c510 disabling pipeline 2018-11-22 21:22:18 +01:00
Rugantio Costa
a153b1fa7a
Update README.md 2018-11-18 00:46:28 +01:00
Rugantio Costa
fdff7b826c
Update README.md 2018-10-22 00:35:56 +02:00
Rugantio Costa
30ba7dd125
new spider is introduced 2018-10-22 00:33:20 +02:00
rugantio
2a4ac3a5e2 new spider for comments 2018-10-22 00:24:57 +02:00
Rugantio Costa
728516617f
Update README.md 2018-09-19 01:39:20 +02:00
rugantio
0477d9a780 final 2018-08-27 02:23:57 +02:00
rugantio
ae2b0698f6 final 2018-08-27 02:23:25 +02:00
rugantio
cf6313e4b1 final 2018-08-27 02:21:51 +02:00
rugantio
8359748b81 final 2018-08-26 17:09:16 +02:00
rugantio
a26cbd969c final 2018-08-26 15:43:06 +02:00
rugantio
1f1d6775af final 2018-08-26 15:41:06 +02:00
rugantio
a6d39a4a7e final 2018-08-26 14:12:47 +02:00
rugantio
31c30c7b52 final 2018-08-26 14:08:36 +02:00
rugantio
888ebeab70 final 2018-08-26 04:22:17 +02:00
rugantio
2614f009ec final 2018-08-26 04:20:47 +02:00
rugantio
8babf7aa1d final 2018-08-26 03:42:52 +02:00
Rugantio Costa
cdf6bdc68e
Update README.md 2018-07-25 22:12:33 +02:00
Rugantio Costa
ecf8273df5
Update README.md 2018-07-25 21:50:16 +02:00
rugantio
b147bcf317 update 2018-07-25 21:44:45 +02:00
rugantio
2e00226d52 update 2018-07-25 21:43:53 +02:00
Rugantio Costa
fa320a2978
Initial commit 2018-07-25 21:36:57 +02:00