Spider/bulk-downloader-for-reddit
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
1,137 Commits 2 Branches 45 Tags 14 MiB
master
Commit Graph

220 Commits

Author SHA1 Message Date
Soulsuck24
0e23dcb8ad
Gfycat/Redgifs coverage 
Coverage for direct gfycat links that redirect to redgifs. The redirect through the sites themselves are broken but this fixes that.

Coverage for o.imgur links and incorrect capitalisation of domains in download_factory.

Changed tests for direct as gfycat is handled by the gfycat downloader.

fix pornhub test as the previous video was removed.
 2023-01-30 14:52:08 -05:00
OMEGARAZER
63b0607f58
bugbear B007 2023-01-28 02:04:06 -05:00
OMEGARAZER
cf5f7bfd16
pep585 and pathlib updates 2023-01-25 22:23:59 -05:00
Serene
e96b167b71
Merge pull request #748 from Soulsuck24/development 2023-01-25 20:25:21 +10:00
Soulsuck24
105ceaf386
More Redgifs coverage 2023-01-25 04:43:06 -05:00
OMEGARAZER
804d0eb661
Fix failing test 
Replace deleted user in test.
 2023-01-21 22:01:18 -05:00
OMEGARAZER
5fbe64dc71
Move Imgur to API 
Moves Imgur to use API with public Client-ID.
 2023-01-21 17:49:31 -05:00
OMEGARAZER
8e60a12517
Imgur thumbnail coverage 
Coverage for links posted to thumbnail variations.
 2023-01-09 20:53:53 -05:00
OMEGARAZER
0f94c98733
Account for new gallery url 
Coverage for gallery urls
 2023-01-09 12:48:24 -05:00
Serene-Arc
8f17bcf43b Modify code to accept pluggable logging handlers 2023-01-08 15:21:57 +10:00
Serene-Arc
9b23082a78 Fix test 2023-01-07 17:21:54 +10:00
Serene-Arc
d18c30b481 Fix failing tests 2023-01-07 17:21:25 +10:00
Soulsuck24
579c5ab8eb
Add new Redgifs subdomain 
Seems there's a v3 subdomain now (looks like it's mostly for mobile)
 2023-01-06 11:56:54 -05:00
Serene-Arc
468b5a33ae Switch to a minute wiggle room 2023-01-05 20:01:49 +10:00
Serene-Arc
37dd7f11a8 Add some wiggle room to test results 2023-01-05 20:01:00 +10:00
Serene-Arc
12311029e4 Conform test name 2023-01-04 19:49:50 +10:00
Serene-Arc
77a01e1627 Add tests for new option 2023-01-04 19:49:09 +10:00
Serene-Arc
b64f508025 Conform path length to filename scheme restriction 2023-01-04 19:37:02 +10:00
Serene-Arc
8c57dc2283 Add missing pytest flags to test 2023-01-04 19:07:31 +10:00
Serene-Arc
f40ac35f4a Add option to determine restriction scheme 2023-01-03 19:16:23 +10:00
OMEGA_RAZER
954df88c86
Merge branch 'aliparlakci:development' into development 2023-01-01 06:51:47 -05:00
Serene
b7aae727e5
Merge pull request #738 from Soulsuck24/development 2023-01-01 19:59:52 +10:00
OMEGARAZER
b6edc36753
Update connector for 7 digit ID's 2023-01-01 04:09:11 -05:00
Soulsuck24
874c7e3117
Redgif fixes 
Missing half of #733
 2022-12-31 08:53:13 -05:00
OMEGARAZER
2bafb1b99b
Consolidate flake8 settings 
Consolidates sane flake8 settings to pyproject with the Flake8-pyproject plugin.

Does not change logic of test workflow but allows base settings to live in pyproject for anyone using flake8 as an external linter (e.g. vscode)

Also fixes some flake8 errors that were not being picked up by current testing, mostly unused imports.
 2022-12-28 10:00:43 -05:00
Soulsuck24
13887ca7e1
Redgif updates 
Coverage for direct links.

The direct link won't work because it will have the wrong auth anyway but this will at least end up with the right API call.
 2022-12-26 20:25:20 -05:00
Soulsuck24
fe9cc7f29f
Redgifs updates 
Update Redgifs regex for further edge case.

Add test for checking ID.
 2022-12-24 20:52:45 -05:00
OMEGA_RAZER
e92929ef07
Merge branch 'aliparlakci:development' into completion 2022-12-22 23:46:06 -05:00
Serene
c63a8842d9
Merge pull request #729 from OMEGARAZER/development 2022-12-22 18:11:16 +10:00
OMEGARAZER
2aea7d0d48
Move completion to pathlib 2022-12-20 13:05:50 -05:00
OMEGARAZER
57ac0130a6
revert init files 2022-12-20 12:16:31 -05:00
OMEGA_RAZER
da74096cde
Update test_download_factory.py 2022-12-19 22:04:49 -05:00
Serene
8d43cdeef7
Merge pull request #726 from Soulsuck24/development 2022-12-20 10:28:10 +10:00
OMEGARAZER
83f45e7f60
Standardize shebang and coding declaration 
Standardizes shebang and coding declarations.

Coding matches what's used by install tools such as pip(x).

Removes a few init files that were not needed.
 2022-12-19 18:32:37 -05:00
OMEGARAZER
5d3a539eda
Fix install of completion if dirs missing 
Fixes situations if completion directories are missing and adds tests for installer.
 2022-12-19 17:54:34 -05:00
Soulsuck24
603e7de04d
Redgifs fix 
Handle redgifs link with trailing / causing id to return empty string.
 2022-12-19 11:02:06 -05:00
OMEGARAZER
af6222e06c
Cleanup tests 
Cleans up test args on new tests.

Add log path to default config test so as not to mangle default log outside of tests.

Match setup functions to archive/clone.

Remove testing marker that was commited in error.
 2022-12-17 20:35:58 -05:00
Serene
58e1d1a8f9
Merge pull request #713 from OMEGARAZER/development 2022-12-16 10:59:35 +10:00
OMEGARAZER
4ba5df6b37
5xx error tests 2022-12-14 23:19:15 -05:00
Soulsuck24
15a9d25a9d
Imgur webp coverage 
update regex to catch _d in webp links from imgur.
 2022-12-11 14:20:04 -05:00
OMEGARAZER
d4bfe8fa19
Formatting cleanup 
Cleanup some formatting from switch to Black
 2022-12-03 14:49:39 -05:00
Serene-Arc
0873a4a2b2 Format according to the black standard 2022-12-03 15:11:17 +10:00
OMEGARAZER
b30ced9be9
Redo Pylance typing changes 2022-11-30 21:48:10 -05:00
Serene
45429be27c
Revert "Pylance typing" 2022-12-01 12:37:03 +10:00
OMEGARAZER
69fa1f3f09
Pylance typing 
Fix Pylance warnings for typing
 2022-11-30 21:29:07 -05:00
OMEGARAZER
ef7fcce1cc
lint tests 
Lint with [refurb](https://github.com/dosisod/refurb) using `--disable 126 --python-version 3.9`

Also update bats to 1.8.2 and bats-assets to 2.1.0. No changes to the tests, all still passing.
 2022-11-30 18:05:10 -05:00
Soulsuck24
48c96beba2
Redgifs improvements 
Add check to verify token was received.
Update headers sent to content API.

Add availability check for videos to resolve last part of #472 where only SD version is available.
 2022-11-27 18:07:43 -05:00
OMEGARAZER
9ee13aea23
Update tests 
Suspended user in two tests.

Updated hashes and yt-dlp version.

Removed success check on known failure.
 2022-11-26 14:36:19 -05:00
Serene-Arc
87104e7e6a Catch exceptions in cloner 2022-11-24 10:48:17 +10:00
OMEGARAZER
42416db8b9
Fix PRAW deprecations 
Fix depreciations in MultiredditHelper and CommentForest.
 2022-11-21 21:37:59 -05:00