Spider/bulk-downloader-for-reddit
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge pull request #34 from aliparlakci/MinimalizeREADME

Browse Source 
Minimalize README
This commit is contained in:
aliparlakci 2018-07-11 23:29:58 +03:00 committed by GitHub
commit 9849c0439b
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
3 changed files with 92 additions and 128 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

134
README.md
View File

@ -1,20 +1,7 @@
# Bulk Downloader for Reddit
This program downloads imgur, gfycat and direct image and video links of saved posts from a reddit account. It is written in Python 3.
**PLEASE** post any issue you had with the script to [Issues](https://github.com/aliparlakci/bulk-downloader-for-reddit/issues) tab. Since I don't have any testers or contributers I need your feedback.
## Table of Contents
- [What it can do?](#what-it-can-do)
- [Requirements](#requirements)
- [Setting up the script](#setting-up-the-script)
- [Creating an imgur app](#creating-an-imgur-app)
- [Program Modes](#program-modes)
- [Running the script](#running-the-script)
- [Using the command line arguments](#using-the-command-line-arguments)
- [Examples](#examples)
- [FAQ](#faq)
- [Changelog](#changelog)
**PLEASE** post any issue you have with the script to [Issues](https://github.com/aliparlakci/bulk-downloader-for-reddit/issues) tab. Since I don't have any testers or contributers I need your feedback.
## What it can do?
### It...
@ -48,130 +35,23 @@ Because this is not a commercial app, you need to create an imgur developer app
It should redirect to a page which shows your **imgur_client_id** and **imgur_client_secret**
\*Select **OAuth 2 authorization without a callback URL** first then select **Anonymous usage without user authorization** if it says *Authorization callback URL: required*
\* Select **OAuth 2 authorization without a callback URL** first then select **Anonymous usage without user authorization** if it says *Authorization callback URL: required*
## Program Modes
- **saved mode**
- Gets posts from given user's saved posts.
- **submitted mode**
- Gets posts from given user's submitted posts.
- **upvoted mode**
- Gets posts from given user's upvoted posts.
- **subreddit mode**
- Gets posts from given subreddit or subreddits that is sorted by given type and limited by given number.
- **multireddit mode**
- Gets posts from given user's given multireddit that is sorted by given type and limited by given number.
- **link mode**
- Gets posts from given reddit link.
- You may customize the behaviour with `--sort`, `--time`, `--limit`.
- This mode is only accessible via command-line arguments. See **[`python script.py --help`](#using-the-command-line-arguments)**
- **log read mode**
- Takes a log file which created by itself (json files), reads posts and tries downloading them again.
- Running log read mode for FAILED.json file once after the download is complete is **HIGHLY** recommended as unexpected problems may occur.
## Running the script
Letting more than one instance of the script run is **not** suggested as it interferes with IMGUR Request Rate.
## Running the script with command line arguments
### Using the command line arguments
If no arguments are passed program will prompt you for arguments below which means you may start up the script with double-clicking on it (at least on Windows for sure).
Open up the [terminal](https://www.reddit.com/r/NSFW411/comments/8vtnl8/meta_i_made_reddit_downloader_that_can_download/e1rnbnl) and navigate to where script.py is. If you are unfamiliar with changing directories in terminal see Change Directories in [this article](https://lifehacker.com/5633909/who-needs-a-mouse-learn-to-use-the-command-line-for-almost-anything).
Run the script.py file from terminal with command-line arguments. Here is the help page:
Use `.\` for current directory and `..\` for upper directory when using short directories, otherwise it might act weird.
```console
$ python script.py --help
usage: script.py [-h] [--link link] [--saved] [--submitted] [--upvoted]
[--log LOG FILE] [--subreddit SUBREDDIT [SUBREDDIT ...]]
[--multireddit MULTIREDDIT] [--user redditor]
[--search query] [--sort SORT TYPE] [--limit Limit]
[--time TIME_LIMIT] [--NoDownload]
DIRECTORY
This program downloads media from reddit posts
positional arguments:
DIRECTORY Specifies the directory where posts will be downloaded
to
optional arguments:
-h, --help show this help message and exit
--link link, -l link Get posts from link
--saved Triggers saved mode
--submitted Gets posts of --user
--upvoted Gets upvoted posts of --user
--log LOG FILE Triggers log read mode and takes a log file
--subreddit SUBREDDIT [SUBREDDIT ...]
Triggers subreddit mode and takes subreddit's name
without r/. use "frontpage" for frontpage
--multireddit MULTIREDDIT
Triggers multireddit mode and takes multireddit's name
without m/
--user redditor reddit username if needed. use "me" for current user
--search query Searches for given query in given subreddits
--sort SORT TYPE Either hot, top, new, controversial, rising or
relevance default: hot
--limit Limit default: unlimited
--time TIME_LIMIT Either hour, day, week, month, year or all. default:
all
--NoDownload Just gets the posts and store them in a file for
downloading later
```
## Examples
- **Don't include `python script.py` part if you start the script by double-clicking**
- **Use `python3` instead of `python` if you are using *MacOS* or *Linux***
```console
python script.py .\\NEW_FOLDER --sort new --time all --limit 10 --link "https://www.reddit.com/r/gifs/search?q=dogs&restrict_sr=on&type=link&sort=new&t=month"
```
```console
python script.py .\\NEW_FOLDER --link "https://www.reddit.com/r/learnprogramming/comments/7mjw12/"
```
```console
python script.py .\\NEW_FOLDER --search cats --sort new --time all --subreddit gifs pics --NoDownload
```
```console
python script.py .\\NEW_FOLDER --user [USER_NAME] --submitted --limit 10
```
```console
python script.py .\\NEW_FOLDER --multireddit good_subs --user [USER_NAME] --sort top --time week --limit 250
```
```console
python script.py .\\NEW_FOLDER\\ANOTHER_FOLDER --saved --limit 1000
```
```console
python script.py C:\\NEW_FOLDER\\ANOTHER_FOLDER --log UNNAMED_FOLDER\\FAILED.json
```
See **[`python script.py --help`](docs/help_page.md)**
## FAQ
### I can't startup the script no matter what.
- Try these:
- **`python`**
- **`python3`**
- **`python3.7`**
- **`python3.6`**
- **`py -3`**
Python have real issues about naming their program
### I can't open the self post files.
- Self posts are held at reddit as styled with markdown. So, the script downloads them as they are in order not to lose their stylings. However, there is a great Chrome extension [here](https://chrome.google.com/webstore/detail/markdown-viewer/ckkdlimhmcjmikdlpkmbgfkaikojcbjk) for viewing Markdown files with its styling. Install it and open the files with Chrome.
- Self posts are held at reddit as styled with markdown. So, the script downloads them as they are in order not to lose their stylings.
However, there is a great Chrome extension [here](https://chrome.google.com/webstore/detail/markdown-viewer/ckkdlimhmcjmikdlpkmbgfkaikojcbjk) for viewing Markdown files with its styling. Install it and open the files with Chrome.
## Changelog
### [11/07/2018]()
- Improvements on UX and UI
- Added logging errors to CONSOLE_LOG.txt
- Using current directory if directory has not been given yet.
### [10/07/2018](https://github.com/aliparlakci/bulk-downloader-for-reddit/tree/ffe3839aee6dc1a552d95154d817aefc2b66af81)
- Added support for *self* post
- Now getting posts is quicker

82
docs/help_page.md Normal file
View File

@ -0,0 +1,82 @@
## python script.py --help
```console
usage: script.py [-h] [--directory DIRECTORY] [--link link] [--saved]
[--submitted] [--upvoted] [--log LOG FILE]
[--subreddit SUBREDDIT [SUBREDDIT ...]]
[--multireddit MULTIREDDIT] [--user redditor]
[--search query] [--sort SORT TYPE] [--limit Limit]
[--time TIME_LIMIT] [--NoDownload]
This program downloads media from reddit posts
optional arguments:
-h, --help show this help message and exit
--directory DIRECTORY
Specifies the directory where posts will be downloaded
to
--link link, -l link Get posts from link
--saved Triggers saved mode
--submitted Gets posts of --user
--upvoted Gets upvoted posts of --user
--log LOG FILE Triggers log read mode and takes a log file
--subreddit SUBREDDIT [SUBREDDIT ...]
Triggers subreddit mode and takes subreddit's name
without r/. use "frontpage" for frontpage
--multireddit MULTIREDDIT
Triggers multireddit mode and takes multireddit's name
without m/
--user redditor reddit username if needed. use "me" for current user
--search query Searches for given query in given subreddits
--sort SORT TYPE Either hot, top, new, controversial, rising or
relevance default: hot
--limit Limit default: unlimited
--time TIME_LIMIT Either hour, day, week, month, year or all. default:
all
--NoDownload Just gets the posts and store them in a file for
downloading later
```
## Examples
- **Don't include `python script.py` part if you start the script by double-clicking**
- **Use `python3` instead of `python` if you are using *MacOS* or *Linux***
```console
python script.py .\\NEW_FOLDER --sort new --time all --limit 10 --link "https://www.reddit.com/r/gifs/search?q=dogs&restrict_sr=on&type=link&sort=new&t=month"
```
```console
python script.py .\\NEW_FOLDER --link "https://www.reddit.com/r/learnprogramming/comments/7mjw12/"
```
```console
python script.py .\\NEW_FOLDER --search cats --sort new --time all --subreddit gifs pics --NoDownload
```
```console
python script.py .\\NEW_FOLDER --user [USER_NAME] --submitted --limit 10
```
```console
python script.py .\\NEW_FOLDER --multireddit good_subs --user [USER_NAME] --sort top --time week --limit 250
```
```console
python script.py .\\NEW_FOLDER\\ANOTHER_FOLDER --saved --limit 1000
```
```console
python script.py C:\\NEW_FOLDER\\ANOTHER_FOLDER --log UNNAMED_FOLDER\\FAILED.json
```
## FAQ
### I can't startup the script no matter what.
- Try these:
- **`python`**
- **`python3`**
- **`python3.7`**
- **`python3.6`**
- **`py -3`**
Python have real issues about naming their program

4
script.py
View File

@ -84,7 +84,9 @@ def parseArguments(arguments=[]):
help="Gets upvoted posts of --user")
parser.add_argument("--log",
help="Triggers log read mode and takes a log file",
help="Takes a log file which created by itself " \
"(json files), reads posts and tries downloadin" \
"g them again.",
# type=argparse.FileType('r'),
metavar="LOG FILE")