evanferrao
/
dump-mirrorbot
mirror of https://github.com/Box-boi/dump-mirrorbot.git
1
0
Fork 0
Code Issues Actions Packages Projects Releases Wiki Activity
272 Commits 2 Branches 21 Tags 2 MiB
Commit Graph

108 Commits

Author SHA1 Message Date
Nageen Chand b6b8a416e9
add ability to unzip files (#93) 
handle unsupported request

use shell utils to extract archives

use shell utils to extract archives

update mirror calls for unzipmirror

fix extract location of files

install deps

handling if archive is not extracted correctly

typo fix

implement Extract Status

handle filenames with spaces

handle special character
 2020-05-28 00:26:22 +05:30
lzzy12 0f7b79f7a4 Improved upload progress implementation 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-05-21 13:08:50 +05:30
jaskaranSM cdefe12348 Implement Centralized Aria Websocket Listener 2020-05-15 13:34:09 +05:30
jaskaranSM 1670b619ae Fix deadlock 2020-05-12 14:27:24 +05:30
jaskaranSM c4e81d5ab1 start Aria2 Websocket listener before adding download in queue. 2020-05-12 13:50:16 +05:30
lzzy12 d3938784ac Remove broken direct download link extracters 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-05-09 00:10:09 +05:30
jaskaranSM 315108e7d1 patch regex for more links 2020-05-05 10:51:55 +05:30
jaskaranSM 4d964b6e29 use regex to extract fileId from GDrive links 2020-05-04 12:25:40 +05:30
jaskaranSM 78b60d6715
import threading lib 2020-04-19 14:58:35 +05:30
jaskaranSM 8624e0174b Fix IO Block on Large Playlists 2020-04-18 14:32:01 +05:30
jaskaranSM 120ca7538d clone: remove forward slash 2020-04-16 12:39:46 +05:30
jaskaranSM e5bb8e55ef Clone fix and added index link support 2020-04-16 12:38:12 +05:30
jaskaranSM d7bc8f3c9a switch to context based callbacks 2020-04-16 12:26:28 +05:30
lzzy12 c6b4c542b7 fixup after e0f4c4347c 
- Also fixed copying of netrc file in the container

Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-04-15 14:47:33 +05:30
lzzy12 e0f4c4347c Separate out youtube-dl command 
- /watch [youtube-dl supported link]

Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-04-14 23:20:52 +05:30
lzzy12 e5364f96e1 Revert "Handle rare bug when download is not deleted from download_dict on error" 
This reverts commit 136924ffd4.

Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-04-13 13:32:39 +05:30
lzzy12 76e3db1946 [Hotfix] Fix bug where the link have 302 request are treated as torrents 
Download.is_torrent introduced in aria2p v0.9.0
https://github.com/pawamoy/aria2p/issues/53

Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-04-08 19:29:59 +05:30
Arsalan ed60afa9d6
fix typo in INDEX_URL (#54) 2020-04-07 20:36:04 +05:30
lzzy12 b2f2ee4335 Fix aria downloads cancellation after b58eb87b11 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-04-06 13:14:40 +05:30
lzzy12 47dd3c2b6f gdriveTools: Handle unhandled exception in clone method 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-04-05 14:12:13 +05:30
lzzy12 0536472e28 Expose message object from Status objects 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-04-05 13:28:35 +05:30
lzzy12 b58eb87b11 Improvements in cancelling mirrors 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-04-05 12:34:08 +05:30
jaskaranSM 708e6df87a youtube-dl: use best format 2020-04-01 12:07:29 +05:30
lzzy12 50ab740219 Correct filename if ffmpeg conversion is invoked by yt-dl 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-31 17:56:57 +05:30
lzzy12 a54fc42e1e Handle some exceptions 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-31 15:42:13 +05:30
lzzy12 e2c5c8beaf Log youtube-dl warnings and errors in bot log 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-31 11:15:09 +05:30
lzzy12 85d4005c16 Use netrc by default for youtube-dl 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-31 10:50:53 +05:30
lzzy12 9fa41f68c1 Some minor Fix-ups 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-30 18:16:54 +05:30
lzzy12 136924ffd4 Handle rare bug when download is not deleted from download_dict on error 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-30 17:46:56 +05:30
jaskaranSM 52e8fbec11 Fixed direct link torrent mirroring 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-30 17:05:55 +05:30
lzzy12 0ec499ff4a Handle weird path naming of youtube_dl 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-30 16:36:12 +05:30
lzzy12 141b13b52d Make ydl object local to the methods 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-30 14:55:29 +05:30
lzzy12 3bf9e0ea31 Use default quality provided by youtube-dl for now 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-28 20:14:43 +05:30
jaskaranSM 6262b26fe5 youtube-dl: fixed downloading from sites that provide encrypted content (m3u8) 2020-03-28 18:44:02 +05:30
lzzy12 7c304d449a Fix uploading for YT-DL 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-28 14:38:08 +05:30
lzzy12 9057d3fc4e Fixed yt-dl support 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-28 13:47:19 +05:30
jaskaranSM 3a06458609 Added Youtube-dl support 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-27 20:04:13 +05:30
lzzy12 2254444378 Handle KeyError exception in Telegram download error 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-27 16:00:37 +05:30
Shivam Jha 6d07c3c8c4
Stop trying to generate sourceforge direct links 2020-03-26 16:02:36 +05:30
jaskaranSM be9c64a9aa Use Service Accounts for Cloning if availble. 
- increase default timeout for network socket interface

Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-25 18:00:33 +05:30
jaskaranSM 08a71e4bd4 TeamDrive List fix 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-24 18:57:10 +05:30
lzzy12 6ca2f479d5 Fix up errors after porting service accounts from develop 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-24 16:30:03 +05:30
lzzy12 25856bfac0 gdrive: Stop spamming logs at INFO log level 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-24 15:49:04 +05:30
jaskaranSM 8d1bf5aa92 recursively try again instead of handling switch with backoff 
- Handling service account switching with exponential backoff implementation limits the number of service accounts to be used. recursively retrying again after switch doesnt effect backoff.

Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-24 15:41:59 +05:30
jaskaranSM 6cd59d5a15 Cleanup drive_list and minor fixes 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-24 15:39:57 +05:30
jaskaranSM 681b8bd47d Added missing property 2020-03-24 15:07:45 +05:30
jaskaranSM 2bbfca0496 Refactor service account implementation 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-24 15:07:23 +05:30
lzzy12 7ad7d9388a Add support for service account 
Adding support service accounts

Signed-off-by: lzzy12 <jhashivam2020@gmail.com>

Added scripts and docs for generating service accounts

Signed-off-by: lzzy12 <jhashivam2020@gmail.com>

gen_sa_accounts: Save credentials with indexed file name

Signed-off-by: lzzy12 <jhashivam2020@gmail.com>

gdriveTools: Avoid using oauth2 library for service accounts

oauth2 library is deprecated

Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-24 15:04:24 +05:30
jaskaranSM 997e572b69 Enable TeamDrive support for clone 2020-03-24 14:17:50 +05:30
lzzy12 94b6be4c03 Add licensing and authorship information for direct_link_generator.py 
Signed-off-by: lzzy12 <jhashivam2020@gmail.com>
 2020-03-23 15:41:53 +05:30