evanferrao
/
dump-mirrorbot
mirror of https://github.com/Box-boi/dump-mirrorbot.git
1
0
Fork 0
Code Issues Actions Packages Projects Releases Wiki Activity

v4.7.1 (#82) 

- Revert to megasdk
- Added Estimated Dyno Expired in usage command

Co-authored-by: Ncode2014 <Ncode2014@users.noreply.github.com>
This commit is contained in:
Hafitz Setya 2021-05-08 14:20:12 +07:00 committed by GitHub
parent d3b8ac6eff
commit 9612d9994b
Signed by untrusted user: GitHub
GPG Key ID: 4AEE18F83AFDEB23
18 changed files with 297 additions and 226 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
.gitignore vendored
View File

@ -11,4 +11,3 @@ data*
authorized_chats.txt
log.txt
accounts/*
venv/

0
netrc → .netrc
View File

5
Dockerfile
View File

@ -3,8 +3,7 @@ FROM ubuntu:20.04
WORKDIR /usr/src/app
RUN chmod 777 /usr/src/app
RUN apt-get -qq update && \
apt-get upgrade -y && apt-get autoremove -y && \
DEBIAN_FRONTEND="noninteractive" apt-get -qq install -y tzdata aria2 wget git python3 python3-pip \
DEBIAN_FRONTEND="noninteractive" apt-get -qq install -y tzdata aria2 git python3 python3-pip \
locales python3-lxml \
curl pv jq ffmpeg \
p7zip-full p7zip-rar \
@ -18,8 +17,6 @@ COPY requirements.txt .
COPY extract /usr/local/bin
COPY pextract /usr/local/bin
RUN chmod +x /usr/local/bin/extract && chmod +x /usr/local/bin/pextract
RUN wget -q https://github.com/P3TERX/aria2.conf/raw/master/dht.dat -O /usr/src/app/dht.dat && \
wget -q https://github.com/P3TERX/aria2.conf/raw/master/dht6.dat -O /usr/src/app/dht6.dat
RUN pip3 install --no-cache-dir -r requirements.txt
RUN locale-gen en_US.UTF-8
ENV LANG en_US.UTF-8

12
README.md
View File

@ -18,7 +18,7 @@ This is a telegram bot writen in python for mirroring files on the internet to o
## From Source Repos
- Mirroring direct download links, Torrent, and Telegram files to Google Drive
- Mirroring Mega.nz links to google drive
- Mirroring Mega.nz links to Google Drive (If your Mega account not premium, it will limit 4-5gb/day)
- Copy files from someone's drive to your drive (Using Autorclone)
- Download/upload progress, speeds and ETAs
- Mirror all youtube-dl supported links
@ -92,12 +92,13 @@ Fill up rest of the fields. Meaning of each fields are discussed below:
- **INDEX_URL**: (Optional field) Refer to https://github.com/maple3142/GDIndex/ The URL should not have any trailing '/'
- **API_KEY**: This is to authenticate to your telegram account for downloading Telegram files. You can get this from https://my.telegram.org DO NOT put this in quotes.
- **API_HASH**: This is to authenticate to your telegram account for downloading Telegram files. You can get this from https://my.telegram.org
- **MEGA_KEY**: Mega.nz api key to mirror mega.nz links. Get it from [Mega SDK Page](https://mega.nz/sdk)
- **MEGA_USERNAME**: Your email id you used to sign up on mega.nz for using premium accounts (Leave th)
- **MEGA_API_KEY**: Mega.nz api key to mirror mega.nz links. Get it from [Mega SDK Page](https://mega.nz/sdk)
- **MEGA_EMAIL_ID**: Your email id you used to sign up on mega.nz for using premium accounts (Leave th)
- **MEGA_PASSWORD**: Your password for your mega.nz account
- **STOP_DUPLICATE_MIRROR**: (Optional field) (Leave empty if unsure) if this field is set to `True` , bot will check file in drive, if it is present in drive, downloading will ne stopped. (Note - File will be checked using filename, not using filehash, so this feature is not perfect yet)
- **ENABLE_FILESIZE_LIMIT**: Set it to `True` if you want to use `MAX_TORRENT_SIZE`.
- **MAX_TORRENT_SIZE**: To limit the torrent mirror size, Fill The amount you want to limit, examples: if you fill `15` it will limit `15gb`.
- **BLOCK_MEGA_FOLDER**: (Optional field) If you want to remove mega.nz folder support, set it to `True`.
- **BLOCK_MEGA_LINKS**: (Optional field) If you want to remove mega.nz mirror support (bcoz it's too much buggy and unstable), set it to `True`.
- **IMAGE_URL**: (Optional field) Show Image/Logo in /start message. Fill value of image your link image, use telegra.ph or any direct link image.
- **UPTOBOX_TOKEN**: Uptobox token to mirror uptobox links. Get it from [Uptobox Premium Account](https://uptobox.com/my_account).
@ -122,7 +123,7 @@ shortzon.com
- Go to the Credentials tab and click Create Credentials -> OAuth Client ID
- Choose Desktop and Create.
- Use the download button to download your credentials.
- Move that file to the root of mirrorbot, and rename it to credentials.json
- Move that file to the root of mirrorbot, and rename it to **credentials.json**
- Visit [Google API page](https://console.developers.google.com/apis/library)
- Search for Drive and enable it if it is disabled
- Finally, run the script to generate **token.pickle** file for Google Drive:
@ -148,7 +149,8 @@ sudo docker run mirrorbot
## Deploying on Heroku
Fork this repo then upload **token.pickle** to your forks
- Fork this repo then upload **token.pickle** to your forks
- Hit the deploy to heroku button and follow the further instructions in the screen
**NOTE**: If you didn't upload **token.pickle**, uploading will not work.
<p><a href="https://heroku.com/deploy"> <img src="https://img.shields.io/badge/Deploy%20To%20Heroku-blueviolet?style=for-the-badge&logo=heroku" width="200""/></a></p>

20
app.json
View File

@ -1,5 +1,5 @@
{
"name": "slam-mirrorbot ",
"name": "slam-mirrorbot",
"description": "A Telegram bot for all your mirror needs",
"logo": "https://telegra.ph/file/6507910fd06d18dfaba82.jpg",
"keywords": [
@ -8,14 +8,14 @@
"python"
],
"repository": "https://github.com/breakdowns/slam-mirrorbot",
"website": "https://t.me/SlamMirrorSupport",
"success_url": "https://github.com/breakdowns/slam-mirrorbot/blob/master/README.md",
"website": "https://github.com/breakdowns/slam-mirrorbot",
"success_url": "https://t.me/SlamMirrorSupport",
"stack": "container",
"env": {
"ENV": {
"description": "Setting this to ANYTHING will enable Webhooks when in env mode",
"value": "ANYTHING"
},
"description": "Setting this to ANYTHING will enable Webhooks when in env mode",
"value": "ANYTHING"
},
"BOT_TOKEN": {
"description": "The telegram bot token that you get from @BotFather.",
"required": true
@ -78,11 +78,11 @@
"description": "Uptobox premium token to mirror uptobox links. Get it from https://uptobox.com/my_account.",
"required": false
},
"MEGA_KEY": {
"MEGA_API_KEY": {
"description": "Mega.nz api key to mirror mega.nz links. Get it from https://mega.nz/sdk.",
"required": false
},
"MEGA_USERNAME": {
"MEGA_EMAIL_ID": {
"description": "Your email id you used to sign up on mega.nz.",
"required": false
},
@ -90,6 +90,10 @@
"description": "Your password for your mega.nz account.",
"required": false
},
"BLOCK_MEGA_FOLDER": {
"description": "If you want to remove mega.nz folder support, set it to True.",
"required": false
},
"BLOCK_MEGA_LINKS": {
"description": "If you want to remove mega.nz mirror support, set it to True.",
"required": false

12
aria.sh
View File

@ -8,14 +8,8 @@ aria2c --enable-rpc --rpc-listen-all=false --rpc-listen-port 6800 --check-certif
--follow-torrent=mem --split=10 \
--daemon=true --allow-overwrite=true --max-overall-download-limit=$MAX_DOWNLOAD_SPEED \
--max-overall-upload-limit=1K --max-concurrent-downloads=$MAX_CONCURRENT_DOWNLOADS \
--peer-id-prefix=-TR2940- --user-agent=Transmission/2.94 --peer-agent=Transmission/2.94 --continue=true \
--disk-cache=64M --file-allocation=prealloc \
--peer-id-prefix=-qB4220- --user-agent=qBittorrent/4.2.2 \
--disk-cache=64M --file-allocation=prealloc --continue=true \
--max-file-not-found=5 --max-tries=20 --auto-file-renaming=true \
--bt-enable-lpd=true --seed-time=0.01 --seed-ratio=1.0 \
--file-allocation=prealloc --max-file-not-found=5 --max-tries=5 --retry-wait=5 \
--auto-file-renaming=true --reuse-uri=true --http-accept-gzip=true --listen-port=49152-65535 \
--content-disposition-default-utf8=true --bt-tracker-connect-timeout=600 \
--dht-listen-port=51513 --enable-dht=true --enable-dht6=true \
--dht-file-path=/usr/src/app/dht.dat --dht-file-path6=/usr/src/app/dht6.dat \
--dht-entry-point=dht.transmissionbt.com:6881 \
--dht-entry-point6=dht.transmissionbt.com:6881
--content-disposition-default-utf8=true --http-accept-gzip=true --reuse-uri=true

51
bot/__init__.py
View File

@ -14,8 +14,6 @@ from telegraph import Telegraph
import socket
import faulthandler
faulthandler.enable()
from megasdkrestclient import MegaSdkRestClient, errors as mega_err
import subprocess
socket.setdefaulttimeout(600)
@ -109,35 +107,18 @@ telegraph_token = telegraph.get_access_token()
LOGGER.info("Telegraph Token Generated: '" + telegraph_token + "'")
try:
MEGA_KEY = getConfig('MEGA_KEY')
MEGA_API_KEY = getConfig('MEGA_API_KEY')
except KeyError:
MEGA_KEY = None
LOGGER.info('MEGA API KEY NOT AVAILABLE')
if MEGA_KEY is not None:
# Start megasdkrest binary
subprocess.Popen(["megasdkrest", "--apikey", MEGA_KEY])
time.sleep(3) # Wait for the mega server to start listening
mega_client = MegaSdkRestClient('http://localhost:6090')
try:
MEGA_USERNAME = getConfig('MEGA_USERNAME')
MEGA_PASSWORD = getConfig('MEGA_PASSWORD')
if len(MEGA_USERNAME) > 0 and len(MEGA_PASSWORD) > 0:
try:
mega_client.login(MEGA_USERNAME, MEGA_PASSWORD)
except mega_err.MegaSdkRestClientException as e:
logging.error(e.message['message'])
exit(0)
else:
LOGGER.info("Mega API KEY provided but credentials not provided. Starting mega in anonymous mode!")
MEGA_USERNAME = None
MEGA_PASSWORD = None
except KeyError:
LOGGER.info("Mega API KEY provided but credentials not provided. Starting mega in anonymous mode!")
MEGA_USERNAME = None
MEGA_PASSWORD = None
else:
MEGA_USERNAME = None
logging.warning('MEGA API KEY not provided!')
MEGA_API_KEY = None
try:
MEGA_EMAIL_ID = getConfig('MEGA_EMAIL_ID')
MEGA_PASSWORD = getConfig('MEGA_PASSWORD')
if len(MEGA_EMAIL_ID) == 0 or len(MEGA_PASSWORD) == 0:
raise KeyError
except KeyError:
logging.warning('MEGA Credentials not provided!')
MEGA_EMAIL_ID = None
MEGA_PASSWORD = None
try:
HEROKU_API_KEY = getConfig('HEROKU_API_KEY')
@ -220,6 +201,14 @@ try:
USE_SERVICE_ACCOUNTS = False
except KeyError:
USE_SERVICE_ACCOUNTS = False
try:
BLOCK_MEGA_FOLDER = getConfig('BLOCK_MEGA_FOLDER')
if BLOCK_MEGA_FOLDER.lower() == 'true':
BLOCK_MEGA_FOLDER = True
else:
BLOCK_MEGA_FOLDER = False
except KeyError:
BLOCK_MEGA_FOLDER = False
try:
BLOCK_MEGA_LINKS = getConfig('BLOCK_MEGA_LINKS')
if BLOCK_MEGA_LINKS.lower() == 'true':
@ -241,6 +230,6 @@ try:
except KeyError:
IMAGE_URL = 'https://telegra.ph/file/db03910496f06094f1f7a.jpg'
updater = tg.Updater(token=BOT_TOKEN,use_context=True)
updater = tg.Updater(token=BOT_TOKEN, use_context=True)
bot = updater.bot
dispatcher = updater.dispatcher

19
bot/helper/ext_utils/bot_utils.py
View File

@ -101,7 +101,7 @@ def get_readable_message():
msg += f"\n<b>Downloaded:</b> {get_readable_file_size(download.processed_bytes())} of {download.size()}"
else:
msg += f"\n<b>Uploaded:</b> {get_readable_file_size(download.processed_bytes())} of {download.size()}"
msg += f"\n<b>Speed:</b> {download.speed()}, \n<b>ETA:</b> {download.eta()} "
msg += f"\n<b>Speed:</b> {download.speed()} | <b>ETA:</b> {download.eta()} "
# if hasattr(download, 'is_torrent'):
try:
msg += f"\n<b>Seeders:</b> {download.aria_download().num_seeders}" \
@ -133,10 +133,6 @@ def get_readable_time(seconds: int) -> str:
return result
def is_mega_link(url: str):
return "mega.nz" in url
def is_url(url: str):
url = re.findall(URL_REGEX, url)
if url:
@ -144,13 +140,24 @@ def is_url(url: str):
return False
def is_mega_link(url: str):
return "mega.nz" in url
def get_mega_link_type(url: str):
if "folder" in url:
return "folder"
elif "file" in url:
return "file"
elif "/#F!" in url:
return "folder"
return "file"
def is_magnet(url: str):
magnet = re.findall(MAGNET_REGEX, url)
if magnet:
return True
return False
def new_thread(fn):
"""To use as decorator to make a function call threaded.
Needs import

7
bot/helper/mirror_utils/download_utils/aria2_download.py
View File

@ -32,9 +32,9 @@ class AriaDownloadHelper(DownloadHelper):
smsg, button = gdrive.drive_list(sname)
if smsg:
dl.getListener().onDownloadError(f'File is already available in drive.\n\n')
sendMarkup("Here are the search results: 👇", dl.getListener().bot, dl.getListener().update, button)
sendMarkup("Here are the search results:", dl.getListener().bot, dl.getListener().update, button)
aria2.remove([download])
size = download.total_length
if ENABLE_FILESIZE_LIMIT:
if size / 1024 / 1024 / 1024 > MAX_TORRENT_SIZE:
@ -71,8 +71,7 @@ class AriaDownloadHelper(DownloadHelper):
def __onDownloadStopped(self, api, gid):
LOGGER.info(f"onDownloadStop: {gid}")
dl = getDownloadByGid(gid)
if dl:
dl.getListener().onDownloadError('Dead Torrent!')
if dl: dl.getListener().onDownloadError('Dead torrent!')
@new_thread
def __onDownloadError(self, api, gid):

4
bot/helper/mirror_utils/download_utils/download_helper.py
View File

@ -9,8 +9,8 @@ class MethodNotImplementedError(NotImplementedError):
class DownloadHelper:
def __init__(self):
self._name = '' # Name of the download; empty string if no download has been started
self._size = 0.0 # Size of the download
self.name = '' # Name of the download; empty string if no download has been started
self.size = 0.0 # Size of the download
self.downloaded_bytes = 0.0 # Bytes downloaded
self.speed = 0.0 # Download speed in bytes per second
self.progress = 0.0

108
bot/helper/mirror_utils/download_utils/mega_download.py
View File

@ -1,108 +0,0 @@
import threading
from bot import LOGGER, download_dict, download_dict_lock
from .download_helper import DownloadHelper
from ..status_utils.mega_status import MegaDownloadStatus
from megasdkrestclient import MegaSdkRestClient, constants
from bot.helper.ext_utils.bot_utils import setInterval
from pathlib import Path
class MegaDownloader:
POLLING_INTERVAL = 2
def __init__(self, listener):
super().__init__()
self.__listener = listener
self.__name = ""
self.__gid = ''
self.__resource_lock = threading.Lock()
self.__mega_client = MegaSdkRestClient('http://localhost:6090')
self.__periodic = None
self.__downloaded_bytes = 0
self.__progress = 0
self.__size = 0
@property
def progress(self):
with self.__resource_lock:
return self.__progress
@property
def downloaded_bytes(self):
with self.__resource_lock:
return self.__downloaded_bytes
@property
def size(self):
with self.__resource_lock:
return self.__size
@property
def gid(self):
with self.__resource_lock:
return self.__gid
@property
def name(self):
with self.__resource_lock:
return self.__name
@property
def download_speed(self):
if self.gid is not None:
return self.__mega_client.getDownloadInfo(self.gid)['speed']
def __onDownloadStart(self, name, size, gid):
self.__periodic = setInterval(self.POLLING_INTERVAL, self.__onInterval)
with download_dict_lock:
download_dict[self.__listener.uid] = MegaDownloadStatus(self, self.__listener)
with self.__resource_lock:
self.__name = name
self.__size = size
self.__gid = gid
self.__listener.onDownloadStarted()
def __onInterval(self):
dlInfo = self.__mega_client.getDownloadInfo(self.gid)
if (dlInfo['state'] == constants.State.TYPE_STATE_COMPLETED or dlInfo[
'state'] == constants.State.TYPE_STATE_CANCELED or dlInfo[
'state'] == constants.State.TYPE_STATE_FAILED) and self.__periodic is not None:
self.__periodic.cancel()
if dlInfo['state'] == constants.State.TYPE_STATE_COMPLETED:
self.__onDownloadComplete()
return
if dlInfo['state'] == constants.State.TYPE_STATE_CANCELED:
self.__onDownloadError('Cancelled by user')
return
if dlInfo['state'] == constants.State.TYPE_STATE_FAILED:
self.__onDownloadError(dlInfo['error_string'])
return
self.__onDownloadProgress(dlInfo['completed_length'], dlInfo['total_length'])
def __onDownloadProgress(self, current, total):
with self.__resource_lock:
self.__downloaded_bytes = current
try:
self.__progress = current / total * 100
except ZeroDivisionError:
self.__progress = 0
def __onDownloadError(self, error):
self.__listener.onDownloadError(error)
def __onDownloadComplete(self):
self.__listener.onDownloadComplete()
def add_download(self, link, path):
Path(path).mkdir(parents=True, exist_ok=True)
dl = self.__mega_client.addDl(link, path)
gid = dl['gid']
info = self.__mega_client.getDownloadInfo(gid)
file_name = info['name']
file_size = info['total_length']
self.__onDownloadStart(file_name, file_size, gid)
LOGGER.info(f'Started mega download with gid: {gid}')
def cancel_download(self):
LOGGER.info(f'Cancelling download on user request: {self.gid}')
self.__mega_client.cancelDl(self.gid)

172
bot/helper/mirror_utils/download_utils/mega_downloader.py Normal file
View File

@ -0,0 +1,172 @@
from bot import LOGGER, MEGA_API_KEY, download_dict_lock, download_dict, MEGA_EMAIL_ID, MEGA_PASSWORD
import threading
from mega import (MegaApi, MegaListener, MegaRequest, MegaTransfer, MegaError)
from bot.helper.telegram_helper.message_utils import update_all_messages
import os
from bot.helper.ext_utils.bot_utils import new_thread, get_mega_link_type
from bot.helper.mirror_utils.status_utils.mega_download_status import MegaDownloadStatus
import random
import string
class MegaDownloaderException(Exception):
pass
class MegaAppListener(MegaListener):
_NO_EVENT_ON = (MegaRequest.TYPE_LOGIN,MegaRequest.TYPE_FETCH_NODES)
NO_ERROR = "no error"
def __init__(self, continue_event: threading.Event, listener):
self.continue_event = continue_event
self.node = None
self.public_node = None
self.listener = listener
self.uid = listener.uid
self.__bytes_transferred = 0
self.is_cancelled = False
self.__speed = 0
self.__name = ''
self.__size = 0
self.error = None
self.gid = ""
super(MegaAppListener, self).__init__()
@property
def speed(self):
"""Returns speed of the download in bytes/second"""
return self.__speed
@property
def name(self):
"""Returns name of the download"""
return self.__name
def setValues(self, name, size, gid):
self.__name = name
self.__size = size
self.gid = gid
@property
def size(self):
"""Size of download in bytes"""
return self.__size
@property
def downloaded_bytes(self):
return self.__bytes_transferred
def onRequestStart(self, api, request):
LOGGER.info('Request start ({})'.format(request))
def onRequestFinish(self, api, request, error):
LOGGER.info('Mega Request finished ({}); Result: {}'
.format(request, error))
if str(error).lower() != "no error":
self.error = error.copy()
return
request_type = request.getType()
if request_type == MegaRequest.TYPE_LOGIN:
api.fetchNodes()
elif request_type == MegaRequest.TYPE_GET_PUBLIC_NODE:
self.public_node = request.getPublicMegaNode()
elif request_type == MegaRequest.TYPE_FETCH_NODES:
LOGGER.info("Fetching Root Node.")
self.node = api.getRootNode()
LOGGER.info(f"Node Name: {self.node.getName()}")
if request_type not in self._NO_EVENT_ON or self.node and "cloud drive" not in self.node.getName().lower():
self.continue_event.set()
def onRequestTemporaryError(self, api, request, error: MegaError):
LOGGER.info(f'Mega Request error in {error}')
if not self.is_cancelled:
self.listener.onDownloadError("RequestTempError: " + error.toString())
self.is_cancelled = True
self.error = error.toString()
self.continue_event.set()
def onTransferStart(self, api: MegaApi, transfer: MegaTransfer):
LOGGER.info(f"Transfer Started: {transfer.getFileName()}")
def onTransferUpdate(self, api: MegaApi, transfer: MegaTransfer):
if self.is_cancelled:
api.cancelTransfer(transfer, None)
self.__speed = transfer.getSpeed()
self.__bytes_transferred = transfer.getTransferredBytes()
def onTransferFinish(self, api: MegaApi, transfer: MegaTransfer, error):
try:
LOGGER.info(f'Transfer finished ({transfer}); Result: {transfer.getFileName()}')
if transfer.isFolderTransfer() and transfer.isFinished() or transfer.getFileName() == self.name and not self.is_cancelled:
self.listener.onDownloadComplete()
self.continue_event.set()
except Exception as e:
LOGGER.error(e)
def onTransferTemporaryError(self, api, transfer, error):
filen = transfer.getFileName()
state = transfer.getState()
errStr = error.toString()
LOGGER.info(f'Mega download error in file {transfer} {filen}: {error}')
if state == 1 or state == 4:
# Sometimes MEGA (offical client) can't stream a node either and raises a temp failed error.
# Don't break the transfer queue if transfer's in queued (1) or retrying (4) state [causes seg fault]
return
self.error = errStr
if not self.is_cancelled:
self.is_cancelled = True
self.listener.onDownloadError(f"TransferTempError: {errStr} ({filen})")
def cancel_download(self):
self.is_cancelled = True
self.listener.onDownloadError("Download Canceled by user")
class AsyncExecutor:
def __init__(self):
self.continue_event = threading.Event()
def do(self, function, args):
self.continue_event.clear()
function(*args)
self.continue_event.wait()
listeners = []
class MegaDownloadHelper:
def __init__(self):
pass
@staticmethod
@new_thread
def add_download(mega_link: str, path: str, listener):
if MEGA_API_KEY is None:
raise MegaDownloaderException('Mega API KEY not provided! Cannot mirror mega links')
executor = AsyncExecutor()
api = MegaApi(MEGA_API_KEY, None, None, 'telegram-mirror-bot')
global listeners
mega_listener = MegaAppListener(executor.continue_event, listener)
listeners.append(mega_listener)
with download_dict_lock:
download_dict[listener.uid] = MegaDownloadStatus(mega_listener, listener)
os.makedirs(path)
api.addListener(mega_listener)
if MEGA_EMAIL_ID is not None and MEGA_PASSWORD is not None:
executor.do(api.login, (MEGA_EMAIL_ID, MEGA_PASSWORD))
link_type = get_mega_link_type(mega_link)
if link_type == "file":
executor.do(api.getPublicNode, (mega_link,))
node = mega_listener.public_node
else:
LOGGER.info("Logging into mega folder")
folder_api = MegaApi(MEGA_API_KEY,None,None,'TgBot')
folder_api.addListener(mega_listener)
executor.do(folder_api.loginToFolder, (mega_link,))
node = folder_api.authorizeNode(mega_listener.node)
if mega_listener.error is not None:
return listener.onDownloadError(str(mega_listener.error))
gid = ''.join(random.SystemRandom().choices(string.ascii_letters + string.digits, k=8))
mega_listener.setValues(node.getName(), api.getSize(node), gid)
executor.do(api.startDownload,(node,path))

66
bot/helper/mirror_utils/status_utils/mega_status.py → bot/helper/mirror_utils/status_utils/mega_download_status.py
View File

@ -1,49 +1,34 @@
from bot.helper.ext_utils.bot_utils import get_readable_file_size,MirrorStatus, get_readable_time
from bot import DOWNLOAD_DIR
from bot.helper.ext_utils.bot_utils import MirrorStatus, get_readable_file_size, get_readable_time
from .status import Status
class MegaDownloadStatus(Status):
def __init__(self, obj, listener):
self.uid = obj.uid
self.listener = listener
self.obj = obj
self.uid = listener.uid
self.message = listener.message
def gid(self):
return self.obj.gid
def path(self):
return f"{DOWNLOAD_DIR}{self.uid}"
def processed_bytes(self):
return self.obj.downloaded_bytes
def size_raw(self):
return self.obj.size
def size(self):
return get_readable_file_size(self.size_raw())
def status(self):
return MirrorStatus.STATUS_DOWNLOADING
def name(self):
def name(self) -> str:
return self.obj.name
def progress_raw(self):
return self.obj.progress
try:
return round(self.processed_bytes() / self.obj.size * 100,2)
except ZeroDivisionError:
return 0.0
def progress(self):
return f'{round(self.progress_raw(), 2)}%'
"""Progress of download in percentage"""
return f"{self.progress_raw()}%"
def speed_raw(self):
"""
:return: Download speed in Bytes/Seconds
"""
return self.obj.download_speed
def status(self) -> str:
return MirrorStatus.STATUS_DOWNLOADING
def speed(self):
return f'{get_readable_file_size(self.speed_raw())}/s'
def processed_bytes(self):
return self.obj.downloaded_bytes
def eta(self):
try:
@ -52,5 +37,26 @@ class MegaDownloadStatus(Status):
except ZeroDivisionError:
return '-'
def size_raw(self):
return self.obj.size
def size(self) -> str:
return get_readable_file_size(self.size_raw())
def downloaded(self) -> str:
return get_readable_file_size(self.obj.downloadedBytes)
def speed_raw(self):
return self.obj.speed
def speed(self) -> str:
return f'{get_readable_file_size(self.speed_raw())}/s'
def gid(self) -> str:
return self.obj.gid
def path(self) -> str:
return f"{DOWNLOAD_DIR}{self.uid}"
def download(self):
return self.obj

3
bot/helper/telegram_helper/message_utils.py
View File

@ -139,5 +139,8 @@ def sendStatusMessage(msg, bot):
except Exception as e:
LOGGER.error(str(e))
del status_reply_dict[msg.message.chat.id]
pass
if len(progress) == 0:
progress = "Starting DL"
message = sendMessage(progress, bot, msg)
status_reply_dict[msg.message.chat.id] = message

24
bot/modules/mirror.py
View File

@ -2,12 +2,13 @@ import requests
from telegram.ext import CommandHandler, run_async
from telegram import InlineKeyboardMarkup
from bot import Interval, INDEX_URL, LOGGER, MEGA_KEY, BUTTON_THREE_NAME, BUTTON_THREE_URL, BUTTON_FOUR_NAME, BUTTON_FOUR_URL, BUTTON_FIVE_NAME, BUTTON_FIVE_URL, BLOCK_MEGA_LINKS
from bot import Interval, INDEX_URL, BUTTON_THREE_NAME, BUTTON_THREE_URL, BUTTON_FOUR_NAME, BUTTON_FOUR_URL, BUTTON_FIVE_NAME, BUTTON_FIVE_URL, BLOCK_MEGA_FOLDER, BLOCK_MEGA_LINKS
from bot import dispatcher, DOWNLOAD_DIR, DOWNLOAD_STATUS_UPDATE_INTERVAL, download_dict, download_dict_lock, SHORTENER, SHORTENER_API
from bot.helper.ext_utils import fs_utils, bot_utils
from bot.helper.ext_utils.bot_utils import setInterval
from bot.helper.ext_utils.bot_utils import setInterval, get_mega_link_type
from bot.helper.ext_utils.exceptions import DirectDownloadLinkException, NotSupportedExtractionArchive
from bot.helper.mirror_utils.download_utils.aria2_download import AriaDownloadHelper
from bot.helper.mirror_utils.download_utils.mega_downloader import MegaDownloadHelper
from bot.helper.mirror_utils.download_utils.direct_link_generator import direct_link_generator
from bot.helper.mirror_utils.download_utils.telegram_downloader import TelegramDownloadHelper
from bot.helper.mirror_utils.status_utils import listeners
@ -19,7 +20,6 @@ from bot.helper.telegram_helper.bot_commands import BotCommands
from bot.helper.telegram_helper.filters import CustomFilters
from bot.helper.telegram_helper.message_utils import *
from bot.helper.telegram_helper import button_build
from bot.helper.mirror_utils.download_utils.mega_download import MegaDownloader
import urllib
import pathlib
import os
@ -269,14 +269,18 @@ def _mirror(bot, update, isTar=False, extract=False):
except DirectDownloadLinkException as e:
LOGGER.info(f'{link}: {e}')
listener = MirrorListener(bot, update, pswd, isTar, tag, extract)
if bot_utils.is_mega_link(link) and MEGA_KEY is not None and not BLOCK_MEGA_LINKS:
mega_dl = MegaDownloader(listener)
mega_dl.add_download(link, f'{DOWNLOAD_DIR}{listener.uid}/')
sendStatusMessage(update, bot)
elif bot_utils.is_mega_link(link) and BLOCK_MEGA_LINKS:
sendMessage("Mega links are blocked. Dont try to mirror mega links.", bot, update)
if bot_utils.is_mega_link(link):
link_type = get_mega_link_type(link)
if link_type == "folder" and BLOCK_MEGA_FOLDER:
sendMessage("Mega folder are blocked!", bot, update)
elif BLOCK_MEGA_LINKS:
sendMessage("Mega links are blocked bcoz mega downloading is too much unstable and buggy. mega support will be added back after fix", bot, update)
else:
mega_dl = MegaDownloadHelper()
mega_dl.add_download(link, f'{DOWNLOAD_DIR}/{listener.uid}/', listener)
sendStatusMessage(update, bot)
else:
ariaDlManager.add_download(link, f'{DOWNLOAD_DIR}{listener.uid}/', listener, name)
ariaDlManager.add_download(link, f'{DOWNLOAD_DIR}/{listener.uid}/', listener, name)
sendStatusMessage(update, bot)
if len(Interval) == 0:
Interval.append(setInterval(DOWNLOAD_STATUS_UPDATE_INTERVAL, update_all_messages))

11
bot/modules/usage.py
View File

@ -47,7 +47,8 @@ def dyno_usage(update, context):
minutes_remain = quota_remain / 60
hours = math.floor(minutes_remain / 60)
minutes = math.floor(minutes_remain % 60)
day = math.floor(hours / 24)
"""App Quota."""
Apps = result["apps"]
for apps in Apps:
@ -63,10 +64,12 @@ def dyno_usage(update, context):
AppMinutes = math.floor(AppQuotaUsed % 60)
sendMessage(
f"<b>Dyno Usage for</b> <code>{app.name}</code> :\n"
f"<b>Dyno Usage for</b> <code>{app.name}</code><b>:</b>\n"
f"• <code>{AppHours}</code> <b>Hours and</b> <code>{AppMinutes}</code> <b>Minutes - {AppPercent}%</b>\n\n"
"<b>Dyno Remaining this month :</b>\n"
f"• <code>{hours}</code> <b>Hours and</b> <code>{minutes}</code> <b>Minutes - {quota_percent}%</b>",
"<b>Dyno Remaining this month:</b>\n"
f"• <code>{hours}</code> <b>Hours and</b> <code>{minutes}</code> <b>Minutes - {quota_percent}%</b>\n\n"
"<b>Estimated Dyno Expired:</b>\n"
f"• <code>{day}</code> <b>Days</b>",
context.bot,
update
)

7
config_sample.env
View File

@ -16,9 +16,10 @@ AUTHORIZED_CHATS = ""
USE_SERVICE_ACCOUNTS = ""
INDEX_URL = ""
UPTOBOX_TOKEN = ""
MEGA_KEY = ""
MEGA_USERNAME = ""
MEGA_PASSWORD = ""
MEGA_API_KEY = ""
MEGA_EMAIL_ID = ""
MEGA_PASSWORD = ""
BLOCK_MEGA_FOLDER = ""
BLOCK_MEGA_LINKS = ""
STOP_DUPLICATE_MIRROR = ""
SHORTENER = ""

1
requirements.txt
View File

@ -24,4 +24,3 @@ js2py
lxml
telegraph
pytz
megasdkrestclient>=0.1.1,<1.0.0