Remove Torrent search

This commit is contained in:
Hafitz Setya 2021-08-29 13:44:36 +07:00 committed by GitHub
parent c49100594c
commit 4ed0c367f0
Signed by untrusted user: GitHub
GPG Key ID: 4AEE18F83AFDEB23
7 changed files with 16 additions and 365 deletions

View File

@ -27,11 +27,6 @@
- View Link button
- Shell and Executor
- Speedtest
- Torrent search Supported:
```
nyaa.si, sukebei, 1337x, piratebay,
tgx, yts, eztv, torlock, rarbg
```
- Direct links Supported:
```
letsupload.io, hxfile.co, anonfiles.com, bayfiles.com, antfiles,
@ -83,7 +78,9 @@ For Debian based distros
```
sudo apt install python3
```
Install Docker by following the [official Docker docs](https://docs.docker.com/engine/install/debian/), or:
Install Docker by following the [official Docker docs](https://docs.docker.com/engine/install/debian/)
OR
```
sudo snap install docker
```
@ -110,7 +107,7 @@ pip3 install -r requirements-cli.txt
- Copy your database url, and fill to `DATABASE_URL` in config
**2. Using Heroku PostgreSQL**
<p><a href="https://dev.to/prisma/how-to-setup-a-free-postgresql-database-on-heroku-1dc1"> <img src="https://img.shields.io/badge/See%20Dev.to-black?style=for-the-badge&logo=dev.to" width="170""/></a></p>
<p><a href="https://dev.to/prisma/how-to-setup-a-free-postgresql-database-on-heroku-1dc1"> <img src="https://img.shields.io/badge/See%20Dev.to-black?style=for-the-badge&logo=dev.to" width="160""/></a></p>
</details>
@ -149,7 +146,7 @@ Fill up rest of the fields. Meaning of each fields are discussed below:
- `MEGA_PASSWORD`: Your password for your mega.nz account
- `BLOCK_MEGA_FOLDER`: If you want to remove mega.nz folder support, set it to `True`.
- `BLOCK_MEGA_LINKS`: If you want to remove mega.nz mirror support, set it to `True`.
- `STOP_DUPLICATE`: (Leave empty if unsure) if this field is set to `True`, bot will check file in Drive, if it is present in Drive, downloading or cloning will be stopped. (**Note**: File will be checked using filename, not using filehash, so this feature is not perfect yet)
- `STOP_DUPLICATE`: (Leave empty if unsure) if this field is set to `True`, bot will check file in Drive, if it is present in Drive, downloading or cloning will be stopped. (**NOTE**: File will be checked using filename, not using filehash, so this feature is not perfect yet)
- `CLONE_LIMIT`: To limit cloning Google Drive (leave space between number and unit, Available units is (gb or GB, tb or TB), Examples: `100 gb, 100 GB, 10 tb, 10 TB`
- `MEGA_LIMIT`: To limit downloading Mega (leave space between number and unit, Available units is (gb or GB, tb or TB), Examples: `100 gb, 100 GB, 10 tb, 10 TB`
- `TORRENT_DIRECT_LIMIT`: To limit the Torrent/Direct mirror size, Leave space between number and unit. Available units is (gb or GB, tb or TB), Examples: `100 gb, 100 GB, 10 tb, 10 TB`
@ -163,7 +160,7 @@ Fill up rest of the fields. Meaning of each fields are discussed below:
- `STATUS_LIMIT`: Status limit with buttons (**NOTE**: Recommend limit status to `4` tasks max).
- `IS_VPS`: (Only for VPS) Don't set this to `True` even if you are using VPS, unless facing error with web server. Also go to start.sh and replace `$PORT` by `80` or any port you want to use.
- `SERVER_PORT`: (Only if IS_VPS is `True`) Base URL Port
- `BASE_URL_OF_BOT`: (Required for Heroku) Valid BASE URL of where the bot is deploy. Ip/domain of your bot like `http://myip` or if you have chosen other port then `80` then `http://myip:port`, for Heroku fill `https://yourappname.herokuapp.com` (**NOTE**: No slash at the end)
- `BASE_URL_OF_BOT`: (Required for Heroku to avoid sleep) Valid BASE URL of where the bot is deploy. Ip/domain of your bot like `http://myip` or if you have chosen other port then `80` then `http://myip:port`, for Heroku fill `https://yourappname.herokuapp.com` (**NOTE**: No slash at the end), still got idling? You can use http://cron-job.org to ping you Heroku app.
- `SHORTENER_API`: Fill your Shortener api key if you are using Shortener.
- `SHORTENER`: if you want to use Shortener in Gdrive and index link, fill Shortener url here. Examples:
```
@ -199,7 +196,7 @@ python3 generate_drive_token.py
## Deploying
**IMPORTANT NOTE**: In start.sh you must replace $PORT with 80 or any other port you want to use
**IMPORTANT NOTE**: In start.sh you must replace `$PORT` with 80 or any other port you want to use
- Start Docker daemon (skip if already running):
```
@ -215,7 +212,7 @@ sudo docker run -p 80:80 mirrorbot
```
OR
**NOTE**: If you want to use port other than 80, so change it in docker-compose.yml
**NOTE**: If you want to use port other than 80, so change it in [docker-compose.yml](https://github.com/SlamDevs/slam-mirrorbot/blob/master/docker-compose.yml)
- Using Docker-compose so you can edit and build your image in seconds:
```
@ -230,11 +227,11 @@ sudo docker-compose up
sudo docker-compose build
sudo docker-compose up
```
or
OR
```
sudo docker-compose up --build
```
- To stop docker run
- To stop Docker run
```
sudo docker ps
```
@ -249,7 +246,8 @@ sudo docker container prune
```
sudo docker image prune -a
```
## [Video From Tortoolkit Repo](https://youtu.be/c8_TU1sPK08)
- Video from Tortoolkit repo
<p><a href="https://youtu.be/c8_TU1sPK08"> <img src="https://img.shields.io/badge/See%20Video-black?style=for-the-badge&logo=YouTube" width="160""/></a></p>
## Deploying on Heroku
- Deploying on Heroku with Github Workflow

View File

@ -17,7 +17,7 @@ from bot.helper.telegram_helper.message_utils import *
from .helper.ext_utils.bot_utils import get_readable_file_size, get_readable_time
from .helper.telegram_helper.filters import CustomFilters
from bot.helper.telegram_helper import button_build
from .modules import authorize, list, cancel_mirror, mirror_status, mirror, clone, watch, shell, eval, torrent_search, delete, speedtest, count, reboot
from .modules import authorize, list, cancel_mirror, mirror_status, mirror, clone, watch, shell, eval, delete, speedtest, count, reboot
def stats(update, context):
@ -136,8 +136,6 @@ def bot_help(update, context):
/{BotCommands.ShellCommand}: Run commands in Shell (Terminal)
/{BotCommands.ExecHelpCommand}: Get help for Executor module (Only Owner)
/{BotCommands.TsHelpCommand}: Get help for Torrent search module
'''
help_string = f'''
@ -168,8 +166,6 @@ def bot_help(update, context):
/{BotCommands.StatsCommand}: Show Stats of the machine the bot is hosted on
/{BotCommands.PingCommand}: Check how long it takes to Ping the Bot
/{BotCommands.TsHelpCommand}: Get help for Torrent search module
'''
if CustomFilters.sudo_user(update) or CustomFilters.owner_filter(update):
@ -196,8 +192,7 @@ botcmds = [
(f'{BotCommands.StatsCommand}','Bot Usage Stats'),
(f'{BotCommands.PingCommand}','Ping the Bot'),
(f'{BotCommands.RestartCommand}','Restart the bot [owner/sudo only]'),
(f'{BotCommands.LogCommand}','Get the Bot Log [owner/sudo only]'),
(f'{BotCommands.TsHelpCommand}','Get help for Torrent search module')
(f'{BotCommands.LogCommand}','Get the Bot Log [owner/sudo only]')
]

View File

@ -27,6 +27,5 @@ class _BotCommands:
self.DeleteCommand = 'del'
self.ShellCommand = 'shell'
self.ExecHelpCommand = 'exechelp'
self.TsHelpCommand = 'tshelp'
BotCommands = _BotCommands()

View File

@ -1,339 +0,0 @@
import os
import time
import html
import asyncio
import aiohttp
import json
import feedparser
import requests
import itertools
from telegram.ext import CommandHandler
from telegram import ParseMode
from urllib.parse import quote as urlencode, urlsplit
from pyrogram import Client, filters, emoji
from pyrogram.parser import html as pyrogram_html
from pyrogram.types import InlineKeyboardMarkup, InlineKeyboardButton
from pyrogram.handlers import MessageHandler, CallbackQueryHandler
from bot import app, dispatcher, bot
from bot.helper import custom_filters
from bot.helper.telegram_helper.bot_commands import BotCommands
from bot.helper.telegram_helper.filters import CustomFilters
from bot.helper.telegram_helper.message_utils import sendMessage
search_lock = asyncio.Lock()
search_info = {False: dict(), True: dict()}
async def return_search(query, page=1, sukebei=False):
page -= 1
query = query.lower().strip()
used_search_info = search_info[sukebei]
async with search_lock:
results, get_time = used_search_info.get(query, (None, 0))
if (time.time() - get_time) > 3600:
results = []
async with aiohttp.ClientSession() as session:
async with session.get(f'https://{"sukebei." if sukebei else ""}nyaa.si/?page=rss&q={urlencode(query)}') as resp:
d = feedparser.parse(await resp.text())
text = ''
a = 0
parser = pyrogram_html.HTML(None)
for i in sorted(d['entries'], key=lambda i: int(i['nyaa_seeders']), reverse=True):
if i['nyaa_size'].startswith('0'):
continue
if not int(i['nyaa_seeders']):
break
link = i['link']
splitted = urlsplit(link)
if splitted.scheme == 'magnet' and splitted.query:
link = f'<code>{link}</code>'
newtext = f'''<b>{a + 1}.</b> <code>{html.escape(i["title"])}</code>
<b>Link:</b> <code>{link}</code>
<b>Size:</b> <code>{i["nyaa_size"]}</code>
<b>Seeders:</b> <code>{i["nyaa_seeders"]}</code>
<b>Leechers:</b> <code>{i["nyaa_leechers"]}</code>
<b>Category:</b> <code>{i["nyaa_category"]}</code>\n\n'''
futtext = text + newtext
if (a and not a % 10) or len((await parser.parse(futtext))['message']) > 4096:
results.append(text)
futtext = newtext
text = futtext
a += 1
results.append(text)
ttl = time.time()
used_search_info[query] = results, ttl
try:
return results[page], len(results), ttl
except IndexError:
return '', len(results), ttl
message_info = dict()
ignore = set()
@app.on_message(filters.command(['nyaasi', f'nyaasi@{bot.username}']))
async def nyaa_search(client, message):
text = message.text.split(' ')
text.pop(0)
query = ' '.join(text)
await init_search(client, message, query, False)
@app.on_message(filters.command(['sukebei', f'sukebei@{bot.username}']))
async def nyaa_search_sukebei(client, message):
text = message.text.split(' ')
text.pop(0)
query = ' '.join(text)
await init_search(client, message, query, True)
async def init_search(client, message, query, sukebei):
result, pages, ttl = await return_search(query, sukebei=sukebei)
if not result:
await message.reply_text('No results found')
else:
buttons = [InlineKeyboardButton(f'1/{pages}', 'nyaa_nop'), InlineKeyboardButton(f'Next', 'nyaa_next')]
if pages == 1:
buttons.pop()
reply = await message.reply_text(result, reply_markup=InlineKeyboardMarkup([
buttons
]))
message_info[(reply.chat.id, reply.message_id)] = message.from_user.id, ttl, query, 1, pages, sukebei
@app.on_callback_query(custom_filters.callback_data('nyaa_nop'))
async def nyaa_nop(client, callback_query):
await callback_query.answer(cache_time=3600)
callback_lock = asyncio.Lock()
@app.on_callback_query(custom_filters.callback_data(['nyaa_back', 'nyaa_next']))
async def nyaa_callback(client, callback_query):
message = callback_query.message
message_identifier = (message.chat.id, message.message_id)
data = callback_query.data
async with callback_lock:
if message_identifier in ignore:
await callback_query.answer()
return
user_id, ttl, query, current_page, pages, sukebei = message_info.get(message_identifier, (None, 0, None, 0, 0, None))
og_current_page = current_page
if data == 'nyaa_back':
current_page -= 1
elif data == 'nyaa_next':
current_page += 1
if current_page < 1:
current_page = 1
elif current_page > pages:
current_page = pages
ttl_ended = (time.time() - ttl) > 3600
if ttl_ended:
text = getattr(message.text, 'html', 'Search expired')
else:
if callback_query.from_user.id != user_id:
await callback_query.answer('...no', cache_time=3600)
return
text, pages, ttl = await return_search(query, current_page, sukebei)
buttons = [InlineKeyboardButton(f'Prev', 'nyaa_back'), InlineKeyboardButton(f'{current_page}/{pages}', 'nyaa_nop'), InlineKeyboardButton(f'Next', 'nyaa_next')]
if ttl_ended:
buttons = [InlineKeyboardButton('Search Expired', 'nyaa_nop')]
else:
if current_page == 1:
buttons.pop(0)
if current_page == pages:
buttons.pop()
if ttl_ended or current_page != og_current_page:
await callback_query.edit_message_text(text, reply_markup=InlineKeyboardMarkup([
buttons
]))
message_info[message_identifier] = user_id, ttl, query, current_page, pages, sukebei
if ttl_ended:
ignore.add(message_identifier)
await callback_query.answer()
# Using upstream API based on: https://github.com/Ryuk-me/Torrents-Api
# Implemented by https://github.com/jusidama18
class TorrentSearch:
index = 0
query = None
message = None
response = None
response_range = None
RESULT_LIMIT = 4
RESULT_STR = None
def __init__(self, command: str, source: str, result_str: str):
self.command = command
self.source = source.rstrip('/')
self.RESULT_STR = result_str
app.add_handler(MessageHandler(self.find, filters.command([command, f'{self.command}@{bot.username}'])))
app.add_handler(CallbackQueryHandler(self.previous, filters.regex(f"{self.command}_previous")))
app.add_handler(CallbackQueryHandler(self.delete, filters.regex(f"{self.command}_delete")))
app.add_handler(CallbackQueryHandler(self.next, filters.regex(f"{self.command}_next")))
@staticmethod
def format_magnet(string: str):
if not string:
return ""
return string.split('&tr', 1)[0]
def get_formatted_string(self, values):
string = self.RESULT_STR.format(**values)
extra = ""
if "Files" in values:
tmp_str = "➲[{Quality} - {Type} ({Size})]({Torrent}): `{magnet}`"
extra += "\n".join(
tmp_str.format(**f, magnet=self.format_magnet(f['Magnet']))
for f in values['Files']
)
else:
magnet = values.get('magnet', values.get('Magnet')) # Avoid updating source dict
if magnet:
extra += f"➲Magnet: `{self.format_magnet(magnet)}`"
if (extra):
string += "\n" + extra
return string
async def update_message(self):
prevBtn = InlineKeyboardButton(f"Prev", callback_data=f"{self.command}_previous")
delBtn = InlineKeyboardButton(f"{emoji.CROSS_MARK}", callback_data=f"{self.command}_delete")
nextBtn = InlineKeyboardButton(f"Next", callback_data=f"{self.command}_next")
inline = []
if (self.index != 0):
inline.append(prevBtn)
inline.append(delBtn)
if (self.index != len(self.response_range) - 1):
inline.append(nextBtn)
res_lim = min(self.RESULT_LIMIT, len(self.response) - self.RESULT_LIMIT*self.index)
result = f"**Page - {self.index+1}**\n\n"
result += "\n\n=======================\n\n".join(
self.get_formatted_string(self.response[self.response_range[self.index]+i])
for i in range(res_lim)
)
await self.message.edit(
result,
reply_markup=InlineKeyboardMarkup([inline]),
parse_mode="markdown",
)
async def find(self, client, message):
if len(message.command) < 2:
await message.reply_text(f"Usage: /{self.command} query")
return
query = urlencode(message.text.split(None, 1)[1])
self.message = await message.reply_text("Searching")
try:
async with aiohttp.ClientSession() as session:
async with session.get(f"{self.source}/{query}") as resp:
if (resp.status != 200):
raise Exception('unsuccessful request')
result = await resp.json()
if (result and isinstance(result[0], list)):
result = list(itertools.chain(*result))
self.response = result
self.response_range = range(0, len(self.response), self.RESULT_LIMIT)
except:
await self.message.edit("No Results Found.")
return
await self.update_message()
async def delete(self, client, message):
index = 0
query = None
message = None
response = None
response_range = None
await self.message.delete()
async def previous(self, client, message):
self.index -= 1
await self.update_message()
async def next(self, client, message):
self.index += 1
await self.update_message()
RESULT_STR_1337 = (
"➲Name: `{Name}`\n"
"➲Size: {Size}\n"
"➲Seeders: {Seeders} || ➲Leechers: {Leechers}"
)
RESULT_STR_PIRATEBAY = (
"➲Name: `{Name}`\n"
"➲Size: {Size}\n"
"➲Seeders: {Seeders} || ➲Leechers: {Leechers}"
)
RESULT_STR_TGX = (
"➲Name: `{Name}`\n"
"➲Size: {Size}\n"
"➲Seeders: {Seeders} || ➲Leechers: {Leechers}"
)
RESULT_STR_YTS = (
"➲Name: `{Name}`\n"
"➲Released on: {ReleasedDate}\n"
"➲Genre: {Genre}\n"
"➲Rating: {Rating}\n"
"➲Likes: {Likes}\n"
"➲Duration: {Runtime}\n"
"➲Language: {Language}"
)
RESULT_STR_EZTV = (
"➲Name: `{Name}`\n"
"➲Size: {Size}\n"
"➲Seeders: {Seeders}"
)
RESULT_STR_TORLOCK = (
"➲Name: `{Name}`\n"
"➲Size: {Size}\n"
"➲Seeders: {Seeders} || ➲Leechers: {Leechers}"
)
RESULT_STR_RARBG = (
"➲Name: `{Name}`\n"
"➲Size: {Size}\n"
"➲Seeders: {Seeders} || ➲Leechers: {Leechers}"
)
RESULT_STR_ALL = (
"➲Name: `{Name}`\n"
"➲Size: {Size}\n"
"➲Seeders: {Seeders} || ➲Leechers: {Leechers}"
)
torrents_dict = {
'1337x': {'source': "https://torrents--api.herokuapp.com/api/1337x/", 'result_str': RESULT_STR_1337},
'piratebay': {'source': "https://torrents--api.herokuapp.com/api/piratebay/", 'result_str': RESULT_STR_PIRATEBAY},
'tgx': {'source': "https://torrents--api.herokuapp.com/api/tgx/", 'result_str': RESULT_STR_TGX},
'yts': {'source': "https://torrents--api.herokuapp.com/api/yts/", 'result_str': RESULT_STR_YTS},
'eztv': {'source': "https://torrents--api.herokuapp.com/api/eztv/", 'result_str': RESULT_STR_EZTV},
'torlock': {'source': "https://torrents--api.herokuapp.com/api/torlock/", 'result_str': RESULT_STR_TORLOCK},
'rarbg': {'source': "https://torrents--api.herokuapp.com/api/rarbg/", 'result_str': RESULT_STR_RARBG},
'ts': {'source': "https://torrents--api.herokuapp.com/api/all/", 'result_str': RESULT_STR_ALL}
}
torrent_handlers = []
for command, value in torrents_dict.items():
torrent_handlers.append(TorrentSearch(command, value['source'], value['result_str']))
def searchhelp(update, context):
help_string = '''
<b>Torrent Search</b>
/nyaasi <i>[search query]</i>
/sukebei <i>[search query]</i>
/1337x <i>[search query]</i>
/piratebay <i>[search query]</i>
/tgx <i>[search query]</i>
/yts <i>[search query]</i>
/eztv <i>[search query]</i>
/torlock <i>[search query]</i>
/rarbg <i>[search query]</i>
/ts <i>[search query]</i>
'''
sendMessage(help_string, context.bot, update)
SEARCHHELP_HANDLER = CommandHandler(BotCommands.TsHelpCommand, searchhelp, filters=(CustomFilters.authorized_chat | CustomFilters.authorized_user) & CustomFilters.mirror_owner_filter, run_async=True)
dispatcher.add_handler(SEARCHHELP_HANDLER)

View File

@ -31,7 +31,7 @@ SHORTENER_API = ""
# qBittorrent
IS_VPS = "" # Don't set this to True even if VPS, unless facing error with web server
SERVER_PORT = "80" # Only if IS_VPS is True
BASE_URL_OF_BOT = "" # Web Link, Required for Heroku to avoid sleep or use worker if u don't want to use web(selection)
BASE_URL_OF_BOT = "" # Web Link, Required for Heroku to avoid sleep or use worker if you don't want to use web (selection)
# If you want to use Credentials externally from Index Links, fill these vars with the direct links
# These are optional, if you don't know, simply leave them, don't fill anything in them.
ACCOUNTS_ZIP_URL = ""

View File

@ -7,5 +7,3 @@ services:
restart: on-failure
ports:
- "80:80"

View File

@ -18,6 +18,7 @@ psutil
psycopg2-binary
pybase64
pyrogram
pyshorteners
python-dotenv
python-magic
python-telegram-bot
@ -30,4 +31,3 @@ TgCrypto
torrentool
urllib3
youtube_dl
pyshorteners