DataScene: fix free detection

I'm 99% sure that since they've resurrected they're fully public (no login needed, no ratio, etc.) and since even Jackett isn't asking for login info, it should be reflected here.

README.md

@@ -1,10 +1,12 @@
-# Jackett
+# Jackett
 
 [![GitHub issues](https://img.shields.io/github/issues/Jackett/Jackett.svg?maxAge=60&style=flat-square)](https://github.com/Jackett/Jackett/issues)
 [![GitHub pull requests](https://img.shields.io/github/issues-pr/Jackett/Jackett.svg?maxAge=60&style=flat-square)](https://github.com/Jackett/Jackett/pulls)
+[![Bountysource](https://img.shields.io/bountysource/team/jackett/activity.svg?style=flat-square)](https://www.bountysource.com/teams/jackett)
 [![Build status](https://ci.appveyor.com/api/projects/status/gaybh5mvyx418nsp/branch/master?svg=true)](https://ci.appveyor.com/project/camjac251/jackett)
 [![Github Releases](https://img.shields.io/github/downloads/Jackett/Jackett/total.svg?maxAge=60&style=flat-square)](https://github.com/Jackett/Jackett/releases/latest)
 [![Docker Pulls](https://img.shields.io/docker/pulls/linuxserver/jackett.svg?maxAge=60&style=flat-square)](https://hub.docker.com/r/linuxserver/jackett/)
+[![Discord](https://img.shields.io/badge/discord-chat-7289DA.svg?maxAge=60&style=flat-square)](https://discord.gg/J865QuA)
 
 This project is a new fork and is recruiting development help.  If you are able to help out please contact us.
 
@@ -16,18 +18,19 @@ Developer note: The software implements the [Torznab](https://github.com/Sonarr/
 
 #### Supported Systems
 * Windows using .NET 4.5
-* Linux and OSX using Mono 4 (mono 3 is no longer supported).
+* Linux and macOS using Mono 4 (mono 3 is no longer supported).
 
 ### Supported Public Trackers
  * Anidex
  * Anime Tosho
  * cpasbien
  * EZTV
+ * Gay-Torrents.net
  * Horrible Subs
  * Il Corsaro Nero <!-- maintained by bonny1992 -->
  * Isohunt
  * KickAssTorrent
- * KickAssTorrent (kat.how clone)
+ * KickAssTorrent (thekat.se clone)
  * LimeTorrents
  * NextTorrent
  * Nyaa.si
@@ -39,6 +42,7 @@ Developer note: The software implements the [Torznab](https://github.com/Sonarr/
  * The Pirate Bay
  * TNTVillage <!-- maintained by bonny1992 -->
  * Tokyo Toshokan
+ * Torlock
  * Torrent Downloads
  * TorrentProject
  * Torrentz2
@@ -65,7 +69,6 @@ Developer note: The software implements the [Torznab](https://github.com/Sonarr/
  * B2S-Share
  * BakaBT  [![(invite needed)][inviteneeded]](#)
  * bB
- * Best Friends
  * BeyondHD
  * BIGTorrent
  * Bit-City Reloaded
@@ -78,7 +81,7 @@ Developer note: The software implements the [Torznab](https://github.com/Sonarr/
  * Blu-bits
  * BlueBird
  * Blutopia
- * BroadcastTheNet  [![(invite needed)][inviteneeded]](#)
+ * BroadcastTheNet
  * BrokenStones
  * BTNext
  * Carpathians
@@ -88,13 +91,13 @@ Developer note: The software implements the [Torznab](https://github.com/Sonarr/
  * CinemaZ
  * Classix
  * CZTeam
+ * CzTorrent
  * DanishBits
  * DataScene
  * Deildu
  * Demonoid
  * Diablo Torrent
  * DigitalHive
- * Dragon World (DTW)
  * Dragonworld Reloaded
  * Dream Team
  * EoT-Forum
@@ -107,6 +110,7 @@ Developer note: The software implements the [Torznab](https://github.com/Sonarr/
  * FunFile
  * FunkyTorrents
  * Fuzer
+ * GayTorrent.ru
  * GFXPeers
  * Ghost City
  * GigaTorrents  [![(invite needed)][inviteneeded]](#)
@@ -141,7 +145,6 @@ Developer note: The software implements the [Torznab](https://github.com/Sonarr/
  * JPopsuki
  * Kapaki
  * Karagarga
- * Le Paradis Du Net
  * LinkoManija
  * LosslessClub
  * M-Team - TP
@@ -171,7 +174,6 @@ Developer note: The software implements the [Torznab](https://github.com/Sonarr/
  * PrivateHD
  * Psytorrents
  * PTFiles
- * QcTorrent
  * Redacted (PassTheHeadphones)
  * RevolutionTT
  * Rockhard Lossless
@@ -187,6 +189,7 @@ Developer note: The software implements the [Torznab](https://github.com/Sonarr/
  * Shazbat
  * Shellife
  * SpeedCD
+ * SpeedTorrent Reloaded
  * SportsCult
  * SportHD
  * Superbits
@@ -227,7 +230,7 @@ Developer note: The software implements the [Torznab](https://github.com/Sonarr/
  * Ultimate Gamer Club
  * ULTRAHDCLUB
  * Waffles
- * World-In-HD  [![(invite needed)][inviteneeded]](#)
+ * World-In-HD
  * WorldOfP2P
  * x264
  * XSpeeds
@@ -269,14 +272,21 @@ Jackett can also be run from the command line if you would like to see log messa
 
 Detailed instructions for [Ubuntu 14.x](http://www.htpcguides.com/install-jackett-on-ubuntu-14-x-for-custom-torrents-in-sonarr/) and [Ubuntu 15.x](http://www.htpcguides.com/install-jackett-ubuntu-15-x-for-custom-torrents-in-sonarr/)
 
-## Installation on OSX
- 1. Install [Mono 4](http://www.mono-project.com/download/#download-mac) or better (version 4.8 is recommended)
-       * Setup ssl support by running
-       ```
-       https://curl.haxx.se/ca/cacert.pem
-       cert-sync --user ~/Downloads/cacert.pem
-       ```
- 1. Download and extract the latest `Jackett.Binaries.Mono.tar.gz` release from the [releases page](https://github.com/Jackett/Jackett/releases) and run Jackett using mono with the command `mono --debug JackettConsole.exe`.
+## Installation on macOS
+
+### Prerequisites
+Install [Mono 4](http://www.mono-project.com/download/#download-mac) or better (version 4.8 is recommended).
+ * Setup ssl support by running `curl -sS https://curl.haxx.se/ca/cacert.pem | cert-sync --user /dev/stdin`
+
+### Install as service
+1. Download and extract the latest `Jackett.Binaries.Mono.tar.gz` release from the [releases page](https://github.com/Jackett/Jackett/releases).
+2. In Terminal, run the install script from the extracted directory using `./install_service_macos.sh`
+       
+The service will start on each logon. You can always stop it by running `launchctl unload ~/Library/LaunchAgents/org.user.Jackett.plist` from Terminal. You can start it again it using `launchctl load ~/Library/LaunchAgents/org.user.Jackett.plist`.
+Logs are stored as usual under `~/.config/Jackett/log.txt`.
+
+### Run without installing as a service
+Download and extract the latest `Jackett.Binaries.Mono.tar.gz` release from the [releases page](https://github.com/Jackett/Jackett/releases) and run Jackett using mono with the command `mono --debug JackettConsole.exe`.
 
 ## Installation using Docker
 Detailed instructions are available at [LinuxServer.io Jackett Docker](https://hub.docker.com/r/linuxserver/jackett/). The Jackett Docker is highly recommended, especially if you are having Mono stability issues or having issues running Mono on your system eg. QNAP, Synology. Thanks to [LinuxServer.io](https://linuxserver.io)
@@ -294,16 +304,16 @@ Jackett is available as beta package from [SynoCommunity](https://synocommunity.
 
   If you're using mono this is often caused by missing ca-certificates.
   Try reimporting the certificates in this case:
-
-  `wget -O - https://curl.haxx.se/ca/cacert.pem | cert-sync /dev/stdin`
+   - On Linux: `wget -O - https://curl.haxx.se/ca/cacert.pem | cert-sync /dev/stdin`
+   - On macOS: `curl -sS https://curl.haxx.se/ca/cacert.pem | cert-sync --user /dev/stdin`
 
   As a option of last resort you can disable certificate validation using the `--IgnoreSslErrors true` option but it's not recommended to use it as it enables Man-in-the-middle attacks on your connections.
 
 *  __Enable logging__
 
-  You can get additional logging with the command line switches `-t -l` or by enabeling `Enhanced logging` via the web interface.
+  You can get additional logging with the command line switches `-t -l` or by enabling `Enhanced logging` via the web interface.
   Please post logs if you are unable to resolve your issue with these switches ensuring to remove your username/password/cookies.
-  The logfiles (log.txt/updater.txt) are stored in `%ProgramData%\Jackett` on Windows and `~/.config/Jackett/` on Linux/OSX.
+  The logfiles (log.txt/updater.txt) are stored in `%ProgramData%\Jackett` on Windows and `~/.config/Jackett/` on Linux/macOS.
 
 ## Creating an issue
 Please supply as much information about the problem you are experiencing as possible. Your issue has a much greater chance of being resolved if logs are supplied so that we can see what is going on. Creating an issue with '### isn't working' doesn't help anyone to fix the problem.

src/Jackett.Console/ConsoleOptions.cs

@@ -60,7 +60,7 @@ namespace Jackett.Console
         [Option('n', "IgnoreSslErrors", HelpText = "[true/false] Ignores invalid SSL certificates")]
         public bool? IgnoreSslErrors { get; set; }
 
-        [Option('d', "DataFolder", HelpText = "Specify the location of the data folder (Must be admin on Windows) eg. --DataFolder=\"D:\\Your Data\\Jackett\\\"")]
+        [Option('d', "DataFolder", HelpText = "Specify the location of the data folder (Must be admin on Windows) eg. --DataFolder=\"D:\\Your Data\\Jackett\\\". Don't use this on Unix (mono) systems. On Unix just adjust the HOME directory of the user to the datedir or set the XDG_CONFIG_HOME environment variable.")]
         public string DataFolder { get; set; }
 
         [Option(HelpText = "Don't restart after update")]

src/Jackett.Console/Jackett.Console.csproj

@@ -153,6 +153,9 @@
   <ItemGroup>
     <None Include="App.config" />
     <None Include="packages.config" />
+    <Content Include="install_service_macos.sh">
+      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
+    </Content>
   </ItemGroup>
   <ItemGroup>
     <ProjectReference Include="..\CurlSharp\CurlSharp.csproj">
@@ -182,4 +185,4 @@
   <Target Name="AfterBuild">
   </Target>
   -->
-</Project>
+</Project>

src/Jackett.Console/install_service_macos.sh

@@ -0,0 +1,67 @@
+#!/bin/bash
+
+# Stop and unload the service if it's running
+launchctl remove org.user.Jackett
+
+# Check if we're running from Jackett's directory
+if [ ! -f ./JackettConsole.exe ]; then
+    echo "Couldn't locate JackettConsole.exe. Are you running from the right directory?"
+    exit 1
+fi
+jackettdir="$(pwd)"
+
+# Check if mono is installed
+command -v mono >/dev/null 2>&1 || { echo >&2 "Jackett requires Mono but it's not installed. Aborting."; exit 1; }
+monodir="$(dirname $(command -v mono))"
+
+# Check that no other service called Jackett is already running
+if [[ $(launchctl list | grep org.user.Jackett) ]]; then
+    echo "Jackett already seems to be running as a service. Please stop it before running this script again."
+    exit 1
+fi
+
+# Write the plist to LaunchAgents
+cat >~/Library/LaunchAgents/org.user.Jackett.plist <<EOL
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+    <key>EnvironmentVariables</key>
+    <dict>
+        <key>PATH</key>
+        <string>/usr/bin:/bin:/usr/sbin:/sbin:${monodir}</string>
+    </dict>
+    <key>KeepAlive</key>
+    <true/>
+    <key>Label</key>
+    <string>org.user.Jackett</string>
+    <key>ProgramArguments</key>
+    <array>
+        <string>${monodir}/mono</string>
+        <string>--debug</string>
+        <string>JackettConsole.exe</string>
+        <string>--NoRestart</string>
+    </array>
+    <key>RunAtLoad</key>
+    <true/>
+    <key>WorkingDirectory</key>
+    <string>${jackettdir}</string>
+</dict>
+</plist>
+
+EOL
+
+# Run the agent
+launchctl load ~/Library/LaunchAgents/org.user.Jackett.plist
+
+# Check that it's running
+if [[ $(launchctl list | grep org.user.Jackett) ]]; then
+    echo "Agent successfully installed and launched!"
+else
+    cat << EOL
+Could not launch agent. The installation might have failed.
+Please open an issue on https://github.com/Jackett/Jackett/issues and paste following information:
+Mono directory: \`${monodir}\`
+Jackett directory: \`${jackettdir}\`
+EOL
+fi

src/Jackett.Updater/Program.cs

@@ -188,6 +188,9 @@ namespace Jackett.Updater
                 "Definitions/maniatorrent.yml",
                 "Definitions/nyaa.yml",
                 "Definitions/nachtwerk.yml",
+                "Definitions/leparadisdunet.yml",
+                "Definitions/qctorrent.yml",
+                "Definitions/dragonworld.yml",
             };
 
             foreach (var oldFIle in oldFiles)

src/Jackett/Content/custom.js

@@ -42,12 +42,9 @@ $(document).ready(function () {
 });
 
 function openSearchIfNecessary() {
-    var parser = document.createElement('a');
-    parser.href = window.location.href;
-
-    if (parser.hash.startsWith("#search")) {
-        var query = parser.hash.split('=')[1];
-        showSearch(null, query);
+    const hashArgs = location.hash.substring(1).split('&').reduce((prev, item) => Object.assign({ [item.split('=')[0]]: (item.split('=').length < 2 ? undefined : decodeURIComponent(item.split('=')[1])) }, prev), {});
+    if ("search" in hashArgs) {
+        showSearch(hashArgs.tracker, hashArgs.search, hashArgs.category);
     }
 }
 
@@ -58,7 +55,7 @@ function insertWordWrap(str) {
 
 function getJackettConfig(callback) {
     api.getServerConfig(callback).fail(function () {
-        doNotify("Error loading Jackett settings, request to Jackett server failed", "danger", "glyphicon glyphicon-alert");
+        doNotify("Error loading Jackett settings, request to Jackett server failed, is server running ?", "danger", "glyphicon glyphicon-alert");
     });
 }
 
@@ -142,7 +139,7 @@ function reloadIndexers() {
         $('#indexers div.dataTables_filter input').focusWithoutScrolling();
         openSearchIfNecessary();
     }).fail(function () {
-        doNotify("Error loading indexers, request to Jackett server failed", "danger", "glyphicon glyphicon-alert");
+        doNotify("Error loading indexers, request to Jackett server failed, is server running ?", "danger", "glyphicon glyphicon-alert");
     });
 }
 
@@ -213,8 +210,13 @@ function displayUnconfiguredIndexersList() {
 		                    }
 		                    doNotify("Configuration failed: " + data.error, "danger", "glyphicon glyphicon-alert");
 		                }
-			        }).fail(function () {
-			            doNotify("Request to Jackett server failed", "danger", "glyphicon glyphicon-alert");
+			        }).fail(function (data) {
+			            if(data.responseJSON.error !== undefined) {
+                doNotify("An error occured while configuring this indexer<br /><b>" + data.responseJSON.error + "</b><br /><i><a href=\"https://github.com/Jackett/Jackett/issues/new?title=[" + indexerId + "] " + data.responseJSON.error + " (Config)\" target=\"_blank\">Click here to open an issue on Github for this indexer.</a><i>", "danger", "glyphicon glyphicon-alert", false);
+            } else {
+                doNotify("An error occured while configuring this indexer, is Jackett server running ?", "danger", "glyphicon glyphicon-alert");
+            }
+                        
 			        });
                 });
             });
@@ -366,6 +368,7 @@ function prepareSearchButtons(element) {
         var $btn = $(btn);
         var id = $btn.data("id");
         $btn.click(function() {
+            window.location.hash = "search&tracker=" +  id;
             showSearch(id);
         });
     });
@@ -429,8 +432,13 @@ function testIndexer(id, notifyResult) {
             if (notifyResult)
                 doNotify("Test failed for " + id + ": \n" + data.error, "danger", "glyphicon glyphicon-alert");
         }
-    }).fail(function () {
-        doNotify("Error testing indexer, request to Jackett server error", "danger", "glyphicon glyphicon-alert");
+    }).fail(function (data) {
+        updateTestState(id, "error", data.error, indexers);
+        if(data.responseJSON.error !== undefined && notifyResult) {
+                doNotify("An error occured while testing this indexer<br /><b>" + data.responseJSON.error + "</b><br /><i><a href=\"https://github.com/Jackett/Jackett/issues/new?title=[" + id + "] " + data.responseJSON.error + " (Test)\" target=\"_blank\">Click here to open an issue on Github for this indexer.</a><i>", "danger", "glyphicon glyphicon-alert", false);
+            } else {
+                doNotify("An error occured while testing indexers, please take a look at indexers with failed test for more informations.", "danger", "glyphicon glyphicon-alert");
+            }
     });
 }
 
@@ -611,8 +619,12 @@ function populateSetupForm(indexerId, name, config, caps, link, alternativesitel
                 }
                 doNotify("Configuration failed: " + data.error, "danger", "glyphicon glyphicon-alert");
             }
-        }).fail(function () {
-            doNotify("Request to Jackett server failed", "danger", "glyphicon glyphicon-alert");
+        }).fail(function (data) {
+            if(data.responseJSON.error !== undefined) {
+                doNotify("An error occured while updating this indexer<br /><b>" + data.responseJSON.error + "</b><br /><i><a href=\"https://github.com/Jackett/Jackett/issues/new?title=[" + indexerId + "] " + data.responseJSON.error + " (Config)\" target=\"_blank\">Click here to open an issue on Github for this indexer.</a><i>", "danger", "glyphicon glyphicon-alert", false);
+            } else {
+                doNotify("An error occured while updating this indexer, request to Jackett server failed, is server running ?", "danger", "glyphicon glyphicon-alert");
+            }
         }).always(function () {
             $goButton.html(originalBtnText);
             $goButton.prop('disabled', false);
@@ -711,7 +723,7 @@ function updateReleasesRow(row)
     }
 }
 
-function showSearch(selectedIndexer, query) {
+function showSearch(selectedIndexer, query, category) {
     $('#select-indexer-modal').remove();
     var releaseTemplate = Handlebars.compile($("#jackett-search").html());
     var releaseDialog = $(releaseTemplate({
@@ -774,7 +786,7 @@ function showSearch(selectedIndexer, query) {
             Tracker: releaseDialog.find('#searchTracker').val().replace("'", "").replace("'", ""),
         };
 
-        window.location.hash = "search=" + searchString;
+        window.location.hash = $.param({ search: queryObj.Query, tracker: queryObj.Tracker, category: queryObj.Category});
 
         $('#jackett-search-perform').html($('#spinner').html());
         $('#searchResults div.dataTables_filter input').val("");
@@ -810,8 +822,12 @@ function showSearch(selectedIndexer, query) {
     clearSearchResultTable($('#searchResults'));
     releaseDialog.modal("show");
 
+    if (category !== undefined) {
+        $('#searchCategory').val(category);
+    }
+
     if (query !== undefined) {
-        queryField.value = decodeURIComponent(query);
+        queryField.value = query;
         searchButton.click();
     }
 }

src/Jackett/Controllers/ResultsController.cs

@@ -19,6 +19,7 @@ using Jackett.Utils;
 using Jackett.Utils.Clients;
 using Newtonsoft.Json;
 using NLog;
+using Jackett.Models.DTO;
 
 namespace Jackett.Controllers.V20
 {
@@ -154,20 +155,67 @@ namespace Jackett.Controllers.V20
         [HttpGet]
         public async Task<Models.DTO.ManualSearchResult> Results([FromUri]Models.DTO.ApiSearch request)
         {
+            var manualResult = new ManualSearchResult();
             var trackers = IndexerService.GetAllIndexers().Where(t => t.IsConfigured);
             if (CurrentIndexer.ID != "all")
                 trackers = trackers.Where(t => t.ID == CurrentIndexer.ID).ToList();
-            trackers = trackers.Where(t => t.IsConfigured && t.CanHandleQuery(CurrentQuery));
+            trackers = trackers.Where(t => t.CanHandleQuery(CurrentQuery));
 
             var tasks = trackers.ToList().Select(t => t.ResultsForQuery(CurrentQuery)).ToList();
-            var aggregateTask = Task.WhenAll(tasks);
-
-            await aggregateTask;
-
-            var results = tasks.Where(t => t.Status == TaskStatus.RanToCompletion).Where(t => t.Result.Count() > 0).SelectMany(t =>
+            try
             {
-                var searchResults = t.Result;
-                var indexer = searchResults.First().Origin;
+                var aggregateTask = Task.WhenAll(tasks);
+                await aggregateTask;
+            }
+            catch (AggregateException aex)
+            {
+                foreach (var ex in aex.InnerExceptions)
+                {
+                    logger.Error(ex);
+                }
+            }
+            catch (Exception ex)
+            {
+                logger.Error(ex);
+            }
+
+            manualResult.Indexers = tasks.Select(t =>
+            {
+                var resultIndexer = new ManualSearchResultIndexer();
+                IIndexer indexer = null;
+                if (t.Status == TaskStatus.RanToCompletion)
+                {
+                    resultIndexer.Status = ManualSearchResultIndexerStatus.OK;
+                    resultIndexer.Results = t.Result.Releases.Count();
+                    resultIndexer.Error = null;
+                    indexer = t.Result.Indexer;
+                }
+                else if (t.Exception.InnerException is IndexerException)
+                {
+                    resultIndexer.Status = ManualSearchResultIndexerStatus.Error;
+                    resultIndexer.Results = 0;
+                    resultIndexer.Error = ((IndexerException)t.Exception.InnerException).ToString();
+                    indexer = ((IndexerException)t.Exception.InnerException).Indexer;
+                }
+                else
+                {
+                    resultIndexer.Status = ManualSearchResultIndexerStatus.Unknown;
+                    resultIndexer.Results = 0;
+                    resultIndexer.Error = null;
+                }
+
+                if (indexer != null)
+                {
+                    resultIndexer.ID = indexer.ID;
+                    resultIndexer.Name = indexer.DisplayName;
+                }
+                return resultIndexer;
+            }).ToList();
+
+            manualResult.Results = tasks.Where(t => t.Status == TaskStatus.RanToCompletion).Where(t => t.Result.Releases.Count() > 0).SelectMany(t =>
+            {
+                var searchResults = t.Result.Releases;
+                var indexer = t.Result.Indexer;
                 cacheService.CacheRssResults(indexer, searchResults);
 
                 return searchResults.Select(result =>
@@ -181,17 +229,7 @@ namespace Jackett.Controllers.V20
                 });
             }).OrderByDescending(d => d.PublishDate).ToList();
 
-            ConfigureCacheResults(results);
-
-            var manualResult = new Models.DTO.ManualSearchResult()
-            {
-                Results = results,
-                Indexers = trackers.Select(t => t.DisplayName).ToList()
-            };
-
-
-            if (manualResult.Indexers.Count() == 0)
-                manualResult.Indexers = new List<string>() { "None" };
+            ConfigureCacheResults(manualResult.Results);
 
             logger.Info(string.Format("Manual search for \"{0}\" on {1} with {2} results.", CurrentQuery.SanitizedSearchTerm, string.Join(", ", manualResult.Indexers), manualResult.Results.Count()));
             return manualResult;
@@ -236,7 +274,7 @@ namespace Jackett.Controllers.V20
                 }
             }
 
-            var releases = await CurrentIndexer.ResultsForQuery(CurrentQuery);
+            var result = await CurrentIndexer.ResultsForQuery(CurrentQuery);
 
             // Some trackers do not support multiple category filtering so filter the releases that match manually.
             int? newItemCount = null;
@@ -244,19 +282,19 @@ namespace Jackett.Controllers.V20
             // Cache non query results
             if (string.IsNullOrEmpty(CurrentQuery.SanitizedSearchTerm))
             {
-                newItemCount = cacheService.GetNewItemCount(CurrentIndexer, releases);
-                cacheService.CacheRssResults(CurrentIndexer, releases);
+                newItemCount = cacheService.GetNewItemCount(CurrentIndexer, result.Releases);
+                cacheService.CacheRssResults(CurrentIndexer, result.Releases);
             }
 
             // Log info
             var logBuilder = new StringBuilder();
             if (newItemCount != null)
             {
-                logBuilder.AppendFormat("Found {0} ({1} new) releases from {2}", releases.Count(), newItemCount, CurrentIndexer.DisplayName);
+                logBuilder.AppendFormat("Found {0} ({1} new) releases from {2}", result.Releases.Count(), newItemCount, CurrentIndexer.DisplayName);
             }
             else
             {
-                logBuilder.AppendFormat("Found {0} releases from {1}", releases.Count(), CurrentIndexer.DisplayName);
+                logBuilder.AppendFormat("Found {0} releases from {1}", result.Releases.Count(), CurrentIndexer.DisplayName);
             }
 
             if (!string.IsNullOrWhiteSpace(CurrentQuery.SanitizedSearchTerm))
@@ -278,7 +316,7 @@ namespace Jackett.Controllers.V20
                 ImageDescription = CurrentIndexer.DisplayName
             });
 
-            var proxiedReleases = releases.Select(r => AutoMapper.Mapper.Map<ReleaseInfo>(r)).Select(r =>
+            var proxiedReleases = result.Releases.Select(r => AutoMapper.Mapper.Map<ReleaseInfo>(r)).Select(r =>
             {
                 r.Link = serverService.ConvertToProxyLink(r.Link, serverUrl, r.Origin.ID, "dl", r.Title + ".torrent");
                 return r;
@@ -316,20 +354,20 @@ namespace Jackett.Controllers.V20
         [JsonResponse]
         public async Task<Models.DTO.TorrentPotatoResponse> Potato([FromUri]Models.DTO.TorrentPotatoRequest request)
         {
-            var releases = await CurrentIndexer.ResultsForQuery(CurrentQuery);
+            var result = await CurrentIndexer.ResultsForQuery(CurrentQuery);
 
             // Cache non query results
             if (string.IsNullOrEmpty(CurrentQuery.SanitizedSearchTerm))
-                cacheService.CacheRssResults(CurrentIndexer, releases);
+                cacheService.CacheRssResults(CurrentIndexer, result.Releases);
 
             // Log info
             if (string.IsNullOrWhiteSpace(CurrentQuery.SanitizedSearchTerm))
-                logger.Info($"Found {releases.Count()} torrentpotato releases from {CurrentIndexer.DisplayName}");
+                logger.Info($"Found {result.Releases.Count()} torrentpotato releases from {CurrentIndexer.DisplayName}");
             else
-                logger.Info($"Found {releases.Count()} torrentpotato releases from {CurrentIndexer.DisplayName} for: {CurrentQuery.GetQueryString()}");
+                logger.Info($"Found {result.Releases.Count()} torrentpotato releases from {CurrentIndexer.DisplayName} for: {CurrentQuery.GetQueryString()}");
 
             var serverUrl = string.Format("{0}://{1}:{2}{3}", Request.RequestUri.Scheme, Request.RequestUri.Host, Request.RequestUri.Port, serverService.BasePath());
-            var potatoReleases = releases.Where(r => r.Link != null || r.MagnetUri != null).Select(r =>
+            var potatoReleases = result.Releases.Where(r => r.Link != null || r.MagnetUri != null).Select(r =>
             {
                 var release = AutoMapper.Mapper.Map<ReleaseInfo>(r);
                 release.Link = serverService.ConvertToProxyLink(release.Link, serverUrl, CurrentIndexer.ID, "dl", release.Title + ".torrent");

src/Jackett/Definitions/2fast4you.yml

@@ -112,6 +112,9 @@
             args: ["torrents-details.php", "download.php"]
       title:
         selector: a[href^="torrents-details.php?id="]
+        filters:
+          - name: replace
+            args: [" - (Nouveau!)", ""]
       category:
         selector: a[href^="torrents.php?cat="]
         attribute: href

src/Jackett/Definitions/b2s-share.yml

@@ -214,8 +214,11 @@
 
   search:
     path: torrents-search.php
+    keywordsfilters: 
+      - name: re_replace
+        args: ["[^a-zA-Z0-9]+", "%"]
     inputs:
-      search: "{{ .Query.Keywords }}"
+      search: "{{ .Keywords }}"
     rows:
       selector: table[class^="ttable_headinner"] > tbody > tr:has(a[href^="torrents-details.php?id="])
     fields:

src/Jackett/Definitions/btnext.yml

@@ -120,7 +120,7 @@
       search_type: "name"
       searchin: "title"
     error:
-      - selector: div.error
+      - selector: div.error:not(:contains("Não existem resultados encontrados."))
     rows:
       selector: table#torrents_table_classic > tbody > tr:has(td.torrent_name)
     fields:

src/Jackett/Definitions/cztorrent.yml

@@ -0,0 +1,125 @@
+---
+  site: cztorrent
+  name: CzTorrent
+  language: cs-cz
+  type: semi-private
+  encoding: UTF-8
+  links:
+    - https://tracker.cztorrent.net/
+
+  caps:
+    categorymappings:
+      - {id: 1, cat: Movies, desc: "Filmy"}
+      - {id: 25, cat: TV, desc: "Seriály"}
+      - {id: 23, cat: TV/Documentary, desc: "Filmy - dokument"}
+      - {id: 22, cat: PC, desc: "Aplikace"}
+      - {id: 36, cat: Movies/3D, desc: "Filmy - 3D"}
+      - {id: 35, cat: Movies/Other, desc: "Filmy - anime"}
+      - {id: 37, cat: Movies/BluRay, desc: "Filmy - Blu-ray"}
+      - {id: 11, cat: Movies/DVD, desc: "Filmy - DVD"}
+      - {id: 30, cat: Movies/DVD, desc: "Filmy - DVD full"}
+      - {id: 5, cat: Movies, desc: "Filmy - kreslené"}
+      - {id: 31, cat: Movies/HD, desc: "HD"}
+      - {id: 38, cat: Movies/HD, desc: "HD-LQ"}
+      - {id: 3, cat: PC/Games, desc: "Hry"}
+      - {id: 2, cat: Audio, desc: "Hudba"}
+      - {id: 34, cat:  Audio/Video, desc: "Hudba DVD/HD"}
+      - {id: 6, cat: Books, desc: "Knihy"}
+      - {id: 13, cat: Console, desc: "Konzole"}
+      - {id: 32, cat: Audio, desc: "Mluvené slovo"}
+      - {id: 16, cat: PC/Phone-Other, desc: "Mobil, PDA"}
+      - {id: 4, cat: Other, desc: "Ostatní"}
+      - {id: 29, cat: Audio, desc: "Soundtrack"}
+      - {id: 19, cat:  Audio/Video, desc: "Videoklipy"}
+      - {id: 24, cat: XXX, desc: "xXx"}
+
+    modes:
+      search: [q]
+      tv-search: [q, season, ep]
+      movie-search: [q]
+
+  login:
+    path: /login-page
+    method: form
+    form: form[action^="/login"]
+    inputs:
+      username: "{{ .Config.username }}"
+      password: "{{ .Config.password }}"
+      persistent_login: "1"
+    error:
+      - selector: div.error
+    test:
+      path: /torrents
+
+  search:
+    path: /torrents
+    inputs:
+      $raw: "{{range .Categories}}c{{.}}=1&{{end}}"
+      s: "{{ .Keywords }}"
+      t: "1"
+    rows:
+      selector: tr.torr_hover
+    fields:
+      title:
+        selector: td.detaily a
+        filters:
+          - name: re_replace
+            args: [".*? / ", ""]
+      category:
+        selector: td:nth-child(2)
+        case:
+          ":contains(\"Filmy\")": 1
+          ":contains(\"Seriály\")": 25
+          ":contains(\"Filmy - dokument\")": 23
+          ":contains(\"Aplikace\")": 22
+          ":contains(\"Filmy - 3D\")": 36
+          ":contains(\"Filmy - anime\")": 35
+          ":contains(\"Filmy - Blu-ray\")": 37
+          ":contains(\"Filmy - DVD\")": 11
+          ":contains(\"Filmy - DVD full\")": 30
+          ":contains(\"Filmy - kreslené\")": 5
+          ":contains(\"HD\")": 31
+          ":contains(\"HD-LQ\")": 38
+          ":contains(\"Hry\")": 3
+          ":contains(\"Hudba\")": 2
+          ":contains(\"Hudba DVD/HD\")": 34
+          ":contains(\"Knihy\")": 6
+          ":contains(\"Konzole\")": 13
+          ":contains(\"Mluvené slovo\")": 32
+          ":contains(\"Mobil, PDA\")": 16
+          ":contains(\"Ostatní\")": 4
+          ":contains(\"Soundtrack\")": 29
+          ":contains(\"Videoklipy\")": 19
+          ":contains(\"xXx\")": 24
+      details:
+        selector: td.detaily a
+        attribute: href
+      download:
+        selector: td.download a
+        attribute: href
+      size:
+        selector: td.detaily
+        filters:
+          - name: split
+            args: [ "|", 0 ]
+          - name: regexp
+            args: "Velikost: (.+?) ?$"
+      date:
+        selector: td:nth-child(4)
+        filters:
+          - name: split
+            args: [ "|", 1 ]
+          - name: regexp
+            args: "Přidán: (.+?) ?$"
+          - name: append
+            args: " +02:00"
+      seeders:
+        selector: td:nth-child(7) span
+      leechers:
+        selector: td:nth-child(8) span
+      downloadvolumefactor:
+        case:
+          "*": "1"
+      uploadvolumefactor:
+        case:
+          "*": "1"

src/Jackett/Definitions/datascene.yml

@@ -67,7 +67,7 @@
       search: "{{ .Query.Keywords }}"
       incldead: 1
     rows:
-      selector: tr:has(a.tname)
+      selector: div.ncls > table > tbody > tr:has(a.tname)
     fields:
       title-attribute:
         selector: a.tname
@@ -86,7 +86,7 @@
           - name: querystring
             args: cat
       download:
-        selector: a[href^="/download.php/"]
+        selector: a[href^="/download.php/"], a[href^="/downloadd.php/"] # some releases use a download link with two d's
         attribute: href
       grabs:
         selector: td:nth-child(8)
@@ -97,10 +97,9 @@
         selector: td:nth-child(7)
       date:
         selector: td:nth-child(2) > right > div:has(font:contains("Uploaded"))
-        remove: div > font
         filters:
-          - name: trim
-            args: ":"
+          - name: replace
+            args: ["Uploaded: ", ""]
       seeders:
         selector: td:nth-child(9)
       leechers:
@@ -117,7 +116,7 @@
         attribute: href
       downloadvolumefactor:
         case:
-          "img[src=\"pic/free.gif\"]": "0"
+          "img[src=\"pic/free.png\"]": "0"
           "*": "1"
       uploadvolumefactor:
         case:

src/Jackett/Definitions/diablotorrent.yml

@@ -58,32 +58,25 @@
       movie-search: [q]
 
   login:
-    path: login.php
-    submitpath: login.php
+    path: login
+    submitpath: login/run
     method: form
     inputs:
       referer: ""
       nev: "{{ .Config.username }}"
       jelszo: "{{ .Config.password }}"
-      fsv: "-1"
-      ssl: "1"
+      fsv: "0"
+      login: "1"
       belepes: "1"
-    getselectorinputs:
-      c:
-        selector: "script:contains(\"login.php\")"
-        filters:
-          - name: regexp
-            args: "login.php\\?c=(.*?)&rhash="
-      rhash:
-        text: "123"
+      rhash: "123"
     test:
-      path: browse.php
+      path: browse
 
   search:
-    path: browse.php
+    path: browse
     inputs:
       $raw: "{{range .Categories}}category[]={{.}}&{{end}}"
-      cim: "{{ .Keywords }}"
+      search: "{{ .Keywords }}"
       s_alapjan: "7"
       s_sorrend: "desc"
     rows:
@@ -93,10 +86,8 @@
       download:
         selector: div.box_download > a
         attribute: href
-      #title: # shortened
-      #  selector: a.box_name_a
       title:
-        selector: div.torrent_details_po + h2
+        selector: a.box_name_a > span
       details:
         selector: a.box_name_a
         attribute: href

src/Jackett/Definitions/dragonworld.yml

@@ -1,154 +0,0 @@
----
-  site: dragonworld
-  name: Dragon World (DTW)
-  language: de-de
-  type: private
-  encoding: UTF-8
-  links:
-    - http://dtw.sytes.net/
-
-  caps:
-    categorymappings:
-      # Dokumentation
-      - {id: 46, cat: TV/Documentary, desc: "Dokumentation"}
-      - {id: 55, cat: TV/Documentary, desc: "Dokumentation/HD"}
-      - {id: 56, cat: TV/Documentary, desc: "Dokumentation/SD"}
-      # Ebooks
-      - {id: 36, cat: Books, desc: "Ebooks"}
-      - {id: 37, cat: Books, desc: "Ebooks"}
-      - {id: 38, cat: Books, desc: "Ebooks/Hoerspiele/Hoerbuecher"}
-      # Games
-      - {id: 21, cat: Console, desc: "Games"}
-      - {id: 24, cat: Console/Other, desc: "Games/Nintendo"}
-      - {id: 22, cat: PC/Games, desc: "Games/PC"}
-      - {id: 23, cat: Console/PS4, desc: "Games/Playstation"}
-      - {id: 25, cat: Console/Xbox, desc: "Games/Xbox"}
-      # Kinder
-      - {id: 10, cat: Other, desc: "Kinder"}
-      - {id: 14, cat: Other, desc: "Kinder/Diverses"}
-      - {id: 12, cat: Movies, desc: "Kinder/Filme"}
-      - {id: 11, cat: PC/Games, desc: "Kinder/Games"}
-      - {id: 13, cat: Audio, desc: "Kinder/Musik"}
-      # Movies
-      - {id: 15, cat: Movies, desc: "Movies"}
-      - {id: 50, cat: Movies/3D, desc: "Movies/3D"}
-      - {id: 48, cat: Movies/HD, desc: "Movies/HD"}
-      - {id: 53, cat: Movies/HD, desc: "Movies/HD Pack"}
-      - {id: 45, cat: Movies/HD, desc: "Movies/Remuxe"}
-      - {id: 17, cat: Movies/SD, desc: "Movies/SD"}
-      - {id: 54, cat: Movies/SD, desc: "Movies/SD Pack"}
-      # Musik
-      - {id: 4, cat: Audio, desc: "Musik"}
-      - {id: 57, cat: Audio, desc: "Musik/Album"}
-      - {id: 8, cat: Audio/Lossless, desc: "Musik/Flac"}
-      - {id: 7, cat: Audio/MP3, desc: "Musik/Mp3"}
-      - {id: 9, cat: Audio/Video, desc: "Musik/Video"}
-      # Serien
-      - {id: 26, cat: TV, desc: "Serien"}
-      - {id: 27, cat: TV/HD, desc: "Serien/HD"}
-      - {id: 28, cat: TV/SD, desc: "Serien/SD"}
-      # Software
-      - {id: 29, cat: PC/0day, desc: "Software"}
-      - {id: 32, cat: PC/0day, desc: "Software/Diverses"}
-      - {id: 31, cat: PC/Mac, desc: "Software/Mac"}
-      - {id: 30, cat: PC/0day, desc: "Software/Windows"}
-      # Sport
-      - {id: 39, cat: TV/Sport, desc: "Sport"}
-      - {id: 40, cat: TV/Sport, desc: "Sport HD"}
-      - {id: 58, cat: TV/Sport, desc: "Sport SD"}
-      # XXX
-      - {id: 33, cat: XXX, desc: "XXX"}
-      - {id: 34, cat: XXX, desc: "XXX/HD"}
-      - {id: 35, cat: XXX, desc: "XXX/SD"}
-
-
-    modes:
-      search: [q]
-      tv-search: [q, season, ep]
-      movie-search: [q]
-  
-  login:
-    path: takelogin.php
-    method: post
-    inputs:
-      username: "{{ .Config.username }}"
-      password: "{{ .Config.password }}"
-    error:
-      - selector: table:has(td:contains("Ein Fehler ist aufgetreten"))
-    test:
-      path: browse.php
-      selector: a[href*="/logout.php"]
-
-  download:
-    before:
-      path: "takethanks.php"
-      method: "post"
-      inputs:
-        torrentid: "{{ .DownloadUri.Query.id }}"
-      
-  search:
-    path: browse.php
-    keywordsfilters:
-      - name: re_replace
-        args: ["[^a-zA-Z0-9]+", "%"]
-    inputs:
-      do: "search"
-      keywords: "{{ .Keywords }}"
-      search_type: "t_name"
-      category: "0" # multi cat search not supported
-      include_dead_torrents: "yes"
-    rows:
-      selector: table#sortabletable > tbody > tr:has(a[href*="/details.php?id="])
-      filters:
-        - name: andmatch
-          args: 66
-    fields:
-      download:
-        selector: a[href*="/download.php?id="]
-        attribute: href
-      magnet:
-        selector: a[href^="magnet:"]
-        attribute: href
-      title:
-        selector: a[href*="/details.php?id="]
-      title:
-        selector: div.tooltip-content > div
-        optional: true
-      details:
-        selector: a[href*="/details.php?id="]
-        attribute: href
-      category:
-        selector: a[href*="/browse.php?category="]
-        attribute: href
-        filters:
-          - name: querystring
-            args: category
-      banner:
-        selector: div.tooltip-content > img
-        attribute: src
-        optional: true
-      size:
-        selector: td:nth-child(5)
-      grabs:
-        selector: td:nth-child(6)
-      seeders:
-        selector: td:nth-child(7)
-      leechers:
-        selector: td:nth-child(8)
-      downloadvolumefactor:
-        case:
-          img[alt^="OnlyUp Torrent"]: "0"
-          img[alt^="50% "]: "0.5"
-          "*": "1"
-      uploadvolumefactor:
-        case:
-          img[alt^="multi2 Torrent"]: "2"
-          "*": "1"
-      date:
-        selector: "td:nth-child(2) > div:has(span[style=\"float: right;\"])"
-        remove: span
-        filters:
-          - name: append
-            args: " +01:00"
-          - name: dateparse
-            args: "02-01-2006 15:04 -07:00"

src/Jackett/Definitions/eotforum.yml

@@ -6,7 +6,7 @@
   type: private
   encoding: windows-1252
   links:
-    - http://eot-forum.net
+    - https://eot-forum.net
 
   caps:
     categorymappings:

src/Jackett/Definitions/gay-torrents.yml

@@ -0,0 +1,114 @@
+  site: gay-torrents
+  name: Gay-Torrents.net
+  description: "Gay Torrents Live Free!"
+  language: en-us
+  type: semi-private
+  encoding: UTF-8
+  links:
+    - https://www.gay-torrents.net/
+
+  caps:
+    categorymappings:
+     - {id: porn , cat: XXX , desc: "Porn"}
+     - {id: porn/Amateur , cat: XXX , desc: "Amateur"}
+     - {id: porn/Anal , cat: XXX , desc: "Anal"}
+     - {id: porn/Asian , cat: XXX , desc: "Asian"}
+     - {id: porn/Bareback , cat: XXX , desc: "Bareback"}
+     - {id: porn/Bears , cat: XXX , desc: "Bears"}
+     - {id: porn/Bisexual , cat: XXX , desc: "Bisexual"}
+     - {id: porn/Black-Men , cat: XXX , desc: "Black-Men"}
+     - {id: porn/Chubs , cat: XXX , desc: "Chubs"}
+     - {id: porn/Clips , cat: XXX/Packs , desc: "Clips"}
+     - {id: porn/Cross-Generation , cat: XXX , desc: "Cross-Generation"}
+     - {id: porn/DVD-R , cat: XXX/DVD , desc: "DVD-R"}
+     - {id: porn/Fetish , cat: XXX , desc: "Fetish"}
+     - {id: porn/Group-Sex , cat: XXX , desc: "Group-Sex"}
+     - {id: porn/HD-Movies , cat: XXX , desc: "HD-Movies"}
+     - {id: porn/Hunks , cat: XXX , desc: "Hunks"}
+     - {id: porn/Images , cat: XXX/Imageset , desc: "Images"}
+     - {id: porn/Interracial , cat: XXX , desc: "Interracial"}
+     - {id: porn/Jocks , cat: XXX , desc: "Jocks"}
+     - {id: porn/Latino , cat: XXX , desc: "Latino"}
+     - {id: porn/Mature , cat: XXX , desc: "Mature"}
+     - {id: porn/Member , cat: XXX , desc: "Member"}
+     - {id: porn/MiddleEast , cat: XXX , desc: "MiddleEast"}
+     - {id: porn/Military , cat: XXX , desc: "Military"}
+     - {id: porn/Muscle , cat: XXX , desc: "Muscle"}
+     - {id: porn/Oral-Sex , cat: XXX , desc: "Oral-Sex"}
+     - {id: porn/Solo , cat: XXX , desc: "Solo"}
+     - {id: porn/Transsexual , cat: XXX , desc: "Transsexual"}
+     - {id: porn/Twinks , cat: XXX , desc: "Twinks"}
+     - {id: porn/Vintage , cat: XXX , desc: "Vintage"}
+     - {id: porn/Wrestling , cat: XXX , desc: "Wrestling"}
+     - {id: nonporn , cat: Other , desc: "NonPorn"}
+     - {id: nonporn/Anime , cat: TV/Anime , desc: "Anime"}
+     - {id: nonporn/Applications , cat: PC , desc: "Applications"}
+     - {id: nonporn/Comedy , cat: Movies , desc: "Comedy"}
+     - {id: nonporn/Comics , cat: Books/Comics , desc: "Comics"}
+     - {id: nonporn/Coming-Out , cat: Movies , desc: "Coming-Out"}
+     - {id: nonporn/Documentary , cat: Movies , desc: "Documentary"}
+     - {id: nonporn/Drama , cat: Movies , desc: "Drama"}
+     - {id: nonporn/DVD-R , cat: Movies/DVD , desc: "DVD-R"}
+     - {id: nonporn/Gay-Movies , cat: Movies , desc: "Gay-Movies"}
+     - {id: nonporn/Misc , cat: Other/Misc , desc: "Misc"}
+     - {id: nonporn/Short-Film , cat: Movies , desc: "Short-Film"}
+     - {id: nonporn/Softcore , cat: Movies , desc: "Softcore"}
+     - {id: nonporn/Thriller , cat: Movies , desc: "Thriller"}
+     - {id: nonporn/TV-Episode , cat: TV , desc: "TV-Episode"}
+    modes:
+      search: [q]
+
+  login:
+    path: login.php
+    method: post
+    inputs:
+      vb_login_username: "{{ .Config.username }}"
+      vb_login_password: "{{ .Config.password }}"
+      cookieuser: "1"
+      do: "login"
+    test:
+      path: torrentslist.php
+      selector: ul.isuser
+
+  search:
+    path: "{{if .Query.Keywords}}search.php{{else}}torrentslist.php{{end}}"
+    inputs:
+      $raw: "{{range .Categories}}type={{.}}&{{end}}"
+      textsearch: "{{ .Query.Keywords }}"
+    rows:
+      selector: ul.TorrentList
+    fields:
+      category:
+        selector: .TorrentList1 > a
+        attribute: href
+        filters:
+          - name: querystring
+            args: type
+      title:
+        selector: .TorrentList2 > a
+      download:
+        selector: .TorrentList2 > a
+        attribute: href
+        filters:
+          - name: append
+            args: "&do=download"
+      details:
+        selector: .TorrentList2 > a
+        attribute: href
+      size:
+        selector: .TorrentList3
+      seeders:
+        selector: .TorrentList6
+      leechers:
+        selector: .TorrentList7
+      date:
+        selector: li.TorrentList8
+        filters:
+          - name: dateparse
+            args: "02 Jan 06, 15:04"
+      downloadvolumefactor:
+        case:
+          ".TorrentList2 > a:contains(\"[FFL]\")": "0"
+          "*": "1"
+      uploadvolumefactor:
+        text: "1"

src/Jackett/Definitions/gaytorrentru.yml

@@ -0,0 +1,113 @@
+  site: gaytorrentru
+  name: GayTorrent.ru
+  description: "World largest gay porn library for free with a stunning forum and 24/7 Chat"
+  language: en-us
+  type: private
+  encoding: UTF-8
+  links:
+    - https://www.gaytorrent.ru/
+
+  caps:
+    categorymappings:
+     - {id: 62 , cat: XXX , desc: "Amateur"}
+     - {id: 29 , cat: XXX , desc: "Anal"}
+     - {id: 46 , cat: XXX , desc: "Anime Games"}
+     - {id: 30 , cat: XXX , desc: "Asian"}
+     - {id: 43 , cat: XXX , desc: "Bareback"}
+     - {id: 19 , cat: XXX , desc: "BDSM"}
+     - {id: 17 , cat: XXX , desc: "Bears"}
+     - {id: 44 , cat: XXX , desc: "Black"}
+     - {id: 50 , cat: Books , desc: "Books & Magazines"}
+     - {id: 9 , cat: XXX , desc: "Chubbies"}
+     - {id: 7 , cat: XXX , desc: "Clips"}
+     - {id: 48 , cat: Books/Comics  , desc: "Comic & Yaoi"}
+     - {id: 5 , cat: XXX , desc: "Daddies / Sons"}
+     - {id: 34 , cat: XXX , desc: "Fetish"}
+     - {id: 27 , cat: XXX , desc: "Grey / Older"}
+     - {id: 32 , cat: XXX , desc: "Group-Sex"}
+     - {id: 63 , cat: XXX , desc: "Homemade"}
+     - {id: 12 , cat: XXX , desc: "Hunks"}
+     - {id: 33 , cat: XXX , desc: "Images"}
+     - {id: 53 , cat: XXX , desc: "Interracial"}
+     - {id: 57 , cat: XXX , desc: "Jocks"}
+     - {id: 35 , cat: XXX , desc: "Latino"}
+     - {id: 36 , cat: XXX , desc: "Mature"}
+     - {id: 58 , cat: PC , desc: "Media Programs"}
+     - {id: 37 , cat: XXX , desc: "Member"}
+     - {id: 54 , cat: XXX , desc: "Middle Eastern"}
+     - {id: 38 , cat: XXX , desc: "Military"}
+     - {id: 39 , cat: XXX , desc: "Oral-Sex"}
+     - {id: 47 , cat: XXX , desc: "Shemale"}
+     - {id: 56 , cat: XXX , desc: "Softcore"}
+     - {id: 40 , cat: XXX , desc: "Solo"}
+     - {id: 45 , cat: Movies , desc: "Themed Movie"}
+     - {id: 1 , cat: TV , desc: "TV / Episodes"}
+     - {id: 41 , cat: XXX , desc: "Twinks"}
+     - {id: 42 , cat: XXX , desc: "Vintage"}
+     - {id: 51 , cat: XXX , desc: "Voyeur"}
+     - {id: 65 , cat: XXX , desc: "Wrestling and Sports"}
+     - {id: 28 , cat: XXX , desc: "Youngblood"}
+    modes:
+      search: [q]
+
+  login:
+    path: takelogin.php
+    method: post
+    inputs:
+      username: "{{ .Config.username }}"
+      password: "{{ .Config.password }}"
+    test:
+      path: browse.php
+
+  search:
+    path: browse.php
+    inputs:
+      $raw: "{{range .Categories}}c{{.}}=1&{{end}}"
+      search: "{{ .Query.Keywords }}"
+      incldead: "0" # Searches only for alive torrents
+    rows:
+      selector: table.browse_result > tbody > tr:has(a[href^="details.php?id="])
+    fields:
+      category:
+        selector: a[href^="browse.php?cat="]
+        attribute: href
+        filters:
+          - name: querystring
+            args: cat
+      title:
+        selector: .browsedesc > a
+      download:
+        selector: a[href^="download.php/"]
+        attribute: href
+      details:
+        selector: a[href^="details.php?id="]
+        attribute: href
+      grabs:
+        selector: .tsnatch
+      files:
+        selector: .tfiles
+        filters:
+          - name: regexp
+            args: ([\d]+)
+      size:
+        selector: .tsize
+      seeders:
+        optional: true
+        selector: a[href$="&toseeders=1"]
+      leechers:
+        optional: true
+        selector: a[href$="&todlers=1"]
+      date:
+        selector: .tadded
+        filters:
+          - name: re_replace
+            args: ["(\\d{4}-\\d{2}-\\d{2})(\\d{2}:\\d{2}:\\d{2})(.*)","$1 $2"]
+          - name: dateparse
+            args: "2006-01-02 15:04:05"
+      downloadvolumefactor:
+        case:
+          "td:nth-child(3) > div > nobr > font[color=\"yellow\"]": "0"
+          "*": "1"
+      uploadvolumefactor:
+        case:
+          "*": "1"

src/Jackett/Definitions/gigatorrents.yml

@@ -70,6 +70,8 @@
       blah: "0"
     rows:
       selector: table#torrenttable > tbody > tr:has()
+    error:
+      - selector: div.content:contains("Meg van vonva a letöltési jogod")
     fields:
       download:
         selector: a[href^="/details.php?id="]

src/Jackett/Definitions/kapaki.yml

@@ -77,7 +77,7 @@
       search_type: "name"
       searchin: "title"
     error:
-      - selector: div#show_error
+      - selector: div#show_error:not(:contains("Ουπς! Λάθος!Δεν βρέθηκαν αποτελέσματα."))
     rows:
       selector: div#content > div.torrent-box[id^="torrent_"]
       filters:

src/Jackett/Definitions/karagarga.yml

@@ -30,9 +30,6 @@
     test:
       path: index.php
 
-  download:
-    selector: a[href^="download.php?id="]
-        
   search:
     path: browse.php
     inputs:
@@ -47,11 +44,8 @@
       title:
         selector: td:nth-child(2) span
       download:
-        selector: a[href^="details.php?id="]
+        selector: a[href^="down.php"]
         attribute: href
-        filters:
-          - name: replace
-            args: ["details.php?id=", "download.php?id="]
       details:
         selector: a[href^="details.php?id="]
         attribute: href

src/Jackett/Definitions/kickasstorrent-kathow.yml

@@ -1,11 +1,11 @@
 ---
   site: kickasstorrent-kathow
-  name: KickAssTorrent (kat.how)
+  name: KickAssTorrent (thekat.se)
   language: en-us
   type: public
   encoding: UTF-8
   links:
-    - https://kat.how
+    - https://thekat.se
 
   caps:
     categories:

src/Jackett/Definitions/kickasstorrent.yml

@@ -2,7 +2,7 @@
   site: kickasstorrent
   name: KickAssTorrent
   language: en-us
-  type: semi-private
+  type: public
   encoding: UTF-8
   links:
     - https://katcr.co/
@@ -85,28 +85,10 @@
       search: [q]
       tv-search: [q, season, ep]
 
-  settings:
-    - name: username
-      type: text
-      label: Username
-    - name: passkey
-      type: text
-      label: Passkey
-
-  login:
-    method: post
-    path: "/new/account-login.php"
-    inputs:
-      username: "{{ .Config.username }}"
-      passkey: "{{ .Config.passkey }}"
-    error:
-      - selector: font.error:contains("Access Denied")
-    test:
-      path: "/new/index.php"
-      selector: a[href="account-logout.php"]
+  settings: []
 
   search:
-    path: "/new/{{if .Query.Keywords}}search-torrents.php{{else}}torrents.php{{end}}"
+    path: "/new/{{if .Query.Keywords}}search-torrents.php{{else}}index.php{{end}}"
     inputs:
       $raw: "{{range .Categories}}c{{.}}=1&{{end}}"
       search: "\"{{ .Query.Keywords }}\""

src/Jackett/Definitions/leparadisdunet.yml

@@ -1,163 +0,0 @@
----
-  site: leparadisdunet
-  name: Le Paradis Du Net
-  language: fr-fr
-  type: semi-private
-  encoding: UTF-8
-  links:
-    - https://le-paradis-du-net.com/
-
-  caps:
-    categorymappings:
-      - {id: 10, cat: Movies/3D, desc: "3D"}
-      - {id: 9, cat: XXX, desc: "Adultes"}
-      - {id: 1, cat: PC, desc: "APPLICATION"}
-      - {id: 33, cat: PC/Phone-Other, desc: "  Iphone/Ipod/Android"}
-      - {id: 3, cat: PC/0day, desc: "  Linux"}
-      - {id: 32, cat: PC/Mac, desc: "  Mac"}
-      - {id: 4, cat: PC/0day, desc: "  Windows"}
-      - {id: 70, cat: Movies/HD, desc: "BDRIP"}
-      - {id: 69, cat: Movies/BluRay, desc: "Blueray"}
-      - {id: 73, cat: Movies/HD, desc: "BRRIP"}
-      - {id: 22, cat: Movies/SD, desc: "CAM TS SCREENER"}
-      - {id: 39, cat: Movies/DVD, desc: "  R5"}
-      - {id: 13, cat: TV/Anime, desc: "DESSINS ANIMES"}
-      - {id: 48, cat: TV/Anime, desc: "  Animations"}
-      - {id: 47, cat: TV/Anime, desc: "  Mangas"}
-      - {id: 14, cat: TV/Documentary, desc: "DOCUMENTAIRE"}
-      - {id: 52, cat: TV/Documentary, desc: "  EmissionsTV"}
-      - {id: 49, cat: TV/Documentary, desc: "  Tv docs"}
-      - {id: 15, cat: Movies/DVD, desc: "DVDR"}
-      - {id: 16, cat: Movies/SD, desc: "DVDRIP"}
-      - {id: 35, cat: Movies, desc: "  Action"}
-      - {id: 64, cat: Movies, desc: "  Autres"}
-      - {id: 36, cat: Movies, desc: "  Aventure"}
-      - {id: 55, cat: Movies, desc: "  Comédie"}
-      - {id: 37, cat: Movies, desc: "  Drame"}
-      - {id: 38, cat: Movies, desc: "  Fantastique"}
-      - {id: 63, cat: Movies, desc: "  Guerre"}
-      - {id: 72, cat: Movies, desc: "  Historique/Biopic"}
-      - {id: 67, cat: Movies, desc: "  Horreur"}
-      - {id: 65, cat: Movies, desc: "  Thriller"}
-      - {id: 71, cat: Movies, desc: "  Werstern"}
-      - {id: 17, cat: Movies/SD, desc: "DVDRIP VOSTFR"}
-      - {id: 19, cat: Books, desc: "EBOOK"}
-      - {id: 54, cat: Books, desc: "  Journaux"}
-      - {id: 81, cat: Books, desc: "  Magazines"}
-      - {id: 82, cat: Books, desc: "  People"}
-      - {id: 40, cat: Movies/HD, desc: "HD"}
-      - {id: 41, cat: Movies/HD, desc: "  1080p"}
-      - {id: 42, cat: Movies/HD, desc: "  720P"}
-      - {id: 77, cat: Movies/HD, desc: "  HD LIGHT"}
-      - {id: 86, cat: TV/SD, desc: "  HDTV"}
-      - {id: 20, cat: Console/Other, desc: "JEUX"}
-      - {id: 56, cat: Console/NDS, desc: "  DS"}
-      - {id: 57, cat: PC/Games, desc: "  Pc"}
-      - {id: 26, cat: Console/PS3, desc: "  PS3"}
-      - {id: 58, cat: Console/PSP, desc: "  PSP"}
-      - {id: 25, cat: Console/Wii, desc: "  Wii"}
-      - {id: 24, cat: Console/Xbox360, desc: "  Xbox360"}
-      - {id: 21, cat: Audio, desc: "MUSIQUES"}
-      - {id: 29, cat: Audio/Video, desc: "  Clip"}
-      - {id: 68, cat: Audio/Video, desc: "  Concert"}
-      - {id: 31, cat: Audio/Lossless, desc: "  FLAC"}
-      - {id: 30, cat: Audio/MP3, desc: "  mp3"}
-      - {id: 28, cat: Audio/Lossless, desc: "  wave"}
-      - {id: 27, cat: Audio/Other, desc: "  wma"}
-      - {id: 5, cat: TV, desc: "SERIES"}
-      - {id: 79, cat: TV/HD, desc: "  SERIES HD 1080P"}
-      - {id: 80, cat: TV/HD, desc: "  SERIES HD 720p"}
-      - {id: 75, cat: TV, desc: "  TV pack"}
-      - {id: 8, cat: TV, desc: "  vf"}
-      - {id: 6, cat: TV, desc: "  vo"}
-      - {id: 7, cat: TV, desc: "  vost"}
-      - {id: 12, cat: TV/Sport, desc: "SPORTS"}
-      - {id: 61, cat: TV/Sport, desc: "  Autres "}
-      - {id: 45, cat: TV/Sport, desc: "  Catch VF"}
-      - {id: 59, cat: TV/Sport, desc: "  Catch VO"}
-      - {id: 44, cat: TV/Sport, desc: "  Football"}
-      - {id: 60, cat: TV/Sport, desc: "  UFC/MMA"}
-      - {id: 76, cat: TV, desc: "Télévision"}
-      - {id: 78, cat: Movies/WEBDL, desc: "WEBRIP"}
-
-    modes:
-      search: [q]
-      tv-search: [q, season, ep]
-
-  login:
-    path: /takelogin.php
-    method: post
-    inputs:
-      username: "{{ .Config.username }}"
-      password: "{{ .Config.password }}"
-    error:
-      - selector: table:has(td:contains("Une erreur est survenue"))
-    test:
-      path: browse.php
-
-  download:
-    before:
-      path: "/takethanks.php"
-      method: "post"
-      inputs:
-        torrentid: "{{ re_replace .DownloadUri.AbsolutePath \"^.*download-torrent-(\\d+)/.*$\" \"$1\" }}"
-
-  search:
-    path: browse.php
-    inputs:
-      do: "chercher"
-      keywords: "{{ .Query.Keywords }}"
-      search_type: "t_name"
-      category: "0" # multi cat search not supported
-      include_dead_torrents: "yes"
-    rows:
-      selector: table#sortabletable > tbody > tr:has(a[href*="/torrent-details-"])
-    fields:
-      download:
-        selector: a[href*="/torrent-details-"]
-        attribute: href
-        filters:
-          - name: replace
-            args: ["torrent-details-", "download-torrent-"]
-      title:
-        selector: a[href*="/torrent-details-"]
-      details:
-        selector: a[href*="/torrent-details-"]
-        attribute: href
-      category:
-        selector: a[href*="/torrent-category-"]
-        attribute: href
-        filters:
-          - name: regexp
-            args: torrent-category-(\d+)
-      size:
-        selector: td:nth-child(4)
-      date:
-        selector: td:nth-child(2) > div > font[color="white"]
-        filters:
-          - name: replace
-            args: ["le ", ""]
-          - name: replace
-            args: [" à ", " "]
-          - name: trim
-            args: "\t"
-          - name: trim
-            args: "\n"
-          - name: append
-            args: " +01:00"
-          - name: dateparse
-            args: "02-01-2006 15:04 -07:00"
-      grabs:
-        selector: td:nth-child(5)
-      seeders:
-        selector: td:nth-child(6)
-      leechers:
-        selector: td:nth-child(7)
-      downloadvolumefactor:
-        case:
-          img[alt^="Free Torrent "]: "0"
-          img[alt^="Silver Torrent "]: "0.5"
-          "*": "1"
-      uploadvolumefactor:
-        case:
-          "*": "1"

src/Jackett/Definitions/manicomioshare.yml

@@ -199,22 +199,26 @@
     modes:
       search: [q]
 
-  settings:
-    - name: cookie
-      type: text
-      label: Cookie
-          
   login:
-    method: cookie
+    path: index.php
+    method: post
     inputs:
-      cookie: "{{ .Config.cookie }}"
+      dados: "ok"
+      username: "{{ .Config.username }}"
+      password: "{{ .Config.password }}"
     test:
-      path: index.php
+      path: torrents.php
 
   search:
     path: pesquisa.php
+    keywordsfilters: 
+      # workaroud to remove year in search keywords
+      - name: re_replace
+        args: ["([1-2]\\d{3})", ""]    
+      - name: re_replace
+        args: ["[^a-zA-Z0-9]+", "%"]
     inputs:
-      busca: "{{ .Query.Keywords }}"
+      busca: "{{ .Keywords }}"
     rows:
       selector: table#tbltorrent > tbody > tr[data-id]
     fields:

src/Jackett/Definitions/nexttorrent.yml

@@ -24,7 +24,7 @@
   search:
     path: "recherche/{{ .Query.Keywords }}"
     rows:
-      selector: div.listing-torrent > table tbody tr
+      selector: div.listing-torrent > table tbody tr:has(a)
     fields:
       site_date:
         selector: td:nth-child(1) a
@@ -85,4 +85,4 @@
       downloadvolumefactor:
         text: "0"
       uploadvolumefactor:
-        text: "1"
+        text: "1"

src/Jackett/Definitions/polishtracker.yml

@@ -6,6 +6,8 @@
   encoding: UTF-8
   links:
     - https://polishtracker.net/
+  certificates:
+    - 95EAB2795D8E35A14A9B35619C2568F974AE4D95 # expired
 
   caps:
     categorymappings:
@@ -57,6 +59,8 @@
       search: "{{ .Query.Keywords }}"
     rows:
       selector: div#bro1 > table.shad1 > tbody > tr[class^="rowfx"]
+    error:
+      - selector: table.textfx2 td:contains("In order to use PolishTracker we require users to familiarize with FAQ, Rules and any News from Home Page.")
     fields:
       download:
         selector: a[href^="/download.php/"]

src/Jackett/Definitions/qctorrent.yml

@@ -1,107 +0,0 @@
----
-  site: qctorrent
-  name: QcTorrent
-  description: "A French gerneral tracker"
-  language: fr-fr
-  type: private
-  encoding: UTF-8
-  links:
-    - http://www.qctorrent.net/
-
-  caps:
-    categorymappings:
-      - {id: 30, cat: PC, desc: "++ Applications"}
-      - {id: 1, cat: PC, desc: "Applications/Divers"}
-      - {id: 2, cat: PC, desc: "Applications/PC ISO"}
-      - {id: 3, cat: PC, desc: "Applications/Portable"}
-      - {id: 31, cat: Movies, desc: "++ Films"}
-      - {id: 4, cat: Movies/BluRay, desc: "Films/Bluray"}
-      - {id: 5, cat: Movies/DVD, desc: "Films/DVDr"}
-      - {id: 6, cat: Movies/HD, desc: "Films/HD Rip"}
-      - {id: 7, cat: Movies/SD, desc: "Films/SD Rip"}
-      - {id: 8, cat: Movies/SD, desc: "Films/VCD"}
-      - {id: 32, cat: Console, desc: "++ Jeux"}
-      - {id: 9, cat: PC/Games, desc: "Jeux/PC"}
-      - {id: 10, cat: Console, desc: "Jeux/Portable"}
-      - {id: 11, cat: Console/PS4, desc: "Jeux/PS"}
-      - {id: 12, cat: Console/Wii, desc: "Jeux/Wii"}
-      - {id: 13, cat: Console/Xbox, desc: "Jeux/Xbox"}
-      - {id: 33, cat: Audio, desc: "++ Musique"}
-      - {id: 14, cat: Audio, desc: "Musique"}
-      - {id: 15, cat: Audio/Video, desc: "Musique/Video"}
-      - {id: 34, cat: TV, desc: "++ Série-Télé"}
-      - {id: 16, cat: TV/HD, desc: "Série-Télé/Bluray"}
-      - {id: 17, cat: TV/SD, desc: "Série-Télé/DVDr"}
-      - {id: 18, cat: TV/HD, desc: "Série-Télé/HD Rip"}
-      - {id: 19, cat: TV/SD, desc: "Série-Télé/SD Rip"}
-      - {id: 20, cat: Books, desc: "E-Books"}
-      - {id: 21, cat: XXX, desc: "XXX"}
-
-    modes:
-      search: [q]
-      tv-search: [q, season, ep]
-
-  login:
-    path: login.php
-    method: post
-    inputs:
-      login-username: "{{ .Config.username }}"
-      login-password: "{{ .Config.password }}"
-      login-remember-me: "on"
-      login: ""
-    error:
-      - selector: "script[type=\"text/javascript\"]:contains(\"$.ambiance({message: \")"
-    test:
-      path: search.php
-      selector: div.top-bar > div.container > div.textleft > div.hidden-sm > font:contains("Ratio:") > font
-
-  ratio:
-    path: search.php
-    selector: div.top-bar > div.container > div.textleft > div.hidden-sm > font:contains("Ratio:") > font
-
-  search:
-    path: search.php
-    inputs:
-      category: "{{range .Categories}}{{.}};{{end}}"
-      title: "{{ .Query.Keywords }}"
-      search: "Recherche"
-    rows:
-      selector: tr[data-snatches]
-    fields:
-      download:
-        selector: td.name > a
-        attribute: href
-        filters:
-          - name: replace
-            args: ["/torrent/", "/dl/"]
-      title:
-        selector: td.name > a
-      category:
-        selector: td.coll-0 > a
-        attribute: href
-        filters:
-          - name: querystring
-            args: category
-      details:
-        selector: td.name > a
-        attribute: href
-      grabs:
-        attribute: data-snatches
-      seeders:
-        selector: td.seeds
-      leechers:
-        selector: td.leeches
-      date:
-        selector: td[data-date]
-        attribute: data-date
-      downloadvolumefactor:
-        case:
-          span[title^="Freeleech:"]: "0"
-          span[title^="Half Freeleech:"]: "0.5"
-          "*": "1"
-      uploadvolumefactor:
-        case:
-          "*": "1"
-      size:
-        selector: td.size
-        remove: span

src/Jackett/Definitions/secretcinema.yml

@@ -6,7 +6,7 @@
   type: private
   encoding: "UTF-8"
   links:
-    - http://www.secret-cinema.net
+    - https://secret-cinema.pw
 
   caps:
     categorymappings:

src/Jackett/Definitions/speedtorrentreloaded.yml

@@ -0,0 +1,122 @@
+---
+  site: speedtorrentreloaded
+  name: SpeedTorrent Reloaded
+  language: de-de
+  type: private
+  encoding: UTF-8
+  links:
+    - https://speedtorrent-tracker.mine.nu/
+
+  caps:
+    categorymappings:
+      - {id: 201, cat: Movies/HD, desc: "Filme: HD 720P / 1080P"}
+      - {id: 103, cat: Movies/DVD, desc: "Filme: DVD"}
+      - {id: 104, cat: Movies/DVD, desc: "Filme: HD2DVD"}
+      - {id: 137, cat: Audio/Video, desc: "Musik: Video"}
+      - {id: 106, cat: Movies/SD, desc: "Filme: DVDRip"}
+      - {id: 107, cat: Movies/SD, desc: "Filme: BDRip "}
+      - {id: 108, cat: Movies/3D, desc: "Filme: 3D"}
+      - {id: 109, cat: Movies/BluRay, desc: "Filme: Blue Ray"}
+      - {id: 111, cat: Movies/SD, desc: "Filme:  SD"}
+      - {id: 112, cat: Movies/Other, desc: "Filme: TV/HDTV"}
+      - {id: 203, cat: XXX, desc: "Erotik: XXX PDF"}
+      - {id: 116, cat: Console, desc: "Spiele: Konsolen"}
+      - {id: 117, cat: PC/Games, desc: "Spiele: Windows / Mac"}
+      - {id: 126, cat: PC/0day, desc: "Software: Mac / Linux"}
+      - {id: 120, cat: Audio/Other, desc: "Musik: Alben/Sampler "}
+      - {id: 121, cat: TV/SD, desc: "Serien: SD"}
+      - {id: 123, cat: TV/Documentary, desc: "Doku: sonstige"}
+      - {id: 124, cat: Console/Other, desc: "Spiele: sonstige"}
+      - {id: 125, cat: PC/0day, desc: "Software: Windows"}
+      - {id: 129, cat: PC/Phone-Other, desc: "Software: Handy / Navi / Sonst"}
+      - {id: 131, cat: TV/HD, desc: "Serien: HD"}
+      - {id: 132, cat: TV, desc: "Serien: Packs"}
+      - {id: 135, cat: Audio, desc: "Musik: Discographie"}
+      - {id: 138, cat: TV/Documentary, desc: "Doku: HD"}
+      - {id: 139, cat: TV/Documentary, desc: "Doku: x264"}
+      - {id: 141, cat: Audio/Audiobook, desc: "A/Ebook: Hoerbook"}
+      - {id: 142, cat: Books, desc: "A/Ebook: EBooks"}
+      - {id: 143, cat: Books, desc: "sonstige:PDF"}
+      - {id: 144, cat: XXX, desc: "Erotik: XXX Pics"}
+      - {id: 202, cat: TV/Sport, desc: "Sport"}
+      - {id: 204, cat: XXX, desc: "XXX-Games"}
+      - {id: 205, cat: Movies/SD, desc: "International-SD"}
+      - {id: 206, cat: movies/HD, desc: "International-HD"}
+
+    modes:
+      search: [q]
+      tv-search: [q, season, ep]
+      movie-search: [q]
+
+  login:
+    path: takelogin.php
+    method: post
+    inputs:
+      username: "{{ .Config.username }}"
+      password: "{{ .Config.password }}"
+    test:
+      path: browse.php
+
+  search:
+    path: browse.php
+    inputs:
+      $raw: "{{range .Categories}}c{{.}}=1&{{end}}"
+      search: "{{ .Query.Keywords }}"
+      incldead: "1"
+      orderby: "added"
+      sort: desc
+    rows:
+      selector: table.tableinborder > tbody > tr > td > table.tableinborder > tbody > tr:has(a[href^="details.php"])
+    fields:
+      title:
+        selector: a[href^="details.php"]
+      banner:
+        selector: a[href^="details.php"][onmouseover]
+        attribute: onmouseover
+        filters:
+          - name: regexp
+            args: "<img src=(.*)>')"
+      category:
+        selector: a[href^="browse.php?cat="]
+        attribute: href
+        filters:
+          - name: querystring
+            args: cat
+      details:
+        selector: a[href^="details.php"]
+        attribute: href
+      comments:
+        selector: a[href*="&tocomm="]
+        attribute: href
+      download:
+        selector: a[href^="download.php"]
+        attribute: href
+      files:
+        selector: td:nth-child(2) > table > tbody > tr:nth-child(2) > td:nth-child(1) > b:nth-child(2)
+      grabs:
+        selector: td:nth-child(2) > table > tbody > tr:nth-child(2) > td:nth-child(3) > b:nth-child(1)
+      size:
+        selector: td:nth-child(2) > table > tbody > tr:nth-child(2) > td:nth-child(1) > b:nth-child(1)
+        filters:
+          - name: replace
+            args: [".", ""]
+          - name: replace
+            args: [",", "."]
+      seeders:
+        selector: td:nth-child(2) > table > tbody > tr:nth-child(2) > td:nth-child(2) > b:nth-child(1)
+      leechers:
+        selector: td:nth-child(2) > table > tbody > tr:nth-child(2) > td:nth-child(2) > b:nth-child(3)
+      date:
+        selector: td:nth-child(2) > table > tbody > tr:nth-child(2) > td:nth-child(5)
+        filters:
+          - name: append
+            args: " +2:00"
+          - name: dateparse
+            args: "02.01.2006 15:04:05 -07:00"
+      downloadvolumefactor:
+        case:
+          img[title="OnlyUp"]: "0"
+          "*": "1"
+      uploadvolumefactor:
+        case:
+          "*": "1"

src/Jackett/Definitions/t411v2.yml

@@ -2,14 +2,21 @@
   site: t411v2
   name: t411 v2
   language: fr-fr
-  type: semi-private
+  type: public
   encoding: UTF-8
   links:
     - https://t411.si
   caps:
     categorymappings:
-      - {id: 1, cat: Movies}
-      - {id: 2, cat: TV}
+      - {id: 1, cat: Movies, desc: Films}
+      - {id: 2, cat: TV, desc: Séries}
+      - {id: 3, cat: TV/Anime, desc: Animes}
+      - {id: 4, cat: Audio, desc: Musique}
+      - {id: 5, cat: Books, desc: Ebooks}
+      - {id: 6, cat: PC/0day, desc: Logiciels}
+      - {id: 7, cat: PC/Games, desc: Jeux}
+      - {id: 8, cat: TV/Documentary, desc: Documentaires}
+      - {id: 9, cat: XXX, desc: XXX}
     modes:
       search: [q]
       tv-search: [q, season, ep]
@@ -21,17 +28,23 @@
   search:
     path: /torrents/search/?search={{ .Keywords}}
     rows:
-      selector: div.isItem
+      selector: tr.isItem.isItemDesk
     fields:
+     category:
+        selector: td.m-cat > a
+        attribute: href
+        filters:
+          - name: querystring
+            args: category
      site_date:
-        selector: div.gname a
+        selector: td.m-name > a
         filters:
           # date is at the end of the title, so we get it and name it 
           # site_date
           - name: regexp
             args: "(\\w+)$"
      title:
-        selector: div.gname a
+        selector: td.m-name > a
         filters:
           # now we put the date at the right place according scene 
           # naming rules using .Result.site_date
@@ -45,13 +58,13 @@
           - name: re_replace
             args: ["(\\w+)$", ""]
      details:
-        selector: div.gname a
+        selector: td.m-name > a
         attribute: href
      download:
-        selector: div.gname a
+        selector: td.m-name > a
         attribute: href
      size:
-        selector: div.gsmall:nth-child(5) span
+        selector: td.m-taille > span
         filters:
           - name: re_replace
             args: [ "\\.(\\w+) K", "$1X00"]
@@ -68,19 +81,19 @@
      seeders:
         text: 0
      seeders:
-        selector: div.gsmall:nth-child(6)
+        selector: td.m-seeders > span
         optional: true
      leechers:
         text: 0
      leechers:
-        selector: div.gsmall:nth-child(7)
+        selector: td.m-leechers > span
         optional: true
      downloadvolumefactor:
         text: "0"
      uploadvolumefactor:
         text: "1"
      date:
-        selector: div.gsmall:nth-child(4) span
+        selector: td.m-age > span
         filters:
           - name: replace
             args: [ " jours", " days"]

src/Jackett/Definitions/tasmanit.yml

@@ -71,6 +71,9 @@
       keywords: "{{ .Keywords }}"
       search_type: "t_name"
       include_dead_torrents: "yes"
+    keywordsfilters: 
+      - name: re_replace
+        args: ["[^a-zA-Z0-9]+", "%"]
     rows:
       selector: table#sortabletable > tbody > tr:has(a[href])
     fields:

src/Jackett/Definitions/thepiratebay.yml

@@ -90,6 +90,9 @@
             args: [ "/", -1 ]
       title:
         selector: .detLink
+        filters:
+          - name: replace
+            args: ["\u000f", ""] # get rid of unwanted character (Example: at the end of https://thepiratebay.org/torrent/18316540/Game.of.Thrones.S07E04.iNTERNAL.1080p.WEBRip.x264-MOROSE_)
       details:
         selector: .detLink
         attribute: href

src/Jackett/Definitions/thetorrents.yml

@@ -65,8 +65,9 @@
       tv-search: [q, season, ep]
 
   login:
-    path: /takelogin.php
-    method: post
+    path: /login.php
+    method: form
+    form: form[action="takelogin.php"]
     inputs:
       username: "{{ .Config.username }}"
       password: "{{ .Config.password }}"
@@ -102,7 +103,7 @@
           - name: querystring
             args: cat
       download:
-        selector: a[href^="/download.php/"]
+        selector: a[href^="/download.php"]
         attribute: href
       description:
         selector: td:nth-child(2) > small

src/Jackett/Definitions/torlock.yml

@@ -0,0 +1,79 @@
+---
+  site: torlock
+  name: Torlock
+  language: en-us
+  type: public
+  encoding: UTF-8
+  links:
+    - https://www.torlock.com/
+
+  caps:
+    categorymappings:
+      - {id: TELEVISION, cat: TV, desc: "TV Shows"}
+      - {id: MOVIES, cat: Movies, desc: "Movies"}
+      - {id: MUSIC, cat: Audio, desc: "Music"}
+      - {id: GAMES, cat: PC/Games, desc: "Games"}
+      - {id: SOFTWARE, cat: PC, desc: "Software"}
+      - {id: ANIME, cat: TV/Anime, desc: "Anime"}
+      - {id: EBOOKS, cat: Books/Ebook, desc: "Books"}
+      - {id: OTHER, cat: Other, desc: "Other"}
+      - {id: ADULT, cat: XXX, desc: "Adult"}
+      - {id: AUDIOBOOK, cat: Audio/Audiobook, desc: "Audiobook"}
+      - {id: IMAGES, cat: Other/Misc, desc: "Images"}
+      
+    modes:
+      search: [q]
+  
+  settings: []
+  
+  search:
+    path: "{{if .Keywords}}/all/torrents/{{ .Keywords }}.html{{else}}/fresh.html{{end}}"
+    keywordsfilters:
+      - name: tolower
+      - name: re_replace
+        args: ["[^a-zA-Z0-9]+", "-"]
+    rows:
+      selector: table > tbody > tr:has(td:has(div:has(a[href^="/torrent/"])))
+    fields:
+      title:
+        selector: td:nth-child(1) > div > a
+      details:
+        selector: td:nth-child(1) > div > a[href^="/torrent/"]
+        attribute: href
+      download:
+        selector: td:nth-child(1) > div > a[href^="/torrent/"]
+        attribute: href
+        filters:
+        - name: replace
+          args: ["/torrent/", "/tor/"]
+        - name: regexp
+          args: (^/tor/\d*)
+        - name: append
+          args: ".torrent"
+      date:
+        selector: td:nth-child(2)
+      size:
+        selector: td:nth-child(3)
+      seeders:
+        selector: td:nth-child(4)
+      leechers:
+        selector: td:nth-child(5)
+      category:
+        selector: span[class^="tv"]
+        attribute: class
+        case:
+          span.tv0: OTHER
+          span.tv1: MOVIES
+          span.tv2: MUSIC
+          span.tv3: TELEVISION
+          span.tv4: GAMES
+          span.tv5: SOFTWARE
+          span.tv6: ANIME
+          span.tv7: ADULT
+          span.tv8: EBOOKS
+          span.tv9: IMAGES
+          span.tv12: AUDIOBOOK
+      downloadvolumefactor:
+        text: "1"
+      uploadvolumefactor:
+        text: "1"

src/Jackett/Definitions/trancetraffic.yml

@@ -7,7 +7,7 @@
   links:
     - https://www.trancetraffic.com
   certificates:
-    - D40789207A75EA36B02E255BF7162C8DF9637750 # expired 13.03.2017
+    - 117B89D7C086F3E051F0A5A3576504667402AE52
 
   caps:
     categories:

src/Jackett/Definitions/waffles.yml

@@ -110,9 +110,6 @@
     test:
       path: index.php
 
-  download:
-    selector: a[href^="download.php?id="]
-        
   search:
     path: browse.php
     inputs:
@@ -130,11 +127,8 @@
       title:
         selector: td:nth-child(2)
       download:
-        selector: a[href^="/details.php?id="]
+        selector: a[href^="/download.php/"]
         attribute: href
-        filters:
-          - name: replace
-            args: ["/details.php?id=", "/download.php?id="]
       details:
         selector: a[href^="/details.php?id="]
         attribute: href

src/Jackett/Definitions/yggtorrent.yml

@@ -43,6 +43,10 @@
     - name: password
       type: password
       label: Password
+    - name: filter_title
+      type: checkbox
+      label: Try to normalize releases names by moving year after the title
+      default: false
   login:
     path: "/user/login"
     method: post
@@ -93,8 +97,23 @@
             args: [ " an", " year"]
           - name: append
             args: " ago"
-      title:
+      title_normal:
         selector: "a.torrent-name"
+      title_filtered:
+        selector: "a.torrent-name"
+        filters:
+          - name: re_replace
+            args: ["(?i)^(?:(.+?)((?:[\\.\\-\\s_\\[]+(?:imax|(?:dvd|bd|tv)(?:rip|scr)|bluray(?:\\-?rip)?|720\\s*p?|1080\\s*p?|vof?|vost(?:fr)?|multi|vf(?:f|q)?[1-3]?|(?:true)?french|eng?)[\\.\\-\\s_\\]]*)*)([\\(\\[]?(?:20|1[7-9])\\d{2}[\\)\\]]?)(.*)$|(.*))$", "$1 $3 $2 $4 $5"]
+          - name: replace
+            args: [".", " "]
+          - name: trim
+          - name: re_replace
+            args: ["(?i)\\s(mkv|avi|divx|xvid|mp4)$", ""]
+          - name: re_replace
+            args: ["(\\s{2,5})", " "]
+          - name: trim
+      title:
+        text: "{{if .Config.filter_title }}{{ .Result.title_filtered }}{{else}}{{ .Result.title_normal }}{{end}}"
       details:
         selector: "a.torrent-name"
         attribute: href

src/Jackett/Definitions/zetorrents.yml

@@ -24,7 +24,7 @@
   search:
     path: "/recherche/{{ .Query.Keywords }}"
     rows:
-      selector: div.content-list-torrent > table tbody tr
+      selector: div.content-list-torrent > table tbody tr:has(a)
     fields:
       site_date:
         selector: td:nth-child(1) a
@@ -85,4 +85,4 @@
       downloadvolumefactor:
         text: "0"
       uploadvolumefactor:
-        text: "1"
+        text: "1"

src/Jackett/IndexerException.cs

@@ -0,0 +1,30 @@
+using Jackett.Indexers;
+using System;
+using System.Collections.Generic;
+using System.Linq;
+using System.Text;
+using System.Threading.Tasks;
+
+namespace Jackett
+{
+    class IndexerException : Exception
+    {
+        public IIndexer Indexer { get; protected set; }
+
+        public IndexerException(IIndexer Indexer, string message, Exception innerException)
+            : base(message, innerException)
+        {
+            this.Indexer = Indexer;
+        }
+
+        public IndexerException(IIndexer Indexer, string message)
+            : this(Indexer, message, null)
+        {
+        }
+
+        public IndexerException(IIndexer Indexer, Exception innerException)
+            : this(Indexer, "Exception (" + Indexer.ID + "): " + innerException.Message, innerException)
+        {
+        }
+    }
+}

src/Jackett/Indexers/Abstract/GazelleTracker.cs

@@ -228,7 +228,8 @@ namespace Jackett.Indexers.Abstract
             {
                 release.DownloadVolumeFactor = 0;
             }
-            if ((bool)torrent["isPersonalFreeleech"])
+            var isPersonalFreeleech = (bool?)torrent["isPersonalFreeleech"];
+            if (isPersonalFreeleech != null && isPersonalFreeleech == true)
             {
                 release.DownloadVolumeFactor = 0;
             }

src/Jackett/Indexers/BaseIndexer.cs

@@ -219,23 +219,30 @@ namespace Jackett.Indexers
 
         public abstract Task<IndexerConfigurationStatus> ApplyConfiguration(JToken configJson);
 
-        public virtual async Task<IEnumerable<ReleaseInfo>> ResultsForQuery(TorznabQuery query)
+        public virtual async Task<IndexerResult> ResultsForQuery(TorznabQuery query)
         {
-            if (!CanHandleQuery(query))
-                return new ReleaseInfo[0];
-            var results = await PerformQuery(query);
-            results = FilterResults(query, results);
-            results = results.Select(r =>
+            try
             {
-                r.Origin = this;
+                if (!CanHandleQuery(query))
+                    return new IndexerResult(this, new ReleaseInfo[0]);
+                var results = await PerformQuery(query);
+                results = FilterResults(query, results);
+                results = results.Select(r =>
+                {
+                    r.Origin = this;
 
-                // Some trackers do not keep their clocks up to date and can be ~20 minutes out!
-                if (r.PublishDate > DateTime.Now)
-                    r.PublishDate = DateTime.Now;
-                return r;
-            });
+                    // Some trackers do not keep their clocks up to date and can be ~20 minutes out!
+                    if (r.PublishDate > DateTime.Now)
+                        r.PublishDate = DateTime.Now;
+                    return r;
+                });
 
-            return results;
+                return new IndexerResult(this, results);
+            }
+            catch (Exception ex)
+            {
+                throw new IndexerException(this, ex);
+            }
         }
         protected abstract Task<IEnumerable<ReleaseInfo>> PerformQuery(TorznabQuery query);
     }
@@ -667,12 +674,12 @@ namespace Jackett.Indexers
             return releases;
         }
 
-        public override async Task<IEnumerable<ReleaseInfo>> ResultsForQuery(TorznabQuery query)
+        public override async Task<IndexerResult> ResultsForQuery(TorznabQuery query)
         {
-            var results = await base.ResultsForQuery(query);
-            results = CleanLinks(results);
+            var result = await base.ResultsForQuery(query);
+            result.Releases = CleanLinks(result.Releases);
 
-            return results;
+            return result;
         }
 
         protected virtual Uri UncleanLink(Uri link)

src/Jackett/Indexers/BestFriends.cs

@@ -1,234 +0,0 @@
-using Jackett.Utils.Clients;
-using NLog;
-using Jackett.Services;
-using Jackett.Utils;
-using Jackett.Models;
-using System.Threading.Tasks;
-using Newtonsoft.Json.Linq;
-using System.Collections.Generic;
-using CsQuery;
-using System;
-using System.Globalization;
-using Jackett.Models.IndexerConfig;
-using System.Collections.Specialized;
-using System.Text;
-
-namespace Jackett.Indexers
-{
-    public class BestFriends : BaseWebIndexer
-    {
-        string LoginUrl { get { return SiteLink + "login.php"; } }
-        string TakeLoginUrl { get { return SiteLink + "takelogin.php"; } }
-        string BrowseUrl { get { return SiteLink + "browse.php"; } }
-
-        new ConfigurationDataCaptchaLogin configData
-        {
-            get { return (ConfigurationDataCaptchaLogin)base.configData; }
-            set { base.configData = value; }
-        }
-
-        public BestFriends(IIndexerConfigurationService configService, IWebClient wc, Logger l, IProtectionService ps)
-            : base(name: "Best Friends",
-                   description: "A German general tracker.",
-                   link: "http://bf.mine.nu/",
-                   caps: TorznabUtil.CreateDefaultTorznabTVCaps(),
-                   configService: configService,
-                   client: wc,
-                   logger: l,
-                   p: ps,
-                   configData: new ConfigurationDataCaptchaLogin())
-        {
-            Encoding = Encoding.GetEncoding("iso-8859-1");
-            Language = "de-de";
-            Type = "private";
-
-            AddCategoryMapping(18, TorznabCatType.TVAnime); // Anime
-            AddCategoryMapping(8,  TorznabCatType.PCMac); // Appz MAC
-            AddCategoryMapping(9,  TorznabCatType.PC); // Appz other
-            AddCategoryMapping(7,  TorznabCatType.PC); // Appz Windows
-            AddCategoryMapping(23, TorznabCatType.TVDocumentary); // Dokumentationen
-            AddCategoryMapping(32, TorznabCatType.Movies3D); // DVD 3D
-            AddCategoryMapping(15, TorznabCatType.Books); // eBooks
-            AddCategoryMapping(12, TorznabCatType.PCGames); // Games PC
-            AddCategoryMapping(37, TorznabCatType.PCPhoneOther); // Handy_Mobile 
-            AddCategoryMapping(24, TorznabCatType.TVHD); // HDTV
-            AddCategoryMapping(22, TorznabCatType.AudioAudiobook); // Hörbücher
-            AddCategoryMapping(1,  TorznabCatType.MoviesHD); // Movies 1080p/1080i
-            AddCategoryMapping(31, TorznabCatType.Movies3D); // Movies 3D
-            AddCategoryMapping(2,  TorznabCatType.MoviesHD); // Movies 720p/720i
-            AddCategoryMapping(21, TorznabCatType.MoviesBluRay); // Movies BluRay
-            AddCategoryMapping(5,  TorznabCatType.MoviesDVD); // Movies DVD/HDDVD
-            AddCategoryMapping(6,  TorznabCatType.MoviesSD); // Movies M/SVCD/Other
-            AddCategoryMapping(4,  TorznabCatType.MoviesSD); // Movies XVID/DIVX/h.264
-            AddCategoryMapping(10, TorznabCatType.Audio); // Music
-            AddCategoryMapping(25, TorznabCatType.AudioVideo); // Musikvideo
-            AddCategoryMapping(29, TorznabCatType.ConsoleNDS); // Nintendo DS
-            AddCategoryMapping(16, TorznabCatType.Other); // other
-            AddCategoryMapping(13, TorznabCatType.ConsolePS4); // Playstation
-            AddCategoryMapping(28, TorznabCatType.TVHD); // Serien HD
-            AddCategoryMapping(11, TorznabCatType.TVSD); // Serien XviD
-            AddCategoryMapping(33, TorznabCatType.Other); // Specials
-            AddCategoryMapping(30, TorznabCatType.TVSport); // Sport
-            AddCategoryMapping(19, TorznabCatType.TVOTHER); // TVRip
-            AddCategoryMapping(38, TorznabCatType.TVDocumentary); // US Dokus
-            AddCategoryMapping(20, TorznabCatType.MoviesForeign); // US Movies
-            AddCategoryMapping(14, TorznabCatType.TVFOREIGN); // US Serien
-            AddCategoryMapping(36, TorznabCatType.Other); // Wallpaper 
-            AddCategoryMapping(26, TorznabCatType.ConsoleWii); // Wii
-            AddCategoryMapping(27, TorznabCatType.ConsoleXbox360); // Xbox 360
-            AddCategoryMapping(3,  TorznabCatType.XXX); // XXX
-        }
-
-        public override async Task<ConfigurationData> GetConfigurationForSetup()
-        {
-            var loginPage = await RequestStringWithCookies(LoginUrl, string.Empty);
-            CQ dom = loginPage.Content;
-            CQ qCaptchaImg = dom.Find("td.tablea > img").First();
-
-            var CaptchaUrl = SiteLink + qCaptchaImg.Attr("src");
-            var captchaImage = await RequestBytesWithCookies(CaptchaUrl, loginPage.Cookies);
-            configData.CaptchaImage.Value = captchaImage.Content;
-            configData.CaptchaCookie.Value = loginPage.Cookies;
-            return configData;
-        }
-
-        public override async Task<IndexerConfigurationStatus> ApplyConfiguration(JToken configJson)
-        {
-            LoadValuesFromJson(configJson);
-
-            var pairs1 = new Dictionary<string, string>
-            {
-                { "proofcode", configData.CaptchaText.Value }
-            };
-            var cookies = configData.CaptchaCookie.Value;
-            var result1 = await RequestLoginAndFollowRedirect(LoginUrl, pairs1, cookies, true, null, LoginUrl, true);
-            if(result1.Content == null || !result1.Content.Contains("takelogin.php"))
-            {
-                CQ dom = result1.Content;
-                var errorMessage = dom["#login_error"].Text().Trim();
-                errorMessage = result1.Content;
-                throw new ExceptionWithConfigData(errorMessage, configData);
-            }
-
-            var pairs2 = new Dictionary<string, string>
-            {
-                { "username", configData.Username.Value },
-                { "password", configData.Password.Value }
-            };
-
-            var result = await RequestLoginAndFollowRedirect(TakeLoginUrl, pairs2, result1.Cookies, true, null, LoginUrl, true);
-            await ConfigureIfOK(result.Cookies, result.Content != null && result.Content.Contains("logout.php"), () =>
-                {
-                    CQ dom = result.Content;
-                    var errorMessage = dom["#login_error"].Text().Trim();
-                    errorMessage = result.Content;
-                    throw new ExceptionWithConfigData(errorMessage, configData);
-                });
-            return IndexerConfigurationStatus.RequiresTesting;
-        }
-
-        protected override async Task<IEnumerable<ReleaseInfo>> PerformQuery(TorznabQuery query)
-        {
-            TimeZoneInfo.TransitionTime startTransition = TimeZoneInfo.TransitionTime.CreateFloatingDateRule(new DateTime(1, 1, 1, 3, 0, 0), 3, 5, DayOfWeek.Sunday);
-            TimeZoneInfo.TransitionTime endTransition = TimeZoneInfo.TransitionTime.CreateFloatingDateRule(new DateTime(1, 1, 1, 4, 0, 0), 10, 5, DayOfWeek.Sunday);
-            TimeSpan delta = new TimeSpan(1, 0, 0);
-            TimeZoneInfo.AdjustmentRule adjustment = TimeZoneInfo.AdjustmentRule.CreateAdjustmentRule(new DateTime(1999, 10, 1), DateTime.MaxValue.Date, delta, startTransition, endTransition);
-            TimeZoneInfo.AdjustmentRule[] adjustments = { adjustment };
-            TimeZoneInfo germanyTz = TimeZoneInfo.CreateCustomTimeZone("W. Europe Standard Time", new TimeSpan(1, 0, 0), "(GMT+01:00) W. Europe Standard Time", "W. Europe Standard Time", "W. Europe DST Time", adjustments);
-
-            var releases = new List<ReleaseInfo>();
-            
-            var searchString = query.GetQueryString();
-            var searchUrl = BrowseUrl;
-            var queryCollection = new NameValueCollection();
-            queryCollection.Add("showsearch", "1");
-            queryCollection.Add("incldead", "1");
-            queryCollection.Add("blah", "0");
-            queryCollection.Add("orderby", "added");
-            queryCollection.Add("sort", "desc");
-
-            if (!string.IsNullOrWhiteSpace(searchString))
-            {
-                queryCollection.Add("search", searchString);
-            }
-
-            foreach (var cat in MapTorznabCapsToTrackers(query))
-            {
-                queryCollection.Add("c" + cat, "1");
-            }
-            searchUrl += "?" + queryCollection.GetQueryString();
-
-            var response = await RequestStringWithCookiesAndRetry(searchUrl, null, BrowseUrl);
-            var results = response.Content;
-            try
-            {
-                CQ dom = results;
-                var rows = dom["table.tableinborder > tbody > tr:has(td.tableb)"];
-
-                foreach (var row in rows)
-                {
-                    var release = new ReleaseInfo();
-                    release.MinimumRatio = 0.75;
-                    release.MinimumSeedTime = 0;
-                    var qRow = row.Cq();
-
-                    var qDetailsLink = qRow.Find("a[href^=details.php?id=]").First();
-                    release.Title = qDetailsLink.Attr("title");
-
-                    if (!query.MatchQueryStringAND(release.Title))
-                        continue;
-
-                    var qCatLink = qRow.Find("a[href^=browse.php?cat=]").First();
-
-                    // use negative indexes as if a user has "Wartezeit" there's an extra column after the title
-                    var qSeeders = qRow.Find("td:nth-last-child(4)");
-                    var qLeechers = qRow.Find("td:nth-last-child(3)");
-                    var qDateStr = qRow.Find("td:nth-last-child(7)");
-                    var qSize = qRow.Find("td:nth-last-child(6)");
-
-                    var torrentId = qDetailsLink.Attr("href").Replace("&hit=1", "").Split('=')[1];
-
-                    var catStr = qCatLink.Attr("href").Split('=')[1];
-                    release.Category = MapTrackerCatToNewznab(catStr);
-
-                    release.Link = new Uri(SiteLink + "download.php?torrent="+torrentId);
-                    release.Comments = new Uri(SiteLink + qDetailsLink.Attr("href"));
-                    release.Guid = release.Link;
-
-                    var sizeStr = qSize.Text();
-                    release.Size = ReleaseInfo.GetBytes(sizeStr.Replace(",", "."));
-
-                    release.Seeders = ParseUtil.CoerceInt(qSeeders.Text());
-                    release.Peers = ParseUtil.CoerceInt(qLeechers.Text()) + release.Seeders;
-
-                    var dateStr = qDateStr.Text();
-                    var dateGerman = DateTime.SpecifyKind(DateTime.ParseExact(dateStr, "dd.MM.yyyyHH:mm:ss", CultureInfo.InvariantCulture), DateTimeKind.Unspecified);
-                    DateTime pubDateUtc = TimeZoneInfo.ConvertTimeToUtc(dateGerman, germanyTz);
-                    release.PublishDate = pubDateUtc;
-
-                    var files = qRow.Find("td:nth-last-child(9)").Text();
-                    release.Files = ParseUtil.CoerceInt(files);
-
-                    var grabs = qRow.Find("td:nth-last-child(5)").Text();
-                    release.Grabs = ParseUtil.CoerceInt(grabs);
-
-                    if (qRow.Find("font[color=\"red\"]:contains(OnlyUp)").Length >= 1)
-                        release.DownloadVolumeFactor = 0;
-                    else
-                        release.DownloadVolumeFactor = 1;
-
-                    release.UploadVolumeFactor = 1;
-
-                    releases.Add(release);
-                }
-            }
-            catch (Exception ex)
-            {
-                OnParseError(results, ex);
-            }
-
-            return releases;
-        }
-    }
-}
-

src/Jackett/Indexers/BroadcastTheNet.cs

@@ -1,27 +1,22 @@
-using CsQuery;
+using System;
+using System.Collections.Generic;
+using System.Linq;
+using System.Text;
+using System.Threading.Tasks;
 using Jackett.Models;
+using Jackett.Models.IndexerConfig;
 using Jackett.Services;
 using Jackett.Utils;
 using Jackett.Utils.Clients;
+using Newtonsoft.Json;
 using Newtonsoft.Json.Linq;
 using NLog;
-using System;
-using System.Collections.Generic;
-using System.Globalization;
-using System.Linq;
-using System.Net;
-using System.Net.Http;
-using System.Text;
-using System.Threading.Tasks;
-using System.Web;
-using Jackett.Models.IndexerConfig;
-using System.Dynamic;
-using Newtonsoft.Json;
 
 namespace Jackett.Indexers
 {
     public class BroadcastTheNet : BaseWebIndexer
     {
+        // Docs at http://apidocs.broadcasthe.net/docs.php
         string APIBASE = "https://api.broadcasthe.net";
 
         new ConfigurationDataAPIKey configData
@@ -32,9 +27,9 @@ namespace Jackett.Indexers
 
         public BroadcastTheNet(IIndexerConfigurationService configService, IWebClient wc, Logger l, IProtectionService ps)
             : base(name: "BroadcastTheNet",
-                description: "Needs no description..",
+                description: null,
                 link: "https://broadcasthe.net/",
-                caps: TorznabUtil.CreateDefaultTorznabTVCaps(),
+                caps: new TorznabCapabilities(),
                 configService: configService,
                 client: wc,
                 logger: l,
@@ -44,6 +39,16 @@ namespace Jackett.Indexers
             Encoding = Encoding.UTF8;
             Language = "en-us";
             Type = "private";
+
+            TorznabCaps.LimitsDefault = 100;
+            TorznabCaps.LimitsMax = 1000;
+
+            AddCategoryMapping("SD", TorznabCatType.TVSD, "SD");
+            AddCategoryMapping("720p", TorznabCatType.TVHD, "720p");
+            AddCategoryMapping("1080p", TorznabCatType.TVHD, "1080p");
+            AddCategoryMapping("1080i", TorznabCatType.TVHD, "1080i");
+            AddCategoryMapping("2160p", TorznabCatType.TVHD, "2160p");
+            AddCategoryMapping("Portable Device", TorznabCatType.TVSD, "Portable Device");
         }
 
         public override async Task<IndexerConfigurationStatus> ApplyConfiguration(JToken configJson)
@@ -59,7 +64,7 @@ namespace Jackett.Indexers
                 IsConfigured = true;
                 SaveConfig();
             }
-            catch(Exception e)
+            catch (Exception e)
             {
                 throw new ExceptionWithConfigData(e.Message, configData);
             }
@@ -81,34 +86,40 @@ namespace Jackett.Indexers
         protected override async Task<IEnumerable<ReleaseInfo>> PerformQuery(TorznabQuery query)
         {
             var searchString = query.GetQueryString();
+            var btnResults = query.Limit;
+            if (btnResults == 0)
+                btnResults = (int)TorznabCaps.LimitsDefault;
+            var btnOffset = query.Offset;
             var releases = new List<ReleaseInfo>();
 
             var parameters = new JArray();
             parameters.Add(new JValue(configData.Key.Value));
             parameters.Add(new JValue(searchString.Trim()));
-            parameters.Add(new JValue(100));
-            parameters.Add(new JValue(0));
+            parameters.Add(new JValue(btnResults));
+            parameters.Add(new JValue(btnOffset));
             var response = await PostDataWithCookiesAndRetry(APIBASE, null, null, null, new Dictionary<string, string>()
             {
                 { "Accept", "application/json-rpc, application/json"},
                 {"Content-Type", "application/json-rpc"}
-            }, JsonRPCRequest("getTorrents", parameters),false);
+            }, JsonRPCRequest("getTorrents", parameters), false);
 
             try
             {
                 var btnResponse = JsonConvert.DeserializeObject<BTNRPCResponse>(response.Content);
 
-                if (btnResponse != null && btnResponse.Result != null)
+                if (btnResponse != null && btnResponse.Result != null && btnResponse.Result.Torrents != null)
                 {
                     foreach (var itemKey in btnResponse.Result.Torrents)
                     {
+                        var descriptions = new List<string>();
                         var btnResult = itemKey.Value;
                         var item = new ReleaseInfo();
                         if (!string.IsNullOrEmpty(btnResult.SeriesBanner))
                             item.BannerUrl = new Uri(btnResult.SeriesBanner);
-                        item.Category = new List<int> { TorznabCatType.TV.ID };
-                        item.Comments = new Uri($"https://broadcasthe.net/torrents.php?id={btnResult.GroupID}&torrentid={btnResult.TorrentID}");
-                        item.Description = btnResult.ReleaseName;
+                        item.Category = MapTrackerCatToNewznab(btnResult.Resolution);
+                        if (item.Category.Count == 0) // default to TV
+                            item.Category.Add(TorznabCatType.TV.ID);
+                        item.Comments = new Uri($"{SiteLink}torrents.php?id={btnResult.GroupID}&torrentid={btnResult.TorrentID}");
                         item.Guid = new Uri(btnResult.DownloadURL);
                         if (!string.IsNullOrWhiteSpace(btnResult.ImdbID))
                             item.Imdb = ParseUtil.CoerceLong(btnResult.ImdbID);
@@ -121,6 +132,29 @@ namespace Jackett.Indexers
                         item.Size = btnResult.Size;
                         item.TVDBId = btnResult.TvdbID;
                         item.Title = btnResult.ReleaseName;
+                        item.UploadVolumeFactor = 1;
+                        item.DownloadVolumeFactor = 0; // ratioless
+                        item.Grabs = btnResult.Snatched;
+
+                        if (!string.IsNullOrWhiteSpace(btnResult.Series))
+                            descriptions.Add("Series: " + btnResult.Series);
+                        if (!string.IsNullOrWhiteSpace(btnResult.GroupName))
+                            descriptions.Add("Group Name: " + btnResult.GroupName);
+                        if (!string.IsNullOrWhiteSpace(btnResult.Source))
+                            descriptions.Add("Source: " + btnResult.Source);
+                        if (!string.IsNullOrWhiteSpace(btnResult.Container))
+                            descriptions.Add("Container: " + btnResult.Container);
+                        if (!string.IsNullOrWhiteSpace(btnResult.Codec))
+                            descriptions.Add("Codec: " + btnResult.Codec);
+                        if (!string.IsNullOrWhiteSpace(btnResult.Resolution))
+                            descriptions.Add("Resolution: " + btnResult.Resolution);
+                        if (!string.IsNullOrWhiteSpace(btnResult.Origin))
+                            descriptions.Add("Origin: " + btnResult.Origin);
+                        if (!string.IsNullOrWhiteSpace(btnResult.Series))
+                            descriptions.Add("Youtube Trailer: <a href=\"" + btnResult.YoutubeTrailer + "\">" + btnResult.YoutubeTrailer + "</a>");
+
+                        item.Description = string.Join("<br />\n", descriptions);
+
                         releases.Add(item);
                     }
                 }

src/Jackett/Indexers/Demonoid.cs

@@ -106,6 +106,10 @@ namespace Jackett.Indexers
                     {
                         // ex: "Monday, Jun 01, 2015", "Monday, Aug 03, 2015"
                         var dateStr = rowA.Cq().Text().Trim().Replace("Added on ", "");
+                        if (string.IsNullOrWhiteSpace(dateStr) || dateStr == "Sponsored links" || dateStr.StartsWith("!function")) // ignore ads
+                        { 
+                            continue;
+                        }
                         if (dateStr.ToLowerInvariant().Contains("today"))
                             lastDateTime = DateTime.Now;
                         else
@@ -145,7 +149,7 @@ namespace Jackett.Indexers
                     release.Guid = release.Comments;
 
                     var qDownload = rowB.ChildElements.ElementAt(2).ChildElements.ElementAt(0).Cq();
-                    release.Link = new Uri(SiteLink + qDownload.Attr("href"));
+                    release.Link = new Uri(qDownload.Attr("href"));
 
                     var sizeStr = rowB.ChildElements.ElementAt(3).Cq().Text();
                     release.Size = ReleaseInfo.GetBytes(sizeStr);

src/Jackett/Indexers/GhostCity.cs

@@ -134,6 +134,12 @@ namespace Jackett.Indexers
 
             var results = await RequestStringWithCookiesAndRetry(searchUrl);
 
+            if (results.Content.Contains("<meta http-equiv=\"refresh\"")) // relogin needed?
+            {
+                await ApplyConfiguration(null);
+                results = await RequestStringWithCookiesAndRetry(searchUrl);
+            }
+
             try
             {
                 CQ dom = results.Content;

src/Jackett/Indexers/Hardbay.cs

@@ -90,50 +90,53 @@ namespace Jackett.Indexers
             {
                 //var json = JArray.Parse(results.Content);
                 dynamic json = JsonConvert.DeserializeObject<dynamic>(results.Content);
-                foreach (var row in json)
-                {
-                    var release = new ReleaseInfo();
-                    var descriptions = new List<string>();
-                    var tags = new List<string>();
-
-                    release.MinimumRatio = 0.5;
-                    release.MinimumSeedTime = 0;
-                    release.Title = row.name;
-                    release.Category = new List<int> { TorznabCatType.Audio.ID };
-                    release.Size = row.size;
-                    release.Seeders = row.seeders;
-                    release.Peers = row.leechers + release.Seeders;
-                    release.PublishDate = DateTime.ParseExact(row.added.ToString() + " +01:00", "yyyy-MM-dd HH:mm:ss zzz", CultureInfo.InvariantCulture);
-                    release.Files = row.numfiles;
-                    release.Grabs = row.times_completed;
-
-                    release.Comments = new Uri(SiteLink + "torrent/" + row.id.ToString() + "/");
-                    release.Link = new Uri(SiteLink + "api/v1/torrents/download/" + row.id.ToString());
-
-                    if (row.frileech == 1)
-                        release.DownloadVolumeFactor = 0;
-                    else
-                        release.DownloadVolumeFactor = 0.33;
-                    release.UploadVolumeFactor = 1;
-
-                    if ((int)row.p2p == 1)
-                        tags.Add("P2P");
-                    if ((int)row.pack == 1)
-                        tags.Add("Pack");
-                    if ((int)row.reqid != 0)
-                        tags.Add("Archive");
-                    if ((int)row.flac != 0)
+                if (json != null) // no results
+                { 
+                    foreach (var row in json)
                     {
-                        tags.Add("FLAC");
-                        release.Category = new List<int> { TorznabCatType.AudioLossless.ID };
+                        var release = new ReleaseInfo();
+                        var descriptions = new List<string>();
+                        var tags = new List<string>();
+
+                        release.MinimumRatio = 0.5;
+                        release.MinimumSeedTime = 0;
+                        release.Title = row.name;
+                        release.Category = new List<int> { TorznabCatType.Audio.ID };
+                        release.Size = row.size;
+                        release.Seeders = row.seeders;
+                        release.Peers = row.leechers + release.Seeders;
+                        release.PublishDate = DateTime.ParseExact(row.added.ToString() + " +01:00", "yyyy-MM-dd HH:mm:ss zzz", CultureInfo.InvariantCulture);
+                        release.Files = row.numfiles;
+                        release.Grabs = row.times_completed;
+
+                        release.Comments = new Uri(SiteLink + "torrent/" + row.id.ToString() + "/");
+                        release.Link = new Uri(SiteLink + "api/v1/torrents/download/" + row.id.ToString());
+
+                        if (row.frileech == 1)
+                            release.DownloadVolumeFactor = 0;
+                        else
+                            release.DownloadVolumeFactor = 0.33;
+                        release.UploadVolumeFactor = 1;
+
+                        if ((int)row.p2p == 1)
+                            tags.Add("P2P");
+                        if ((int)row.pack == 1)
+                            tags.Add("Pack");
+                        if ((int)row.reqid != 0)
+                            tags.Add("Archive");
+                        if ((int)row.flac != 0)
+                        {
+                            tags.Add("FLAC");
+                            release.Category = new List<int> { TorznabCatType.AudioLossless.ID };
+                        }
+
+                        if (tags.Count > 0)
+                            descriptions.Add("Tags: " + string.Join(", ", tags));
+
+                        release.Description = string.Join("<br>\n", descriptions);
+
+                        releases.Add(release);
                     }
-
-                    if (tags.Count > 0)
-                        descriptions.Add("Tags: " + string.Join(", ", tags));
-
-                    release.Description = string.Join("<br>\n", descriptions);
-
-                    releases.Add(release);
                 }
             }
             catch (Exception ex)

src/Jackett/Indexers/Hounddawgs.cs

@@ -151,8 +151,11 @@ namespace Jackett.Indexers
                     var cat = catUrl.Substring(catUrl.LastIndexOf('[') + 1).Trim(']');
                     release.Category = MapTrackerCatToNewznab(cat);
 
+                    // support both date format (profile settings)
                     var qAdded = row.ChildElements.ElementAt(4).ChildElements.ElementAt(0).Cq();
                     var addedStr = qAdded.Attr("title");
+                    if (!addedStr.Contains(","))
+                        addedStr = qAdded.Text();
                     release.PublishDate = DateTime.ParseExact(addedStr, "MMM dd yyyy, HH:mm", CultureInfo.InvariantCulture);
 
                     var overlayScript = qRow.Find("script:contains(\"var overlay\")").Text();

src/Jackett/Indexers/IIndexer.cs

@@ -10,6 +10,18 @@ using System.Web.UI.WebControls;
 
 namespace Jackett.Indexers
 {
+    public class IndexerResult
+    {
+        public IIndexer Indexer { get; set; }
+        public IEnumerable<ReleaseInfo> Releases { get; set; }
+
+        public IndexerResult(IIndexer Indexer, IEnumerable<ReleaseInfo> Releases)
+        {
+            this.Indexer = Indexer;
+            this.Releases = Releases;
+        }
+    }
+
     public interface IIndexer
     {
         string SiteLink { get; }
@@ -39,7 +51,7 @@ namespace Jackett.Indexers
 
         void Unconfigure();
 
-        Task<IEnumerable<ReleaseInfo>> ResultsForQuery(TorznabQuery query);
+        Task<IndexerResult> ResultsForQuery(TorznabQuery query);
 
         bool CanHandleQuery(TorznabQuery query);
     }

src/Jackett/Indexers/ImmortalSeed.cs

@@ -115,7 +115,7 @@ namespace Jackett.Indexers
             {
                 CQ dom = results.Content;
 
-                var rows = dom["#sortabletable tr"];
+                var rows = dom["#sortabletable tr:has(a[href*=\"details.php?id=\"])"];
                 foreach (var row in rows.Skip(1))
                 {
                     var release = new ReleaseInfo();

src/Jackett/Indexers/Meta/BaseMetaIndexer.cs

@@ -27,25 +27,32 @@ namespace Jackett.Indexers.Meta
             return Task.FromResult(IndexerConfigurationStatus.Completed);
         }
 
-        public override async Task<IEnumerable<ReleaseInfo>> ResultsForQuery(TorznabQuery query)
+        public override async Task<IndexerResult> ResultsForQuery(TorznabQuery query)
         {
-            if (!CanHandleQuery(query))
-                return new ReleaseInfo[0];
-            var results = await PerformQuery(query);
-            var correctedResults = results.Select(r =>
+            try
             {
-                if (r.PublishDate > DateTime.Now)
-                    r.PublishDate = DateTime.Now;
-                return r;
-            });
+                if (!CanHandleQuery(query))
+                    return new IndexerResult(this, new ReleaseInfo[0]);
+                var results = await PerformQuery(query);
+                var correctedResults = results.Select(r =>
+                {
+                    if (r.PublishDate > DateTime.Now)
+                        r.PublishDate = DateTime.Now;
+                    return r;
+                });
 
-            return correctedResults;
+                return new IndexerResult(this, correctedResults);
+            }
+            catch (Exception ex)
+            {
+                throw new IndexerException(this, ex);
+            }
         }
 
         protected override async Task<IEnumerable<ReleaseInfo>> PerformQuery(TorznabQuery query)
         {
             var indexers = validIndexers;
-            IEnumerable<Task<IEnumerable<ReleaseInfo>>> supportedTasks = indexers.Where(i => i.CanHandleQuery(query)).Select(i => i.ResultsForQuery(query)).ToList(); // explicit conversion to List to execute LINQ query
+            IEnumerable<Task<IndexerResult>> supportedTasks = indexers.Where(i => i.CanHandleQuery(query)).Select(i => i.ResultsForQuery(query)).ToList(); // explicit conversion to List to execute LINQ query
 
             var fallbackStrategies = fallbackStrategyProvider.FallbackStrategiesForQuery(query);
             var fallbackQueries = fallbackStrategies.Select(async f => await f.FallbackQueries()).SelectMany(t => t.Result);
@@ -72,7 +79,7 @@ namespace Jackett.Indexers.Meta
                 logger.Error(aggregateTask.Exception, "Error during request in metaindexer " + ID);
             }
 
-            var unorderedResult = aggregateTask.Result.Flatten();
+            var unorderedResult = aggregateTask.Result.Select(r => r.Releases).Flatten();
             var resultFilters = resultFilterProvider.FiltersForQuery(query);
             var filteredResults = resultFilters.Select(async f => await f.FilterResults(unorderedResult)).SelectMany(t => t.Result);
             var uniqueFilteredResults = filteredResults.Distinct();

src/Jackett/Indexers/Meta/MetaIndexers.cs

@@ -25,5 +25,16 @@ namespace Jackett.Indexers.Meta
             : base("AggregateSearch", "This feed includes all configured trackers", fallbackStrategyProvider, resultFilterProvider, configService, wc, l, new ConfigurationData(), ps, x => true)
         {
         }
+
+        public override TorznabCapabilities TorznabCaps
+        {
+            get
+            {
+                // increase the limits (workaround until proper paging is supported, issue #1661)
+                var caps = base.TorznabCaps;
+                caps.LimitsMax = caps.LimitsDefault = 1000;
+                return caps;
+            }
+        }
     }
 }

src/Jackett/Indexers/NewRealWorld.cs

@@ -29,7 +29,7 @@ namespace Jackett.Indexers
         public NewRealWorld(IIndexerConfigurationService configService, IWebClient wc, Logger l, IProtectionService ps)
             : base(name: "New Real World",
                    description: "A German general tracker.",
-                   link: "http://nrw-tracker.eu/",
+                   link: "https://nrw-tracker.eu/",
                    caps: TorznabUtil.CreateDefaultTorznabTVCaps(),
                    configService: configService,
                    client: wc,
@@ -151,6 +151,13 @@ namespace Jackett.Indexers
             searchUrl += "?" + queryCollection.GetQueryString();
 
             var response = await RequestStringWithCookies(searchUrl);
+            if (response.IsRedirect)
+            {
+                // re-login
+                await ApplyConfiguration(null);
+                response = await RequestStringWithCookies(searchUrl);
+            }
+
             var results = response.Content;
             try
             {

src/Jackett/Indexers/Norbits.cs

@@ -16,6 +16,7 @@ using Newtonsoft.Json;
 using Newtonsoft.Json.Linq;
 using NLog;
 using System.Text;
+using System.IO;
 
 namespace Jackett.Indexers
 {
@@ -33,7 +34,7 @@ namespace Jackett.Indexers
         private bool Latency => ConfigData.Latency.Value;
         private bool DevMode => ConfigData.DevMode.Value;
         private bool CacheMode => ConfigData.HardDriveCache.Value;
-        private static string Directory => System.IO.Path.GetTempPath() + "Jackett\\" + MethodBase.GetCurrentMethod().DeclaringType?.Name + "\\";
+        private static string Directory => Path.Combine(Path.GetTempPath(), "Jackett", MethodBase.GetCurrentMethod().DeclaringType?.Name);
 
         private readonly Dictionary<string, string> _emulatedBrowserHeaders = new Dictionary<string, string>();
         private CQ _fDom;
@@ -49,7 +50,6 @@ namespace Jackett.Indexers
                 client: w,
                 logger: l,
                 p: ps,
-                downloadBase: "https://norbits.net/download.php?id=",
                 configData: new ConfigurationDataNorbits())
         {
             Encoding = Encoding.GetEncoding("iso-8859-1");

src/Jackett/Indexers/SceneAccess.cs

@@ -129,8 +129,7 @@ namespace Jackett.Indexers
 
                     release.MinimumRatio = 1;
                     release.MinimumSeedTime = 129600;
-                    release.Title = qRow.Find(".ttr_name > a").Text();
-                    release.Description = release.Title;
+                    release.Title = qRow.Find(".ttr_name > a").Attr("title");
                     release.Guid = new Uri(SiteLink  + qRow.Find(".ttr_name > a").Attr("href"));
                     release.Comments = release.Guid;
                     release.Link = new Uri(SiteLink + qRow.Find(".td_dl > a").Attr("href"));

src/Jackett/Indexers/SceneFZ.cs

@@ -1,186 +1,801 @@
-using Jackett.Utils.Clients;
-using NLog;
+using System;
+using System.Collections.Generic;
+using System.Collections.Specialized;
+using System.Globalization;
+using System.IO;
+using System.Linq;
+using System.Reflection;
+using System.Text;
+using System.Text.RegularExpressions;
+using System.Threading.Tasks;
+using System.Web;
+using CsQuery;
+using Jackett.Models;
+using Jackett.Models.IndexerConfig.Bespoke;
 using Jackett.Services;
 using Jackett.Utils;
-using Jackett.Models;
-using System.Threading.Tasks;
+using Jackett.Utils.Clients;
+using Newtonsoft.Json;
 using Newtonsoft.Json.Linq;
-using System.Collections.Generic;
-using CsQuery;
-using System.Web;
-using System;
-using System.Text;
-using System.Globalization;
-using Jackett.Models.IndexerConfig;
+using NLog;
 
 namespace Jackett.Indexers
 {
-    public class SceneFZ : BaseWebIndexer
+    /// <summary>
+    /// Provider for SceneFZ
+    /// </summary>
+    public class SceneFZ : BaseCachingWebIndexer
     {
-        string LoginUrl { get { return SiteLink + "takelogin.php"; } }
+        private string LoginUrl { get { return SiteLink + "login.php"; } }
+        private string LoginCheckUrl { get { return SiteLink + "takelogin.php"; } }
+        private string SearchUrl { get { return SiteLink + "torrents.php"; } }
+        private bool Latency { get { return ConfigData.Latency.Value; } }
+        private bool DevMode { get { return ConfigData.DevMode.Value; } }
+        private bool CacheMode { get { return ConfigData.HardDriveCache.Value; } }
+        private static string Directory => Path.Combine(Path.GetTempPath(), Assembly.GetExecutingAssembly().GetName().Name.ToLower(), MethodBase.GetCurrentMethod().DeclaringType?.Name.ToLower());
 
-        string BrowseUrl { get { return SiteLink + "ajax_browse.php"; } }
+        private Dictionary<string, string> emulatedBrowserHeaders = new Dictionary<string, string>();
+        private CQ fDom = null;
 
-        new ConfigurationDataBasicLogin configData
+        private ConfigurationDataSceneFZ ConfigData
         {
-            get { return (ConfigurationDataBasicLogin)base.configData; }
+            get { return (ConfigurationDataSceneFZ)configData; }
             set { base.configData = value; }
         }
 
-        public SceneFZ(IIndexerConfigurationService configService, IWebClient wc, Logger l, IProtectionService ps)
-            : base(name: "SceneFZ",
-                   description: "Torrent tracker. Tracking over 50.000 torrent files.",
-                   link: "http://scenefz.me/",
-                   caps: TorznabUtil.CreateDefaultTorznabTVCaps(),
-                   configService: configService,
-                   client: wc,
-                   logger: l,
-                   p: ps,
-                   configData: new ConfigurationDataBasicLogin())
+        public SceneFZ(IIndexerConfigurationService configService, IWebClient w, Logger l, IProtectionService ps)
+            : base(
+                name: "SceneFZ",
+                description: "Torrent tracker. Tracking over 50.000 torrent files.",
+                link: "https://scenefz.me/",
+                caps: TorznabUtil.CreateDefaultTorznabTVCaps(),
+                configService: configService,
+                client: w,
+                logger: l,
+                p: ps,
+                configData: new ConfigurationDataSceneFZ())
         {
             Encoding = Encoding.UTF8;
             Language = "ro-ro";
             Type = "private";
 
-            this.configData.Instructions.Value = "The published date is only available if you set \"Torrent Listing\" to Complex is your profile.";
+            // Clean capabilities
+            TorznabCaps.Categories.Clear();
 
-            AddCategoryMapping("mc32", TorznabCatType.Movies); // Movies
-            AddCategoryMapping("scat22", TorznabCatType.MoviesBluRay); // BluRay
-            AddCategoryMapping("scat40", TorznabCatType.MoviesSD); // Xvid
-            AddCategoryMapping("scat41", TorznabCatType.MoviesDVD); // Dvd
-            AddCategoryMapping("scat47", TorznabCatType.MoviesHD); // HD
-            AddCategoryMapping("scat52", TorznabCatType.MoviesDVD); // DVD-RO
-            AddCategoryMapping("scat53", TorznabCatType.MoviesHD); // HD-RO
-            AddCategoryMapping("scat54", TorznabCatType.MoviesBluRay); // BluRay-RO
-            AddCategoryMapping("scat55", TorznabCatType.MoviesSD); // XVID-RO
-            AddCategoryMapping("scat60", TorznabCatType.MoviesOther); // Sport
-            AddCategoryMapping("mc33", TorznabCatType.TV); // TV
-            AddCategoryMapping("scat66", TorznabCatType.TVSD); // SD
-            AddCategoryMapping("scat67", TorznabCatType.TVSD); // SD-RO
-            AddCategoryMapping("scat68", TorznabCatType.TVHD); // HD
-            AddCategoryMapping("scat69", TorznabCatType.TVHD); // HDTV-RO
-            AddCategoryMapping("mc30", TorznabCatType.Console); // Games
-            AddCategoryMapping("scat58", TorznabCatType.ConsolePS3); // PS2
-            AddCategoryMapping("scat16", TorznabCatType.PCGames); // Pc-Iso
-            AddCategoryMapping("scat17", TorznabCatType.Console); // Misc
-            AddCategoryMapping("scat18", TorznabCatType.PCGames); // Pc-Rip
-            AddCategoryMapping("scat19", TorznabCatType.Console); // Consoles
-            AddCategoryMapping("scat57", TorznabCatType.ConsoleXbox360); // Xbox 360
-            AddCategoryMapping("scat46", TorznabCatType.Console); // Oldies
-            AddCategoryMapping("scat59", TorznabCatType.ConsolePS3); // PS3
-            AddCategoryMapping("mc31", TorznabCatType.PC); // Soft
-            AddCategoryMapping("scat20", TorznabCatType.PC); // Pc-Iso
-            AddCategoryMapping("scat21", TorznabCatType.PC); // Misc
-            AddCategoryMapping("scat48", TorznabCatType.PCMac); // Mac OS
-            AddCategoryMapping("mc27", TorznabCatType.Audio); // Music
-            AddCategoryMapping("scat8", TorznabCatType.AudioMP3); // MP3
-            AddCategoryMapping("scat45", TorznabCatType.AudioVideo); // Videoclips
-            AddCategoryMapping("scat61", TorznabCatType.AudioLossless); // FLAC
-            AddCategoryMapping("mc35", TorznabCatType.PCPhoneOther); // Mobile
-            AddCategoryMapping("scat44", TorznabCatType.PCPhoneOther); // Misc
-            AddCategoryMapping("scat64", TorznabCatType.PCPhoneIOS); // iOS
-            AddCategoryMapping("scat65", TorznabCatType.PCPhoneAndroid); // Android
-            AddCategoryMapping("mc28", TorznabCatType.TVAnime); // Anime
-            AddCategoryMapping("scat13", TorznabCatType.TVAnime); // Tv-Eps
-            AddCategoryMapping("scat12", TorznabCatType.TVAnime); // Cartoons
-            AddCategoryMapping("mc29", TorznabCatType.TVDocumentary); // Docs
-            AddCategoryMapping("scat14", TorznabCatType.Books); // Books
-            AddCategoryMapping("scat15", TorznabCatType.Other); // Misc
-            AddCategoryMapping("mc36", TorznabCatType.PC0day); // 0Day
-            AddCategoryMapping("mc34", TorznabCatType.XXX); // XXX 18
-            AddCategoryMapping("scat33", TorznabCatType.Other); // Images
-            AddCategoryMapping("scat34", TorznabCatType.Other); // Video
+            AddCategoryMapping(1, TorznabCatType.TVAnime);
+            AddCategoryMapping(49, TorznabCatType.Other);
+            AddCategoryMapping(43, TorznabCatType.TVDocumentary);
+            AddCategoryMapping(35, TorznabCatType.Console);
+            AddCategoryMapping(30, TorznabCatType.PC);
+            AddCategoryMapping(13, TorznabCatType.TVHD);
+            AddCategoryMapping(14, TorznabCatType.TVHD);
+            AddCategoryMapping(44, TorznabCatType.Other);
+            AddCategoryMapping(45, TorznabCatType.PC);
+            AddCategoryMapping(15, TorznabCatType.PCPhoneOther);
+            AddCategoryMapping(61, TorznabCatType.Movies3D);
+            AddCategoryMapping(62, TorznabCatType.Movies3D);
+            AddCategoryMapping(5, TorznabCatType.MoviesBluRay);
+            AddCategoryMapping(6, TorznabCatType.MoviesBluRay);
+            AddCategoryMapping(9, TorznabCatType.MoviesDVD);
+            AddCategoryMapping(10, TorznabCatType.MoviesDVD);
+            AddCategoryMapping(11, TorznabCatType.MoviesHD);
+            AddCategoryMapping(12, TorznabCatType.MoviesHD);
+            AddCategoryMapping(24, TorznabCatType.MoviesSD);
+            AddCategoryMapping(25, TorznabCatType.MoviesSD);
+            AddCategoryMapping(27, TorznabCatType.XXX);
+            AddCategoryMapping(28, TorznabCatType.Audio);
+            AddCategoryMapping(64, TorznabCatType.AudioLossless);
+            AddCategoryMapping(29, TorznabCatType.AudioVideo);
+            AddCategoryMapping(26, TorznabCatType.PCISO);
+            AddCategoryMapping(22, TorznabCatType.TVSport);
+            AddCategoryMapping(20, TorznabCatType.TVSD);
+            AddCategoryMapping(21, TorznabCatType.TVSD);
         }
 
+        /// <summary>
+        /// Configure our SceneFZ Provider
+        /// </summary>
+        /// <param name="configJson">Our params in Json</param>
+        /// <returns>Configuration state</returns>
         public override async Task<IndexerConfigurationStatus> ApplyConfiguration(JToken configJson)
         {
-            LoadValuesFromJson(configJson);
-            var pairs = new Dictionary<string, string>
+            try
             {
-                { "username", configData.Username.Value },
-                { "password", configData.Password.Value }
+                // Retrieve config values set by Jackett's user
+                LoadValuesFromJson(configJson);
+
+                // Check & Validate Config
+                validateConfig();
+
+                // Setting our data for a better emulated browser (maximum security)
+                // TODO: Encoded Content not supported by Jackett at this time
+                // emulatedBrowserHeaders.Add("Accept-Encoding", "gzip, deflate");
+
+                // If we want to simulate a browser
+                if (ConfigData.Browser.Value)
+                {
+
+                    // Clean headers
+                    emulatedBrowserHeaders.Clear();
+
+                    // Inject headers
+                    emulatedBrowserHeaders.Add("Accept", ConfigData.HeaderAccept.Value);
+                    emulatedBrowserHeaders.Add("Accept-Language", ConfigData.HeaderAcceptLang.Value);
+                    emulatedBrowserHeaders.Add("DNT", Convert.ToInt32(ConfigData.HeaderDNT.Value).ToString());
+                    emulatedBrowserHeaders.Add("Upgrade-Insecure-Requests", Convert.ToInt32(ConfigData.HeaderUpgradeInsecure.Value).ToString());
+                    emulatedBrowserHeaders.Add("User-Agent", ConfigData.HeaderUserAgent.Value);
+                }
+
+                // Building login form data
+                var pairs = new Dictionary<string, string> {
+                { "username", ConfigData.Username.Value },
+                { "password", ConfigData.Password.Value },
+                { "submitme", "X" },
+                { "returnto", "%2F" }
             };
 
-            var result = await RequestLoginAndFollowRedirect(LoginUrl, pairs, null, true, null, LoginUrl);
-            await ConfigureIfOK(result.Cookies, result.Content != null && result.Content.Contains("Please wait..."), () =>
+
+                // Getting login form to retrieve PHPSESSID
+                var myRequest = new WebRequest()
                 {
-                    CQ dom = result.Content;
-                    var errorMessage = dom[".tableinborder:eq(1) td"].Text().Trim();
-                    throw new ExceptionWithConfigData(errorMessage, configData);
+                    Url = LoginUrl
+                };
+
+                // Add our headers to request
+                myRequest.Headers = emulatedBrowserHeaders;
+
+                // Get login page
+                var loginPage = await webclient.GetString(myRequest);
+
+                // Do the login
+                var request = new Utils.Clients.WebRequest()
+                {
+                    Cookies = loginPage.Cookies,
+                    PostData = pairs,
+                    Referer = LoginUrl,
+                    Type = RequestType.POST,
+                    Url = LoginCheckUrl,
+                    Headers = emulatedBrowserHeaders
+                };
+
+                // Perform loggin
+                latencyNow();
+                output("\nPerform login.. with " + LoginCheckUrl);
+                // Get login page
+                var response = await webclient.GetString(request);
+                output("\nTesting if we are logged...");
+                await ConfigureIfOK(response.Cookies, response.Cookies != null && response.Cookies.Contains("uid="), () =>
+                {
+                    output("-> Login Failed: Wrong username or Password");
+                    throw new ExceptionWithConfigData("Wrong username or Password", configData);
                 });
-            return IndexerConfigurationStatus.RequiresTesting;
+
+                output("-> Login Success");
+
+                return IndexerConfigurationStatus.RequiresTesting;
+            }
+            catch (Exception e)
+            {
+                IsConfigured = false;
+                Console.WriteLine("Exception thrown : {0}.", e.Message);
+                Console.WriteLine(e.StackTrace);
+                throw e;
+            }
         }
 
+        /// <summary>
+        /// Execute our search query
+        /// </summary>
+        /// <param name="query">Query</param>
+        /// <returns>Releases</returns>
         protected override async Task<IEnumerable<ReleaseInfo>> PerformQuery(TorznabQuery query)
         {
             var releases = new List<ReleaseInfo>();
-            var searchUrl = BrowseUrl;
-            var searchString = query.GetQueryString();
+            var torrentRowList = new List<CQ>();
+            var searchTerm = query.GetQueryString();
+            var searchUrl = SearchUrl;
+            //int nbResults = 0;
+            //int pageLinkCount = 0;
 
-            var cats = MapTorznabCapsToTrackers(query);
-            string cat = "a1";
-
-            if (cats.Count == 1)
+            // Check cache first so we don't query the server (if search term used or not in dev mode)
+            if (!DevMode && !string.IsNullOrEmpty(searchTerm))
             {
-                cat = cats[0];
+                lock (cache)
+                {
+                    // Remove old cache items
+                    CleanCache();
+
+                    // Search in cache
+                    var cachedResult = cache.Where(i => i.Query == searchTerm).FirstOrDefault();
+                    if (cachedResult != null)
+                        return cachedResult.Results.Select(s => (ReleaseInfo)s.Clone()).ToArray();
+                }
             }
 
-            searchUrl += string.Format("?search={0}&param_val=0&complex_search=0&incldead={1}&orderby=added&sort=desc", HttpUtility.UrlEncode(searchString), cat);
+            // Build our query
+            var request = buildQuery(searchTerm, query, searchUrl);
+
+            // Getting results & Store content
+            fDom = await queryExec(request);
 
-            var response = await RequestStringWithCookiesAndRetry(searchUrl, null, BrowseUrl);
-            var results = response.Content;
             try
             {
-                CQ dom = results;
-                var rows = dom["table#torrenttable > tbody > tr:has(td.tablea), table#torrents_table > tbody > tr#torrent-row"]; // selector for old and new style
+                // Find number of results
+                //nbResults = ParseUtil.CoerceInt(Regex.Match(fDom["div.ajaxtotaltorrentcount"].Text(), @"\d+").Value);
 
-                foreach (var row in rows)
+                // Find torrent rows
+                var firstPageRows = findTorrentRows();
+
+                // Add them to torrents list
+                torrentRowList.AddRange(firstPageRows.Select(fRow => fRow.Cq()));
+
+                // Check if there are pagination links at bottom
+                /*Boolean pagination = (nbResults != 0);
+
+                // If pagination available
+                if (pagination)
                 {
-                    var release = new ReleaseInfo();
-                    var qRow = row.Cq();
-                    var qTitleLink = qRow.Find("a[href^=\"details\"]").First();
-                    if (qTitleLink.HasAttr("title"))
-                        release.Title = qTitleLink.Attr("title");
+                    // Calculate numbers of pages available for this search query (Based on number results and number of torrents on first page)
+                    pageLinkCount = (int)Math.Ceiling((double)nbResults / firstPageRows.Length);
+                }
+                else
+                {
+                    // Check if we have a minimum of one result
+                    if (firstPageRows.Length >= 1)
+                    {
+                        // Set page count arbitrary to one
+                        pageLinkCount = 1;
+                    }
                     else
-                        release.Title = qTitleLink.Text();
-                    release.Description = qRow.Find("small > i").Text();
-                    release.Guid = new Uri(SiteLink + qTitleLink.Attr("href"));
-                    release.Comments = release.Guid;
+                    {
+                        output("\nNo result found for your query, please try another search term ...\n", "info");
+                        // No result found for this query
+                        return releases;
+                    }
+                }
+                output("\nFound " + nbResults + " result(s) in " + pageLinkCount + " page(s) for this query !");
+                output("\nThere are " + firstPageRows.Length + " results on the first page !");
 
-                    // date is only available with Complex listing
-                    var dateStr = qRow.Find("table > tbody > tr:nth-child(2) > td:nth-child(5)").Html().Replace("&nbsp;", " ");
-                    if (!string.IsNullOrEmpty(dateStr))
-                        release.PublishDate = DateTime.ParseExact(dateStr + " +0200", "dd.MM.yyyy HH:mm:ss zzz", CultureInfo.InvariantCulture);
+                // If we have a term used for search and pagination result superior to one
+                if (!string.IsNullOrWhiteSpace(query.GetQueryString()) && pageLinkCount > 1)
+                {
+                    // Starting with page #2
+                    for (int i = 2; i <= Math.Min(Int32.Parse(ConfigData.Pages.Value), pageLinkCount); i++)
+                    {
+                        output("\nProcessing page #" + i);
 
-                    var qLink = qRow.Find("a[href^=\"download/\"]");
-                    release.Link = new Uri(SiteLink + qLink.Attr("href"));
+                        // Request our page
+                        latencyNow();
 
-                    var sizeStr = qRow.Find("td[nowrap]:nth-child(3), table > tbody > tr:nth-child(2) > td:nth-child(1) > b").Text().Trim();
-                    release.Size = ReleaseInfo.GetBytes(sizeStr.Replace(".", "").Replace(",", "."));
+                        // Build our query
+                        var pageRequest = buildQuery(searchTerm, query, searchUrl, i);
 
-                    release.Seeders = ParseUtil.CoerceInt(qRow.Find("td > a[href*=\"&toseeders=1\"]:first-child, td:has(a[href*=\"&toseeders=1\"]) > b:nth-child(1)").Text());
-                    release.Peers = ParseUtil.CoerceInt(qRow.Find("td > a[href*=\"&todlers=1\"]:last-child, a[href*=\"&toseeders=1\"] + b").Text().Replace("L.", "")) + release.Seeders;
-                    release.Grabs = ParseUtil.CoerceLong(qRow.Find("td[style]:has(a[href*=\"tosnatchers=1\"])").Text().Replace(" Completed", ""));
+                        // Getting results & Store content
+                        fDom = await queryExec(pageRequest);
 
-                    release.DownloadVolumeFactor = 0;
-                    release.UploadVolumeFactor = 1;
+                        // Process page results
+                        var additionalPageRows = findTorrentRows();
 
-                    var catLink = qRow.Find("a[onclick^=\"bparam(\"][onclick*=\"cat\"]");
-                    var catId = catLink.Attr("onclick").Split('=')[1].Replace("');", "");
-                    if (!catId.StartsWith("scat"))
-                        catId = "mc" + catId;
-                    release.Category = MapTrackerCatToNewznab(catId);
+                        // Add them to torrents list
+                        torrentRowList.AddRange(additionalPageRows.Select(fRow => fRow.Cq()));
+                    }
+                }*/
 
+                // Loop on results
+                foreach (CQ tRow in torrentRowList)
+                {
+                    output("\n=>> Torrent #" + (releases.Count + 1));
+
+                    // Release Name
+                    string name = tRow.Find("td:nth-child(2) > a > b").Text();
+                    output("Release: " + name);
+
+                    // Category
+                    string categoryID = tRow.Find("td:nth-child(1) > a").Attr("href").Split('=').Last();
+                    string categoryName = tRow.Find("td:nth-child(1) > a > img").Attr("alt");
+                    output("Category: " + MapTrackerCatToNewznab(categoryID).First().ToString() + " (" + categoryName + ")");
+
+                    // Uploader
+                    string uploader = tRow.Find("td.tt_row > span").Text();
+                    output("Uploader: " + uploader);
+
+                    // Seeders
+                    int seeders = ParseUtil.CoerceInt(Regex.Match(tRow.Find("td:nth-child(9)").Text().Trim(), @"\d+").Value);
+                    output("Seeders: " + seeders);
+
+                    // Leechers
+                    int leechers = ParseUtil.CoerceInt(Regex.Match(tRow.Find("td:nth-child(10)").Text().Trim(), @"\d+").Value);
+                    output("Leechers: " + leechers);
+
+                    // Completed
+                    int completed = ParseUtil.CoerceInt(Regex.Match(tRow.Find("td:nth-child(8)").Text().Trim(), @"\d+").Value);
+                    output("Completed: " + completed);
+
+                    // Comments
+                    int comments = ParseUtil.CoerceInt(Regex.Match(tRow.Find("td:nth-child(5)").Text().Trim(), @"\d+").Value);
+                    output("Comments: " + comments);
+
+                    // Files
+                    int files = ParseUtil.CoerceInt(Regex.Match(tRow.Find("td:nth-child(4)").Text().Trim(), @"\d+").Value);
+                    output("Files: " + files);
+
+                    // Size
+                    string humanSize = tRow.Find("td:nth-child(7)").Text().Trim();
+                    var size = ReleaseInfo.GetBytes(humanSize);
+                    output("Size: " + humanSize + " (" + size + " bytes)");
+
+                    // Publish Date
+                    var clock = DateTime.ParseExact(tRow.Find("td:nth-child(6) > nobr").Text().Trim(), "dd-MM-yyyyHH:mm:ss", CultureInfo.InvariantCulture);
+                    output("Released on: " + clock.ToShortDateString());
+
+                    // Torrent Details URL
+                    string details = tRow.Find("td:nth-child(2) > a").Attr("href").ToString().TrimStart('/');
+                    Uri detailsLink = new Uri(SiteLink + details);
+                    output("Details: " + detailsLink.AbsoluteUri);
+
+                    // Torrent Comments URL
+                    Uri commentsLink = new Uri(SiteLink + details + "&hit=1&tocomm=1");
+                    output("Comments Link: " + commentsLink.AbsoluteUri);
+
+                    // Torrent Download URL
+                    string download = tRow.Find("td:nth-child(3) > a").Attr("href");
+                    Uri downloadLink = new Uri(SiteLink + download);
+                    output("Download Link: " + downloadLink.AbsoluteUri);
+
+                    // Freeleech
+                    int downloadVolumeFactor = 1;
+                    if (tRow.Find("img[title^=\"FreeLeech\"]").Length >= 1)
+                    {
+                        downloadVolumeFactor = 0;
+                        output("FreeLeech =)");
+                    }
+
+                    // Building release infos
+                    var release = new ReleaseInfo()
+                    {
+                        Category = MapTrackerCatToNewznab(categoryID),
+                        Title = name,
+                        Seeders = seeders,
+                        Peers = seeders + leechers,
+                        Grabs = completed,
+                        MinimumRatio = 1,
+                        MinimumSeedTime = 345600,
+                        PublishDate = clock,
+                        Files = files,
+                        Size = size,
+                        Guid = detailsLink,
+                        Comments = commentsLink,
+                        Link = downloadLink,
+                        UploadVolumeFactor = 1,
+                        DownloadVolumeFactor = downloadVolumeFactor
+                    };
                     releases.Add(release);
                 }
+
             }
             catch (Exception ex)
             {
-                OnParseError(results, ex);
+                OnParseError("Error, unable to parse result \n" + ex.StackTrace, ex);
             }
 
+            // Return found releases
             return releases;
         }
+
+        /// <summary>
+        /// Build query to process
+        /// </summary>
+        /// <param name="term">Term to search</param>
+        /// <param name="query">Torznab Query for categories mapping</param>
+        /// <param name="url">Search url for provider</param>
+        /// <param name="page">Page number to request</param>
+        /// <returns>URL to query for parsing and processing results</returns>
+        private string buildQuery(string term, TorznabQuery query, string url, int page = 1)
+        {
+            var queryCollection = new NameValueCollection();
+
+            // Check if we are processing a new page
+            /*if (page > 1)
+            {
+                // Adding page number to query
+                url += "/" + page.ToString();
+            }*/
+
+            // If search term provided
+            if (!string.IsNullOrWhiteSpace(term))
+            {
+                // Add search term
+                queryCollection.Add("search", HttpUtility.UrlEncode(term));
+                queryCollection.Add("searchin", "0");
+                queryCollection.Add("cat", "0");
+                queryCollection.Add("incldead", "0");
+
+                // Building our query 
+                url += "?" + queryCollection.GetQueryString();
+            }
+            else
+            {
+                // Showing all torrents (just for output function)
+                term = "all";
+            }
+
+            output("\nBuilded query for \"" + term + "\"... " + url);
+
+            // Return our search url
+            return url;
+        }
+
+        /// <summary>
+        /// Switch Method for Querying
+        /// </summary>
+        /// <param name="request">URL created by Query Builder</param>
+        /// <returns>Results from query</returns>
+        private async Task<String> queryExec(string request)
+        {
+            String results = null;
+
+            // Switch in we are in DEV mode with Hard Drive Cache or not
+            if (DevMode && CacheMode)
+            {
+                // Check Cache before querying and load previous results if available
+                results = await queryCache(request);
+            }
+            else
+            {
+                // Querying tracker directly
+                results = await queryTracker(request);
+            }
+            return results;
+        }
+
+        /// <summary>
+        /// Get Torrents Page from Cache by Query Provided
+        /// </summary>
+        /// <param name="request">URL created by Query Builder</param>
+        /// <returns>Results from query</returns>
+        private async Task<String> queryCache(string request)
+        {
+            String results;
+
+            // Create Directory if not exist
+            System.IO.Directory.CreateDirectory(Directory);
+
+            // Clean Storage Provider Directory from outdated cached queries
+            cleanCacheStorage();
+
+            // File Name
+            string fileName = StringUtil.HashSHA1(request) + ".json";
+
+            // Create fingerprint for request
+            string file = Path.Combine(Directory, fileName);
+
+            // Checking modes states
+            if (File.Exists(file))
+            {
+                // File exist... loading it right now !
+                output("Loading results from hard drive cache ..." + fileName);
+                try
+                {
+                    using (StreamReader fileReader = File.OpenText(file))
+                    {
+                        JsonSerializer serializer = new JsonSerializer();
+                        results = (String)serializer.Deserialize(fileReader, typeof(String));
+                    }
+                }
+                catch (Exception e)
+                {
+                    output("Error loading cached results ! " + e.Message, "error");
+                    results = null;
+                }
+            }
+            else
+            {
+                // No cached file found, querying tracker directly
+                results = await queryTracker(request);
+
+                // Cached file didn't exist for our query, writing it right now !
+                output("Writing results to hard drive cache ..." + fileName);
+                using (StreamWriter fileWriter = File.CreateText(file))
+                {
+                    JsonSerializer serializer = new JsonSerializer();
+                    serializer.Serialize(fileWriter, results);
+                }
+                output("Writed to " + Directory + fileName);
+            }
+            return results;
+        }
+
+        /// <summary>
+        /// Get Torrents Page from Tracker by Query Provided
+        /// </summary>
+        /// <param name="request">URL created by Query Builder</param>
+        /// <returns>Results from query</returns>
+        private async Task<String> queryTracker(string request)
+        {
+            WebClientStringResult results = null;
+
+            // Cache mode not enabled or cached file didn't exist for our query
+            output("\nQuerying tracker for results....");
+
+            // Request our first page
+            latencyNow();
+            results = await RequestStringWithCookiesAndRetry(request, null, null, emulatedBrowserHeaders);
+
+            // Return results from tracker
+            return results.Content;
+        }
+
+        /// <summary>
+        /// Clean Hard Drive Cache Storage
+        /// </summary>
+        /// <param name="force">Force Provider Folder deletion</param>
+        private void cleanCacheStorage(bool force = false)
+        {
+            // Check cleaning method
+            if (force)
+            {
+                // Deleting Provider Storage folder and all files recursively
+                output("\nDeleting Provider Storage folder and all files recursively ...");
+
+                // Check if directory exist
+                if (System.IO.Directory.Exists(Directory))
+                {
+                    // Delete storage directory of provider
+                    System.IO.Directory.Delete(Directory, true);
+                    output("-> Storage folder deleted successfully.");
+                }
+                else
+                {
+                    // No directory, so nothing to do
+                    output("-> No Storage folder found for this provider !");
+                }
+            }
+            else
+            {
+                var i = 0;
+                // Check if there is file older than ... and delete them
+                output("\nCleaning Provider Storage folder... in progress.");
+                System.IO.Directory.GetFiles(Directory)
+                .Select(f => new FileInfo(f))
+                .Where(f => f.LastAccessTime < DateTime.Now.AddMilliseconds(-Convert.ToInt32(ConfigData.HardDriveCacheKeepTime.Value)))
+                .ToList()
+                .ForEach(f =>
+                {
+                    output("Deleting cached file << " + f.Name + " >> ... done.");
+                    f.Delete();
+                    i++;
+                });
+
+                // Inform on what was cleaned during process
+                if (i > 0)
+                {
+                    output("-> Deleted " + i + " cached files during cleaning.");
+                }
+                else
+                {
+                    output("-> Nothing deleted during cleaning.");
+                }
+            }
+        }
+
+        /// <summary>
+        /// Generate a random fake latency to avoid detection on tracker side
+        /// </summary>
+        private void latencyNow()
+        {
+            // Need latency ?
+            if (Latency)
+            {
+                // Generate a random value in our range
+                var random = new Random(DateTime.Now.Millisecond);
+                int waiting = random.Next(Convert.ToInt32(ConfigData.LatencyStart.Value), Convert.ToInt32(ConfigData.LatencyEnd.Value));
+                output("\nLatency Faker => Sleeping for " + waiting + " ms...");
+
+                // Sleep now...
+                System.Threading.Thread.Sleep(waiting);
+            }
+        }
+
+        /// <summary>
+        /// Find torrent rows in search pages
+        /// </summary>
+        /// <returns>JQuery Object</returns>
+        private CQ findTorrentRows()
+        {
+            // Return all occurencis of torrents found
+            return fDom["#minions .browse,.sticky"];
+        }
+
+        /// <summary>
+        /// Output message for logging or developpment (console)
+        /// </summary>
+        /// <param name="message">Message to output</param>
+        /// <param name="level">Level for Logger</param>
+        private void output(string message, string level = "debug")
+        {
+            // Check if we are in dev mode
+            if (DevMode)
+            {
+                // Output message to console
+                Console.WriteLine(message);
+            }
+            else
+            {
+                // Send message to logger with level
+                switch (level)
+                {
+                    default:
+                        goto case "debug";
+                    case "debug":
+                        // Only if Debug Level Enabled on Jackett
+                        if (Engine.Logger.IsDebugEnabled)
+                        {
+                            logger.Debug(message);
+                        }
+                        break;
+                    case "info":
+                        logger.Info(message);
+                        break;
+                    case "error":
+                        logger.Error(message);
+                        break;
+                }
+            }
+        }
+
+        /// <summary>
+        /// Validate Config entered by user on Jackett
+        /// </summary>
+        private void validateConfig()
+        {
+            output("\nValidating Settings ... \n");
+
+            // Check Username Setting
+            if (string.IsNullOrEmpty(ConfigData.Username.Value))
+            {
+                throw new ExceptionWithConfigData("You must provide a username for this tracker to login !", ConfigData);
+            }
+            else
+            {
+                output("Validated Setting -- Username (auth) => " + ConfigData.Username.Value.ToString());
+            }
+
+            // Check Password Setting
+            if (string.IsNullOrEmpty(ConfigData.Password.Value))
+            {
+                throw new ExceptionWithConfigData("You must provide a password with your username for this tracker to login !", ConfigData);
+            }
+            else
+            {
+                output("Validated Setting -- Password (auth) => " + ConfigData.Password.Value.ToString());
+            }
+
+            // Check Max Page Setting
+            /*if (!string.IsNullOrEmpty(ConfigData.Pages.Value))
+            {
+                try
+                {
+                    output("Validated Setting -- Max Pages => " + Convert.ToInt32(ConfigData.Pages.Value));
+                }
+                catch (Exception)
+                {
+                    throw new ExceptionWithConfigData("Please enter a numeric maximum number of pages to crawl !", ConfigData);
+                }
+            }
+            else
+            {
+                throw new ExceptionWithConfigData("Please enter a maximum number of pages to crawl !", ConfigData);
+            }*/
+
+            // Check Latency Setting
+            if (ConfigData.Latency.Value)
+            {
+                output("\nValidated Setting -- Latency Simulation enabled");
+
+                // Check Latency Start Setting
+                if (!string.IsNullOrEmpty(ConfigData.LatencyStart.Value))
+                {
+                    try
+                    {
+                        output("Validated Setting -- Latency Start => " + Convert.ToInt32(ConfigData.LatencyStart.Value));
+                    }
+                    catch (Exception)
+                    {
+                        throw new ExceptionWithConfigData("Please enter a numeric latency start in ms !", ConfigData);
+                    }
+                }
+                else
+                {
+                    throw new ExceptionWithConfigData("Latency Simulation enabled, Please enter a start latency !", ConfigData);
+                }
+
+                // Check Latency End Setting
+                if (!string.IsNullOrEmpty(ConfigData.LatencyEnd.Value))
+                {
+                    try
+                    {
+                        output("Validated Setting -- Latency End => " + Convert.ToInt32(ConfigData.LatencyEnd.Value));
+                    }
+                    catch (Exception)
+                    {
+                        throw new ExceptionWithConfigData("Please enter a numeric latency end in ms !", ConfigData);
+                    }
+                }
+                else
+                {
+                    throw new ExceptionWithConfigData("Latency Simulation enabled, Please enter a end latency !", ConfigData);
+                }
+            }
+
+            // Check Browser Setting
+            if (ConfigData.Browser.Value)
+            {
+                output("\nValidated Setting -- Browser Simulation enabled");
+
+                // Check ACCEPT header Setting
+                if (string.IsNullOrEmpty(ConfigData.HeaderAccept.Value))
+                {
+                    throw new ExceptionWithConfigData("Browser Simulation enabled, Please enter an ACCEPT header !", ConfigData);
+                }
+                else
+                {
+                    output("Validated Setting -- ACCEPT (header) => " + ConfigData.HeaderAccept.Value.ToString());
+                }
+
+                // Check ACCEPT-LANG header Setting
+                if (string.IsNullOrEmpty(ConfigData.HeaderAcceptLang.Value))
+                {
+                    throw new ExceptionWithConfigData("Browser Simulation enabled, Please enter an ACCEPT-LANG header !", ConfigData);
+                }
+                else
+                {
+                    output("Validated Setting -- ACCEPT-LANG (header) => " + ConfigData.HeaderAcceptLang.Value.ToString());
+                }
+
+                // Check USER-AGENT header Setting
+                if (string.IsNullOrEmpty(ConfigData.HeaderUserAgent.Value))
+                {
+                    throw new ExceptionWithConfigData("Browser Simulation enabled, Please enter an USER-AGENT header !", ConfigData);
+                }
+                else
+                {
+                    output("Validated Setting -- USER-AGENT (header) => " + ConfigData.HeaderUserAgent.Value.ToString());
+                }
+            }
+
+            // Check Dev Cache Settings
+            if (ConfigData.HardDriveCache.Value == true)
+            {
+                output("\nValidated Setting -- DEV Hard Drive Cache enabled");
+
+                // Check if Dev Mode enabled !
+                if (!ConfigData.DevMode.Value)
+                {
+                    throw new ExceptionWithConfigData("Hard Drive is enabled but not in DEV MODE, Please enable DEV MODE !", ConfigData);
+                }
+
+                // Check Cache Keep Time Setting
+                if (!string.IsNullOrEmpty(ConfigData.HardDriveCacheKeepTime.Value))
+                {
+                    try
+                    {
+                        output("Validated Setting -- Cache Keep Time (ms) => " + Convert.ToInt32(ConfigData.HardDriveCacheKeepTime.Value));
+                    }
+                    catch (Exception)
+                    {
+                        throw new ExceptionWithConfigData("Please enter a numeric hard drive keep time in ms !", ConfigData);
+                    }
+                }
+                else
+                {
+                    throw new ExceptionWithConfigData("Hard Drive Cache enabled, Please enter a maximum keep time for cache !", ConfigData);
+                }
+            }
+            else
+            {
+                // Delete cache if previously existed
+                cleanCacheStorage(true);
+            }
+        }
     }
 }
-

src/Jackett/Indexers/T411.cs

@@ -1,278 +0,0 @@
-using Jackett.Models;
-using Jackett.Services;
-using Jackett.Utils;
-using Jackett.Utils.Clients;
-using Newtonsoft.Json.Linq;
-using NLog;
-using System;
-using System.Collections.Generic;
-using System.Globalization;
-using System.Linq;
-using System.Text;
-using System.Threading.Tasks;
-using System.Web;
-using Jackett.Models.IndexerConfig;
-
-namespace Jackett.Indexers
-{
-    public class T411 : BaseWebIndexer
-    {
-        const string ApiUrl = "https://api.t411.al";
-        const string AuthUrl = ApiUrl + "/auth";
-        const string SearchUrl = ApiUrl + "/torrents/search/";
-        const string TermsUrl = ApiUrl + "/terms/tree";
-        const string DownloadUrl = ApiUrl + "/torrents/download/";
-        private string CommentsUrl { get { return SiteLink + "torrents/"; } }
-
-        new ConfigurationDataLoginTokin configData
-        {
-            get { return (ConfigurationDataLoginTokin)base.configData; }
-            set { base.configData = value; }
-        }
-
-        private Dictionary<int, List<int>> _mediaCategoryMapping = new Dictionary<int, List<int>>();
-
-        public T411(IIndexerConfigurationService configService, IWebClient wc, Logger l, IProtectionService ps)
-            : base(name: "T411",
-                description: "French Torrent Tracker",
-                link: "https://t411.al/",
-                caps: TorznabUtil.CreateDefaultTorznabTVCaps(),
-                configService: configService,
-                client: wc,
-                logger: l,
-                p: ps,
-                configData: new ConfigurationDataLoginTokin())
-        {
-            Encoding = Encoding.UTF8;
-            Type = "semi-private";
-            Language = "fr-fr";
-
-            // 210, FilmVidéo
-            AddCategoryMapping(210, 402, TorznabCatType.Movies, "Vidéoclips");
-            AddCategoryMapping(210, 433, TorznabCatType.TV, "Série TV");
-            AddCategoryMapping(210, 455, TorznabCatType.TVAnime, "Animation");
-            AddCategoryMapping(210, 631, TorznabCatType.Movies, "Film");
-            AddCategoryMapping(210, 633, TorznabCatType.Movies, "Concert");
-            AddCategoryMapping(210, 634, TorznabCatType.TVDocumentary, "Documentaire");
-            AddCategoryMapping(210, 635, TorznabCatType.TV, "Spectacle");
-            AddCategoryMapping(210, 636, TorznabCatType.TVSport, "Sport");
-            AddCategoryMapping(210, 637, TorznabCatType.TVAnime, "Animation Série");
-            AddCategoryMapping(210, 639, TorznabCatType.TV, "Emission TV");
-
-            // 233, Application
-            AddCategoryMapping(233, 234, TorznabCatType.PC, "Linux");
-            AddCategoryMapping(233, 235, TorznabCatType.PCMac, "MacOS");
-            AddCategoryMapping(233, 236, TorznabCatType.PC, "Windows");
-            AddCategoryMapping(233, 625, TorznabCatType.PCPhoneOther, "Smartphone");
-            AddCategoryMapping(233, 627, TorznabCatType.PCPhoneOther, "Tablette");
-            AddCategoryMapping(233, 629, TorznabCatType.PC, "Autre");
-            AddCategoryMapping(233, 638, TorznabCatType.PC, "Formation");
-
-            // 395, Audio
-            AddCategoryMapping(395, 400, TorznabCatType.Audio, "Karaoke");
-            AddCategoryMapping(395, 403, TorznabCatType.Audio, "Samples");
-            AddCategoryMapping(395, 623, TorznabCatType.Audio, "Musique");
-            AddCategoryMapping(395, 642, TorznabCatType.Audio, "Podcast Radio");
-
-            // 404, eBook
-            AddCategoryMapping(404, 405, TorznabCatType.Books, "Audio");
-            AddCategoryMapping(404, 406, TorznabCatType.Books, "Bds");
-            AddCategoryMapping(404, 407, TorznabCatType.Books, "Comics");
-            AddCategoryMapping(404, 408, TorznabCatType.Books, "Livres");
-            AddCategoryMapping(404, 409, TorznabCatType.Books, "Mangas");
-            AddCategoryMapping(404, 410, TorznabCatType.Books, "Presse");
-
-            // 456, xXx
-            AddCategoryMapping(456, 461, TorznabCatType.XXX, "eBooks");
-            AddCategoryMapping(456, 462, TorznabCatType.XXX, "Jeux vidéo");
-            AddCategoryMapping(456, 632, TorznabCatType.XXX, "Video");
-            AddCategoryMapping(456, 641, TorznabCatType.XXX, "Animation");
-
-            // 624, Jeu vidéo
-            AddCategoryMapping(624, 239, TorznabCatType.PCGames, "Linux");
-            AddCategoryMapping(624, 245, TorznabCatType.PCMac, "MacOS");
-            AddCategoryMapping(624, 246, TorznabCatType.PCGames, "Windows");
-            AddCategoryMapping(624, 307, TorznabCatType.ConsoleNDS, "Nintendo");
-            AddCategoryMapping(624, 308, TorznabCatType.ConsolePS4, "Sony");
-            AddCategoryMapping(624, 309, TorznabCatType.ConsoleXbox, "Microsoft");
-            AddCategoryMapping(624, 626, TorznabCatType.PCPhoneOther, "Smartphone");
-            AddCategoryMapping(624, 628, TorznabCatType.PCPhoneOther, "Tablette");
-            AddCategoryMapping(624, 630, TorznabCatType.ConsoleOther, "Autre");
-        }
-
-        private void AddCategoryMapping(int trackerMediaCategory, int trackerCategory, TorznabCategory newznabCategory, string trackerCategoryDesc = null)
-        {
-            AddCategoryMapping(trackerCategory, newznabCategory, trackerCategoryDesc);
-            if (!_mediaCategoryMapping.ContainsKey(trackerMediaCategory))
-                _mediaCategoryMapping.Add(trackerMediaCategory, new List<int>());
-            _mediaCategoryMapping[trackerMediaCategory].Add(trackerCategory);
-        }
-
-        private KeyValuePair<int, List<int>> GetCategoryFromSubCat(int subCategory)
-        {
-            try
-            {
-                return _mediaCategoryMapping.First(pair => pair.Value.Contains(subCategory));
-            }
-            catch (Exception)
-            {
-                return new KeyValuePair<int, List<int>>(0, new List<int>() { 0 }); //If the provided category does not exist, we return 0 (ALL)
-            }
-        }
-
-        async Task<string> GetAuthToken(bool forceFetch = false)
-        {
-            if (!forceFetch && configData.LastTokenFetchDateTime > DateTime.Now - TimeSpan.FromHours(48))
-            {
-                return configData.ApiToken.Value;
-            }
-
-            var pairs = new Dictionary<string, string> {
-                { "username", configData.Username.Value },
-                { "password", configData.Password.Value }
-            };
-
-            var response = await PostDataWithCookies(AuthUrl, pairs);
-            var responseContent = response.Content;
-            var jsonResponse = JObject.Parse(responseContent);
-            if (jsonResponse["error"] != null)
-            {
-                throw new ApplicationException((string)jsonResponse["error"]);
-            }
-            configData.ApiToken.Value = (string)jsonResponse["token"];
-            configData.LastTokenFetchDateTime = DateTime.Now;
-            return configData.ApiToken.Value;
-        }
-
-        public override async Task<IndexerConfigurationStatus> ApplyConfiguration(JToken configJson)
-        {
-            configData.LoadValuesFromJson(configJson);
-
-            Exception tokenFetchEx = null;
-            try
-            {
-                await GetAuthToken(true);
-            }
-            catch (Exception ex)
-            {
-                tokenFetchEx = new ExceptionWithConfigData(ex.Message, configData);
-            }
-
-            await ConfigureIfOK(string.Empty, tokenFetchEx == null, () =>
-            {
-                throw tokenFetchEx;
-            });
-
-            return IndexerConfigurationStatus.RequiresTesting;
-        }
-
-        protected override async Task<IEnumerable<ReleaseInfo>> PerformQuery(TorznabQuery query)
-        {
-            var releases = new List<ReleaseInfo>();
-            // API doesn't support getting the latest torrents, searching for the empty string will cause an error and all torrents returned
-            var searchUrl = SearchUrl + HttpUtility.UrlEncode(query.SanitizedSearchTerm).Replace("+", "%20");
-            searchUrl += "?offset=0&limit=200&cat=" + GetCategoryFromSubCat(query.Categories.FirstOrDefault()).Key;
-
-            // handle special term search for tvsearch
-            var queryStringOverride = query.SanitizedSearchTerm;
-            switch (query.QueryType)
-            {
-                case "tvsearch":
-                    // T411 make the difference beetween Animation Movies and TV Animation Series, while Torznab does not.
-                    // So here we take LastOrDefault from the category ids, so if the query specify an animation tv serie, we select the correct id.
-                    searchUrl += "&subcat=" + GetCategoryFromSubCat(query.Categories.FirstOrDefault()).Value.LastOrDefault();
-                    if (query.Season >= 1 && query.Season <= 30)
-                    {
-                        var seasonTermValue = 967 + query.Season;
-                        searchUrl += "&term[45][]=" + seasonTermValue;
-                        queryStringOverride += " " + query.Season;
-                    }
-
-                    if (query.Episode != null)
-                    {
-                        int episodeInt;
-                        int episodeCategoryOffset = 936;
-                        ParseUtil.TryCoerceInt(query.Episode, out episodeInt);
-                        if (episodeInt >= 1 && episodeInt <= 8)
-                            episodeCategoryOffset = 936;
-                        else if (episodeInt >= 9 && episodeInt <= 30)
-                            episodeCategoryOffset = 937;
-                        else if (episodeInt >= 31)
-                            episodeCategoryOffset = 1057;
-                        searchUrl += "&term[46][]=" + (episodeCategoryOffset + episodeInt);
-                        queryStringOverride += " " + query.Episode;
-                    }
-                    break;
-                case "movie":
-                    // T411 make the difference beetween Animation Movies and TV Animation Series, while Torznab does not.
-                    // So here we take FirstOrDefault from the category ids, so if the query specify an animation movie, we select the correct id.
-                    searchUrl += "&subcat=" + GetCategoryFromSubCat(query.Categories.FirstOrDefault()).Value.FirstOrDefault();
-                    break;
-            }
-
-            var headers = new Dictionary<string, string>();
-            headers.Add("Authorization", await GetAuthToken());
-
-            var response = await RequestStringWithCookies(searchUrl, null, null, headers);
-            var results = response.Content;
-
-            var jsonStart = results.IndexOf('{');
-            var jsonLength = results.Length - jsonStart;
-            var jsonResult = JObject.Parse(results.Substring(jsonStart));
-            try
-            {
-                var items = (JArray)jsonResult["torrents"];
-                foreach (var item in items)
-                {
-                    if (item.GetType() == typeof(JValue))
-                    {
-                        logger.Debug(string.Format("{0}: skipping torrent ID {1} (pending release without details)", ID, item.ToString()));
-                        continue;
-                    }
-                    var release = new ReleaseInfo();
-
-                    release.MinimumRatio = 1;
-                    release.MinimumSeedTime = 172800;
-                    release.DownloadVolumeFactor = 0;
-                    release.DownloadVolumeFactor = 1;
-                    var torrentId = (string)item["id"];
-                    release.Link = new Uri(DownloadUrl + torrentId);
-                    release.Title = (string)item["name"];
-
-                    if ((query.ImdbID == null || !TorznabCaps.SupportsImdbSearch) && !query.MatchQueryStringAND(release.Title, null, queryStringOverride))
-                        continue;
-
-                    if ((string)item["isVerified"] == "1")
-                        release.Description = "Verified";
-                    release.Comments = new Uri(CommentsUrl + (string)item["rewritename"]);
-                    release.Guid = release.Comments;
-
-                    var dateUtc = DateTime.ParseExact((string)item["added"], "yyyy-MM-dd HH:mm:ss", CultureInfo.InvariantCulture);
-                    release.PublishDate = DateTime.SpecifyKind(dateUtc, DateTimeKind.Utc).ToLocalTime();
-
-                    release.Seeders = ParseUtil.CoerceInt((string)item["seeders"]);
-                    release.Peers = ParseUtil.CoerceInt((string)item["leechers"]) + release.Seeders;
-                    release.Size = ParseUtil.CoerceLong((string)item["size"]);
-                    release.Category = MapTrackerCatToNewznab((string)item["category"]);
-                    release.Grabs = ParseUtil.CoerceLong((string)item["times_completed"]);
-
-                    releases.Add(release);
-                }
-            }
-            catch (Exception ex)
-            {
-                OnParseError(results, ex);
-            }
-            return releases;
-        }
-
-        public override async Task<byte[]> Download(Uri link)
-        {
-            var headers = new Dictionary<string, string>();
-            headers.Add("Authorization", await GetAuthToken());
-
-            var response = await RequestBytesWithCookies(link.AbsoluteUri, null, RequestType.GET, null, null, headers);
-            return response.Content;
-        }
-    }
-}

src/Jackett/Indexers/TorrentNetwork.cs

@@ -6,19 +6,24 @@ using Jackett.Models;
 using System.Threading.Tasks;
 using Newtonsoft.Json.Linq;
 using System.Collections.Generic;
-using CsQuery;
 using System;
-using System.Globalization;
 using Jackett.Models.IndexerConfig;
 using System.Collections.Specialized;
 using System.Text;
+using Newtonsoft.Json;
+using static Jackett.Models.IndexerConfig.ConfigurationData;
 
 namespace Jackett.Indexers
 {
     public class TorrentNetwork : BaseWebIndexer
     {
-        string LoginUrl { get { return SiteLink + "takelogin.php"; } }
-        string BrowseUrl { get { return SiteLink + "browse.php"; } }
+        string APIUrl { get { return SiteLink + "api/"; } }
+        private string passkey;
+
+        private Dictionary<string, string> APIHeaders = new Dictionary<string, string>()
+        {
+            {"Content-Type", "application/json"},
+        };
 
         new ConfigurationDataBasicLoginWithRSSAndDisplay configData
         {
@@ -28,9 +33,9 @@ namespace Jackett.Indexers
 
         public TorrentNetwork(IIndexerConfigurationService configService, IWebClient wc, Logger l, IProtectionService ps)
             : base(name: "Torrent Network",
-                   description: "A German general tracker.",
+                   description: null,
                    link: "https://tntracker.org/",
-                   caps: TorznabUtil.CreateDefaultTorznabTVCaps(),
+                   caps: new TorznabCapabilities(),
                    configService: configService,
                    client: wc,
                    logger: l,
@@ -41,51 +46,93 @@ namespace Jackett.Indexers
             Language = "de-de";
             Type = "private";
 
-            AddCategoryMapping(1,  TorznabCatType.AudioAudiobook); // aBook
-            AddCategoryMapping(4,  TorznabCatType.PCMac); // App|Mac
-            AddCategoryMapping(5,  TorznabCatType.PC); // App|Win
-            AddCategoryMapping(7,  TorznabCatType.TVDocumentary); // Docu|HD
-            AddCategoryMapping(6,  TorznabCatType.TVDocumentary); // Docu|SD
-            AddCategoryMapping(8,  TorznabCatType.Books); // eBook
-            AddCategoryMapping(10, TorznabCatType.PCGames); // Game|PC
-            AddCategoryMapping(13, TorznabCatType.ConsolePS4); // Game|PSX
-            AddCategoryMapping(12, TorznabCatType.ConsoleWii); // Game|Wii
-            AddCategoryMapping(14, TorznabCatType.ConsoleXbox); // Game|XBOX
-            AddCategoryMapping(30, TorznabCatType.Other); // Misc
-            AddCategoryMapping(17, TorznabCatType.MoviesHD); // Movie|DE|1080p
-            AddCategoryMapping(20, TorznabCatType.MoviesHD); // Movie|DE|2160p
-            AddCategoryMapping(36, TorznabCatType.Movies3D); // Movie|DE|3D
-            AddCategoryMapping(18, TorznabCatType.MoviesHD); // Movie|DE|720p
-            AddCategoryMapping(34, TorznabCatType.TVAnime); // Movie|DE|Anime
-            AddCategoryMapping(19, TorznabCatType.MoviesBluRay); // Movie|DE|BluRay
-            AddCategoryMapping(45, TorznabCatType.Movies); // Movie|DE|Remux
-            AddCategoryMapping(24, TorznabCatType.MoviesSD); // Movie|DE|SD
-            AddCategoryMapping(39, TorznabCatType.Movies); // Movie|EN/JP|Anime
-            AddCategoryMapping(43, TorznabCatType.MoviesHD); // Movie|EN|1080p
-            AddCategoryMapping(37, TorznabCatType.MoviesHD); // Movie|EN|2160p
-            AddCategoryMapping(35, TorznabCatType.MoviesHD); // Movie|EN|720p
-            AddCategoryMapping(38, TorznabCatType.MoviesBluRay); // Movie|EN|BluRay
-            AddCategoryMapping(46, TorznabCatType.Movies); // Movie|EN|Remux
-            AddCategoryMapping(22, TorznabCatType.MoviesSD); // Movie|EN|SD
-            AddCategoryMapping(44, TorznabCatType.AudioLossless); // Music|Flac
-            AddCategoryMapping(25, TorznabCatType.AudioMP3); // Music|MP3
-            AddCategoryMapping(26, TorznabCatType.AudioVideo); // Music|Video
-            AddCategoryMapping(31, TorznabCatType.TVSport); // Sport
-            AddCategoryMapping(2,  TorznabCatType.TVAnime); // TV|DE|Anime
-            AddCategoryMapping(28, TorznabCatType.TVHD); // TV|DE|HD
-            AddCategoryMapping(16, TorznabCatType.TV); // TV|DE|Pack
-            AddCategoryMapping(27, TorznabCatType.TVSD); // TV|DE|SD
-            AddCategoryMapping(41, TorznabCatType.TVAnime); // TV|EN/JP|Anime
-            AddCategoryMapping(40, TorznabCatType.TVHD); // TV|EN|HD
-            AddCategoryMapping(42, TorznabCatType.TV); // TV|EN|Pack
-            AddCategoryMapping(29, TorznabCatType.TVSD); // TV|EN|SD
-            AddCategoryMapping(33, TorznabCatType.XXX); // XXX|HD
-            AddCategoryMapping(32, TorznabCatType.XXX); // XXX|SD
+            configData.AddDynamic("token", new HiddenItem() { Name = "token" });
+            configData.AddDynamic("passkey", new HiddenItem() { Name = "passkey" });
+
+            AddCategoryMapping(24, TorznabCatType.MoviesSD, "Movies GER/SD");
+            AddCategoryMapping(18, TorznabCatType.MoviesHD, "Movies GER/720p");
+            AddCategoryMapping(17, TorznabCatType.MoviesHD, "Movies GER/1080p");
+            AddCategoryMapping(20, TorznabCatType.MoviesHD, "Movies GER/2160p");
+            AddCategoryMapping(45, TorznabCatType.MoviesOther, "Movies GER/Remux");
+            AddCategoryMapping(19, TorznabCatType.MoviesBluRay, "Movies GER/BluRay");
+            AddCategoryMapping(34, TorznabCatType.TVAnime, "Movies GER/Anime");
+            AddCategoryMapping(36, TorznabCatType.Movies3D, "Movies GER/3D");
+
+            AddCategoryMapping(22, TorznabCatType.MoviesSD, "Movies ENG/SD");
+            AddCategoryMapping(35, TorznabCatType.MoviesHD, "Movies ENG/720p");
+            AddCategoryMapping(43, TorznabCatType.MoviesHD, "Movies ENG/1080p");
+            AddCategoryMapping(37, TorznabCatType.MoviesHD, "Movies ENG/2160p");
+            AddCategoryMapping(46, TorznabCatType.MoviesOther, "Movies ENG/Remux");
+            AddCategoryMapping(38, TorznabCatType.MoviesBluRay, "Movies ENG/BluRay");
+            AddCategoryMapping(39, TorznabCatType.TVAnime, "Movies ENG/Anime");
+
+            AddCategoryMapping(27, TorznabCatType.TVSD, "Series GER/SD");
+            AddCategoryMapping(28, TorznabCatType.TVHD, "Series GER/HD");
+            AddCategoryMapping(2, TorznabCatType.TVAnime, "Series GER/Anime");
+            AddCategoryMapping(16, TorznabCatType.TV, "Series GER/Pack");
+            AddCategoryMapping(6, TorznabCatType.TVDocumentary, "Docu/SD");
+            AddCategoryMapping(7, TorznabCatType.TVDocumentary, "Docu/HD");
+
+            AddCategoryMapping(29, TorznabCatType.TVSD, "Series ENG/SD");
+            AddCategoryMapping(40, TorznabCatType.TVHD, "Series ENG/HD");
+            AddCategoryMapping(41, TorznabCatType.TVAnime, "Series ENG/Anime");
+            AddCategoryMapping(42, TorznabCatType.TV, "Series ENG/Pack");
+            AddCategoryMapping(31, TorznabCatType.TVSport, "Sport");
+
+
+            AddCategoryMapping(10, TorznabCatType.PCGames, "Games/Win");
+            AddCategoryMapping(12, TorznabCatType.ConsoleWii, "Games/Wii");
+            AddCategoryMapping(13, TorznabCatType.ConsolePS4, "Games/PSX");
+            AddCategoryMapping(14, TorznabCatType.ConsoleXbox, "Games/XBOX");
+
+            AddCategoryMapping(4, TorznabCatType.PCMac, "Apps/Mac");
+            AddCategoryMapping(5, TorznabCatType.PC0day, "Apps/Win");
+
+            AddCategoryMapping(1, TorznabCatType.AudioAudiobook, "Misc/aBook");
+            AddCategoryMapping(8, TorznabCatType.Books, "Misc/eBook");
+            AddCategoryMapping(30, TorznabCatType.Other, "Misc/Sonstiges");
+
+            AddCategoryMapping(44, TorznabCatType.AudioLossless, "Musik/Flac");
+            AddCategoryMapping(25, TorznabCatType.AudioMP3, "Musik/MP3");
+            AddCategoryMapping(26, TorznabCatType.AudioVideo, "Musik/Video");
+
+            AddCategoryMapping(32, TorznabCatType.XXX, "XXX/XXX");
+            AddCategoryMapping(33, TorznabCatType.XXX, "XXX/XXX|HD");
+        }
+
+        public override void LoadValuesFromJson(JToken jsonConfig, bool useProtectionService = false)
+        {
+            base.LoadValuesFromJson(jsonConfig, useProtectionService);
+
+            var tokenItem = (HiddenItem)configData.GetDynamic("token");
+            if (tokenItem != null)
+            {
+                var token = tokenItem.Value;
+                if (!string.IsNullOrWhiteSpace(token))
+                    APIHeaders["Authorization"] = token;
+            }
+
+            var passkeyItem = (HiddenItem)configData.GetDynamic("passkey");
+            if (passkeyItem != null)
+            {
+                passkey = passkeyItem.Value;
+            }
+        }
+
+        private async Task<dynamic> SendAPIRequest(string endpoint, object data)
+        {
+            var jsonData = JsonConvert.SerializeObject(data);
+            var result = await PostDataWithCookies(APIUrl + endpoint, null, null, SiteLink, APIHeaders, jsonData);
+            if (!result.Content.StartsWith("{")) // not JSON => error
+                throw new ExceptionWithConfigData(result.Content, configData);
+            dynamic json = JsonConvert.DeserializeObject<dynamic>(result.Content);
+            return json;
         }
 
         public override async Task<IndexerConfigurationStatus> ApplyConfiguration(JToken configJson)
         {
             LoadValuesFromJson(configJson);
+            APIHeaders.Remove("Authorization"); // remove any token from the headers
 
             var pairs = new Dictionary<string, string>
             {
@@ -93,113 +140,100 @@ namespace Jackett.Indexers
                 { "password", configData.Password.Value }
             };
 
-            var result = await RequestLoginAndFollowRedirect(LoginUrl, pairs, null, true, null, LoginUrl, true);
-            await ConfigureIfOK(result.Cookies, result.Content != null && result.Content.Contains("logout.php"), () =>
+            var json = await SendAPIRequest("auth", pairs);
+            string token = json.token;
+
+            APIHeaders["Authorization"] = token;
+
+            var curuser = await SendAPIRequest("curuser", null);
+
+            var passkeyItem = (HiddenItem)configData.GetDynamic("passkey");
+            passkeyItem.Value = curuser.passkey;
+
+            var tokenItem = (HiddenItem)configData.GetDynamic("token");
+            tokenItem.Value = token;
+
+            await ConfigureIfOK("", token.Length > 0, () =>
             {
-                CQ dom = result.Content;
-                var errorMessage = dom["table.tableinborder"].Html();
-                errorMessage = result.Content;
-                throw new ExceptionWithConfigData(errorMessage, configData);
+                throw new ExceptionWithConfigData(json.ToString(), configData);
             });
+
             return IndexerConfigurationStatus.RequiresTesting;
         }
 
         protected override async Task<IEnumerable<ReleaseInfo>> PerformQuery(TorznabQuery query)
         {
-            TimeZoneInfo.TransitionTime startTransition = TimeZoneInfo.TransitionTime.CreateFloatingDateRule(new DateTime(1, 1, 1, 3, 0, 0), 3, 5, DayOfWeek.Sunday);
-            TimeZoneInfo.TransitionTime endTransition = TimeZoneInfo.TransitionTime.CreateFloatingDateRule(new DateTime(1, 1, 1, 4, 0, 0), 10, 5, DayOfWeek.Sunday);
-            TimeSpan delta = new TimeSpan(1, 0, 0);
-            TimeZoneInfo.AdjustmentRule adjustment = TimeZoneInfo.AdjustmentRule.CreateAdjustmentRule(new DateTime(1999, 10, 1), DateTime.MaxValue.Date, delta, startTransition, endTransition);
-            TimeZoneInfo.AdjustmentRule[] adjustments = { adjustment };
-            TimeZoneInfo germanyTz = TimeZoneInfo.CreateCustomTimeZone("W. Europe Standard Time", new TimeSpan(1, 0, 0), "(GMT+01:00) W. Europe Standard Time", "W. Europe Standard Time", "W. Europe DST Time", adjustments);
-
             var releases = new List<ReleaseInfo>();
-            
+
+            var searchUrl = "browse";
             var searchString = query.GetQueryString();
-            var searchUrl = BrowseUrl;
             var queryCollection = new NameValueCollection();
-            queryCollection.Add("incldead", "1");
-            queryCollection.Add("_by", "0");
-            queryCollection.Add("sort", "4");
-            queryCollection.Add("type", "desc");
+            queryCollection.Add("orderC", "4");
+            queryCollection.Add("orderD", "desc");
+            queryCollection.Add("start", "0");
+            queryCollection.Add("length", "100");
 
             if (!string.IsNullOrWhiteSpace(searchString))
-            {
                 queryCollection.Add("search", searchString);
-            }
 
-            foreach (var cat in MapTorznabCapsToTrackers(query))
-            {
-                queryCollection.Add("c" + cat, "1");
-            }
+            var cats = MapTorznabCapsToTrackers(query);
+            if (cats.Count > 0)
+                queryCollection.Add("cats", string.Join(",", cats));
+
             searchUrl += "?" + queryCollection.GetQueryString();
 
-            var response = await RequestStringWithCookies(searchUrl);
-            var results = response.Content;
+            var result = await SendAPIRequest(searchUrl, null);
             try
             {
-                CQ dom = results;
-                var rows = dom["table[border=1] > tbody > tr:has(td.torrenttable)"];
+                if (result["error"] != null)
+                    throw new Exception(result["error"].ToString());
 
-                foreach (var row in rows)
+                var data = (JArray)result.data;
+                
+                foreach (JArray torrent in data)
                 {
                     var release = new ReleaseInfo();
                     release.MinimumRatio = 0.8;
                     release.MinimumSeedTime = 48 * 60 * 60;
 
-                    var qRow = row.Cq();
+                    release.Category = MapTrackerCatToNewznab(torrent[0].ToString());
+                    release.Title = torrent[1].ToString();
+                    var torrentID = (long)torrent[2];
+                    release.Comments = new Uri(SiteLink + "torrent/" + torrentID);
+                    release.Guid = release.Comments;
+                    release.Link = new Uri(SiteLink + "sdownload/" + torrentID + "/" + passkey);
+                    release.PublishDate = DateTimeUtil.UnixTimestampToDateTime((double)torrent[3]).ToLocalTime();
+                    //var preDelaySeconds = (long)torrent[4];
+                    release.Size = (long)torrent[5];
+                    release.Seeders = (int)torrent[6];
+                    release.Peers = release.Seeders + (int)torrent[7];
+                    //var imdbRating = (double)torrent[8] / 10;
+                    var genres = (string)torrent[9];
+                    if (!string.IsNullOrWhiteSpace(genres))
+                        release.Description = "Genres: " + genres;
 
-                    var qDetailsLink = qRow.Find("a[href^=details.php?id=]").First();
-                    var qTitle = qDetailsLink.Find("b").First();
-                    release.Title = qTitle.Text();
-
-                    var qCatLink = qRow.Find("a[href^=browse.php?cat=]").First();
-                    var qDLLink = qRow.Find("a.download").First();
-                    var qSeeders = qRow.Find("td.torrenttable:eq(7)");
-                    var qLeechers = qRow.Find("td.torrenttable:eq(8)");
-                    var qDateStr = qRow.Find("td.torrenttable:eq(4)").First();
-                    var qSize = qRow.Find("td.torrenttable:eq(5)").First();
-
-                    var catStr = qCatLink.Attr("href").Split('=')[1].Split('\'')[0];
-                    release.Category = MapTrackerCatToNewznab(catStr);
-
-                    release.Link = new Uri(SiteLink + qDLLink.Attr("href"));
-                    release.Comments = new Uri(SiteLink + qDetailsLink.Attr("href"));
-                    release.Guid = release.Link;
-
-                    var sizeStr = qSize.Text();
-                    release.Size = ReleaseInfo.GetBytes(sizeStr);
-
-                    release.Seeders = ParseUtil.CoerceInt(qSeeders.Text());
-                    release.Peers = ParseUtil.CoerceInt(qLeechers.Text()) + release.Seeders;
-
-                    var dateStr = qDateStr.Text().Trim();
-                    DateTime dateGerman = DateTime.SpecifyKind(DateTime.ParseExact(dateStr, "MMM d yyyy HH:mm", CultureInfo.InvariantCulture), DateTimeKind.Unspecified);
-
-                    DateTime pubDateUtc = TimeZoneInfo.ConvertTimeToUtc(dateGerman, germanyTz);
-                    release.PublishDate = pubDateUtc.ToLocalTime();
-
-                    var files = qRow.Find("td:nth-child(4)").Text();
-                    release.Files = ParseUtil.CoerceInt(files);
-
-                    var grabs = qRow.Find("td:nth-child(8)").Get(0).FirstChild.ToString();
-                    release.Grabs = ParseUtil.CoerceInt(grabs);
-
-                    if (qRow.Find("img[src=\"pic/torrent_ou.gif\"]").Length >= 1)
+                    var DownloadVolumeFlag = (long)torrent[10];
+                    release.UploadVolumeFactor = 1;
+                    if (DownloadVolumeFlag == 2) // Only Up
                         release.DownloadVolumeFactor = 0;
-                    else if (qRow.Find("font[color=\"gray\"]:contains(50% Down)").Length >= 1)
+                    else if (DownloadVolumeFlag == 1) // 50 % Down
                         release.DownloadVolumeFactor = 0.5;
-                    else
+                    else if (DownloadVolumeFlag == 0)
                         release.DownloadVolumeFactor = 1;
 
-                    release.UploadVolumeFactor = 1;
+                    release.Grabs = (long)torrent[11];
+
+                    // 12/13/14 unknown, probably IDs/name of the uploader
+                    //var row12 = (long)torrent[12];
+                    //var row13 = (string)torrent[13];
+                    //var row14 = (long)torrent[14];
 
                     releases.Add(release);
                 }
             }
             catch (Exception ex)
             {
-                OnParseError(results, ex);
+                OnParseError(result.ToString(), ex);
             }
 
             return releases;

src/Jackett/Indexers/WiHD.cs

@@ -15,6 +15,7 @@ using Jackett.Utils.Clients;
 using Newtonsoft.Json;
 using Newtonsoft.Json.Linq;
 using NLog;
+using System.IO;
 
 namespace Jackett.Indexers
 {
@@ -29,7 +30,7 @@ namespace Jackett.Indexers
         private bool Latency { get { return ConfigData.Latency.Value; } }
         private bool DevMode { get { return ConfigData.DevMode.Value; } }
         private bool CacheMode { get { return ConfigData.HardDriveCache.Value; } }
-        private string directory { get { return System.IO.Path.GetTempPath() + "Jackett\\" + MethodBase.GetCurrentMethod().DeclaringType.Name + "\\"; } }
+        private static string Directory => Path.Combine(Path.GetTempPath(), Assembly.GetExecutingAssembly().GetName().Name.ToLower(), MethodBase.GetCurrentMethod().DeclaringType?.Name.ToLower());
 
         private Dictionary<string, string> emulatedBrowserHeaders = new Dictionary<string, string>();
         private CQ fDom = null;
@@ -44,13 +45,13 @@ namespace Jackett.Indexers
             : base(
                 name: "WiHD",
                 description: "Your World in High Definition",
-                link: "http://world-in-hd.net/",
+                link: "https://world-in-hd.net/",
                 caps: new TorznabCapabilities(),
                 configService: configService,
                 client: w,
                 logger: l,
                 p: ps,
-                downloadBase: "http://world-in-hd.net/torrents/download/",
+                downloadBase: "https://world-in-hd.net/torrents/download/",
                 configData: new ConfigurationDataWiHD())
         {
             Encoding = Encoding.UTF8;
@@ -215,8 +216,7 @@ namespace Jackett.Indexers
             var request = buildQuery(searchTerm, query, searchUrl);
 
             // Getting results & Store content
-            WebClientStringResult results = await queryExec(request);
-            fDom = results.Content;
+            fDom = await queryExec(request);
 
             try
             {
@@ -271,10 +271,7 @@ namespace Jackett.Indexers
                         var pageRequest = buildQuery(searchTerm, query, searchUrl, i);
 
                         // Getting results & Store content
-                        WebClientStringResult pageResults = await queryExec(pageRequest);
-
-                        // Assign response
-                        fDom = pageResults.Content;
+                        fDom = await queryExec(pageRequest);
 
                         // Process page results
                         var additionalPageRows = findTorrentRows();
@@ -296,7 +293,7 @@ namespace Jackett.Indexers
                     // Category
                     string categoryID = tRow.Find(".category > img").Attr("src").Split('/').Last().ToString();
                     string categoryName = tRow.Find(".category > img").Attr("title").ToString();
-                    output("Category: " + MapTrackerCatToNewznab(mediaToCategory(categoryID, categoryName)) + " (" + categoryName + ")");
+                    output("Category: " + MapTrackerCatToNewznab(mediaToCategory(categoryID, categoryName)).First().ToString() + " (" + categoryName + ")");
 
                     // Uploader
                     string uploader = tRow.Find(".uploader > span > a").Attr("title").ToString();
@@ -345,19 +342,31 @@ namespace Jackett.Indexers
                     Uri downloadLink = new Uri(SiteLink + download);
                     output("Download Link: " + downloadLink.AbsoluteUri);
 
+                    // Freeleech
+                    int downloadVolumeFactor = 1;
+                    if (tRow.Find(".fl-item").Length >= 1)
+                    {
+                        downloadVolumeFactor = 0;
+                        output("FreeLeech =)");
+                    }
+
                     // Building release infos
-                    var release = new ReleaseInfo();
-                    release.Category = MapTrackerCatToNewznab(mediaToCategory(categoryID, categoryName));
-                    release.Title = name;
-                    release.Seeders = seeders;
-                    release.Peers = seeders + leechers;
-                    release.MinimumRatio = 1;
-                    release.MinimumSeedTime = 345600;
-                    release.PublishDate = clock;
-                    release.Size = size;
-                    release.Guid = detailsLink;
-                    release.Comments = commentsLink;
-                    release.Link = downloadLink;
+                    var release = new ReleaseInfo()
+                    {
+                        Category = MapTrackerCatToNewznab(mediaToCategory(categoryID, categoryName)),
+                        Title = name,
+                        Seeders = seeders,
+                        Peers = seeders + leechers,
+                        MinimumRatio = 1,
+                        MinimumSeedTime = 345600,
+                        PublishDate = clock,
+                        Size = size,
+                        Guid = detailsLink,
+                        Comments = commentsLink,
+                        Link = downloadLink,
+                        UploadVolumeFactor = 1,
+                        DownloadVolumeFactor = downloadVolumeFactor
+                    };
                     releases.Add(release);
                 }
 
@@ -448,9 +457,9 @@ namespace Jackett.Indexers
         /// </summary>
         /// <param name="request">URL created by Query Builder</param>
         /// <returns>Results from query</returns>
-        private async Task<WebClientStringResult> queryExec(string request)
+        private async Task<String> queryExec(string request)
         {
-            WebClientStringResult results = null;
+            String results = null;
 
             // Switch in we are in DEV mode with Hard Drive Cache or not
             if (DevMode && CacheMode)
@@ -471,28 +480,40 @@ namespace Jackett.Indexers
         /// </summary>
         /// <param name="request">URL created by Query Builder</param>
         /// <returns>Results from query</returns>
-        private async Task<WebClientStringResult> queryCache(string request)
+        private async Task<String> queryCache(string request)
         {
-            WebClientStringResult results = null;
+            String results;
 
             // Create Directory if not exist
-            System.IO.Directory.CreateDirectory(directory);
+            System.IO.Directory.CreateDirectory(Directory);
 
             // Clean Storage Provider Directory from outdated cached queries
             cleanCacheStorage();
 
-            // Remove timestamp from query
-            string file = string.Join("&", Regex.Split(request, "&").Where(s => !Regex.IsMatch(s, @"_=\d")).ToList());
+            // File Name
+            string fileName = StringUtil.HashSHA1(request) + ".json";
 
             // Create fingerprint for request
-            file = directory + file.GetHashCode() + ".json";
+            string file = Path.Combine(Directory, fileName);
 
             // Checking modes states
-            if (System.IO.File.Exists(file))
+            if (File.Exists(file))
             {
                 // File exist... loading it right now !
-                output("Loading results from hard drive cache ..." + request.GetHashCode() + ".json");
-                results = JsonConvert.DeserializeObject<WebClientStringResult>(System.IO.File.ReadAllText(file));
+                output("Loading results from hard drive cache ..." + fileName);
+                try
+                {
+                    using (StreamReader fileReader = File.OpenText(file))
+                    {
+                        JsonSerializer serializer = new JsonSerializer();
+                        results = (String)serializer.Deserialize(fileReader, typeof(String));
+                    }
+                }
+                catch (Exception e)
+                {
+                    output("Error loading cached results ! " + e.Message, "error");
+                    results = null;
+                }
             }
             else
             {
@@ -500,8 +521,12 @@ namespace Jackett.Indexers
                 results = await queryTracker(request);
 
                 // Cached file didn't exist for our query, writing it right now !
-                output("Writing results to hard drive cache ..." + request.GetHashCode() + ".json");
-                System.IO.File.WriteAllText(file, JsonConvert.SerializeObject(results));
+                output("Writing results to hard drive cache ..." + fileName);
+                using (StreamWriter fileWriter = File.CreateText(file))
+                {
+                    JsonSerializer serializer = new JsonSerializer();
+                    serializer.Serialize(fileWriter, results);
+                }
             }
             return results;
         }
@@ -511,7 +536,7 @@ namespace Jackett.Indexers
         /// </summary>
         /// <param name="request">URL created by Query Builder</param>
         /// <returns>Results from query</returns>
-        private async Task<WebClientStringResult> queryTracker(string request)
+        private async Task<String> queryTracker(string request)
         {
             WebClientStringResult results = null;
 
@@ -523,14 +548,14 @@ namespace Jackett.Indexers
             results = await RequestStringWithCookiesAndRetry(request, null, null, emulatedBrowserHeaders);
 
             // Return results from tracker
-            return results;
+            return results.Content;
         }
 
         /// <summary>
         /// Clean Hard Drive Cache Storage
         /// </summary>
         /// <param name="force">Force Provider Folder deletion</param>
-        private void cleanCacheStorage(Boolean force = false)
+        private void cleanCacheStorage(bool force = false)
         {
             // Check cleaning method
             if (force)
@@ -539,10 +564,10 @@ namespace Jackett.Indexers
                 output("\nDeleting Provider Storage folder and all files recursively ...");
 
                 // Check if directory exist
-                if (System.IO.Directory.Exists(directory))
+                if (System.IO.Directory.Exists(Directory))
                 {
                     // Delete storage directory of provider
-                    System.IO.Directory.Delete(directory, true);
+                    System.IO.Directory.Delete(Directory, true);
                     output("-> Storage folder deleted successfully.");
                 }
                 else
@@ -553,11 +578,11 @@ namespace Jackett.Indexers
             }
             else
             {
-                int i = 0;
+                var i = 0;
                 // Check if there is file older than ... and delete them
                 output("\nCleaning Provider Storage folder... in progress.");
-                System.IO.Directory.GetFiles(directory)
-                .Select(f => new System.IO.FileInfo(f))
+                System.IO.Directory.GetFiles(Directory)
+                .Select(f => new FileInfo(f))
                 .Where(f => f.LastAccessTime < DateTime.Now.AddMilliseconds(-Convert.ToInt32(ConfigData.HardDriveCacheKeepTime.Value)))
                 .ToList()
                 .ForEach(f =>

src/Jackett/Indexers/Xthor.cs

@@ -228,7 +228,7 @@ namespace Jackett.Indexers
             }
             catch (Exception ex)
             {
-                OnParseError("Error, unable to parse result \n" + ex.StackTrace, ex);
+                OnParseError("Unable to parse result \n" + ex.StackTrace, ex);
             }
 
             // Return found releases
@@ -466,7 +466,7 @@ namespace Jackett.Indexers
                 case 1:
                     // Passkey not found
                     Output("\nAPI State : Error, Passkey not found in tracker's database, aborting... -> " + state.descr);
-                    throw new Exception("API State : Error, Passkey not found in tracker's database, aborting... -> " + state.descr);
+                    throw new Exception("Passkey not found in tracker's database");
                 case 2:
                     // No results
                     Output("\nAPI State : No results for query ... -> " + state.descr);
@@ -478,11 +478,11 @@ namespace Jackett.Indexers
                 case 4:
                     // DDOS Attack, API disabled
                     Output("\nAPI State : Tracker is under DDOS attack, API disabled, aborting ... -> " + state.descr);
-                    throw new Exception("\nAPI State : Tracker is under DDOS attack, API disabled, aborting ... -> " + state.descr);
+                    throw new Exception("Tracker is under DDOS attack, API disabled");
                 default:
                     // Unknown state
                     Output("\nAPI State : Unknown state, aborting querying ... -> " + state.descr);
-                    throw new Exception("API State : Unknown state, aborting querying ... -> " + state.descr);
+                    throw new Exception("Unknown state, aborting querying");
             }
         }
 

src/Jackett/Indexers/myAmity.cs

@@ -113,7 +113,7 @@ namespace Jackett.Indexers
             searchUrl += "?" + queryCollection.GetQueryString();
 
             var response = await RequestStringWithCookies(searchUrl);
-            if (response.IsRedirect)
+            if (response.IsRedirect || response.Cookies != null && response.Cookies.Contains("pass=deleted;"))
             {
                 // re-login
                 await ApplyConfiguration(null);

src/Jackett/Jackett.csproj

@@ -173,6 +173,7 @@
   <ItemGroup>
     <Compile Include="AuthenticationException.cs" />
     <Compile Include="CacheControlAttribute.cs" />
+    <Compile Include="IndexerException.cs" />
     <Compile Include="Controllers\BlackholeController.cs" />
     <Compile Include="Controllers\DownloadController.cs" />
     <Compile Include="Engine.cs" />
@@ -193,7 +194,6 @@
     <Compile Include="Indexers\AnimeTorrents.cs" />
     <Compile Include="Indexers\7tor.cs" />
     <Compile Include="Indexers\EliteTracker.cs" />
-    <Compile Include="Indexers\T411.cs" />
     <Compile Include="Indexers\Torrentech.cs" />
     <Compile Include="Indexers\notwhatcd.cs" />
     <Compile Include="Indexers\Redacted.cs" />
@@ -208,7 +208,6 @@
     <Compile Include="Indexers\Andraste.cs" />
     <Compile Include="Indexers\TorrentHeaven.cs" />
     <Compile Include="Indexers\NewRealWorld.cs" />
-    <Compile Include="Indexers\BestFriends.cs" />
     <Compile Include="Indexers\BitCityReloaded.cs" />
     <Compile Include="Indexers\FunFile.cs" />
     <Compile Include="Indexers\GhostCity.cs" />
@@ -254,8 +253,6 @@
     <Compile Include="Models\GitHub\Release.cs" />
     <Compile Include="Models\IndexerConfig\Bespoke\ConfigurationDataNorbits.cs" />
     <Compile Include="Models\IndexerConfig\Bespoke\ConfigurationDataXthor.cs" />
-    <Compile Include="Models\IndexerConfig\Bespoke\ConfigurationDataPhxBit.cs" />
-    <Compile Include="Models\IndexerConfig\Bespoke\ConfigurationDataBlueTigers.cs" />
     <Compile Include="Models\IndexerConfig\Bespoke\ConfigurationDataAbnormal.cs" />
     <Compile Include="Models\IndexerConfig\Bespoke\ConfigurationDataWiHD.cs" />
     <Compile Include="Models\IndexerConfig\ConfigurationDataBasicLoginWithFilterAndPasskey.cs" />
@@ -271,7 +268,6 @@
     <Compile Include="Indexers\Pretome.cs" />
     <Compile Include="Indexers\PrivateHD.cs" />
     <Compile Include="Indexers\SceneAccess.cs" />
-    <Compile Include="Indexers\SceneFZ.cs" />
     <Compile Include="Indexers\SceneTime.cs" />
     <Compile Include="Indexers\SpeedCD.cs" />
     <Compile Include="Indexers\TorrentDay.cs" />
@@ -288,7 +284,6 @@
     <Compile Include="Models\IndexerConfig\Bespoke\ConfigurationDataNCore.cs" />
     <Compile Include="Models\IndexerConfig\ConfigurationDataCaptchaLogin.cs" />
     <Compile Include="Models\IndexerConfig\Bespoke\ConfigurationDataAnimeBytes.cs" />
-    <Compile Include="Models\IndexerConfig\Bespoke\ConfigurationDataStrike.cs" />
     <Compile Include="Models\IndexerConfig\ConfigurationDataUrl.cs" />
     <Compile Include="Models\IndexerConfig\ISerializableConfig.cs" />
     <Compile Include="Models\IndexerConfig\ConfigurationDataPinNumber.cs" />
@@ -324,7 +319,6 @@
     <Compile Include="Models\IndexerConfig\ConfigurationData.cs" />
     <Compile Include="Models\IndexerConfig\ConfigurationDataBasicLogin.cs" />
     <Compile Include="Models\IndexerConfig\ConfigurationDataCookie.cs" />
-    <Compile Include="Models\IndexerConfig\Bespoke\ConfigurationDataRuTor.cs" />
     <Compile Include="CookieContainerExtensions.cs" />
     <Compile Include="Utils\Clients\WebRequest.cs" />
     <Compile Include="Utils\DataUrl.cs" />
@@ -387,6 +381,8 @@
     <Compile Include="Models\DTO\TorrentPotatoResponse.cs" />
     <Compile Include="Models\DTO\TorrentPotatoResponseItem.cs" />
     <Compile Include="Models\DTO\TorznabRequest.cs" />
+    <Compile Include="Indexers\SceneFZ.cs" />
+    <Compile Include="Models\IndexerConfig\Bespoke\ConfigurationDataSceneFZ.cs" />
   </ItemGroup>
   <ItemGroup>
     <None Include="App.config">

src/Jackett/Models/DTO/Config.cs

@@ -5,5 +5,8 @@ namespace Jackett.Models.DTO
     {
         public string id { get; set; }
         public string value { get; set; }
+        public string cookie { get; set; } // for cookie alternative login (captcha needed + remote host)
+        public string challenge { get; set; } // for reCaptcha V1 compatibility
+        public string version { get; set; } // for reCaptcha V1 compatibility
     }
 }

src/Jackett/Models/DTO/Indexer.cs

@@ -35,7 +35,7 @@ namespace Jackett.Models.DTO
             site_link = indexer.SiteLink;
             language = indexer.Language;
             last_error = indexer.LastError;
-            potatoenabled = indexer.TorznabCaps.Categories.Select(c => c.ID).Any(i => i == TorznabCatType.Movies.ID);
+            potatoenabled = indexer.TorznabCaps.Categories.Any(i => TorznabCatType.Movies.Contains(i));
 
             alternativesitelinks = indexer.AlternativeSiteLinks;
 

src/Jackett/Models/DTO/ManualSearchResult.cs

@@ -7,9 +7,20 @@ using System.Threading.Tasks;
 
 namespace Jackett.Models.DTO
 {
+    public enum ManualSearchResultIndexerStatus { Unknown = 0, Error = 1, OK = 2 };
+
+    public class ManualSearchResultIndexer
+    {
+        public string ID { get; set; }
+        public string Name { get; set; }
+        public ManualSearchResultIndexerStatus Status { get; set; }
+        public int Results { get; set; }
+        public string Error { get; set; }
+    }
+
     public class ManualSearchResult
     {
         public IEnumerable<TrackerCacheResult> Results { get; set; }
-        public IEnumerable<string> Indexers { get; set; }
+        public IList<ManualSearchResultIndexer> Indexers { get; set; }
     }
 }

src/Jackett/Models/IndexerConfig/Bespoke/ConfigurationDataBlueTigers.cs

@@ -1,63 +0,0 @@
-using Newtonsoft.Json;
-using Newtonsoft.Json.Linq;
-
-namespace Jackett.Models.IndexerConfig.Bespoke
-{
-    public class ConfigurationDataBlueTigers : ConfigurationData
-    {
-        public StringItem Username { get; private set; }
-        public StringItem Password { get; private set; }
-        public DisplayItem Instructions { get; set; }
-        public BoolItem French { get; set; }
-        public BoolItem English { get; set; }
-        public BoolItem Spanish { get; set; }
-
-        public ConfigurationDataBlueTigers(string displayInstructions)
-        {
-            Username = new StringItem { Name = "Username", Value = "" };
-            Password = new StringItem { Name = "Password", Value = "" };
-            Instructions = new DisplayItem(displayInstructions) { Name = "" };
-            French = new BoolItem { Name = "French", Value = true };
-            English = new BoolItem { Name = "English", Value = true };
-            Spanish = new BoolItem { Name = "Spanish", Value = true };
-        }
-
-        public ConfigurationDataBlueTigers(JToken json)
-        {
-            ConfigurationDataNCore configData = new ConfigurationDataNCore();
-
-            dynamic configArray = JsonConvert.DeserializeObject(json.ToString());
-            foreach (var config in configArray)
-            {
-                string propertyName = UppercaseFirst((string)config.id);
-                switch (propertyName)
-                {
-                    case "Username":
-                        Username = new StringItem { Name = propertyName, Value = config.value };
-                        break;
-                    case "Password":
-                        Password = new StringItem { Name = propertyName, Value = config.value };
-                        break;
-                    case "French":
-                        French = new BoolItem { Name = propertyName, Value = config.value };
-                        break;
-                    case "English":
-                        English = new BoolItem { Name = propertyName, Value = config.value };
-                        break;
-                    case "Spanish":
-                        Spanish = new BoolItem { Name = propertyName, Value = config.value };
-                        break;
-                    default:
-                        break;
-                }
-            }
-        }
-
-        static string UppercaseFirst(string s)
-        {
-            if (string.IsNullOrEmpty(s))
-                return string.Empty;
-            return char.ToUpper(s[0]) + s.Substring(1);
-        }
-    }
-}

src/Jackett/Models/IndexerConfig/Bespoke/ConfigurationDataRuTor.cs

@@ -1,27 +0,0 @@
-using Newtonsoft.Json;
-using System;
-using System.Collections.Generic;
-using System.Linq;
-using System.Text;
-using System.Threading.Tasks;
-
-namespace Jackett.Models.IndexerConfig.Bespoke
-{
-    public class ConfigurationDataRuTor : ConfigurationData
-    {
-        [JsonProperty]
-        public StringItem Url { get; private set; }
-        [JsonProperty]
-        public BoolItem StripRussian { get; private set; }
-
-        public ConfigurationDataRuTor()
-        {
-        }
-
-        public ConfigurationDataRuTor(string defaultUrl)
-        {
-            Url = new StringItem { Name = "Url", Value = defaultUrl };
-            StripRussian = new BoolItem() { Name = "StripRusNamePrefix", Value = true };
-        }
-    }
-}

src/Jackett/Models/IndexerConfig/Bespoke/ConfigurationDataSceneFZ.cs

@@ -1,13 +1,13 @@
 namespace Jackett.Models.IndexerConfig.Bespoke
 {
-    class ConfigurationDataPhxBit : ConfigurationData
+    class ConfigurationDataSceneFZ : ConfigurationData
     {
-        public HiddenItem PassKey { get; set; }
         public DisplayItem CredentialsWarning { get; private set; }
         public StringItem Username { get; private set; }
         public StringItem Password { get; private set; }
         public DisplayItem PagesWarning { get; private set; }
-        public StringItem Pages { get; private set; }
+        //public StringItem Pages { get; private set; }
+        public BoolItem Dead { get; private set; }
         public DisplayItem SecurityWarning { get; private set; }
         public BoolItem Latency { get; private set; }
         public BoolItem Browser { get; private set; }
@@ -25,18 +25,18 @@
         public BoolItem HardDriveCache { get; private set; }
         public StringItem HardDriveCacheKeepTime { get; private set; }
 
-        public ConfigurationDataPhxBit()
+        public ConfigurationDataSceneFZ()
             : base()
         {
-            PassKey = new HiddenItem { Name = "PassKey", Value = "" };
             CredentialsWarning = new DisplayItem("<b>Credentials Configuration</b> (<i>Private Tracker</i>),<br /><br /> <ul><li><b>Username</b> is your account name on this tracker.</li><li><b>Password</b> is your password associated to your account name.</li></ul>") { Name = "Credentials" };
             Username = new StringItem { Name = "Username (Required)", Value = "" };
             Password = new StringItem { Name = "Password (Required)", Value = "" };
-            PagesWarning = new DisplayItem("<b>Preferences Configuration</b> (<i>Tweak your search settings</i>),<br /><br /> <ul><li><b>Max Pages to Process</b> let you specify how many page (max) Jackett can process when doing a search. Setting a value <b>higher than 4 is dangerous</b> for you account ! (<b>Result of too many requests to tracker...that <u>will be suspect</u></b>).</li></ul>") { Name  = "Preferences" };
-            Pages = new StringItem { Name = "Max Pages to Process (Required)", Value = "4" };
+            PagesWarning = new DisplayItem("<b>Preferences Configuration</b> (<i>Tweak your search settings</i>),<br /><br /> <ul><li><b>Include Dead</b> let you search <u>including</u> torrents which are marked as Dead.</li></ul>") { Name = "Preferences" };
+            //Pages = new StringItem { Name = "Max Pages to Process (Required)", Value = "4" };
+            Dead = new BoolItem() { Name = "Include Dead (Optional)", Value = false };
             SecurityWarning = new DisplayItem("<b>Security Configuration</b> (<i>Read this area carefully !</i>),<br /><br /> <ul><li><b>Latency Simulation</b> will simulate human browsing with Jacket by pausing Jacket for an random time between each request, to fake a real content browsing.</li><li><b>Browser Simulation</b> will simulate a real human browser by injecting additionals headers when doing requests to tracker.</li></ul>") { Name = "Security" };
             Latency = new BoolItem() { Name = "Latency Simulation (Optional)", Value = false };
-            Browser = new BoolItem() { Name = "Browser Simulation (Optional)", Value = true };
+            Browser = new BoolItem() { Name = "Browser Simulation (Optional)", Value = false };
             LatencyWarning = new DisplayItem("<b>Latency Configuration</b> (<i>Required if latency simulation enabled</i>),<br /><br/> <ul><li>By filling this range, <b>Jackett will make a random timed pause</b> <u>between requests</u> to tracker <u>to simulate a real browser</u>.</li><li>MilliSeconds <b>only</b></li></ul>") { Name = "Simulate Latency" };
             LatencyStart = new StringItem { Name = "Minimum Latency (ms)", Value = "1589" };
             LatencyEnd = new StringItem { Name = "Maximum Latency (ms)", Value = "3674" };
@@ -47,9 +47,9 @@
             HeaderUpgradeInsecure = new BoolItem { Name = "Upgrade-Insecure-Requests", Value = false };
             HeaderUserAgent = new StringItem { Name = "User-Agent", Value = "" };
             DevWarning = new DisplayItem("<b>Development Facility</b> (<i>For Developers ONLY</i>),<br /><br /> <ul><li>By enabling development mode, <b>Jackett will bypass his cache</b> and will <u>output debug messages to console</u> instead of his log file.</li><li>By enabling Hard Drive Cache, <b>This provider</b> will <u>save each query answers from tracker</u> in temp directory, in fact this reduce drastically HTTP requests when building a provider at parsing step for example. So, <b> Jackett will search for a cached query answer on hard drive before executing query on tracker side !</b> <i>DEV MODE must be enabled to use it !</li></ul>") { Name = "Development" };
-            DevMode = new BoolItem { Name = "Enable DEV MODE (Developers ONLY)", Value = false };
-            HardDriveCache = new BoolItem { Name = "Enable HARD DRIVE CACHE (Developers ONLY)", Value = false };
+            DevMode = new BoolItem { Name = "Enable DEV MODE (Developers ONLY)", Value = true };
+            HardDriveCache = new BoolItem { Name = "Enable HARD DRIVE CACHE (Developers ONLY)", Value = true };
             HardDriveCacheKeepTime = new StringItem { Name = "Keep Cached files for (ms)", Value = "300000" };
-            }
+        }
     }
 }

src/Jackett/Models/IndexerConfig/Bespoke/ConfigurationDataStrike.cs

@@ -1,18 +0,0 @@
-using System;
-using System.Collections.Generic;
-using System.Linq;
-using System.Text;
-using System.Threading.Tasks;
-
-namespace Jackett.Models.IndexerConfig.Bespoke
-{
-    public class ConfigurationDataStrike : ConfigurationDataUrl
-    {
-        public DisplayItem StrikeWarning { get; private set; }
-
-        public ConfigurationDataStrike(string url) : base(url)
-        {
-            StrikeWarning = new DisplayItem("This indexer does not support RSS Sync, only Search") { Name = "Warning" };
-        }
-    }
-}

src/Jackett/Models/IndexerConfig/Bespoke/ConfigurationDataWiHD.cs

@@ -30,10 +30,10 @@
         public ConfigurationDataWiHD()
             : base()
         {
-            CredentialsWarning = new DisplayItem("<b>Credentials Configuration</b> (<i>Private Tracker</i>),<br /><br /> <ul><li><b>Username</b> is your account name on this tracker.</li><li><b>Password</b> is your password associated to your account name.</li></ul>") { Name = "Credentials" };
+            CredentialsWarning = new DisplayItem("<b> YOU MUST USE CLASSIC VIEW ON WIHD !</b><br /> In order to use this provider with Jackett, take a look in your settings on the tracker before proceeding...<br /><br /><b>Credentials Configuration</b> (<i>Private Tracker</i>),<br /><br /> <ul><li><b>Username</b> is your account name on this tracker.</li><li><b>Password</b> is your password associated to your account name.</li></ul>") { Name = "Credentials" };
             Username = new StringItem { Name = "Username (Required)", Value = "" };
             Password = new StringItem { Name = "Password (Required)", Value = "" };
-            PagesWarning = new DisplayItem("<b>Preferences Configuration</b> (<i>Tweak your search settings</i>),<br /><br /> <ul><li><b>Max Pages to Process</b> let you specify how many page (max) Jackett can process when doing a search. Setting a value <b>higher than 4 is dangerous</b> for you account ! (<b>Result of too many requests to tracker...that <u>will be suspect</u></b>).</li><li><b>Exclusive Only</b> let you search <u>only</u> for torrents which are marked Exclusive.</li><li><b>Freeleech Only</b> let you search <u>only</u> for torrents which are marked Freeleech.</li><li><b>Reseed Only</b> let you search <u>only</u> for torrents which need to be seeded.</li></ul>") { Name  = "Preferences" };
+            PagesWarning = new DisplayItem("<b>Preferences Configuration</b> (<i>Tweak your search settings</i>),<br /><br /> <ul><li><b>Max Pages to Process</b> let you specify how many page (max) Jackett can process when doing a search. Setting a value <b>higher than 4 is dangerous</b> for you account ! (<b>Result of too many requests to tracker...that <u>will be suspect</u></b>).</li><li><b>Exclusive Only</b> let you search <u>only</u> for torrents which are marked Exclusive.</li><li><b>Freeleech Only</b> let you search <u>only</u> for torrents which are marked Freeleech.</li><li><b>Reseed Only</b> let you search <u>only</u> for torrents which need to be seeded.</li></ul>") { Name = "Preferences" };
             Pages = new StringItem { Name = "Max Pages to Process (Required)", Value = "4" };
             Exclusive = new BoolItem() { Name = "Exclusive Only (Optional)", Value = false };
             Freeleech = new BoolItem() { Name = "Freeleech Only (Optional)", Value = false };

src/Jackett/Models/ReleaseInfo.cs

@@ -3,6 +3,7 @@ using System;
 using System.Collections.Generic;
 using System.Linq;
 using Jackett.Indexers;
+using Newtonsoft.Json;
 
 namespace Jackett.Models
 {
@@ -32,6 +33,7 @@ namespace Jackett.Models
         public long? MinimumSeedTime { get; set; }
         public double? DownloadVolumeFactor { get; set; }
         public double? UploadVolumeFactor { get; set; }
+        [JsonIgnore] // don't export the Origin to the manul search API, otherwise each result line contains a full recursive indexer JSON structure
         public IIndexer Origin;
 
         public double? Gain

src/Jackett/Models/TorznabCapabilities.cs

@@ -10,6 +10,8 @@ namespace Jackett.Models
 {
     public class TorznabCapabilities
     {
+        public int? LimitsMax { get; set; } = null;
+        public int? LimitsDefault { get; set; } = null;
 
         public bool SearchAvailable { get; set; }
 
@@ -66,15 +68,18 @@ namespace Jackett.Models
             }
         }
 
-        public bool SupportsCategories (int[] categories)
+        public bool SupportsCategories(int[] categories)
         {
-            return Categories.Count(i => categories.Any(c => c == i.ID)) > 0;
+            var subCategories = Categories.SelectMany(c => c.SubCategories);
+            var allCategories = Categories.Concat(subCategories);
+            var supportsCategory = allCategories.Any(i => categories.Any(c => c == i.ID));
+            return supportsCategory;
         }
 
         public JArray CapsToJson()
         {
             var jArray = new JArray();
-            foreach (var cat in Categories.GroupBy(p => p.ID).Select(g => g.First()).OrderBy(c=> c.ID < 100000 ? "z"+c.ID.ToString() : c.Name))
+            foreach (var cat in Categories.GroupBy(p => p.ID).Select(g => g.First()).OrderBy(c => c.ID < 100000 ? "z" + c.ID.ToString() : c.Name))
             {
                 jArray.Add(cat.ToJson());
             }
@@ -86,6 +91,15 @@ namespace Jackett.Models
             var xdoc = new XDocument(
                 new XDeclaration("1.0", "UTF-8", null),
                 new XElement("caps",
+                    new XElement("server",
+                        new XAttribute("title", "Jackett")
+                    ),
+                    LimitsMax != null || LimitsDefault != null ?
+                        new XElement("limits",
+                            LimitsMax != null ? new XAttribute("max", LimitsMax) : null,
+                            LimitsDefault != null ? new XAttribute("default", LimitsDefault) : null
+                        )
+                    : null,
                     new XElement("searching",
                         new XElement("search",
                             new XAttribute("available", SearchAvailable ? "yes" : "no"),
@@ -124,7 +138,7 @@ namespace Jackett.Models
             lhs.MovieSearchAvailable = lhs.MovieSearchAvailable || rhs.MovieSearchAvailable;
             lhs.SupportsTVRageSearch = lhs.SupportsTVRageSearch || rhs.SupportsTVRageSearch;
             lhs.SupportsImdbSearch = lhs.SupportsImdbSearch || rhs.SupportsImdbSearch;
-            lhs.Categories.AddRange (rhs.Categories.Except (lhs.Categories));
+            lhs.Categories.AddRange(rhs.Categories.Where(x => x.ID < 100000).Except(lhs.Categories)); // exclude indexer specific categories (>= 100000)
 
             return lhs;
         }

src/Jackett/Services/IndexerManagerService.cs

@@ -207,11 +207,11 @@ namespace Jackett.Services
             browseQuery.QueryType = "search";
             browseQuery.SearchTerm = "";
             browseQuery.IsTest = true;
-            var results = await indexer.ResultsForQuery(browseQuery);
-            logger.Info(string.Format("Found {0} releases from {1}", results.Count(), indexer.DisplayName));
-            if (results.Count() == 0)
+            var result = await indexer.ResultsForQuery(browseQuery);
+            logger.Info(string.Format("Found {0} releases from {1}", result.Releases.Count(), indexer.DisplayName));
+            if (result.Releases.Count() == 0)
                 throw new Exception("Found no results while trying to browse this tracker");
-            cacheService.CacheRssResults(indexer, results);
+            cacheService.CacheRssResults(indexer, result.Releases);
         }
 
         public void DeleteIndexer(string name)

src/Jackett/Startup.cs

@@ -43,6 +43,7 @@ namespace Jackett
 
             json["result"] = "error";
             json["error"] = msg;
+            json["stacktrace"] = exception.StackTrace;
 
             var response = actionExecutedContext.Request.CreateResponse();
             response.Content = new JsonContent(json);

src/Jackett/Utils/Clients/HttpWebClient.cs

@@ -196,6 +196,15 @@ namespace Jackett.Utils.Clients
                                 {
                                     result.RedirectingTo = response.Headers.Location.ToString();
                                 }
+                                // Mono won't add the baseurl to relative redirects.
+                                // e.g. a "Location: /index.php" header will result in the Uri "file:///index.php"
+                                // See issue #1200
+                                if (result.RedirectingTo != null && result.RedirectingTo.StartsWith("file://"))
+                                {
+                                    var newRedirectingTo = result.RedirectingTo.Replace("file://", request.RequestUri.Scheme + "://" + request.RequestUri.Host);
+                                    logger.Debug("[MONO relative redirect bug] Rewriting relative redirect URL from " + result.RedirectingTo + " to " + newRedirectingTo);
+                                    result.RedirectingTo = newRedirectingTo;
+                                }
                                 result.Status = response.StatusCode;
 
                                 // Compatiblity issue between the cookie format and httpclient

src/Jackett/Utils/Clients/HttpWebClient2.cs

@@ -207,6 +207,15 @@ namespace Jackett.Utils.Clients
             {
                 result.RedirectingTo = response.Headers.Location.ToString();
             }
+            // Mono won't add the baseurl to relative redirects.
+            // e.g. a "Location: /index.php" header will result in the Uri "file:///index.php"
+            // See issue #1200
+            if (result.RedirectingTo != null && result.RedirectingTo.StartsWith("file://"))
+            {
+                var newRedirectingTo = result.RedirectingTo.Replace("file://", request.RequestUri.Scheme + "://" + request.RequestUri.Host);
+                logger.Debug("[MONO relative redirect bug] Rewriting relative redirect URL from " + result.RedirectingTo + " to " + newRedirectingTo);
+                result.RedirectingTo = newRedirectingTo;
+            }
             result.Status = response.StatusCode;
 
             // Compatiblity issue between the cookie format and httpclient