CurlHelper: Add NotImplemented/InternalServerError debugging (#730)

* Cardigann: Use GET for landing site request

* Cardigann: Add support for magnet links

README.md

@@ -2,9 +2,9 @@
 
 This project is a new fork and is recruiting development help.  If you are able to help out please contact us.
 
-Jackett works as a proxy server: it translates queries from apps (Sonarr, SickRage, CouchPotato, Mylar, etc) into tracker-site-specific http queries, parses the html response, then sends results back to the requesting software. This allows for getting recent uploads (like RSS) and performing searches. Jackett is a single repository of maintained indexer scraping & translation logic - removing the burden from other apps. 
+Jackett works as a proxy server: it translates queries from apps (Sonarr, SickRage, CouchPotato, Mylar, etc) into tracker-site-specific http queries, parses the html response, then sends results back to the requesting software. This allows for getting recent uploads (like RSS) and performing searches. Jackett is a single repository of maintained indexer scraping & translation logic - removing the burden from other apps.
 
-Developer note: The software implements the [Torznab](https://github.com/Sonarr/Sonarr/wiki/Implementing-a-Torznab-indexer) (with [nZEDb](https://github.com/nZEDb/nZEDb/blob/master/docs/newznab_api_specification.txt) category numbering) and [TorrentPotato](https://github.com/RuudBurger/CouchPotatoServer/wiki/Couchpotato-torrent-provider) APIs.
+Developer note: The software implements the [Torznab](https://github.com/Sonarr/Sonarr/wiki/Implementing-a-Torznab-indexer) (with [nZEDb](https://github.com/nZEDb/nZEDb/blob/dev/docs/newznab_api_specification.txt) category numbering) and [TorrentPotato](https://github.com/RuudBurger/CouchPotatoServer/wiki/Couchpotato-torrent-provider) APIs.
 
 
 
@@ -16,65 +16,105 @@ Developer note: The software implements the [Torznab](https://github.com/Sonarr/
 #### Supported Private Trackers
  * Abnormal
  * AlphaRatio
+ * AlphaReign
+ * Andraste
  * AnimeBytes
+ * AnimeTorrents
  * Avistaz
+ * BakaBT
  * bB
+ * Best Friends
  * BeyondHD
+ * Bit-City Reloaded
  * BIT-HDTV
  * BitMeTV
  * BitSoup
- * BlueTigers
  * BTN
+ * CinemaZ
  * DanishBits
  * Demonoid
- * EuTorrents
+ * DigitalHive
+ * EoT-Forum
+ * Ethor.net (Thor's Land)
  * FileList
- * French-ADN
  * Freshon
+ * FunFile
  * Fuzer
+ * Ghost City
+ * HD4Free
  * HD-Space
  * HD-Torrents
+ * Hebits
+ * New Real World
  * Hounddawgs
+ * House-of-Torrents
  * ILoveTorrents
  * Immortalseed
  * IPTorrents
+ * PassThePopcorn
+ * PirateTheNet
  * MoreThanTV
  * MyAnonamouse
+ * myAmity
  * NCore
  * NextGen
- * PhxBit
  * Pretome
  * PrivateHD
  * RevolutionTT
  * SceneAccess
  * SceneFZ
  * SceneTime
+ * Shareisland
  * Shazbat
  * SpeedCD
+ * The Horror Charnel
+ * The New Retro
  * TehConnection
+ * Torrent Network
+ * Torrent Sector Crew
  * TorrentBytes
  * TorrentDay
+ * TorrentHeaven
  * TorrentLeech
  * TorrentShack
+ * Torrent-Syndikat
+ * TransmitheNet
+ * Tspate
  * TV Chaos UK
+ * TV-Vault
+ * UHDBits
  * World-In-HD
+ * WorldOfP2P
+ * x264
  * XSpeeds
  * Xthor
 
 #### Installation on Windows
 
-We recommend you install Jackett as a Windows service using the supplied installer.  When installed as a service the tray icon acts as a way to open/start/stop Jackett. If you opted to not install it as a service then Jackett will run its web server from the tray tool.
+We recommend you install Jackett as a Windows service using the supplied installer. You may also download the zipped version if you would like to configure everything manually.
 
-Jackett can also be run from the command line using JackettConsole.exe if you would like to see log messages (Ensure the server isn't already running from the tray/service).
+To get started with using the installer for Jackett, follow the steps below:
+
+1. Download the latest version of the Windows installer, "Jackett.Installer.Windows.exe" from the [releases](https://github.com/Jackett/Jackett/releases/latest) page.
+2. When prompted if you would like this app to make changes to your computer, select "yes".
+3. If you would like to install Jackett as a Windows Service, make sure the "Install as Windows Service" checkbox is filled.
+4. Once the installation has finished, check the "Launch Jackett" box to get started.
+5. Navigate your web browser to: http://127.0.0.1:9117
+6. You're now ready to begin adding your trackers and using Jackett.
+
+When installed as a service the tray icon acts as a way to open/start/stop Jackett. If you opted to not install it as a service then Jackett will run its web server from the tray tool.
+
+Jackett can also be run from the command line if you would like to see log messages (Ensure the server isn't already running from the tray/service). This can be done by using "JackettConsole.exe" (for Command Prompt), found in the Jackett data folder: "%ProgramData%\Jackett". 
 
 #### Installation on Linux/OSX
  1. Install [Mono 4](http://www.mono-project.com/download/) or better
+       * Follow the instructions on the mono website and install the `mono-devel` package.
  2. Install  libcurl:
-       * Debian/Ubunutu: apt-get install libcurl-dev
-       * Redhat/Fedora: yum install libcurl-devel
+       * Debian/Ubunutu: `apt-get install libcurl-dev`
+       * Redhat/Fedora: `yum install libcurl-devel`
        * For other distros see the  [Curl docs](http://curl.haxx.se/dlwiz/?type=devel).
- 3. Download and extract the latest ```.tar.gz``` release from the [releases page](https://github.com/Jackett/Jackett/releases) and run Jackett using mono with the command "mono JackettConsole.exe".
- 
+ 3. Download and extract the latest `Jackett.Binaries.Mono.tar.gz` release from the [releases page](https://github.com/Jackett/Jackett/releases) and run Jackett using mono with the command `mono JackettConsole.exe`.
+
 Detailed instructions for [Ubuntu 14.x](http://www.htpcguides.com/install-jackett-on-ubuntu-14-x-for-custom-torrents-in-sonarr/) and [Ubuntu 15.x](http://www.htpcguides.com/install-jackett-ubuntu-15-x-for-custom-torrents-in-sonarr/)
 
 #### Installation on Synology
@@ -98,9 +138,9 @@ You can get additional logging with the switches "-t -l".  Please post logs if y
 Please supply as much information about the problem you are experiencing as possible. Your issue has a much greater chance of being resolved if logs are supplied so that we can see what is going on. Creating an issue with '### isn't working' doesn't help anyone to fix the problem.
 
 ### Contributing
-All contributions are welcome just send a pull request.  Jackett's framework allows our team (and any other volunteering dev) to implement new trackers in an hour or two. If you'd like support for a new tracker but are not a developer then feel free to leave a request on the [issues page](https://github.com/zone117x/Jackett/issues).  It is recommended to use Visual studio 2015 when making code changes in this project.  We currently only support private trackers.
+All contributions are welcome just send a pull request.  Jackett's framework allows our team (and any other volunteering dev) to implement new trackers in an hour or two. If you'd like support for a new tracker but are not a developer then feel free to leave a request on the [issues page](https://github.com/Jackett/Jackett/issues).  It is recommended to use Visual studio 2015 when making code changes in this project.  We currently only support private trackers.
 
 
 ### Screenshots
 
-![screenshot](http://i.imgur.com/t1sVva6.png "screenshot")
+![screenshot](https://i.imgur.com/SCGLzFq.png "screenshot")

src/Jackett.Console/App.config

@@ -7,7 +7,7 @@
     <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
       <dependentAssembly>
         <assemblyIdentity name="Newtonsoft.Json" publicKeyToken="30ad4fe6b2a6aeed" culture="neutral" />
-        <bindingRedirect oldVersion="0.0.0.0-7.0.0.0" newVersion="7.0.0.0" />
+        <bindingRedirect oldVersion="0.0.0.0-9.0.0.0" newVersion="9.0.0.0" />
       </dependentAssembly>
       <dependentAssembly>
         <assemblyIdentity name="System.Web.Http" publicKeyToken="31bf3856ad364e35" culture="neutral" />

src/Jackett.Console/ConsoleOptions.cs

@@ -57,9 +57,12 @@ namespace Jackett.Console
         [Option('f', "SSLFix", HelpText = "[true/false] Linux Libcurl NSS Missing ECC Ciphers workaround (Use if you can't access some trackers) .")]
         public bool? SSLFix { get; set; }
 
-        [Option('n', "IgnoreSslErrors", HelpText = "[true/false] Linux Libcurl - Ignores invalid SSL certificates")]
+        [Option('n', "IgnoreSslErrors", HelpText = "[true/false] Ignores invalid SSL certificates")]
         public bool? IgnoreSslErrors { get; set; }
 
+        [Option('d', "DataFolder", HelpText = "Specify the location of the data folder (Must be admin on Windows) eg. --DataFolder=\"D:\\Your Data\\Jackett\\\"")]
+        public string DataFolder { get; set; }
+
         [ParserState]
         public IParserState LastParserState { get; set; }
     }

src/Jackett.Console/Jackett.Console.csproj

@@ -94,12 +94,12 @@
       <HintPath>..\packages\Microsoft.Owin.StaticFiles.3.0.1\lib\net45\Microsoft.Owin.StaticFiles.dll</HintPath>
       <Private>True</Private>
     </Reference>
-    <Reference Include="Newtonsoft.Json, Version=7.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed, processorArchitecture=MSIL">
-      <HintPath>..\packages\Newtonsoft.Json.7.0.1\lib\net45\Newtonsoft.Json.dll</HintPath>
+    <Reference Include="Newtonsoft.Json, Version=9.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed, processorArchitecture=MSIL">
+      <HintPath>..\packages\Newtonsoft.Json.9.0.1\lib\net45\Newtonsoft.Json.dll</HintPath>
       <Private>True</Private>
     </Reference>
     <Reference Include="NLog, Version=4.0.0.0, Culture=neutral, PublicKeyToken=5120e14c03d0593c, processorArchitecture=MSIL">
-      <HintPath>..\packages\NLog.4.2.3\lib\net45\NLog.dll</HintPath>
+      <HintPath>..\packages\NLog.4.3.7\lib\net45\NLog.dll</HintPath>
       <Private>True</Private>
     </Reference>
     <Reference Include="Owin, Version=1.0.0.0, Culture=neutral, PublicKeyToken=f0ebd12fd5e55cc5, processorArchitecture=MSIL">

src/Jackett.Console/Program.cs

@@ -9,6 +9,7 @@ using System.Collections.Generic;
 using System.Diagnostics;
 using System.IO;
 using System.Linq;
+using System.Net;
 using System.Reflection;
 using System.Text;
 using System.Text.RegularExpressions;
@@ -96,7 +97,14 @@ namespace JackettConsole
                     Startup.IgnoreSslErrors = options.IgnoreSslErrors;
                     if (options.IgnoreSslErrors == true)
                     {
-                        Engine.Logger.Info("Curl will ignore SSL certificate errors.");
+                        Engine.Logger.Info("Jackett will ignore SSL certificate errors.");
+                    }
+
+                    // Choose Data Folder
+                    if (!string.IsNullOrWhiteSpace(options.DataFolder))
+                    {
+                        Startup.CustomDataFolder = options.DataFolder.Replace("\"", string.Empty).Replace("'", string.Empty).Replace(@"\\", @"\");
+                        Engine.Logger.Info("Jackett Data will be stored in: " + Startup.CustomDataFolder);
                     }
 
                     /*  ======     Actions    =====  */

src/Jackett.Console/packages.config

@@ -21,7 +21,7 @@
   <package id="Microsoft.Owin.Host.SystemWeb" version="3.0.1" targetFramework="net45" />
   <package id="Microsoft.Owin.Hosting" version="3.0.1" targetFramework="net45" />
   <package id="Microsoft.Owin.StaticFiles" version="3.0.1" targetFramework="net45" />
-  <package id="Newtonsoft.Json" version="7.0.1" targetFramework="net45" />
-  <package id="NLog" version="4.2.3" targetFramework="net45" />
+  <package id="Newtonsoft.Json" version="9.0.1" targetFramework="net45" />
+  <package id="NLog" version="4.3.7" targetFramework="net45" />
   <package id="Owin" version="1.0" targetFramework="net45" />
 </packages>

src/Jackett.Service/App.config

@@ -23,7 +23,7 @@
       </dependentAssembly>
       <dependentAssembly>
         <assemblyIdentity name="Newtonsoft.Json" publicKeyToken="30ad4fe6b2a6aeed" culture="neutral" />
-        <bindingRedirect oldVersion="0.0.0.0-7.0.0.0" newVersion="7.0.0.0" />
+        <bindingRedirect oldVersion="0.0.0.0-9.0.0.0" newVersion="9.0.0.0" />
       </dependentAssembly>
     </assemblyBinding>
   </runtime>

src/Jackett.Service/Jackett.Service.csproj

@@ -75,12 +75,12 @@
       <HintPath>..\packages\Microsoft.Owin.StaticFiles.3.0.1\lib\net45\Microsoft.Owin.StaticFiles.dll</HintPath>
       <Private>True</Private>
     </Reference>
-    <Reference Include="Newtonsoft.Json, Version=7.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed, processorArchitecture=MSIL">
-      <HintPath>..\packages\Newtonsoft.Json.7.0.1\lib\net45\Newtonsoft.Json.dll</HintPath>
+    <Reference Include="Newtonsoft.Json, Version=9.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed, processorArchitecture=MSIL">
+      <HintPath>..\packages\Newtonsoft.Json.9.0.1\lib\net45\Newtonsoft.Json.dll</HintPath>
       <Private>True</Private>
     </Reference>
     <Reference Include="NLog, Version=4.0.0.0, Culture=neutral, PublicKeyToken=5120e14c03d0593c, processorArchitecture=MSIL">
-      <HintPath>..\packages\NLog.4.2.3\lib\net45\NLog.dll</HintPath>
+      <HintPath>..\packages\NLog.4.3.7\lib\net45\NLog.dll</HintPath>
       <Private>True</Private>
     </Reference>
     <Reference Include="Owin, Version=1.0.0.0, Culture=neutral, PublicKeyToken=f0ebd12fd5e55cc5, processorArchitecture=MSIL">

src/Jackett.Service/packages.config

@@ -17,7 +17,7 @@
   <package id="Microsoft.Owin.Host.HttpListener" version="3.0.1" targetFramework="net45" />
   <package id="Microsoft.Owin.Hosting" version="3.0.1" targetFramework="net45" />
   <package id="Microsoft.Owin.StaticFiles" version="3.0.1" targetFramework="net45" />
-  <package id="Newtonsoft.Json" version="7.0.1" targetFramework="net45" />
-  <package id="NLog" version="4.2.3" targetFramework="net45" />
+  <package id="Newtonsoft.Json" version="9.0.1" targetFramework="net45" />
+  <package id="NLog" version="4.3.7" targetFramework="net45" />
   <package id="Owin" version="1.0" targetFramework="net45" />
 </packages>

src/Jackett.Test/Jackett.Test.csproj

@@ -62,12 +62,12 @@
       <HintPath>..\packages\CsQuery.1.3.4\lib\net40\CsQuery.dll</HintPath>
       <Private>True</Private>
     </Reference>
-    <Reference Include="FluentAssertions, Version=4.1.1.0, Culture=neutral, PublicKeyToken=33f2691a05b67b6a, processorArchitecture=MSIL">
-      <HintPath>..\packages\FluentAssertions.4.1.1\lib\net45\FluentAssertions.dll</HintPath>
+    <Reference Include="FluentAssertions, Version=4.13.0.0, Culture=neutral, PublicKeyToken=33f2691a05b67b6a, processorArchitecture=MSIL">
+      <HintPath>..\packages\FluentAssertions.4.13.0\lib\net45\FluentAssertions.dll</HintPath>
       <Private>True</Private>
     </Reference>
-    <Reference Include="FluentAssertions.Core, Version=4.1.1.0, Culture=neutral, PublicKeyToken=33f2691a05b67b6a, processorArchitecture=MSIL">
-      <HintPath>..\packages\FluentAssertions.4.1.1\lib\net45\FluentAssertions.Core.dll</HintPath>
+    <Reference Include="FluentAssertions.Core, Version=4.13.0.0, Culture=neutral, PublicKeyToken=33f2691a05b67b6a, processorArchitecture=MSIL">
+      <HintPath>..\packages\FluentAssertions.4.13.0\lib\net45\FluentAssertions.Core.dll</HintPath>
       <Private>True</Private>
     </Reference>
     <Reference Include="Microsoft.AspNet.Identity.Core, Version=2.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL">
@@ -87,12 +87,12 @@
       <HintPath>..\packages\Microsoft.Owin.Hosting.3.0.1\lib\net45\Microsoft.Owin.Hosting.dll</HintPath>
       <Private>True</Private>
     </Reference>
-    <Reference Include="Newtonsoft.Json, Version=7.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed, processorArchitecture=MSIL">
-      <HintPath>..\packages\Newtonsoft.Json.7.0.1\lib\net45\Newtonsoft.Json.dll</HintPath>
+    <Reference Include="Newtonsoft.Json, Version=9.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed, processorArchitecture=MSIL">
+      <HintPath>..\packages\Newtonsoft.Json.9.0.1\lib\net45\Newtonsoft.Json.dll</HintPath>
       <Private>True</Private>
     </Reference>
     <Reference Include="NLog, Version=4.0.0.0, Culture=neutral, PublicKeyToken=5120e14c03d0593c, processorArchitecture=MSIL">
-      <HintPath>..\packages\NLog.4.2.3\lib\net45\NLog.dll</HintPath>
+      <HintPath>..\packages\NLog.4.3.7\lib\net45\NLog.dll</HintPath>
       <Private>True</Private>
     </Reference>
     <Reference Include="nunit.core, Version=2.6.4.14350, Culture=neutral, PublicKeyToken=96d09a1eb7f44a77, processorArchitecture=MSIL">
@@ -103,8 +103,8 @@
       <HintPath>..\packages\NUnitTestAdapter.2.0.0\lib\nunit.core.interfaces.dll</HintPath>
       <Private>True</Private>
     </Reference>
-    <Reference Include="nunit.framework, Version=3.0.5813.39031, Culture=neutral, PublicKeyToken=2638cd05610744eb, processorArchitecture=MSIL">
-      <HintPath>..\packages\NUnit.3.0.1\lib\net45\nunit.framework.dll</HintPath>
+    <Reference Include="nunit.framework, Version=3.4.1.0, Culture=neutral, PublicKeyToken=2638cd05610744eb, processorArchitecture=MSIL">
+      <HintPath>..\packages\NUnit.3.4.1\lib\net45\nunit.framework.dll</HintPath>
       <Private>True</Private>
     </Reference>
     <Reference Include="nunit.util, Version=2.6.4.14350, Culture=neutral, PublicKeyToken=96d09a1eb7f44a77, processorArchitecture=MSIL">
@@ -164,6 +164,7 @@
     <Compile Include="TestIIndexerManagerServiceHelper.cs" />
     <Compile Include="TestUtil.cs" />
     <Compile Include="TestWebClient.cs" />
+    <Compile Include="Util\ParseUtilTests.cs" />
     <Compile Include="Util\ServerUtilTests.cs" />
     <Compile Include="Util\TvCategoryParserTests.cs" />
   </ItemGroup>
@@ -182,6 +183,9 @@
   <ItemGroup>
     <Folder Include="Indexers\" />
   </ItemGroup>
+  <ItemGroup>
+    <EmbeddedResource Include="Util\Invalid-RSS.xml" />
+  </ItemGroup>
   <Choose>
     <When Condition="'$(VisualStudioVersion)' == '10.0' And '$(IsCodedUITest)' == 'True'">
       <ItemGroup>

src/Jackett.Test/TestIIndexerManagerServiceHelper.cs

@@ -42,5 +42,10 @@ namespace JackettTest
         {
             throw new NotImplementedException();
         }
+
+        public void InitCardigannIndexers(string path)
+        {
+            throw new NotImplementedException();
+        }
     }
 }

src/Jackett.Test/Util/Invalid-RSS.xml

@@ -0,0 +1,20 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<rss version="2.0">
+  <channel>
+    <title>RSS Syndicator</title>
+    <link>http://somewebsite.com</link>
+    <description>
+      &lt;br /&gt;
+      Enjoy!&lt;br /&gt;
+      &lt;br /&gt;
+      -&lt;br /&gt;
+      &lt;br /&gt;
+       group info&lt;br /&gt;
+      &lt;br /&gt;
+      Know Your Role and Shut Your Mouth!&lt;br /&gt;
+      &lt;br /&gt;
+       we are now looking for...&lt;br /&gt;
+      &lt;br /&gt;
+    </description>
+  </channel>
+</rss>

src/Jackett.Test/Util/ParseUtilTests.cs

@@ -0,0 +1,46 @@
+using System;
+using System.Collections.Generic;
+using System.IO;
+using System.Linq;
+using System.Text;
+using System.Threading.Tasks;
+using System.Xml.Linq;
+using System.Xml.XPath;
+using FluentAssertions;
+using Jackett.Utils;
+using NUnit.Framework;
+
+namespace JackettTest.Util
+{
+    [TestFixture]
+    public class ParseUtilTests
+    {
+        private static string InvalidRssXml
+        {
+            get
+            {
+                var type = typeof(ParseUtilTests);
+                using (var resourceStream = type.Assembly.GetManifestResourceStream($"{type.Namespace}.Invalid-RSS.xml"))
+                using (var sr = new StreamReader(resourceStream))
+                {
+                    return sr.ReadToEnd();
+                }
+                
+            }
+        }
+
+        [Test]
+        public void Invalid_RSS_should_parse_after_removing_invalid_chars()
+        {
+            var invalidRss = InvalidRssXml;
+            Action parseAction = () => XDocument.Parse(invalidRss);
+            parseAction.ShouldThrow<Exception>().WithMessage("'\a', hexadecimal value 0x07, is an invalid character. Line 12, position 7.");
+
+            var validRSs = ParseUtil.RemoveInvalidXmlChars(invalidRss);
+            var rssDoc = XDocument.Parse(validRSs);
+            rssDoc.Root.Should().NotBeNull();
+            var description = rssDoc.Root.XPathSelectElement("//description");
+            description.Value.Should().Contain("Know Your Role and Shut Your Mouth!");
+        }
+    }
+}

src/Jackett.Test/app.config

@@ -20,7 +20,7 @@
       </dependentAssembly>
       <dependentAssembly>
         <assemblyIdentity name="Newtonsoft.Json" publicKeyToken="30ad4fe6b2a6aeed" culture="neutral" />
-        <bindingRedirect oldVersion="0.0.0.0-7.0.0.0" newVersion="7.0.0.0" />
+        <bindingRedirect oldVersion="0.0.0.0-9.0.0.0" newVersion="9.0.0.0" />
       </dependentAssembly>
       <dependentAssembly>
         <assemblyIdentity name="AutoMapper" publicKeyToken="be96cd2c38ef1005" culture="neutral" />

src/Jackett.Test/packages.config

@@ -6,7 +6,7 @@
   <package id="Autofac.WebApi2.Owin" version="3.3.0" targetFramework="net45" />
   <package id="AutoMapper" version="4.1.1" targetFramework="net45" />
   <package id="CsQuery" version="1.3.4" targetFramework="net45" />
-  <package id="FluentAssertions" version="4.1.1" targetFramework="net45" />
+  <package id="FluentAssertions" version="4.13.0" targetFramework="net45" />
   <package id="Microsoft.AspNet.Identity.Core" version="2.2.1" targetFramework="net45" />
   <package id="Microsoft.AspNet.WebApi.Client" version="5.2.3" targetFramework="net45" />
   <package id="Microsoft.AspNet.WebApi.Core" version="5.2.3" targetFramework="net45" />
@@ -18,9 +18,9 @@
   <package id="Microsoft.Owin" version="3.0.1" targetFramework="net45" />
   <package id="Microsoft.Owin.Host.HttpListener" version="3.0.1" targetFramework="net45" />
   <package id="Microsoft.Owin.Hosting" version="3.0.1" targetFramework="net45" />
-  <package id="Newtonsoft.Json" version="7.0.1" targetFramework="net45" />
-  <package id="NLog" version="4.2.3" targetFramework="net45" />
-  <package id="NUnit" version="3.0.1" targetFramework="net45" />
+  <package id="Newtonsoft.Json" version="9.0.1" targetFramework="net45" />
+  <package id="NLog" version="4.3.7" targetFramework="net45" />
+  <package id="NUnit" version="3.4.1" targetFramework="net45" />
   <package id="NUnitTestAdapter" version="2.0.0" targetFramework="net45" />
   <package id="Owin" version="1.0" targetFramework="net45" />
 </packages>

src/Jackett.Tray/App.config

@@ -23,7 +23,7 @@
       </dependentAssembly>
       <dependentAssembly>
         <assemblyIdentity name="Newtonsoft.Json" publicKeyToken="30ad4fe6b2a6aeed" culture="neutral" />
-        <bindingRedirect oldVersion="0.0.0.0-7.0.0.0" newVersion="7.0.0.0" />
+        <bindingRedirect oldVersion="0.0.0.0-9.0.0.0" newVersion="9.0.0.0" />
       </dependentAssembly>
       <dependentAssembly>
         <assemblyIdentity name="AutoMapper" publicKeyToken="be96cd2c38ef1005" culture="neutral" />

src/Jackett.Tray/Jackett.Tray.csproj

@@ -79,12 +79,12 @@
       <HintPath>..\packages\Microsoft.Owin.StaticFiles.3.0.1\lib\net45\Microsoft.Owin.StaticFiles.dll</HintPath>
       <Private>True</Private>
     </Reference>
-    <Reference Include="Newtonsoft.Json, Version=7.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed, processorArchitecture=MSIL">
-      <HintPath>..\packages\Newtonsoft.Json.7.0.1\lib\net45\Newtonsoft.Json.dll</HintPath>
+    <Reference Include="Newtonsoft.Json, Version=9.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed, processorArchitecture=MSIL">
+      <HintPath>..\packages\Newtonsoft.Json.9.0.1\lib\net45\Newtonsoft.Json.dll</HintPath>
       <Private>True</Private>
     </Reference>
     <Reference Include="NLog, Version=4.0.0.0, Culture=neutral, PublicKeyToken=5120e14c03d0593c, processorArchitecture=MSIL">
-      <HintPath>..\packages\NLog.4.2.3\lib\net45\NLog.dll</HintPath>
+      <HintPath>..\packages\NLog.4.3.7\lib\net45\NLog.dll</HintPath>
       <Private>True</Private>
     </Reference>
     <Reference Include="Owin, Version=1.0.0.0, Culture=neutral, PublicKeyToken=f0ebd12fd5e55cc5, processorArchitecture=MSIL">

src/Jackett.Tray/packages.config

@@ -18,7 +18,7 @@
   <package id="Microsoft.Owin.Host.HttpListener" version="3.0.1" targetFramework="net45" />
   <package id="Microsoft.Owin.Hosting" version="3.0.1" targetFramework="net45" />
   <package id="Microsoft.Owin.StaticFiles" version="3.0.1" targetFramework="net45" />
-  <package id="Newtonsoft.Json" version="7.0.1" targetFramework="net45" />
-  <package id="NLog" version="4.2.3" targetFramework="net45" />
+  <package id="Newtonsoft.Json" version="9.0.1" targetFramework="net45" />
+  <package id="NLog" version="4.3.7" targetFramework="net45" />
   <package id="Owin" version="1.0" targetFramework="net45" />
 </packages>

src/Jackett.Updater/App.config

@@ -23,7 +23,7 @@
       </dependentAssembly>
       <dependentAssembly>
         <assemblyIdentity name="Newtonsoft.Json" publicKeyToken="30ad4fe6b2a6aeed" culture="neutral" />
-        <bindingRedirect oldVersion="0.0.0.0-7.0.0.0" newVersion="7.0.0.0" />
+        <bindingRedirect oldVersion="0.0.0.0-9.0.0.0" newVersion="9.0.0.0" />
       </dependentAssembly>
     </assemblyBinding>
   </runtime>

src/Jackett.Updater/Jackett.Updater.csproj

@@ -48,7 +48,7 @@
       <Private>True</Private>
     </Reference>
     <Reference Include="NLog, Version=4.0.0.0, Culture=neutral, PublicKeyToken=5120e14c03d0593c, processorArchitecture=MSIL">
-      <HintPath>..\packages\NLog.4.2.3\lib\net45\NLog.dll</HintPath>
+      <HintPath>..\packages\NLog.4.3.7\lib\net45\NLog.dll</HintPath>
       <Private>True</Private>
     </Reference>
     <Reference Include="System" />

src/Jackett.Updater/packages.config

@@ -5,5 +5,5 @@
   <package id="Microsoft.Bcl" version="1.1.10" targetFramework="net45" />
   <package id="Microsoft.Bcl.Build" version="1.0.21" targetFramework="net45" />
   <package id="Microsoft.Net.Http" version="2.2.29" targetFramework="net45" />
-  <package id="NLog" version="4.2.3" targetFramework="net45" />
+  <package id="NLog" version="4.3.7" targetFramework="net45" />
 </packages>

src/Jackett/App.config

@@ -9,7 +9,7 @@
     <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
       <dependentAssembly>
         <assemblyIdentity name="Newtonsoft.Json" publicKeyToken="30ad4fe6b2a6aeed" culture="neutral" />
-        <bindingRedirect oldVersion="0.0.0.0-7.0.0.0" newVersion="7.0.0.0" />
+        <bindingRedirect oldVersion="0.0.0.0-9.0.0.0" newVersion="9.0.0.0" />
       </dependentAssembly>
       <dependentAssembly>
         <assemblyIdentity name="System.Web.Http" publicKeyToken="31bf3856ad364e35" culture="neutral" />

src/Jackett/Content/custom.js

@@ -200,9 +200,35 @@ function populateConfigItems(configForm, config) {
             var template = setupItemTemplate(item);
             $formItemContainer.append(template);
             if (item.type === 'recaptcha') {
-                grecaptcha.render($('.jackettrecaptcha')[0], {
-                    'sitekey': item.sitekey
-                });
+                var jackettrecaptcha = $('.jackettrecaptcha');
+                jackettrecaptcha.data("version", item.version);
+                switch (item.version) {
+                    case "1":
+                        // The v1 reCAPTCHA code uses document.write() calls to write the CAPTCHA to the location where the script was loaded.
+                        // As it's loaded async this doesn't work.
+                        // We use an iframe to work around this problem.
+                        var html = '<script type="text/javascript" src="https://www.google.com/recaptcha/api/challenge?k='+encodeURIComponent(item.sitekey)+'"></script>';
+                        var frame = document.createElement('iframe');
+                        frame.id = "jackettrecaptchaiframe";
+                        frame.style.height = "145px";
+                        frame.style.weight = "326px";
+                        frame.style.border = "none";
+                        frame.onload = function () {
+                            // auto resize iframe to content
+                            frame.style.height = frame.contentWindow.document.body.scrollHeight + 'px';
+                            frame.style.width = frame.contentWindow.document.body.scrollWidth + 'px';
+                        }
+                        jackettrecaptcha.append(frame);
+                        frame.contentDocument.open();
+                        frame.contentDocument.write(html);
+                        frame.contentDocument.close();
+                        break;
+                    case "2":
+                        grecaptcha.render(jackettrecaptcha[0], {
+                            'sitekey': item.sitekey
+                        });
+                        break;
+                }
             }
         }
     }
@@ -235,7 +261,18 @@ function getConfigModalJson(configForm) {
                 break;
             case "recaptcha":
                 if (window.jackettIsLocal) {
-                    itemEntry.value = $('.g-recaptcha-response').val();
+                    var version = $el.find('.jackettrecaptcha').data("version");
+                    switch (version) {
+                        case "1":
+                            var frameDoc = $("#jackettrecaptchaiframe")[0].contentDocument;
+                            itemEntry.version = version;
+                            itemEntry.challenge = $("#recaptcha_challenge_field", frameDoc).val()
+                            itemEntry.value = $("#recaptcha_response_field", frameDoc).val()
+                            break;
+                        case "2":
+                            itemEntry.value = $('.g-recaptcha-response').val();
+                            break;
+                    }
                 } else {
                     itemEntry.cookie = $el.find(".setup-item-recaptcha input").val();
                 }
@@ -308,6 +345,32 @@ function clearNotifications() {
     $('[data-notify="container"]').remove();
 }
 
+function updateReleasesRow(row)
+{    
+    var labels = $(row).find("span.release-labels");
+    var DownloadVolumeFactor = parseFloat($(row).find("td.DownloadVolumeFactor").html());
+    var UploadVolumeFactor = parseFloat($(row).find("td.UploadVolumeFactor").html());
+
+    labels.empty();
+
+    if (!isNaN(DownloadVolumeFactor)) {
+        if (DownloadVolumeFactor == 0) {
+            labels.append('\n<span class="label label-success">FREELEECH</span>');
+        } else if (DownloadVolumeFactor < 1) {
+            labels.append('\n<span class="label label-primary">' + DownloadVolumeFactor * 100 + '%DL</span>');
+        } else if (DownloadVolumeFactor > 1) {
+            labels.append('\n<span class="label label-danger">' + DownloadVolumeFactor * 100 + '%DL</span>');
+        }
+    }
+
+    if (!isNaN(UploadVolumeFactor)) {
+        if (UploadVolumeFactor == 0) {
+            labels.append('\n<span class="label label-warning">NO UPLOAD</span>');
+        } else if (UploadVolumeFactor != 1) {
+            labels.append('\n<span class="label label-info">' + UploadVolumeFactor * 100 + '%UL</span>');
+        }
+    }
+}
 
 function bindUIButtons() {
     $('body').on('click', '.downloadlink', function (e, b) {
@@ -335,6 +398,7 @@ function bindUIButtons() {
             var releaseTemplate = Handlebars.compile($("#jackett-releases").html());
             var item = { releases: data, Title: 'Releases' };
             var releaseDialog = $(releaseTemplate(item));
+            releaseDialog.find('tr.jackett-releases-row').each(function () { updateReleasesRow(this); });
             releaseDialog.find('table').DataTable(
                  {
                      "pageLength": 20,
@@ -382,7 +446,7 @@ function bindUIButtons() {
                          var count = 0;
                          this.api().columns().every(function () {
                              count++;
-                             if (count === 5 || count === 9) {
+                             if (count === 5 || count === 10) {
                                  var column = this;
                                  var select = $('<select><option value=""></option></select>')
                                      .appendTo($(column.footer()).empty())
@@ -430,6 +494,10 @@ function bindUIButtons() {
             var releaseDialog = $(releaseTemplate({ indexers: indexers }));
             $("#modals").append(releaseDialog);
             releaseDialog.modal("show");
+            
+            releaseDialog.on('shown.bs.modal', function() {
+                releaseDialog.find('#searchquery').focus();
+            });
 
             var setCategories = function (tracker, items) {
                 var cats = {};
@@ -455,6 +523,13 @@ function bindUIButtons() {
                 setCategories(trackerId, this.items);
             }, scope));
 
+            document.getElementById("searchquery")
+            .addEventListener("keyup", function (event) {
+                event.preventDefault();
+                if (event.keyCode == 13) {
+                    document.getElementById("jackett-search-perform").click();
+                }
+            });
 
             $('#jackett-search-perform').click(function () {
                 if ($('#jackett-search-perform').text().trim() !== 'Search trackers') {
@@ -474,6 +549,7 @@ function bindUIButtons() {
                     var resultsTemplate = Handlebars.compile($("#jackett-search-results").html());
                     var results = $('#searchResults');
                     results.html($(resultsTemplate(data)));
+                    results.find('tr.jackett-search-results-row').each(function () { updateReleasesRow(this); });
 
                     results.find('table').DataTable(
                     {
@@ -510,7 +586,7 @@ function bindUIButtons() {
                             var count = 0;
                             this.api().columns().every(function () {
                                 count++;
-                                if (count === 3 || count === 7) {
+                                if (count === 3 || count === 8) {
                                     var column = this;
                                     var select = $('<select><option value=""></option></select>')
                                         .appendTo($(column.footer()).empty())
@@ -622,4 +698,4 @@ function bindUIButtons() {
             doNotify("Request to Jackett server failed", "danger", "glyphicon glyphicon-alert");
         });
     });
-}
+}

src/Jackett/Content/index.html

@@ -236,26 +236,34 @@
                                     <th>Name</th>
                                     <th>Size</th>
                                     <th>Size</th>
+                                    <th>Files</th>
                                     <th>Category</th>
+                                    <th>Grabs</th>
                                     <th>Seeds</th>
                                     <th>Leechers</th>
+                                    <th>DL Factor</th>
+                                    <th>UL Factor</th>
                                     <th>Download</th>
                                 </tr>
                             </thead>
                             <tbody>
                                 {{#each releases}}
-                                <tr>
+                                <tr class="jackett-releases-row">
                                     <td>{{PublishDate}}</td>
                                     <td>{{FirstSeen}}</td>
                                     <td>{{jacketTimespan PublishDate}}</td>
                                     <td>{{jacketTimespan FirstSeen}}</td>
                                     <td>{{Tracker}}</td>
-                                    <td><a href="{{Comments}}">{{Title}}</a></td>
+                                    <td><a href="{{Comments}}">{{Title}}</a> <span class="release-labels"></span></td>
                                     <td>{{Size}}</td>
                                     <td>{{jacketSize Size}}</td>
+                                    <td>{{Files}}</td>
                                     <td>{{CategoryDesc}}</td>
+                                    <td>{{Grabs}}</td>
                                     <td>{{Seeders}}</td>
                                     <td>{{Peers}}</td>
+                                    <td class="DownloadVolumeFactor">{{DownloadVolumeFactor}}</td>
+                                    <td class="UploadVolumeFactor">{{UploadVolumeFactor}}</td>
                                     <td class="downloadcolumn">
                                         <a class="downloadlink" title="Download locally" href="{{Link}}"><i class="fa fa-download"></i></a>
                                         {{#if BlackholeLink}}
@@ -277,6 +285,12 @@
                                     <td></td>
                                     <td></td>
                                     <td></td>
+                                    <td></td>
+                                    <td></td>
+                                    <td></td>
+                                    <td></td>
+                                    <td></td>
+                                    <td></td>
                                 </tr>
                             </tfoot>
                         </table>
@@ -334,24 +348,32 @@
                     <th>Name</th>
                     <th>Size</th>
                     <th>Size</th>
+                    <th>Files</th>
                     <th>Category</th>
+                    <th>Grabs</th>
                     <th>Seeds</th>
                     <th>Leechers</th>
+                    <th>DL Factor</th>
+                    <th>UL Factor</th>
                     <th>Download</th>
                 </tr>
             </thead>
             <tbody>
                 {{#each Results}}
-                <tr>
+                <tr class="jackett-search-results-row">
                     <td>{{PublishDate}}</td>
                     <td>{{jacketTimespan PublishDate}}</td>
                     <td>{{Tracker}}</td>
-                    <td><a href="{{Comments}}">{{Title}}</a></td>
+                    <td><a href="{{Comments}}">{{Title}}</a> <span class="release-labels"></span></td>
                     <td>{{Size}}</td>
                     <td>{{jacketSize Size}}</td>
+                    <td>{{Files}}</td>
                     <td>{{CategoryDesc}}</td>
+                    <td>{{Grabs}}</td>
                     <td>{{Seeders}}</td>
                     <td>{{Peers}}</td>
+                    <td class="DownloadVolumeFactor">{{DownloadVolumeFactor}}</td>
+                    <td class="UploadVolumeFactor">{{UploadVolumeFactor}}</td>
                     <td class="downloadcolumn">
                         <a class="downloadlink" title="Download locally" href="{{Link}}"><i class="fa fa-download"></i></a>
                         {{#if BlackholeLink}}
@@ -371,6 +393,12 @@
                     <td></td>
                     <td></td>
                     <td></td>
+                    <td></td>
+                    <td></td>
+                    <td></td>
+                    <td></td>
+                    <td></td>
+                    <td></td>
                 </tr>
             </tfoot>
         </table>

src/Jackett/Controllers/PotatoController.cs

@@ -100,7 +100,8 @@ namespace Jackett.Controllers
                 ApiKey =  request.passkey,
                 Categories = MOVIE_CATS,
                 SearchTerm = request.search,
-                ImdbID = request.imdbid
+                ImdbID = request.imdbid,
+                QueryType = "TorrentPotato"
             };
 
             IEnumerable<ReleaseInfo> releases = new List<ReleaseInfo>();

src/Jackett/CurlHelper.cs

@@ -173,7 +173,7 @@ namespace Jackett
                 var headerParts = headerString.Split(new char[] { '\n', '\r' }, StringSplitOptions.RemoveEmptyEntries);
                 var headers = new List<string[]>();
                 var headerCount = 0;
-                HttpStatusCode status = HttpStatusCode.InternalServerError;
+                HttpStatusCode status = HttpStatusCode.NotImplemented;
                 var cookieBuilder = new StringBuilder();
                 var cookies = new List<Tuple<string, string>>();
                 foreach (var headerPart in headerParts)
@@ -216,6 +216,31 @@ namespace Jackett
                     cookieBuilder.AppendFormat("{0} ", cookieGroup.Last().Item2);
                 }
 
+                // add some debug output to track down the problem causing people getting InternalServerError results
+                if (status == HttpStatusCode.NotImplemented || status == HttpStatusCode.InternalServerError)
+                {
+                    try
+                    {
+                        OnErrorMessage("got NotImplemented/InternalServerError");
+                        OnErrorMessage("request.Method: " + curlRequest.Method);
+                        OnErrorMessage("request.Url: " + curlRequest.Url);
+                        OnErrorMessage("request.Cookies: " + curlRequest.Cookies);
+                        OnErrorMessage("request.Referer: " + curlRequest.Referer);
+                        OnErrorMessage("request.RawPOSTDdata: " + curlRequest.RawPOSTDdata);
+                        OnErrorMessage("cookies: "+ cookieBuilder.ToString().Trim());
+                        OnErrorMessage("headerString:\n" + headerString);
+                    
+                        foreach (var headerPart in headerParts)
+                        {
+                            OnErrorMessage("headerParts: "+headerPart);
+                        }
+                    }
+                    catch (Exception ex)
+                    {
+                        OnErrorMessage(string.Format("CurlHelper: error while handling NotImplemented/InternalServerError:\n{0}", ex));
+                    }
+                }
+                
                 var contentBytes = Combine(contentBuffers.ToArray());
                 var curlResponse = new CurlResponse(headers, contentBytes, status, cookieBuilder.ToString().Trim());
                 return curlResponse;

src/Jackett/Definitions/alphareign.yml

@@ -0,0 +1,81 @@
+---
+  site: alphareign
+  name: AlphaReign
+  description: "A DHT only tracker"
+  language: en-us
+
+  links:
+    - https://alphareign.se
+
+  caps:
+    categories:
+      Movies: Movies
+      Movies/HD: Movies/HD
+      Movies/SD: Movies/SD
+      TV: TV
+      TV/HD: TV/HD
+      TV/SD: TV/SD
+      Other: Other
+
+    modes:
+      search: [q]
+      tv-search: [q, season, ep]
+
+  login:
+    path: /login
+    method: form
+    form: form[action^="/login"]
+    inputs:
+      username: "{{ .Config.username }}"
+      password: "{{ .Config.password }}"
+    error:
+      - path: /login
+        selector: div.auth-form > div > div.alert-danger
+
+    test:
+      path: settings
+
+  ratio:
+    text: "∞"
+
+  search:
+    path: /
+    method: get
+    inputs:
+      query: "{{ .Keywords }}"
+    rows:
+      selector: .result
+    fields:
+      title:
+        attribute: data-name
+      details:
+        selector: a[href^="/torrent"]
+        attribute: href
+      download:
+        selector: a[href^="magnet:"]
+        attribute: href
+      date:
+        attribute: data-added
+      size:
+        attribute: data-size
+      seeders:
+        attribute: data-seeders
+      leechers:
+        attribute: data-leechers
+      downloadvolumefactor:
+        case:
+          "*": "1"
+      uploadvolumefactor:
+        case:
+          "*": "1"
+      category:
+        case:
+          "span[class=\"label label-primary\"]:contains(\"video\") + span[class=\"label label-success\"]:contains(\"movie\") + span[class=\"label label-info\"]:contains(\"HD\")": "Movies/HD"
+          "span[class=\"label label-primary\"]:contains(\"video\") + span[class=\"label label-success\"]:contains(\"movie\") + span[class=\"label label-info\"]:contains(\"720p\")": "Movies/HD"
+          "span[class=\"label label-primary\"]:contains(\"video\") + span[class=\"label label-success\"]:contains(\"movie\") + span[class=\"label label-info\"]:contains(\"1080p\")": "Movies/HD"
+          "span[class=\"label label-primary\"]:contains(\"video\") + span[class=\"label label-success\"]:contains(\"movie\") + span[class=\"label label-info\"]:contains(\"SD\")": "Movies/SD"
+          "span[class=\"label label-primary\"]:contains(\"video\") + span[class=\"label label-success\"]:contains(\"movie\")": "Movies"
+          "span[class=\"label label-primary\"]:contains(\"video\") + span[class=\"label label-success\"]:contains(\"show\") + span[class=\"label label-info\"]:contains(\"HD\")": "TV/HD"
+          "span[class=\"label label-primary\"]:contains(\"video\") + span[class=\"label label-success\"]:contains(\"show\") + span[class=\"label label-info\"]:contains(\"SD\")": "TV/SD"
+          "span[class=\"label label-primary\"]:contains(\"video\") + span[class=\"label label-success\"]:contains(\"show\")": "TV"
+          "*": "Other"

src/Jackett/Definitions/eotforum.yml

@@ -0,0 +1,176 @@
+---
+  site: eotforum
+  name: EoT-Forum
+  description: "A German gerneral tracker"
+  language: de-de
+  links:
+    - http://eot-forum.net
+
+  caps:
+    categories:
+      # Filme
+      14: Movies/SD # SD XviD
+      15: Movies/SD # SD x264
+      16: Movies/HD # HD
+      68: Movies/HD # UHD
+      17: Movies/3D # 3D
+      18: Movies/DVD # DVD-R
+      19: Movies # Pack
+      20: Movies # International
+      21: XXX # XXX
+
+      # Serien/TV
+      23: TV/SD # SD XviD
+      24: TV/SD # SD x264
+      25: TV/HD # HD
+      26: TV/SD # DVD-R
+      27: TV # Pack
+      28: TV # International
+      29: TV/Sport # Sport
+
+      # Dokus
+      31: TV/Documentary # SD XviD
+      32: TV/Documentary # SD x264
+      33: TV/Documentary # HD
+      34: TV/Documentary # 3D
+      35: TV/Documentary # Pack
+      67: TV/Documentary # DVD-R
+      36: TV/Documentary # International
+
+      # Audio
+      38: Audio # Charts
+      39: Audio/MP3 # MP3
+      40: Audio/Lossless # Flac
+      41: Audio # Pack
+      42: Audio/Video # MusikVideo
+      43: Audio/Audiobook # Hörbücher
+
+      # Spiele
+      45: PC/Games # Windows
+      46: PC/Mac # MacOS
+      47: Console/PS4 # Sony PS
+      48: Console/Xbox  # Microsoft XBox
+      49: Console/NDS # Nintendo
+      50: PC/Games # Linux
+      51: Console # Andere
+
+      # Software
+      53: PC # Windows
+      54: PC/Mac # MacOS
+      55: PC # Linux
+      56: PC/Phone-Android # Android
+      57: PC/Phone-IOS # Apple IOS
+      58: PC/Phone-Other # Andere
+
+      # Sonstiges
+      60: Books # EBooks
+      61: Other # Bilder
+      62: TV/Anime # Anime
+      63: Other # MISC
+      64: XXX # XXX-Bilder/EBooks/Audio
+
+      # EOT-Specials
+      66: Other # Special
+
+    modes:
+      search: [q]
+      tv-search: [q, season, ep]
+
+  login:
+    path: index.php?page=login
+    method: post
+    inputs:
+      uid: "{{ .Config.username }}"
+      pwd: "{{ .Config.password }}"
+      rememberme: "forever"
+      submit: "Login"
+    error:
+    - selector: td.lista[align="center"][colspan="2"] > span
+    test:
+      path: index.php
+      selector: img[alt="Ratio"]
+
+  ratio:
+    path: index.php
+    selector: img[alt="Ratio"] + font
+
+  search:
+    path: index.php
+    inputs:
+      page: "torrents"
+      search: "{{ .Query.Keywords }}"
+      options: "0"
+      active: "0"
+      gold: "0"
+    rows:
+      selector: table.lista > tbody > tr:has(a[href^="index.php?page=torrent-details&id="])
+      dateheaders:
+        selector: ":has(td.header > b)"
+        filters:
+          - name: replace
+            args: ["Torrents vom ", ""]
+          - name: replace
+            args: ["Januar", "January"]
+          - name: replace
+            args: ["Februar", "February"]
+          - name: replace
+            args: ["March", "März"]
+          - name: replace
+            args: ["Mai", "May"]
+          - name: replace
+            args: ["Juni", "June"]
+          - name: replace
+            args: ["Juli", "July"]
+          - name: replace
+            args: ["Oktober", "October"]
+          - name: replace
+            args: ["Dezember", "December"]
+          - name: dateparse
+            args: "02.January.2006"
+    fields:
+      download:
+        selector: a[href^="download.php?id="]
+        attribute: href
+      title:
+        selector: a[href^="index.php?page=torrent-details&id="]
+        attribute: title
+        filters:
+          - name: replace
+            args: ["Details anzeigen: ", ""]
+      category:
+        selector: a[href^="index.php?page=torrents&category="]
+        attribute: href
+        filters:
+          - name: querystring
+            args: category
+      comments:
+        selector: a[href*="#comments"]
+        attribute: href
+      size:
+        selector: td:nth-child(3)
+      grabs:
+        selector: td:nth-child(5) > font:nth-child(3)
+        filters:
+          - name: replace
+            args: ["---", "0"]
+      seeders:
+        selector: td:nth-child(5) > font:nth-child(1)
+      leechers:
+        selector: td:nth-child(5) > font:nth-child(2)
+      downloadvolumefactor:
+        case:
+          img[alt="gold"]: "0"
+          img[alt="silver"]: "0.5"
+          "*": "1"
+      uploadvolumefactor:
+        case:
+          img[alt="2x Upload Multiplier"]: "2"
+          img[alt="3x Upload Multiplier"]: "3"
+          img[alt="4x Upload Multiplier"]: "4"
+          img[alt="5x Upload Multiplier"]: "5"
+          img[alt="6x Upload Multiplier"]: "6"
+          img[alt="7x Upload Multiplier"]: "7"
+          img[alt="8x Upload Multiplier"]: "8"
+          img[alt="9x Upload Multiplier"]: "9"
+          img[alt="10x Upload Multiplier"]: "10"
+          "*": "1"

src/Jackett/Definitions/ethor.yml

@@ -0,0 +1,113 @@
+---
+  site: ethor
+  name: Ethor.net (Thor's Land)
+  description: "A French gerneral tracker"
+  language: fr-fr
+  links:
+    - https://ethor.net/
+
+  caps:
+    categories:
+      22: PC # Applications/Divers
+      1: PC # Applications/PC ISO
+      44: PC # Applications/Portable
+      47: Movies/BluRay # Films/Bluray
+      20: Movies/DVD # Films/DVDr
+      42: Movies/HD # Films/HD Rip
+      19: Movies/SD # Films/SD Rip
+      5: Movies/SD # Films/VCD
+      4: PC/Games # Jeux/PC
+      41: Console # Jeux/Portable
+      34: Console/PS4 # Jeux/PS2-PS3
+      38: Console/Wii # Jeux/Wii-GC
+      40: Console/Xbox # Jeux/Xbox360
+      6: Audio # Musique
+      37: Audio/Video # Musique/Video
+      48: TV/HD # Série-Télé/Bluray
+      45: TV/SD # Série-Télé/DVDr
+      43: TV/HD # Série-Télé/HD Rip
+      7: TV/SD # Série-Télé/SD Rip
+      23: Books # E-Books
+      46: Other # Évé. sportif
+      36: Other # Kidz
+      25: Other # Misc
+      9: XXX # XXX
+
+    modes:
+      search: [q]
+      tv-search: [q, season, ep]
+
+  settings:
+    - name: cookie
+      type: text
+      label: Cookie
+
+  login:
+    method: cookie
+    inputs:
+      cookie: "{{ .Config.cookie }}"
+    test:
+      path: browse.php
+
+  ratio:
+    path: browse.php
+    selector: span#ratioRatio
+
+  search:
+    path: browse.php
+    inputs:
+      $raw: "{{range .Categories}}c{{.}}=1&{{end}}"
+      search: "{{ .Query.Keywords }}"
+      advcat: "0"
+      incldead: "1"
+      stype: "b"
+      dp: "0"
+      isUserClick: "0"
+    rows:
+      selector: p + table > tbody > tr:has(a[href^="/details.php"])
+    fields:
+      download:
+        selector: a[href^="/details.php"]:has(b)
+        attribute: href
+        filters:
+          - name: replace
+            args: ["/details.php", "/download.php"]
+      title:
+        selector: a[href^="/details.php"]:has(b)
+      category:
+        selector: a[href^="/browse.php?cat="]
+        attribute: href
+        filters:
+          - name: querystring
+            args: cat
+      comments:
+        selector: a[href^="/details.php"]:has(b)
+        attribute: href
+      files:
+        selector: a[href*="#filelist"]
+      size:
+        selector: td:nth-child(6)
+      grabs:
+        selector: td:nth-child(7)
+        filters:
+          - name: regexp
+            args: "(\\d+)"
+      seeders:
+        selector: td:nth-child(8)
+      leechers:
+        selector: td:nth-child(9)
+      date:
+        selector: td:nth-child(5)
+        filters:
+          - name: append
+            args: " -05:00" # timezone offset
+          - name: dateparse
+            args: "2006-01-0215:04:05 -07:00"
+      downloadvolumefactor:
+        case:
+          "img[title^=\"Freeleech: \"]": "0"
+          "img[title^=\"Half Freeleech: \"]": "0.5"
+          "*": "1"
+      uploadvolumefactor:
+        case:
+          "*": "1"

src/Jackett/Definitions/hdme.yml

@@ -0,0 +1,101 @@
+---
+  site: hdme
+  name: HDME
+  language: en-us
+  links:
+    - https://hdme.eu
+
+  caps:
+    categories:
+      24: TV/Anime       # Anime
+      25: PC/0day        # Appz
+      47: Movies/HD      # AVCHD
+      26: Movies/BluRay  # Bluray
+      54: Movies/HD      # dbREMUX
+      41: Movies/HD      # Documentaries
+      50: Movies/HD      # FourGHD
+      44: Movies/HD      # HDME
+      28: Audio/Lossless # HQ Music
+      48: Movies/HD      # iCandy
+      45: Movies/HD      # INtL
+      29: Other          # Misc
+      49: PC/Phone-Other # Mobile
+      30: Movies/HD      # Movie 1080i
+      31: Movies/HD      # Movie 1080p
+      32: Movies/HD      # Movie 720p
+      33: Audio/Video    # Music Videos
+      34: TV             # Packs
+      53: Movies/HD      # Remux
+      56: Movies/HD      # RUXi
+      55: Movies/HD      # SiNiSteR
+      36: TV/Sport       # Sports
+      37: TV/HD          # TV Series 1080i
+      38: TV/HD          # TV Series 1080p
+      39: TV/HD          # TV Series 720p
+      57: Movies         # UHD 2160p
+      40: XXX            # XXX
+
+    modes:
+      search: [q]
+      tv-search: [q, season, ep]
+
+  login:
+    path: /takelogin.php
+    method: post
+    form: form
+    inputs:
+      username: "{{ .Config.username }}"
+      password: "{{ .Config.password }}"
+    error:
+      - selector: td.embedded
+        message:
+          selector: td.text
+    test:
+      path: /my.php
+
+  ratio:
+    path: /my.php
+    selector: span.smallfont > font
+    filters:
+      - name: regexp
+        args: "Ratio:(.+?)Uploaded"
+      - name: replace
+        args: [",", ""]
+
+  search:
+    path: /browse.php
+    inputs:
+      $raw: "{{range .Categories}}c{{.}}=1&{{end}}"
+      search: "{{ .Keywords }}"
+      incldead: "1"
+      blah: "0"
+    rows:
+      selector: table[width="100%"] > tbody > tr:has(td.bottom[background="_images/bg_torrent.jpg"])
+    fields:
+      category:
+        selector: td:nth-child(2) a
+        attribute: href
+        filters:
+          - name: querystring
+            args: cat
+      title:
+        selector: td:nth-child(3) > a
+        attribute: title
+      comments:
+        selector: td:nth-child(3) > a
+        attribute: href
+      download:
+        selector: td:nth-child(11) > a
+        attribute: href
+      size:
+        selector: td:nth-child(6)
+        remove: br
+      date:
+        selector: td:nth-child(3)
+        filters:
+          - name: regexp
+            args: "Added: (.+?)\n"
+      seeders:
+        selector: td:nth-child(8)
+      leechers:
+        selector: td:nth-child(9)

src/Jackett/Definitions/newretro.yml

@@ -0,0 +1,117 @@
+---
+  site: newretro
+  name: The New Retro
+  description: "A German gerneral tracker"
+  language: de-de
+  links:
+    - http://new-retro.ddns.net/
+
+  caps:
+    categories:
+      101: TV/Anime # Filme - Animie
+      102: Movies/BluRay # Filme - Bluray
+      131: Movies/Other # Filme - Bollywood
+      103: Movies/DVD # Filme - DVD
+      104: Movies/DVD # Filme - DVD-R
+      132: Movies/DVD # Filme - HD2DVD
+      130: Movies # Filme - Klassiker
+      105: Movies # Filme - x264
+      106: Movies # Filme - XviD / DivX
+      69: XXX #  XXX
+      124: Audio # Musik - Alben
+      122: Audio/Audiobook # Musik - Hörbuch
+      123: Audio # Musik - Mixe
+      133: Audio/MP3 # Musik - MP3
+      125: Audio/Video # Musik - Video
+      113: PC # Programme - Linux
+      114: PC/Mac # Programme - Mac
+      115: PC # Programme - Windows
+      117: TV # Allgemein - Serien
+      116: TV/Documentary # Serien - Dokus
+      118: TV/Sport # Serien - Sport
+      119: Other # Bilder
+      120: Books # Ebook
+      127: Other # Für Unsere kleinsten
+      121: Other # Handy Stuff
+      129: Other # Sonstiges
+      109: Other # Spiele - Handy
+      112: Console # Spiele - Konsolen
+      111: PC/Games # Spiele - Mac / Linux
+      110: PC/Games # Spiele - PC
+
+    modes:
+      search: [q]
+      tv-search: [q, season, ep]
+
+  login:
+    path: /takelogin.php
+    method: post
+    inputs:
+      username: "{{ .Config.username }}"
+      password: "{{ .Config.password }}"
+      returnto: "/"
+    error:
+    - selector: table.tableinborder:contains("Anmeldung Gescheitert!") > tbody > tr > td.tablea
+    test:
+      path: /usercp.php
+
+  ratio:
+    path: /usercp.php
+    selector: div#lmtd table > tbody > tr:contains("Ratio:") > td:nth-child(2)
+    filters:
+      - name: replace
+        args: [".", ""]
+      - name: replace
+        args: [",", "."]
+
+  search:
+    path: /browse.php
+    inputs:
+      $raw: "{{range .Categories}}c{{.}}=1&{{end}}"
+      search: "{{ .Query.Keywords }}"
+      showsearch: "1"
+      orderby: "added"
+      sort: "desc"
+      incldead: "1"
+
+    rows:
+      selector: table.tableinborder[summary] > tbody > tr
+    fields:
+      download:
+        selector: a[href^="download.php?torrent="]
+        attribute: href
+      title:
+        selector: a[href^="details.php?id="]:has(b)
+      category:
+        selector: a[href^="browse.php?cat="]
+        attribute: href
+        filters:
+          - name: querystring
+            args: cat
+      comments:
+        selector: td.tablea > table > tbody > tr:nth-child(2) > td:nth-child(4) > a
+        attribute: href
+      size:
+        selector: td.tablea > table > tbody > tr:nth-child(2) > td:nth-child(1) > b:nth-child(1)
+      grabs:
+        selector: td.tablea > table > tbody > tr:nth-child(2) > td:nth-child(3) > b
+      files:
+        selector: td.tablea > table > tbody > tr:nth-child(2) > td:nth-child(1) > b:nth-child(2)
+      seeders:
+        selector: td.tablea > table > tbody > tr:nth-child(2) > td:nth-child(2) > b:nth-child(1)
+      leechers:
+        selector: td.tablea > table > tbody > tr:nth-child(2) > td:nth-child(2) > b:nth-child(3)
+      date:
+        selector: td.tablea > table > tbody > tr:nth-child(2) > td:nth-child(5)
+        filters:
+          - name: replace
+            args: ["\u00a0", " "]
+          - name: dateparse
+            args: "02.01.2006 15:04:05"
+      downloadvolumefactor:
+        case:
+          "font[color=\"red\"]:contains(\"Only Upload\")": "0"
+          "*": "1"
+      uploadvolumefactor:
+        case:
+          "*": "1"

src/Jackett/Definitions/shareisland.yml

@@ -0,0 +1,150 @@
+---
+  site: shareisland
+  name: Shareisland
+  description: "A general italian tracker"
+  language: it-it
+  links:
+    - http://shareisland.org
+
+  caps:
+    categories:
+       32: Other # Vip
+       45: XXX # Vip XXX
+       33: Other # ex Vip
+       1: Movies # Movies
+       49: Movies/HD # H-264
+       51: Movies/HD # H-265
+       41: Movies/Other # Cartoons
+       14: Movies/SD # DivX
+       16: Movies/Other # Cine News
+       39: Movies/HD # 720p
+       40: Movies/HD # 1080p
+       46: Movies/BluRay # Blu Ray Disk
+       31: Movies/HD # BDRip
+       17: Movies/DVD # DVD
+       43: Movies/SD # DVDRip
+       6: PC # Applications
+       18: PC/0day # PC Applications
+       19: PC/Mac # Macintosh Applications
+       44: PC/Phone-Android # Android applications
+       7: Audio # Music
+       20: Audio/Video # Video
+       21: Audio/MP3 # Mp3
+       2: Console # Games
+       3: Console/PS4 # Sony PS
+       4: Console/Wii # Wii
+       26: Console/Xbox # XboX
+       27: PC/Games # PC
+       28: Console/NDS # Nintendo
+       34: Books # Edicola
+       52: Books # Quotidiani
+       53: Books # Libreria
+       35: TV # SerieTV
+       55: TV/HD # Serie Tv HD
+       36: Other # Rip By ShareIsland
+       47: Other # Disclaimer
+       48: Other # P2P network 
+
+    modes:
+      search: [q]
+      tv-search: [q, season, ep]
+
+  login:
+    path: /ajax/login.php
+    method: post
+    form: form
+    inputs:
+      loginbox_membername: "{{ .Config.username }}"
+      loginbox_password: "{{ .Config.password }}"
+      action: "login"
+      loginbox_remember: "true"
+    error:
+      - selector: div.error
+    test:
+      path: /?p=home&pid=1
+      selector: div#member_info_bar
+
+  ratio:
+    path: /?p=home&pid=1
+    selector: img[title="Ratio"] + span
+
+  search:
+    path: /
+    inputs:
+      p: "torrents"
+      pid: "32"
+      $raw: "{{range .Categories}}cid[]:{{.}}&{{end}}"
+      keywords: "{{ .Query.Keywords }}"
+      search_type: "name"
+      searchin: "title"
+
+    rows:
+      selector: table#torrents_table_classic > tbody > tr:not([id="torrents_table_classic_head"])
+    fields:
+      title:
+        selector: td.torrent_name > a
+      category:
+        selector: div.category_image > a
+        attribute: href
+        filters:
+          - name: querystring
+            args: cid
+      comments:
+        selector: td.torrent_name > a
+        attribute: href
+      download:
+        selector: a:has(img[title="Download Torrent"])
+        attribute: href
+      size:
+        selector: td.size
+      seeders:
+        selector: td.seeders
+      leechers:
+        selector: td.leechers
+      grabs:
+        selector: td.completed
+      downloadvolumefactor:
+        case:
+          "img[title=\"FREE!\"]": "0"
+          "img[title=\"Download Multiplier: 0.5\"]": "0.5"
+          "img[title=\"Moltiplicatore Download: 0.5\"]": "0.5"
+          "*": "1"
+      uploadvolumefactor:
+        case:
+          "img[title=\"Upload Multiplier: 3\"]": "3"
+          "img[title=\"Upload Multiplier: 2\"]": "2"
+          "img[title=\"Upload Multiplier: 1.5\"]": "1.5"
+          "img[title=\"Moltiplicatore Upload: 3\"]": "3"
+          "img[title=\"Moltiplicatore Upload: 2\"]": "2"
+          "img[title=\"Moltiplicatore Upload: 1.5\"]": "1.5"
+          "*": "1"
+      date:
+        selector: td.torrent_name
+        remove: a, span, div, br
+        filters:
+          - name: replace
+            args: ["Uploaded ", ""]
+          - name: replace
+            args: [" by", ""]
+          - name: replace
+            args: [" at", ""]
+          - name: replace
+            args: ["Oggi", "Today"]
+          - name: replace
+            args: ["Ieri", "Yesterday"]
+          - name: replace
+            args: ["lunedì", "Monday"]
+          - name: replace
+            args: ["martedì", "Tuesday"]
+          - name: replace
+            args: ["Mercoledì", "Wednesday"]
+          - name: replace
+            args: ["Giovedì", "Thursday"]
+          - name: replace
+            args: ["Venerdì", "Friday"]
+          - name: replace
+            args: ["Sabato", "Saturday"]
+          - name: replace
+            args: ["Domenica", "Sunday"]
+          - name: replace
+            args: [" alle", ""]

src/Jackett/Definitions/thehorrorcharnel.yml

@@ -0,0 +1,101 @@
+---
+  site: thehorrorcharnel
+  name: The Horror Charnel
+  description: "A Horror, Cult, Sleaze, Sci-Fi & more tracker"
+  language: en-us
+  links:
+    - https://horrorcharnel.org
+
+  caps:
+    categories:
+      1: Movies
+      2: TV
+
+    modes:
+      search: [q]
+      tv-search: [q, season, ep]
+
+  settings:
+    - name: username
+      type: text
+      label: Username
+    - name: password
+      type: password
+      label: Password
+
+  login:
+    path: loginto.php
+    method: form
+    form: form
+    inputs:
+      username: "{{ .Config.username }}"
+      password: "{{ .Config.password }}"
+      use_sslvalue=: "on"
+      perm_ssl: "1"
+      returnto: "/"
+    error:
+      - selector: "div#base_content > table.mainouter > tbody > tr > td.outer > table.main > tbody > tr > td:has(h2)"
+    test:
+      path: usercp.php
+      
+  ratio:
+    path: /my.php
+    selector: td.navi_top:contains("Deine Ratio:")
+    filters:
+      - name: replace
+        args: ["Deine Ratio: ", ""]
+      - name: replace
+        args: [".", ""]
+      - name: replace
+        args: [",", "."]
+
+  search:
+    path: /browse.php
+    inputs:
+      #$raw: "{{range .Categories}}c{{.}}=1&{{end}}"
+      search: "{{ .Query.Keywords }}"
+      incldead: 1
+
+    rows:
+      selector: p + table > tbody > tr:has(a[href^="details.php?id="])
+      after: 1
+    fields:
+      title:
+        selector: a[onmouseover][href^="details.php?id="]
+        attribute: onmouseover
+        filters:
+          - name: regexp
+            args: <b>(.*?)</b>
+      description:
+        selector: td[colspan="13"]
+        remove: a
+      category:
+        text: 1
+      comments:
+        selector: a[onmouseover][href^="details.php"]
+        attribute: href
+      download:
+        selector: a[href^="download.php"]
+        attribute: href
+      files:
+        selector: td:nth-child(6)
+      size:
+        selector: td:nth-child(9)
+      seeders:
+        selector: td:nth-child(11)
+      leechers:
+        selector: td:nth-child(12)
+      date:
+        selector: td:nth-child(8)
+      grabs:
+        selector: td:nth-child(10)
+        filters:
+          - name: regexp
+            args: (\d+)
+      downloadvolumefactor:
+        case:
+          img[src="free.gif"]: "0"
+          "*": "1"
+      uploadvolumefactor:
+        case:
+          "*": "1"

src/Jackett/Definitions/torrentsectorcrew.yml

@@ -0,0 +1,175 @@
+---
+  site: torrentsectorcrew
+  name: Torrent Sector Crew
+  description: "A German general tracker"
+  language: de-de
+  links:
+    - https://tsctracker.net/
+
+  caps:
+    categories:
+      # Apps
+      65: PC/Phone-Android # Android
+      83: PC/Phone-IOS # iOS
+      107: PC/0day # Linux
+      48: PC/Mac # MAC
+      109: PC # Sonstige
+      22: PC/0day # Win
+
+      # Audio
+      24: Audio/Audiobook # aBooks
+      104: Audio # Disco's
+      38: Audio/Audiobook # Hörspiel
+      6: Audio # Musik
+      82: Audio # Tracks
+      29: Audio/Video # Videos
+
+      # Doku
+      113: TV/Documentary # 3D
+      76: TV/Documentary # HD
+      78: TV/Documentary # Packs
+      75: TV/Documentary # SD
+      114: TV/Documentary # Sonstige
+      77: TV/Documentary # Untouched
+
+      # Filme
+      54: Movies/HD # 1080p
+      5: Movies/3D # 3D
+      55: Movies/HD # 720p
+      111: Movies # Anime
+      43: Movies/BluRay # BluRay
+      20: Movies/DVD # DVDR
+      120: Movies/Foreign # Int.
+      119: Movies # Remux
+      121: Movies/HD # UHD
+      36: Movies/HD # x264
+      19: Movies/SD # XviD
+
+      # Serien
+      112: TV/Anime # Anime
+      69: TV/HD # HD
+      72: TV/Foreign # Int.
+      68: TV # Packs
+      66: TV/SD # SD
+      108: TV # TV-Shows
+
+      # Sonstige
+      117: Other # Diverses
+      28: Books # eBooks
+      42: TV/Sport # Sport
+      103: Other # Tutorials
+      9: Other # Wallpaper
+      64: XXX # XXX
+
+      # Spiele
+      115: PC/Mac # MAC
+      37: Console/NDS # Nintendo
+      4: PC/Games # PC
+      58: Console/PS4 # PSX
+      116: Other # Sonstige
+      50: Console/Xbox # XBOX
+
+
+    modes:
+      search: [q]
+      tv-search: [q, season, ep]
+
+  settings:
+    - name: pin
+      type: text
+      label: Pin
+    - name: username
+      type: text
+      label: Username
+    - name: password
+      type: password
+      label: Password
+
+  login:
+    path: landing.php
+    method: form
+    form: form
+    inputs:
+      username: "{{ .Config.username }}"
+      password: "{{ .Config.password }}"
+      pin: "{{ .Config.pin }}"
+    error:
+      - selector: "#login_box_desc"
+    test:
+      path: my.php
+      
+  ratio:
+    path: /my.php
+    selector: td.navi_top:contains("Deine Ratio:")
+    filters:
+      - name: replace
+        args: ["Deine Ratio: ", ""]
+      - name: replace
+        args: [".", ""]
+      - name: replace
+        args: [",", "."]
+
+  search:
+    path: browse.php
+    inputs:
+      $raw: "{{range .Categories}}c{{.}}=1&{{end}}"
+      search: "{{ .Query.Keywords }}"
+      incldead: "1"
+      orderby: "added"
+      sort: desc
+    rows:
+      selector: "h2 +p + br + table.tablebrowse > tbody > tr[style=\"height: 45px;\"], tr:contains(\"Weiter\") > td > table.tablebrowse > tbody > tr[style=\"height: 45px;\"]"
+    fields:
+      title:
+        selector: a[title][href^="details.php"]
+        attribute: title
+      category:
+        selector: a[href*="cat="]
+        attribute: href
+        filters:
+          - name: querystring
+            args: cat
+      comments:
+        selector: a[href*="&tocomm="]
+        attribute: href
+      download:
+        selector: a[href^="download_ssl.php"]
+        attribute: href
+      files:
+        selector: td:nth-child(3)
+      grabs:
+        selector: td:nth-child(9)
+        filters:
+          - name: replace
+            args: ["-mal", ""]
+      size:
+        selector: td:nth-child(6)
+        filters:
+          - name: replace
+            args: [".", ""]
+          - name: replace
+            args: [",", "."]
+      seeders:
+        selector: td:nth-child(7)
+        filters:
+          - name: regexp
+            args: "(\\d+)"
+      leechers:
+        selector: td:nth-child(8)
+        filters:
+          - name: regexp
+            args: "(\\d+)"
+      date:
+        selector: td:nth-child(5)
+        remove: br
+        filters:
+          - name: dateparse
+            args: "02.01.200615:04:05"
+      downloadvolumefactor:
+        case:
+          "font:contains(\"[OnlyUpload]\")": "0"
+          "font:contains(\"[-40 Download]\")": "0.6"
+          "*": "1"
+      uploadvolumefactor:
+        case:
+          "*": "1"

src/Jackett/Definitions/tspate.yml

@@ -0,0 +1,143 @@
+---
+  site: tspate
+  name: Tspate
+  description: "A French gerneral tracker"
+  language: fr-fr
+  links:
+    - https://www.tspate.me/
+
+  caps:
+    categories:
+      50: TV/Anime # Anime: Tout
+      40: PC # Apps: Windows
+      41: PC/Mac # Apps: Mac
+      42: PC/Phone-Android # Apps: Android
+      43: PC/Phone-IOS # Apps: Ipod/Iphone
+      44: PC # Apps: Autres
+      45: PC # Apps: Linux
+      27: Books # E-Books: Livres
+      28: Books # E-Books: Audio
+      29: Books/Comics # E-Books: Comics
+      30: Books # E-Books: Mangas
+      32: Books # E-Books: Bds
+      31: Books # E-Books: Presse
+      63: Books # E-Books: ePUB
+      1: Movies/SD # Films: DVDRip
+      2: Movies/SD # Films: DVDRip VOSTFR
+      3: Movies/DVD # Films: DVD-R
+      4: Movies/HD # Films: 720p
+      5: Movies/HD # Films: 1080p
+      6: Movies/BluRay # Films: BluRay x264 et x265
+      7: Movies/3D # Films: BluRay-3D
+      11: Movies/HD # Films: BDRip
+      9: Movies/DVD # Films: R5
+      10: Movies/SD # Films: DVDSCR
+      56: Movies/HD # Films: mHD
+      12: Movies/HD # Films: BRRip
+      13: Movies/HD # Films: WEBRip
+      58: Movies/HD # Films: WEBRip 720p
+      59: Movies/HD # Films: WEBRip 1080p
+      8: Movies/SD # Films: Cam / TS
+      62: Movies/HD # Films: Uhd4k
+      35: Console/Xbox # Jeux: XBOX
+      36: Console/Wii # Jeux: WII
+      37: Console/PSP # Jeux: PSP
+      38: Console/NDS # Jeux: DS
+      39: PC/Phone-Android # Jeux: Android
+      34: Console/PS3 # Jeux: PS3
+      55: PC/Mac # Jeux: Mac
+      33: PC/Games # Jeux: PC
+      46: Audio/MP3 # Musiques: MP3
+      47: Audio/Lossless # Musiques: FLAC
+      48: Audio # Musiques: WMA
+      49: Audio # Musiques: Autres
+      14: TV/SD # Series: HDTV
+      15: TV # Series: TV VOSTFR
+      16: TV # Series: TV VOSTA
+      17: TV # Series: TV PACK
+      18: TV/HD # Series: HDTV 1080p
+      57: TV/HD # Series: TVHD VOSTFR
+      20: TV # Series: TV VO
+      19: TV/HD # Series: HDTV 720p
+      64: TV/SD # Series: WEBRip
+      60: TV/HD # Series: WEBRip 720p
+      61: TV/HD # Series: WEBRip 1080p
+      21: TV/SD # Series: DVD-R TV
+      26: TV # Television: Television
+      24: TV # Television: Spectacles
+      23: TV/Sport # Television: Sports
+      25: TV/Anime # Television: Animes
+      22: TV/Documentary # Television: Documentaires
+      51: XXX # XXX: Tous
+
+    modes:
+      search: [q]
+      tv-search: [q, season, ep]
+
+  login:
+    path: account-login.php
+    method: post
+    inputs:
+      username: "{{ .Config.username }}"
+      password: "{{ .Config.password }}"
+      remember: "yes"
+      returnto: "/"
+    error:
+    - selector: div#nfobar > p#msgError
+    test:
+      path: torrents-search.php
+
+  ratio:
+    path: torrents-search.php
+    selector: div#infobar0 > ul > li:nth-child(1) > font:last-child
+
+  search:
+    path: torrents-search.php
+    inputs:
+      $raw: "{{range .Categories}}c{{.}}=1&{{end}}"
+      search: "{{ .Query.Keywords }}"
+      incldead: "1"
+
+    rows:
+      selector: table.ttable_headinner > tbody > tr.t-row
+      after: 1
+    fields:
+      download:
+        selector: a[href^="download.php?id="]
+        attribute: href
+      title:
+        selector: a[href^="torrents-details.php?id="]
+        attribute: title
+      category:
+        selector: a[href^="torrents.php?cat="]
+        attribute: href
+        filters:
+          - name: querystring
+            args: cat
+      comments:
+        selector: a[href^="comments.php"]
+        attribute: href
+      size:
+        selector: td:nth-child(7)
+      grabs:
+        selector: td:nth-child(8)
+      seeders:
+        selector: td:nth-child(9)
+      leechers:
+        selector: td:nth-child(10)
+      date:
+        selector: small > i
+        filters:
+          - name: replace
+            args: ["Date: ", ""]
+          - name: replace
+            args: ["le ", ""]
+          - name: dateparse
+            args: "15:04:05 02-01-2006"
+      downloadvolumefactor:
+        case:
+          img[src="images/free.gif"]: "0"
+          "*": "1"
+      uploadvolumefactor:
+        case:
+          "*": "1"

src/Jackett/Definitions/uhdbits.yml

@@ -0,0 +1,123 @@
+---
+  site: uhdbits
+  name: UHDBits
+  description: "A vietnamese general tracker"
+  language: en-us
+  links:
+    - https://uhdbits.org/
+
+  caps:
+    categories:
+      1: Movies
+      2: Audio
+      3: TV
+      4: Audio
+      5: PC
+      6: Other
+
+    modes:
+      search: [q]
+      tv-search: [q, season, ep]
+
+  login:
+    path: /login.php
+    form: form.auth_form
+    inputs:
+      username: "{{ .Config.username }}"
+      password: "{{ .Config.password }}"
+      keeplogged: "1"
+    error:
+    - selector: .auth_form > .warning
+      message:
+        selector: ".auth_form > .warning"
+    test:
+      path: /top10.php
+
+  ratio:
+    path: /top10.php
+    selector: span.r99
+
+  search:
+    path: /torrents.php
+    inputs:
+      $raw: "{{range .Categories}}filter_cat[{{.}}]=1&{{end}}"
+      searchstr: "{{ .Query.Keywords }}"
+      tags_type: "1"
+      order_by: "time"
+      order_way: "desc"
+      action: "advanced"
+      searchsubmit: "1"
+
+    rows:
+      selector: table#torrent_table > tbody > tr.torrent
+    fields:
+      description:
+        selector: div.tags
+      download:
+        selector: a[title="Download"]
+        attribute: href
+        filters:
+          - name: replace
+            args: ["\t", " "]
+      title:
+        selector: div.group_info
+        remove: span, div.tags
+        filters:
+          - name: replace
+            args: [" / Free", ""]
+          - name: replace
+            args: [" / ViE", ""]
+          - name: replace
+            args: [" / User", ""]
+          - name: replace
+            args: [" / Exclusive!", ""]
+          - name: replace
+            args: [" / ↓25%", ""]
+          - name: replace
+            args: [" / ↓50%", ""]
+          - name: replace
+            args: [" / ↓75%", ""]
+          - name: replace
+            args: [" / 2x 50%", ""]
+          - name: replace
+            args: [" / 2x Free", ""]
+          - name: replace
+            args: [" / 2x", ""]
+      downloadvolumefactor:
+        case:
+          "strong.blink_me:contains(\"Free\")": "0"
+          "strong.blink_me:contains(\"50\")": "0.5"
+          "strong.blink_me:contains(\"25\")": "0.75"
+          "strong.blink_me:contains(\"75\")": "0.25"
+          "*": "1"
+      uploadvolumefactor:
+        case:
+          "strong.blink_me:contains(\"2x\")": "2"
+          "*": "1"
+      category:
+        selector: a[href^="torrents.php?filter_cat"]
+        attribute: href
+        filters:
+          - name: regexp
+            args: "\\[(\\d+?)\\]"
+      comments:
+        selector: a.torrent_name
+        attribute: href
+        filters:
+          - name: replace
+            args: ["\t", " "]
+      size:
+        selector: td:nth-child(5)
+      grabs:
+        selector: td:nth-child(6)
+      files:
+        selector: td:nth-child(3)
+      seeders:
+        selector: td:nth-child(7)
+      leechers:
+        selector: td:nth-child(8)
+      date:
+        selector: td:nth-child(4)
+        filters:
+          - name: append
+            args: " ago"

src/Jackett/Definitions/worldofp2p.yml

@@ -0,0 +1,141 @@
+---
+  site: worldofp2p
+  name: WorldOfP2P
+  description: "A general tracker"
+  language: en-us
+  links:
+    - https://worldofp2p.net
+
+  caps:
+    categories:
+      44: TV/Anime # Anime
+      22: PC # Applications
+      43: Audio/Audiobook # Audio Books
+      27: Books # Ebook
+      4:  PC/Games # Games
+      40: Other/Misc # Miscellaneous
+      19: Movies # Movies
+      6:  Audio # Music
+      31: PC/Phone-Other # Portable
+      7:  TV # TV
+      9:  TV/Anime  # Anime
+      15: PC/0day # Apps-Linux
+      16: PC/Mac # Apps-Macintosh
+      1: PC/0day # Apps-Misc
+      17: PC/Phone-Other # Apps-Mobile
+      49: Audio # Audio Tracks
+      51: Audio/Audiobook # AudioBook
+      50: Books # Ebooks
+      23: Console # Games-Mixed
+      32: Other # Games-Packs
+      2: PC/Games # Games-PC
+      12: PC/Games # Games-PC Rips
+      20: Console/Other # Games-PS1
+      8: Console/Other # Games-PS2
+      21: Console/PS3 # Games-PS3
+      22: Console/PS4 # Games-PS4
+      7: Console/PSP # Games-PSP
+      14: Console/Wii # Games-Wii
+      44: Console/Xbox360 # Games-Xbox 360
+      45: Console/Xbox # Games-Xbox One
+      43: Console/Xbox # Gamex-Xbox
+      30: Movies/HD # Movies-1080p
+      24: Movies/3D # Movies-3D
+      53: Movies/SD # Movies-480p
+      52: Movies/SD # Movies-576p
+      25: Movies/HD # Movies-720p
+      11: Movies/BluRay # Movies-Bluray
+      26: Movies/HD # Movies-BRRip
+      27: Movies/SD # Movies-Camera
+      10: Movies/DVD # Movies-DVDR
+      28: Movies/Other # Movies-Oldies
+      31: Movies/Other # Movies-Packs
+      33: Movies/Other # Movies-Sport
+      29: Movies/WEBDL # Movies-Web/DL
+      3: Movies/SD # Movies-XviD
+      13: Audio/Lossless # Music-Flac
+      4: Audio/MP3 # Music-MP3
+      18: Audio # Music-Packs
+      19: Audio/Video # Music-Videos
+      38: TV/HD # TV-Bluray
+      35: TV/SD # TV-DVDR
+      36: TV/SD # TV-DVDRip
+      37: TV/HD # TV-HD-x264
+      41: TV # TV-Packs
+      39: TV/SD # TV-SD-x264
+      42: TV/WEB-DL # TV-Web/DL
+      5:  TV/SD # TV-XviD
+      46: XXX # xXx-HD
+      47: XXX/Imageset # xXx-Images
+      48: XXX/Packs # xXx-Packs
+      6: XXX/XviD # xXx-XviD
+
+    modes:
+      search: [q]
+      tv-search: [q, season, ep]
+
+  login:
+    path: /takelogin.php
+    method: post
+    form: form
+    inputs:
+      username: "{{ .Config.username }}"
+      password: "{{ .Config.password }}"
+      login: "Login"
+    error:
+      - selector: td.stdmsg2
+    test:
+      path: /usercp.php?action=default
+
+  ratio:
+    path: /usercp.php?action=default
+    selector: div#sts_tex1 > span.colorblue > span
+
+  search:
+    path: /browse.php
+    inputs:
+      $raw: "{{range .Categories}}c{{.}}=1&{{end}}"
+      search: "{{ .Query.Keywords }}"
+      incldead: "1"
+      searchin: "title"
+
+    rows:
+      selector: div#browse > form > div > table.browsewidth100 > tbody > tr:has(a[href^="download.php?torrent="])
+    fields:
+      title:
+        selector: a[href^="details.php?id="]
+      category:
+        selector: a[href^="browse.php?cat="]
+        attribute: href
+        filters:
+          - name: querystring
+            args: cat
+      comments:
+        selector: a[href^="details.php?id="]
+        attribute: href
+      download:
+        selector: a[href^="download.php?torrent="]
+        attribute: href
+      files:
+        selector: td:nth-child(5)
+      size:
+        selector: td:nth-child(8)
+      seeders:
+        selector: td:nth-child(10)
+      leechers:
+        selector: td:nth-child(11)
+      date:
+        selector: td:nth-child(7)
+      grabs:
+        selector: a[href^="snatches.php?id="]
+        filters:
+          - name: regexp
+            args: "(\\d+)"
+      downloadvolumefactor:
+        case:
+          a.info:contains("Free"): "0"
+          img[src*="/free.png"]: "0"
+          "*": "1"
+      uploadvolumefactor:
+        case:
+          "*": "1"

src/Jackett/Indexers/Abstract/AvistazTracker.cs

@@ -52,13 +52,13 @@ namespace Jackett.Indexers
         {
             configData.LoadValuesFromJson(configJson);
             var loginPage = await RequestStringWithCookies(LoginUrl, string.Empty);
-            var token = new Regex("Avz.CSRF_TOKEN = '(.*?)';").Match(loginPage.Content).Groups[1].ToString();
+            var token = new Regex("<meta name=\"_token\" content=\"(.*?)\">").Match(loginPage.Content).Groups[1].ToString();
             var pairs = new Dictionary<string, string> {
                 { "_token", token },
-                { "username_email", configData.Username.Value },
+                { "email_username", configData.Username.Value },
                 { "password", configData.Password.Value },
-                { "remember", "on" }
-            };
+                { "remember", "1" }
+            };
 
             var result = await RequestLoginAndFollowRedirect(LoginUrl, pairs, loginPage.Cookies, true, null, LoginUrl);
             await ConfigureIfOK(result.Cookies, result.Content != null && result.Content.Contains("auth/logout"), () =>
@@ -91,7 +91,7 @@ namespace Jackett.Indexers
             try
             {
                 CQ dom = response.Content;
-                var rows = dom["table > tbody > tr"];
+                var rows = dom["table:has(thead) > tbody > tr"];
                 foreach (var row in rows)
                 {
                     CQ qRow = row.Cq();
@@ -100,29 +100,47 @@ namespace Jackett.Indexers
                     release.MinimumRatio = 1;
                     release.MinimumSeedTime = 172800;
 
-                    var qLink = row.ChildElements.ElementAt(1).FirstElementChild.Cq();
+                    var qLink = qRow.Find("a.torrent-filename"); ;
                     release.Title = qLink.Text().Trim();
                     release.Comments = new Uri(qLink.Attr("href"));
                     release.Guid = release.Comments;
 
-                    var qDownload = row.ChildElements.ElementAt(3).FirstElementChild.Cq();
+                    var qDownload = qRow.Find("a.torrent-download-icon"); ;
                     release.Link = new Uri(qDownload.Attr("href"));
 
-                    var dateStr = row.ChildElements.ElementAt(5).Cq().Text().Trim();
+                    var dateStr = qRow.Find("td:eq(3) > span").Text().Trim();
                     release.PublishDate = DateTimeUtil.FromTimeAgo(dateStr);
 
-                    var sizeStr = row.ChildElements.ElementAt(6).Cq().Text();
+                    var sizeStr = qRow.Find("td:eq(5) > span").Text().Trim();
                     release.Size = ReleaseInfo.GetBytes(sizeStr);
 
-                    release.Seeders = ParseUtil.CoerceInt(row.ChildElements.ElementAt(8).Cq().Text());
-                    release.Peers = ParseUtil.CoerceInt(row.ChildElements.ElementAt(9).Cq().Text()) + release.Seeders;
+                    release.Seeders = ParseUtil.CoerceInt(qRow.Find("td:eq(6)").Text().Trim());
+                    release.Peers = ParseUtil.CoerceInt(qRow.Find("td:eq(7)").Text().Trim()) + release.Seeders;
 
                     var cat = row.Cq().Find("td:eq(0) i").First().Attr("class")
-                                            .Replace("gi gi-film", "1")
-                                            .Replace("gi gi-tv", "2")
-                                            .Replace("gi gi-music", "3")
+                                            .Replace("torrent-icon", string.Empty)
+                                            .Replace("fa fa-", string.Empty)
+                                            .Replace("film", "1")
+                                            .Replace("tv", "2")
+                                            .Replace("music", "3")
                                             .Replace("text-pink", string.Empty);
                     release.Category = MapTrackerCatToNewznab(cat.Trim());
+
+                    var grabs = row.Cq().Find("td:nth-child(9)").Text();
+                    release.Grabs = ParseUtil.CoerceInt(grabs);
+
+                    if (row.Cq().Find("i.fa-star").Any())
+                        release.DownloadVolumeFactor = 0;
+                    else if (row.Cq().Find("i.fa-star-half-o").Any())
+                        release.DownloadVolumeFactor = 0.5;
+                    else
+                        release.DownloadVolumeFactor = 1;
+
+                    if (row.Cq().Find("i.fa-diamond").Any())
+                        release.UploadVolumeFactor = 2;
+                    else
+                        release.UploadVolumeFactor = 1;
+
                     releases.Add(release);
                 }
             }

src/Jackett/Indexers/AlphaRatio.cs

@@ -30,7 +30,7 @@ namespace Jackett.Indexers
             : base(name: "AlphaRatio",
                 description: "Legendary",
                 link: "https://alpharatio.cc/",
-                caps: TorznabUtil.CreateDefaultTorznabTVCaps(),
+                caps: new TorznabCapabilities(),
                 manager: i,
                 client: w,
                 logger: l,
@@ -56,13 +56,18 @@ namespace Jackett.Indexers
             AddCategoryMapping(23, TorznabCatType.Audio);
         }
 
+        new ConfigurationDataBasicLogin configData
+        {
+            get { return (ConfigurationDataBasicLogin)base.configData; }
+            set { base.configData = value; }
+        }
+
         public async Task<IndexerConfigurationStatus> ApplyConfiguration(JToken configJson)
         {
-            var incomingConfig = new ConfigurationDataBasicLogin();
-            incomingConfig.LoadValuesFromJson(configJson);
+            configData.LoadValuesFromJson(configJson);
             var pairs = new Dictionary<string, string> {
-                { "username", incomingConfig.Username.Value },
-                { "password", incomingConfig.Password.Value },
+                { "username", configData.Username.Value },
+                { "password", configData.Password.Value },
                 { "login", "Login" },
                 { "keeplogged", "1" }
             };
@@ -74,7 +79,7 @@ namespace Jackett.Indexers
                    CQ dom = response.Content;
                    dom["#loginform > table"].Remove();
                    var errorMessage = dom["#loginform"].Text().Trim().Replace("\n\t", " ");
-                   throw new ExceptionWithConfigData(errorMessage, (ConfigurationData)incomingConfig);
+                   throw new ExceptionWithConfigData(errorMessage, configData);
                });
             return IndexerConfigurationStatus.RequiresTesting;
         }
@@ -88,6 +93,25 @@ namespace Jackett.Indexers
             release.Guid = new Uri(GuidUrl + id);
             release.Comments = release.Guid;
             release.Link = new Uri(DownloadUrl + id);
+            release.Category = MapTrackerCatToNewznab(CategoryReverseMapper((string)r["category"]));
+            release.Files = (int)r["fileCount"];
+            release.Grabs = (int)r["snatches"];
+            release.DownloadVolumeFactor = 1;
+            release.UploadVolumeFactor = 1;
+            if ((bool)r["isFreeleech"])
+            {
+                release.DownloadVolumeFactor = 0;
+            }
+            if ((bool)r["isPersonalFreeleech"])
+            {
+                release.DownloadVolumeFactor = 0;
+            }
+            if ((bool)r["isNeutralLeech"])
+            {
+                release.DownloadVolumeFactor = 0;
+                release.UploadVolumeFactor = 0;
+            }
+
         }
 
         public async Task<IEnumerable<ReleaseInfo>> PerformQuery(TorznabQuery query)
@@ -159,7 +183,23 @@ namespace Jackett.Indexers
         {
             DateTime unixStart = new DateTime(1970, 1, 1, 0, 0, 0, 0, System.DateTimeKind.Utc);
             long unixTimeStampInTicks = (long)(unixTime * TimeSpan.TicksPerSecond);
-            return new DateTime(unixStart.Ticks + unixTimeStampInTicks);
+            return new DateTime(unixStart.Ticks + unixTimeStampInTicks, DateTimeKind.Utc).ToLocalTime();
+        }
+
+        static string CategoryReverseMapper(string categoryName)
+        {
+            Dictionary<string, string> dictionary = new Dictionary<string, string>();
+
+            dictionary.Add("TvSD", "1");
+            dictionary.Add("TvHD", "2");
+            dictionary.Add("MovieSD", "6");
+            dictionary.Add("MovieHD", "7");
+
+            if (dictionary.ContainsKey(categoryName))
+            {
+                return dictionary[categoryName];
+            }
+            return string.Empty;
         }
     }
 }

src/Jackett/Indexers/Andraste.cs

@@ -0,0 +1,205 @@
+using Jackett.Utils.Clients;
+using NLog;
+using Jackett.Services;
+using Jackett.Utils;
+using Jackett.Models;
+using System.Threading.Tasks;
+using Newtonsoft.Json.Linq;
+using System.Collections.Generic;
+using CsQuery;
+using System;
+using System.Globalization;
+using Jackett.Models.IndexerConfig;
+using System.Collections.Specialized;
+using System.Text;
+using System.Linq;
+
+namespace Jackett.Indexers
+{
+    public class Andraste : BaseIndexer, IIndexer
+    {
+        string LoginUrl { get { return SiteLink + "takelogin.php"; } }
+        string BrowseUrl { get { return SiteLink + "browse.php"; } }
+
+        new ConfigurationDataBasicLoginWithRSSAndDisplay configData
+        {
+            get { return (ConfigurationDataBasicLoginWithRSSAndDisplay)base.configData; }
+            set { base.configData = value; }
+        }
+
+        public Andraste(IIndexerManagerService i, IWebClient wc, Logger l, IProtectionService ps)
+            : base(name: "Andraste",
+                   description: "A German general tracker.",
+                   link: "https://andraste.io/",
+                   caps: TorznabUtil.CreateDefaultTorznabTVCaps(),
+                   manager: i,
+                   client: wc,
+                   logger: l,
+                   p: ps,
+                   configData: new ConfigurationDataBasicLoginWithRSSAndDisplay())
+        {
+            AddCategoryMapping(9,  TorznabCatType.Other); // Anderes
+            AddCategoryMapping(23, TorznabCatType.TVAnime); // Animation - Film &; Serie
+            AddCategoryMapping(1,  TorznabCatType.PC); // Appz
+            AddCategoryMapping(52, TorznabCatType.Other); // Botuploads
+            AddCategoryMapping(25, TorznabCatType.TVDocumentary); // Doku - Alle Formate
+            AddCategoryMapping(27, TorznabCatType.Books); // E-Books
+            AddCategoryMapping(51, TorznabCatType.Movies3D); // Film/3D
+            AddCategoryMapping(20, TorznabCatType.MoviesDVD); // Film/DVDr
+            AddCategoryMapping(37, TorznabCatType.MoviesHD); // Film/HD 1080p++
+            AddCategoryMapping(38, TorznabCatType.MoviesSD); // Film/HD 720p
+            AddCategoryMapping(36, TorznabCatType.Movies); // Film/im Kino
+            AddCategoryMapping(19, TorznabCatType.Movies); // Film/XviD,DivX,x264
+            AddCategoryMapping(4,  TorznabCatType.PCGames); // Games/PC
+            AddCategoryMapping(12, TorznabCatType.ConsolePS4); // Games/Playstation
+            AddCategoryMapping(22, TorznabCatType.ConsoleWii); // Games/Wii & DS
+            AddCategoryMapping(21, TorznabCatType.ConsoleXbox); // Games/Xbox & 360
+            AddCategoryMapping(48, TorznabCatType.PCPhoneAndroid); // Handy & PDA/Android
+            AddCategoryMapping(47, TorznabCatType.PCPhoneIOS); // Handy & PDA/iOS
+            AddCategoryMapping(44, TorznabCatType.PCMac); // Macintosh
+            AddCategoryMapping(41, TorznabCatType.Other); // MegaPack
+            AddCategoryMapping(24, TorznabCatType.AudioAudiobook); // Musik/Hörbuch & Hörspiel
+            AddCategoryMapping(46, TorznabCatType.Audio); // Musik/HQ 320++
+            AddCategoryMapping(6,  TorznabCatType.Audio); // Musik/Musik
+            AddCategoryMapping(26, TorznabCatType.AudioVideo); // Musik/Musikvideos
+            AddCategoryMapping(29, TorznabCatType.TVSD); // Serien/DVDr
+            AddCategoryMapping(35, TorznabCatType.TVHD); // Serien/HD 720p++
+            AddCategoryMapping(7,  TorznabCatType.TV); // Serien/XviD,DivX,x264
+            AddCategoryMapping(45, TorznabCatType.TV); // Shows
+            AddCategoryMapping(40, TorznabCatType.TVSport); // Sport
+            AddCategoryMapping(32, TorznabCatType.XXX); // XXX
+        }
+
+        public async Task<IndexerConfigurationStatus> ApplyConfiguration(JToken configJson)
+        {
+            configData.LoadValuesFromJson(configJson);
+
+            var pairs = new Dictionary<string, string>
+            {
+                { "username", configData.Username.Value },
+                { "password", configData.Password.Value }
+            };
+
+            var result = await RequestLoginAndFollowRedirect(LoginUrl, pairs, null, true, null, LoginUrl, true);
+            await ConfigureIfOK(result.Cookies, result.Content != null && result.Content.Contains("logout.php"), () =>
+            {
+                CQ dom = result.Content;
+                var errorMessage = dom["table.tableinborder"].Html();
+                errorMessage = result.Content;
+                throw new ExceptionWithConfigData(errorMessage, configData);
+            });
+            return IndexerConfigurationStatus.RequiresTesting;
+        }
+
+        public async Task<IEnumerable<ReleaseInfo>> PerformQuery(TorznabQuery query)
+        {
+            TimeZoneInfo.TransitionTime startTransition = TimeZoneInfo.TransitionTime.CreateFloatingDateRule(new DateTime(1, 1, 1, 3, 0, 0), 3, 5, DayOfWeek.Sunday);
+            TimeZoneInfo.TransitionTime endTransition = TimeZoneInfo.TransitionTime.CreateFloatingDateRule(new DateTime(1, 1, 1, 4, 0, 0), 10, 5, DayOfWeek.Sunday);
+            TimeSpan delta = new TimeSpan(1, 0, 0);
+            TimeZoneInfo.AdjustmentRule adjustment = TimeZoneInfo.AdjustmentRule.CreateAdjustmentRule(new DateTime(1999, 10, 1), DateTime.MaxValue.Date, delta, startTransition, endTransition);
+            TimeZoneInfo.AdjustmentRule[] adjustments = { adjustment };
+            TimeZoneInfo germanyTz = TimeZoneInfo.CreateCustomTimeZone("W. Europe Standard Time", new TimeSpan(1, 0, 0), "(GMT+01:00) W. Europe Standard Time", "W. Europe Standard Time", "W. Europe DST Time", adjustments);
+
+            var releases = new List<ReleaseInfo>();
+            
+            var searchString = query.GetQueryString();
+            var searchUrl = BrowseUrl;
+            var queryCollection = new NameValueCollection();
+            queryCollection.Add("showsearch", "1");
+            queryCollection.Add("incldead", "1");
+            queryCollection.Add("orderby", "added");
+            queryCollection.Add("sort", "desc");
+
+            if (!string.IsNullOrWhiteSpace(searchString))
+            {
+                queryCollection.Add("search", searchString);
+            }
+
+            foreach (var cat in MapTorznabCapsToTrackers(query))
+            {
+                queryCollection.Add("c" + cat, "1");
+            }
+            searchUrl += "?" + queryCollection.GetQueryString();
+
+            var response = await RequestBytesWithCookies(searchUrl);
+            var results = Encoding.GetEncoding("iso-8859-1").GetString(response.Content);
+            try
+            {
+                CQ dom = results;
+                var rows = dom["table.tableinborder > tbody > tr:has(td.tableb)"];
+
+                foreach (var row in rows)
+                {
+                    var release = new ReleaseInfo();
+                    var qRow = row.Cq();
+
+                    var qDetailsLink = qRow.Find("a[href^=details.php?id=]").First();
+                    release.Title = qDetailsLink.Attr("title");
+
+                    if (!query.MatchQueryStringAND(release.Title))
+                        continue;
+
+                    var qCatLink = qRow.Find("a[href^=browse.php?cat=]").First();
+                    var qDLLink = qRow.Find("a[href^=download.php?torrent=]").First();
+                    var qSeeders = qRow.Find("span:contains(Seeder) > b:eq(0)");
+                    var qLeechers = qRow.Find("span:contains(Seeder) > b:eq(1)");
+                    var qDateStr = qRow.Find("td > table > tbody > tr > td:eq(7)").First();
+                    var qSize = qRow.Find("span:contains(Volumen) > b:eq(0)").First();
+                    var qOnlyUpload = qRow.Find("img[title=OnlyUpload]");
+
+                    if(qOnlyUpload.Any())
+                    {
+                        release.MinimumRatio = 2;
+                        release.MinimumSeedTime = 144 * 60 * 60;
+                    }
+                    else
+                    {
+                        release.MinimumRatio = 1;
+                        release.MinimumSeedTime = 72 * 60 * 60;
+                    }
+
+                    var catStr = qCatLink.Attr("href").Split('=')[1];
+                    release.Category = MapTrackerCatToNewznab(catStr);
+
+                    release.Link = new Uri(SiteLink + qDLLink.Attr("href"));
+                    release.Comments = new Uri(SiteLink + qDetailsLink.Attr("href"));
+                    release.Guid = release.Link;
+
+                    var sizeStr = qSize.Text();
+                    release.Size = ReleaseInfo.GetBytes(sizeStr);
+
+                    release.Seeders = ParseUtil.CoerceInt(qSeeders.Text());
+                    release.Peers = ParseUtil.CoerceInt(qLeechers.Text()) + release.Seeders;
+
+                    var dateStr = qDateStr.Text().Trim().Replace('\xA0', ' ');
+                    DateTime dateGerman = DateTime.SpecifyKind(DateTime.ParseExact(dateStr, "dd.MM.yyyy HH:mm", CultureInfo.InvariantCulture), DateTimeKind.Unspecified);
+
+                    DateTime pubDateUtc = TimeZoneInfo.ConvertTimeToUtc(dateGerman, germanyTz);
+                    release.PublishDate = pubDateUtc.ToLocalTime();
+
+                    var files = qRow.Find("a[href*=\"&filelist=1\"] ~ font ~ b").Text();
+                    release.Files = ParseUtil.CoerceInt(files);
+
+                    var grabs = qRow.Find("a[href*=\"&tosnatchers=1\"] ~ font ~ b").Text();
+                    release.Grabs = ParseUtil.CoerceInt(grabs);
+
+                    if (qRow.Find("img[alt=\"OU\"]").Length >= 1)
+                        release.DownloadVolumeFactor = 0;
+                    else
+                        release.DownloadVolumeFactor = 1;
+
+                    release.UploadVolumeFactor = 1;
+
+                    releases.Add(release);
+                }
+            }
+            catch (Exception ex)
+            {
+                OnParseError(results, ex);
+            }
+
+            return releases;
+        }
+    }
+}
+

src/Jackett/Indexers/AnimeBytes.cs

@@ -24,8 +24,15 @@ namespace Jackett.Indexers
 {
     public class AnimeBytes : BaseIndexer, IIndexer
     {
+        enum SearchType
+        {
+            Video,
+            Audio
+        }
+
         private string LoginUrl { get { return SiteLink + "user/login"; } }
         private string SearchUrl { get { return SiteLink + "torrents.php?"; } }
+        private string MusicSearchUrl { get { return SiteLink + "torrents2.php?"; } }
         public bool AllowRaws { get { return configData.IncludeRaw.Value; } }
         public bool InsertSeason { get { return configData.InsertSeason!=null && configData.InsertSeason.Value; } }
 
@@ -46,7 +53,10 @@ namespace Jackett.Indexers
                                               TorznabCatType.BooksComics,
                                               TorznabCatType.ConsolePSP,
                                               TorznabCatType.ConsoleOther,
-                                              TorznabCatType.PCGames),
+                                              TorznabCatType.PCGames,
+                                              TorznabCatType.AudioMP3,
+                                              TorznabCatType.AudioLossless,
+                                              TorznabCatType.AudioOther),
                 logger: l,
                 p: ps,
                 configData: new ConfigurationDataAnimeBytes())
@@ -55,7 +65,7 @@ namespace Jackett.Indexers
         }
 
 
-        public IEnumerable<ReleaseInfo> FilterResults(TorznabQuery query, IEnumerable<ReleaseInfo> input)
+        public override IEnumerable<ReleaseInfo> FilterResults(TorznabQuery query, IEnumerable<ReleaseInfo> input)
         {
             // Prevent filtering
             return input;
@@ -139,8 +149,16 @@ namespace Jackett.Indexers
         {
             // The result list
             var releases = new List<ReleaseInfo>();
+            
+            if (ContainsMusicCategories(query.Categories))
+            {
+                foreach (var result in await GetResults(SearchType.Audio, query.SanitizedSearchTerm))
+                {
+                    releases.Add(result);
+                }
+            }
 
-            foreach (var result in await GetResults(StripEpisodeNumber(query.SanitizedSearchTerm)))
+            foreach (var result in await GetResults(SearchType.Video, StripEpisodeNumber(query.SanitizedSearchTerm)))
             {
                 releases.Add(result);
             }
@@ -148,31 +166,45 @@ namespace Jackett.Indexers
             return releases.ToArray();
         }
 
-        public async Task<IEnumerable<ReleaseInfo>> GetResults(string searchTerm)
+        private bool ContainsMusicCategories(int[] categories)
+        {
+            var music = new[]
+            {
+                TorznabCatType.Audio.ID,
+                TorznabCatType.AudioMP3.ID,
+                TorznabCatType.AudioLossless.ID,
+                TorznabCatType.AudioOther.ID,
+                TorznabCatType.AudioForeign.ID
+            };
+
+            return categories.Length == 0 || music.Any(categories.Contains);
+        }
+
+        private async Task<IEnumerable<ReleaseInfo>> GetResults(SearchType searchType, string searchTerm)
         {
             var cleanSearchTerm = HttpUtility.UrlEncode(searchTerm);
 
             // The result list
             var releases = new List<ReleaseInfo>();
 
+            var queryUrl = searchType == SearchType.Video ? SearchUrl : MusicSearchUrl;
+            // Only include the query bit if its required as hopefully the site caches the non query page
+            if (!string.IsNullOrWhiteSpace(searchTerm))
+            {
+                queryUrl += string.Format("searchstr={0}&action=advanced&search_type=title&year=&year2=&tags=&tags_type=0&sort=time_added&way=desc&hentai=2&releasegroup=&epcount=&epcount2=&artbooktitle=", cleanSearchTerm);
+            }
+
             // Check cache first so we don't query the server for each episode when searching for each episode in a series.
             lock (cache)
             {
                 // Remove old cache items
                 CleanCache();
 
-                var cachedResult = cache.Where(i => i.Query == searchTerm).FirstOrDefault();
+                var cachedResult = cache.Where(i => i.Query == queryUrl).FirstOrDefault();
                 if (cachedResult != null)
                     return cachedResult.Results.Select(s => (ReleaseInfo)s.Clone()).ToArray();
             }
-
-            var queryUrl = SearchUrl;
-            // Only include the query bit if its required as hopefully the site caches the non query page
-            if (!string.IsNullOrWhiteSpace(searchTerm))
-            {
-                queryUrl += string.Format("searchstr={0}&action=advanced&search_type=title&year=&year2=&tags=&tags_type=0&sort=time_added&way=desc&hentai=2&releasegroup=&epcount=&epcount2=&artbooktitle=", cleanSearchTerm);
-            }
-
+            
             // Get the content from the tracker
             var response = await RequestStringWithCookiesAndRetry(queryUrl);
             CQ dom = response.Content;
@@ -191,7 +223,11 @@ namespace Jackett.Indexers
                     var seriesCq = series.Cq();
 
                     var synonyms = new List<string>();
-                    var mainTitle = seriesCq.Find(".group_title strong a").First().Text().Trim();
+                    string mainTitle;
+                    if (searchType == SearchType.Video)
+                        mainTitle = seriesCq.Find(".group_title strong a").First().Text().Trim();
+                    else
+                        mainTitle = seriesCq.Find(".group_title strong").Text().Trim();
 
                     var yearStr = seriesCq.Find(".group_title strong").First().Text().Trim().Replace("]", "").Trim();
                     int yearIndex = yearStr.LastIndexOf("[");
@@ -282,33 +318,52 @@ namespace Jackett.Indexers
                                 release.Guid = new Uri(SiteLink + infoLink.Attributes.GetAttribute("href") + "&nh=" + StringUtil.Hash(title)); // Sonarr should dedupe on this url - allow a url per name.
                                 release.Link = new Uri(downloadLink.Attributes.GetAttribute("href"), UriKind.Relative);
 
-                                var category = seriesCq.Find("a[title=\"View Torrent\"]").Text().Trim();
-                                if (category == "TV Series")
-                                    release.Category = TorznabCatType.TVAnime.ID;
-
-                                // Ignore these categories as they'll cause hell with the matcher
-                                // TV Special, OVA, ONA, DVD Special, BD Special
-
-                                if (category == "Movie")
-                                    release.Category = TorznabCatType.Movies.ID;
-
-                                if (category == "Manga" || category == "Oneshot" || category == "Anthology" || category == "Manhwa" || category == "Manhua" || category == "Light Novel")
-                                    release.Category = TorznabCatType.BooksComics.ID;
-
-                                if (category == "Novel" || category == "Artbook")
-                                    release.Category = TorznabCatType.BooksComics.ID;
-
-                                if (category == "Game" || category == "Visual Novel")
+                                string category = null;
+                                if (searchType == SearchType.Video)
                                 {
-                                    var description = rowCq.Find(".torrent_properties a:eq(1)").Text();
-                                    if (description.Contains(" PSP "))
-                                        release.Category = TorznabCatType.ConsolePSP.ID;
-                                    if (description.Contains("PSX"))
-                                        release.Category = TorznabCatType.ConsoleOther.ID;
-                                    if (description.Contains(" NES "))
-                                        release.Category = TorznabCatType.ConsoleOther.ID;
-                                    if (description.Contains(" PC "))
-                                        release.Category = TorznabCatType.PCGames.ID;
+                                    category = seriesCq.Find("a[title=\"View Torrent\"]").Text().Trim();
+                                    if (category == "TV Series")
+                                        release.Category = TorznabCatType.TVAnime.ID;
+
+                                    // Ignore these categories as they'll cause hell with the matcher
+                                    // TV Special, OVA, ONA, DVD Special, BD Special
+
+                                    if (category == "Movie")
+                                        release.Category = TorznabCatType.Movies.ID;
+
+                                    if (category == "Manga" || category == "Oneshot" || category == "Anthology" || category == "Manhwa" || category == "Manhua" || category == "Light Novel")
+                                        release.Category = TorznabCatType.BooksComics.ID;
+
+                                    if (category == "Novel" || category == "Artbook")
+                                        release.Category = TorznabCatType.BooksComics.ID;
+
+                                    if (category == "Game" || category == "Visual Novel")
+                                    {
+                                        var description = rowCq.Find(".torrent_properties a:eq(1)").Text();
+                                        if (description.Contains(" PSP "))
+                                            release.Category = TorznabCatType.ConsolePSP.ID;
+                                        if (description.Contains("PSX"))
+                                            release.Category = TorznabCatType.ConsoleOther.ID;
+                                        if (description.Contains(" NES "))
+                                            release.Category = TorznabCatType.ConsoleOther.ID;
+                                        if (description.Contains(" PC "))
+                                            release.Category = TorznabCatType.PCGames.ID;
+                                    }
+                                }
+
+                                if (searchType == SearchType.Audio)
+                                {
+                                    category = seriesCq.Find(".group_img .cat a").Text();
+                                    if (category == "Single" || category == "Album" || category == "Compilation" || category == "Soundtrack" || category == "Remix CD")
+                                    {
+                                        var description = rowCq.Find(".torrent_properties a:eq(1)").Text();
+                                        if (description.Contains(" Lossless "))
+                                            release.Category = TorznabCatType.AudioLossless.ID;
+                                        else if (description.Contains("MP3"))
+                                            release.Category = TorznabCatType.AudioMP3.ID;
+                                        else
+                                            release.Category = TorznabCatType.AudioOther.ID;
+                                    }
                                 }
 
 
@@ -371,6 +426,17 @@ namespace Jackett.Indexers
                                 release.Seeders = ParseUtil.CoerceInt(rowCq.Find(".torrent_seeders").Text());
                                 release.Peers = release.Seeders + ParseUtil.CoerceInt(rowCq.Find(".torrent_leechers").Text());
 
+                                // grabs
+                                var grabs = rowCq.Find("td.torrent_snatched").Text();
+                                release.Grabs = ParseUtil.CoerceInt(grabs);
+
+                                // freeleech
+                                if (rowCq.Find("img[alt=\"Freeleech!\"]").Length >= 1)
+                                    release.DownloadVolumeFactor = 0;
+                                else
+                                    release.DownloadVolumeFactor = 1;
+                                release.UploadVolumeFactor = 1;
+
                                 if (release.Category != 0)
                                     releases.Add(release);
                             }
@@ -386,7 +452,7 @@ namespace Jackett.Indexers
             // Add to the cache
             lock (cache)
             {
-                cache.Add(new CachedQueryResult(searchTerm, releases));
+                cache.Add(new CachedQueryResult(queryUrl, releases));
             }
 
             return releases.Select(s => (ReleaseInfo)s.Clone());

src/Jackett/Indexers/AnimeTorrents.cs

@@ -0,0 +1,204 @@
+using CsQuery;
+using Jackett.Models;
+using Jackett.Services;
+using Jackett.Utils;
+using Jackett.Utils.Clients;
+using Newtonsoft.Json.Linq;
+using NLog;
+using System;
+using System.Collections.Generic;
+using System.IO;
+using System.Linq;
+using System.Net;
+using System.Net.Http;
+using System.Text;
+using System.Threading.Tasks;
+using System.Web;
+using Jackett.Models.IndexerConfig;
+using System.Collections.Specialized;
+using System.Globalization;
+
+namespace Jackett.Indexers
+{
+    public class AnimeTorrents : BaseIndexer, IIndexer
+    {
+        private string LoginUrl { get { return SiteLink + "login.php"; } }
+        private string SearchUrl { get { return SiteLink + "ajax/torrents_data.php"; } }
+        private string SearchUrlReferer { get { return SiteLink + "torrents.php?cat=0&searchin=filename&search="; } }
+
+        new ConfigurationDataBasicLogin configData
+        {
+            get { return (ConfigurationDataBasicLogin)base.configData; }
+            set { base.configData = value; }
+        }
+
+        public AnimeTorrents(IIndexerManagerService i, HttpWebClient c, Logger l, IProtectionService ps)
+            : base(name: "AnimeTorrents",
+                description: "Definitive source for anime and manga",
+                link: "http://animetorrents.me/",
+                caps: new TorznabCapabilities(),
+                manager: i,
+                client: c, // Forced HTTP client for custom headers
+                logger: l,
+                p: ps,
+                configData: new ConfigurationDataBasicLogin())
+        {
+            AddCategoryMapping(1, TorznabCatType.MoviesSD); // Anime Movie
+            AddCategoryMapping(6, TorznabCatType.MoviesHD); // Anime Movie HD
+            AddCategoryMapping(2, TorznabCatType.TVAnime); // Anime Series
+            AddCategoryMapping(7, TorznabCatType.TVAnime); // Anime Series HD
+            AddCategoryMapping(5, TorznabCatType.XXXDVD); // Hentai (censored)
+            AddCategoryMapping(9, TorznabCatType.XXXDVD); // Hentai (censored) HD
+            AddCategoryMapping(4, TorznabCatType.XXXDVD); // Hentai (un-censored)
+            AddCategoryMapping(8, TorznabCatType.XXXDVD); // Hentai (un-censored) HD
+            AddCategoryMapping(13, TorznabCatType.BooksForeign); // Light Novel
+            AddCategoryMapping(3, TorznabCatType.BooksComics); // Manga
+            AddCategoryMapping(10, TorznabCatType.BooksComics); // Manga 18+
+            AddCategoryMapping(11, TorznabCatType.TVAnime); // OVA
+            AddCategoryMapping(12, TorznabCatType.TVAnime); // OVA HD
+            AddCategoryMapping(14, TorznabCatType.BooksComics); // Doujin Anime
+            AddCategoryMapping(15, TorznabCatType.XXXDVD); // Doujin Anime 18+
+            AddCategoryMapping(16, TorznabCatType.AudioForeign); // Doujin Music
+            AddCategoryMapping(17, TorznabCatType.BooksComics); // Doujinshi
+            AddCategoryMapping(18, TorznabCatType.BooksComics); // Doujinshi 18+
+            AddCategoryMapping(19, TorznabCatType.Audio); // OST
+        }
+
+        public async Task<IndexerConfigurationStatus> ApplyConfiguration(JToken configJson)
+        {
+            configData.LoadValuesFromJson(configJson);
+            var pairs = new Dictionary<string, string> {
+                { "username", configData.Username.Value },
+                { "password", configData.Password.Value },
+                { "form", "login" },
+                { "rememberme[]", "1" }
+            };
+
+            var loginPage = await RequestStringWithCookiesAndRetry(LoginUrl, null, null);
+
+            var result = await RequestLoginAndFollowRedirect(LoginUrl, pairs, loginPage.Cookies, true);
+            await ConfigureIfOK(result.Cookies, result.Content != null && result.Content.Contains("logout.php"), () =>
+            {
+                CQ dom = result.Content;
+                var errorMessage = dom[".ui-state-error"].Text().Trim();
+                throw new ExceptionWithConfigData(errorMessage, configData);
+            });
+
+            return IndexerConfigurationStatus.RequiresTesting;
+        }
+
+        public async Task<IEnumerable<ReleaseInfo>> PerformQuery(TorznabQuery query)
+        {
+            var releases = new List<ReleaseInfo>();
+            var searchString = query.GetQueryString();
+            var searchUrl = SearchUrl;
+            var queryCollection = new NameValueCollection();
+
+            queryCollection.Add("total", "146"); // Not sure what this is about but its required!
+
+            var cat = "0";
+            var queryCats = MapTorznabCapsToTrackers(query);
+            if (queryCats.Count == 1)
+            {
+                cat = queryCats.First().ToString();
+            }
+
+            queryCollection.Add("cat", cat);
+            queryCollection.Add("searchin", "filename");
+            queryCollection.Add("search", searchString);
+            queryCollection.Add("page", "1");
+            searchUrl += "?" + queryCollection.GetQueryString();
+
+            var extraHeaders = new Dictionary<string, string>()
+            {
+                { "X-Requested-With", "XMLHttpRequest" }
+            };
+
+            var response = await RequestStringWithCookiesAndRetry(searchUrl, null, SearchUrlReferer, extraHeaders);
+
+            var results = response.Content;
+            try
+            {
+                CQ dom = results;
+
+                var rows = dom["tr"];
+                foreach (var row in rows.Skip(1))
+                {
+                    var release = new ReleaseInfo();
+                    var qRow = row.Cq();
+                    var qTitleLink = qRow.Find("td:eq(1) a:eq(0)").First();
+                    release.Title = qTitleLink.Find("strong").Text().Trim();
+
+                    // If we search an get no results, we still get a table just with no info.
+                    if (string.IsNullOrWhiteSpace(release.Title))
+                    {
+                        break;
+                    }
+
+                    release.Description = release.Title;
+                    release.Guid = new Uri(qTitleLink.Attr("href"));
+                    release.Comments = release.Guid;
+
+                    var dateString = qRow.Find("td:eq(4)").Text();
+                    release.PublishDate = DateTime.ParseExact(dateString, "dd MMM yy", CultureInfo.InvariantCulture);
+
+                    var qLink = qRow.Find("td:eq(2) a");
+                    if (qLink.Length != 0) // newbie users don't see DL links
+                    {
+                        release.Link = new Uri(qLink.Attr("href"));
+                    }
+                    else
+                    {
+                        // use comments link as placeholder
+                        // null causes errors during export to torznab
+                        // skipping the release prevents newbie users from adding the tracker (empty result)
+                        release.Link = release.Comments;
+                    }
+
+                    var sizeStr = qRow.Find("td:eq(5)").Text();
+                    release.Size = ReleaseInfo.GetBytes(sizeStr);
+
+                    var connections = qRow.Find("td:eq(7)").Text().Trim().Split("/".ToCharArray(), StringSplitOptions.RemoveEmptyEntries);
+
+                    release.Seeders = ParseUtil.CoerceInt(connections[0].Trim());
+                    release.Peers = ParseUtil.CoerceInt(connections[1].Trim()) + release.Seeders;
+                    release.Grabs = ParseUtil.CoerceLong(connections[2].Trim());
+
+                    var rCat = row.Cq().Find("td:eq(0) a").First().Attr("href");
+                    var rCatIdx = rCat.IndexOf("cat=");
+                    if (rCatIdx > -1)
+                    {
+                        rCat = rCat.Substring(rCatIdx + 4);
+                    }
+
+                    release.Category = MapTrackerCatToNewznab(rCat);
+
+                    if (qRow.Find("img[alt=\"Gold Torrent\"]").Length >= 1)
+                        release.DownloadVolumeFactor = 0;
+                    else if (qRow.Find("img[alt=\"Silver Torrent\"]").Length >= 1)
+                        release.DownloadVolumeFactor = 0.5;
+                    else
+                        release.DownloadVolumeFactor = 1;
+
+                    var ULFactorImg = qRow.Find("img[alt*=\"x Multiplier Torrent\"]");
+                    if (ULFactorImg.Length >= 1)
+                    {
+                        release.UploadVolumeFactor = ParseUtil.CoerceDouble(ULFactorImg.Attr("alt").Split('x')[0]);
+                    }
+                    else
+                    {
+                        release.UploadVolumeFactor = 1;
+                    }
+                    
+                    releases.Add(release);
+                }
+            }
+            catch (Exception ex)
+            {
+                OnParseError(results, ex);
+            }
+
+            return releases;
+        }
+    }
+}

src/Jackett/Indexers/BB.cs

@@ -48,7 +48,7 @@ namespace Jackett.Indexers
             AddCategoryMapping(2, TorznabCatType.PC);
             AddCategoryMapping(3, TorznabCatType.BooksEbook);
             AddCategoryMapping(4, TorznabCatType.AudioAudiobook);
-            AddCategoryMapping(6, TorznabCatType.BooksComics);
+            AddCategoryMapping(7, TorznabCatType.BooksComics);
             AddCategoryMapping(8, TorznabCatType.TVAnime);
             AddCategoryMapping(9, TorznabCatType.Movies);
             AddCategoryMapping(10, TorznabCatType.TVHD);
@@ -59,7 +59,8 @@ namespace Jackett.Indexers
 
         public async Task<IndexerConfigurationStatus> ApplyConfiguration(JToken configJson)
         {
-            configData.LoadValuesFromJson(configJson);
+            if (configJson != null)
+                configData.LoadValuesFromJson(configJson);
             var pairs = new Dictionary<string, string> {
                 { "username", configData.Username.Value },
                 { "password", configData.Password.Value },
@@ -107,6 +108,13 @@ namespace Jackett.Indexers
 
             var results = await RequestStringWithCookiesAndRetry(searchUrl);
 
+            // Occasionally the cookies become invalid, login again if that happens
+            if (results.IsRedirect)
+            {
+                await ApplyConfiguration(null);
+                results = await RequestStringWithCookiesAndRetry(searchUrl);
+            }
+
             try
             {
                 CQ dom = results.Content;
@@ -137,9 +145,21 @@ namespace Jackett.Indexers
                     var sizeStr = row.ChildElements.ElementAt(4).Cq().Text();
                     release.Size = ReleaseInfo.GetBytes(sizeStr);
 
+                    release.Files = ParseUtil.CoerceInt(row.ChildElements.ElementAt(2).Cq().Text().Trim());
+                    release.Grabs = ParseUtil.CoerceInt(row.ChildElements.ElementAt(6).Cq().Text().Trim());
                     release.Seeders = ParseUtil.CoerceInt(row.ChildElements.ElementAt(7).Cq().Text().Trim());
                     release.Peers = ParseUtil.CoerceInt(row.ChildElements.ElementAt(8).Cq().Text().Trim()) + release.Seeders;
 
+                    var grabs = qRow.Find("td:nth-child(6)").Text();
+                    release.Grabs = ParseUtil.CoerceInt(grabs);
+
+                    if (qRow.Find("strong:contains(\"Freeleech!\")").Length >= 1)
+                        release.DownloadVolumeFactor = 0;
+                    else
+                        release.DownloadVolumeFactor = 1;
+
+                    release.UploadVolumeFactor = 1;
+
                     releases.Add(release);
                 }
             }

src/Jackett/Indexers/BJShare.cs

@@ -0,0 +1,319 @@
+using Jackett.Utils.Clients;
+using NLog;
+using Jackett.Services;
+using Jackett.Utils;
+using Jackett.Models;
+using System.Threading.Tasks;
+using Newtonsoft.Json.Linq;
+using System.Collections.Generic;
+using System;
+using System.Globalization;
+using Jackett.Models.IndexerConfig;
+using System.Collections.Specialized;
+using AngleSharp.Parser.Html;
+using AngleSharp.Dom;
+using System.Text.RegularExpressions;
+
+namespace Jackett.Indexers
+{
+    public class BJShare : BaseIndexer, IIndexer
+    {
+        string LoginUrl { get { return SiteLink + "login.php"; } }
+        string BrowseUrl { get { return SiteLink + "torrents.php"; } }
+        string TodayUrl { get { return SiteLink + "torrents.php?action=today"; } }
+
+        new ConfigurationDataBasicLoginWithRSSAndDisplay configData
+        {
+            get { return (ConfigurationDataBasicLoginWithRSSAndDisplay)base.configData; }
+            set { base.configData = value; }
+        }
+
+        public BJShare(IIndexerManagerService i, IWebClient wc, Logger l, IProtectionService ps)
+            : base(name: "BJ-Share",
+                   description: "A brazilian tracker.",
+                   link: "https://bj-share.me/",
+                   caps: TorznabUtil.CreateDefaultTorznabTVCaps(),
+                   manager: i,
+                   client: wc,
+                   logger: l,
+                   p: ps,
+                   configData: new ConfigurationDataBasicLoginWithRSSAndDisplay())
+        {
+            AddCategoryMapping(14, TorznabCatType.TVAnime); // Anime
+            AddCategoryMapping(3, TorznabCatType.PC0day); // Aplicativos
+            AddCategoryMapping(8, TorznabCatType.Other); // Apostilas/Tutoriais
+            AddCategoryMapping(19, TorznabCatType.AudioAudiobook); // Audiobook
+            AddCategoryMapping(16, TorznabCatType.TVOTHER); // Desenho Animado
+            AddCategoryMapping(18, TorznabCatType.TVDocumentary); // Documentários
+            AddCategoryMapping(10, TorznabCatType.Books); // E-Books
+            AddCategoryMapping(20, TorznabCatType.TVSport); // Esportes
+            AddCategoryMapping(1, TorznabCatType.Movies); // Filmes
+            AddCategoryMapping(12, TorznabCatType.MoviesOther); // Histórias em Quadrinhos
+            AddCategoryMapping(5, TorznabCatType.Audio); // Músicas
+            AddCategoryMapping(7, TorznabCatType.Other); // Outros
+            AddCategoryMapping(9, TorznabCatType.BooksMagazines); // Revistas
+            AddCategoryMapping(2, TorznabCatType.TV); // Seriados
+            AddCategoryMapping(17, TorznabCatType.TV); // Shows
+            AddCategoryMapping(13, TorznabCatType.TV); // Stand Up Comedy
+            AddCategoryMapping(11, TorznabCatType.Other); // Video-Aula
+            AddCategoryMapping(6, TorznabCatType.TV); // Vídeos de TV
+            AddCategoryMapping(4, TorznabCatType.Other); // Jogos
+            AddCategoryMapping(199, TorznabCatType.XXX); // Filmes Adultos
+            AddCategoryMapping(200, TorznabCatType.XXX); // Jogos Adultos
+            AddCategoryMapping(201, TorznabCatType.XXXImageset); // Fotos Adultas
+        }
+
+        public async Task<IndexerConfigurationStatus> ApplyConfiguration(JToken configJson)
+        {
+            configData.LoadValuesFromJson(configJson);
+
+            var pairs = new Dictionary<string, string>
+            {
+                { "username", configData.Username.Value },
+                { "password", configData.Password.Value },
+                { "keeplogged", "1" }
+            };
+
+            var result = await RequestLoginAndFollowRedirect(LoginUrl, pairs, null, true, null, LoginUrl, true);
+            await ConfigureIfOK(result.Cookies, result.Content != null && result.Content.Contains("logout.php"), () =>
+            {
+                var errorMessage = result.Content;
+                throw new ExceptionWithConfigData(errorMessage, configData);
+            });
+            return IndexerConfigurationStatus.RequiresTesting;
+        }
+
+        private string StripSearchString(string term)
+        {
+            // Search does not support searching with episode numbers so strip it if we have one
+            // Ww AND filter the result later to archive the proper result
+            term = Regex.Replace(term, @"[S|E]\d\d", string.Empty);
+            return term.Trim();
+        }
+
+        public async Task<IEnumerable<ReleaseInfo>> PerformQuery(TorznabQuery query)
+        {
+            var releases = new List<ReleaseInfo>();
+            
+            var searchString = query.GetQueryString();
+            
+            // if the search string is empty use the "last 24h torrents" view
+            if (string.IsNullOrWhiteSpace(searchString))
+            {
+                var results = await RequestStringWithCookies(TodayUrl);
+                try
+                {
+                    string RowsSelector = "table.torrent_table > tbody > tr:not(tr.colhead)";
+
+                    var SearchResultParser = new HtmlParser();
+                    var SearchResultDocument = SearchResultParser.Parse(results.Content);
+                    var Rows = SearchResultDocument.QuerySelectorAll(RowsSelector);
+                    foreach (var Row in Rows)
+                    {
+                        try
+                        {
+                            var release = new ReleaseInfo();
+
+                            release.MinimumRatio = 1;
+                            release.MinimumSeedTime = 0;
+
+                            var qDetailsLink = Row.QuerySelector("a.BJinfoBox");
+                            var qTitle = qDetailsLink.QuerySelector("font");
+                            release.Title = qTitle.TextContent;
+
+                            var qBJinfoBox = qDetailsLink.QuerySelector("span");
+                            var qCatLink = Row.QuerySelector("a[href^=\"/torrents.php?filter_cat\"]");
+                            var qDLLink = Row.QuerySelector("a[href^=\"torrents.php?action=download\"]");
+                            var qSeeders = Row.QuerySelector("td:nth-child(4)");
+                            var qLeechers = Row.QuerySelector("td:nth-child(5)");
+                            var qFreeLeech = Row.QuerySelector("font[color=\"green\"]:contains(Free)");
+
+                            release.Description = "";
+                            foreach (var Child in qBJinfoBox.ChildNodes)
+                            {
+                                var type = Child.NodeType;
+                                if (type != NodeType.Text)
+                                    continue;
+
+                                var line = Child.TextContent;
+                                if (line.StartsWith("Tamanho:"))
+                                {
+                                    string Size = line.Substring("Tamanho: ".Length); ;
+                                    release.Size = ReleaseInfo.GetBytes(Size);
+                                }
+                                else if (line.StartsWith("Lançado em: "))
+                                {
+                                    string PublishDateStr = line.Substring("Lançado em: ".Length).Replace("às ", "");
+                                    PublishDateStr += " +0";
+                                    var PublishDate = DateTime.SpecifyKind(DateTime.ParseExact(PublishDateStr, "dd/MM/yyyy HH:mm z", CultureInfo.InvariantCulture), DateTimeKind.Unspecified);
+                                    release.PublishDate = PublishDate.ToLocalTime();
+                                }
+                                else
+                                {
+                                    release.Description += line + "\n";
+                                }
+                            }
+
+
+                            var catStr = qCatLink.GetAttribute("href").Split('=')[1];
+                            release.Category = MapTrackerCatToNewznab(catStr);
+
+                            release.Link = new Uri(SiteLink + qDLLink.GetAttribute("href"));
+                            release.Comments = new Uri(SiteLink + qDetailsLink.GetAttribute("href"));
+                            release.Guid = release.Link;
+
+                            release.Seeders = ParseUtil.CoerceInt(qSeeders.TextContent);
+                            release.Peers = ParseUtil.CoerceInt(qLeechers.TextContent) + release.Seeders;
+
+
+                            if (qFreeLeech != null)
+                                release.DownloadVolumeFactor = 0;
+                            else
+                                release.DownloadVolumeFactor = 1;
+
+                            release.UploadVolumeFactor = 1;
+
+                            releases.Add(release);
+                        }
+                        catch (Exception ex)
+                        {
+                            logger.Error(string.Format("{0}: Error while parsing row '{1}': {2}", ID, Row.OuterHtml, ex.Message));
+                        }
+                    }
+                }
+                catch (Exception ex)
+                {
+                    OnParseError(results.Content, ex);
+                }
+            }
+            else // use search
+            {
+                var searchUrl = BrowseUrl;
+
+                var queryCollection = new NameValueCollection();
+                queryCollection.Add("searchstr", StripSearchString(searchString));
+                queryCollection.Add("order_by", "time");
+                queryCollection.Add("order_way", "desc");
+                queryCollection.Add("group_results", "1");
+                queryCollection.Add("action", "basic");
+                queryCollection.Add("searchsubmit", "1");
+
+                foreach (var cat in MapTorznabCapsToTrackers(query))
+                {
+                    queryCollection.Add("filter_cat["+cat+"]", "1");
+                }
+
+                searchUrl += "?" + queryCollection.GetQueryString();
+
+                var results = await RequestStringWithCookies(searchUrl);
+                try
+                {
+                    string RowsSelector = "table.torrent_table > tbody > tr:not(tr.colhead)";
+
+                    var SearchResultParser = new HtmlParser();
+                    var SearchResultDocument = SearchResultParser.Parse(results.Content);
+                    var Rows = SearchResultDocument.QuerySelectorAll(RowsSelector);
+
+                    int GroupCategory = 0;
+                    string GroupTitle = null;
+                    string GroupYearStr = null;
+                    Nullable<DateTime> GroupPublishDate = null;
+
+                    foreach (var Row in Rows)
+                    {
+                        try
+                        {
+                            var qDetailsLink = Row.QuerySelector("a[href^=\"torrents.php?id=\"]");
+                            string Title = qDetailsLink.TextContent;
+                            int Category = 0;
+                            string YearStr = null;
+                            Nullable<DateTime> YearPublishDate = null;
+
+                            if (Row.ClassList.Contains("group") || Row.ClassList.Contains("torrent")) // group/ungrouped headers
+                            {
+                                var qCatLink = Row.QuerySelector("a[href^=\"/torrents.php?filter_cat\"]");
+                                string CategoryStr = qCatLink.GetAttribute("href").Split('=')[1].Split('&')[0];
+                                Category = MapTrackerCatToNewznab(CategoryStr);
+                                YearStr = qDetailsLink.NextSibling.TextContent.Trim().TrimStart('[').TrimEnd(']');
+                                YearPublishDate = DateTime.SpecifyKind(DateTime.ParseExact(YearStr, "yyyy", CultureInfo.InvariantCulture), DateTimeKind.Unspecified);
+
+                                if (Row.ClassList.Contains("group")) // group headers
+                                {
+                                    GroupCategory = Category;
+                                    GroupTitle = Title;
+                                    GroupYearStr = YearStr;
+                                    GroupPublishDate = YearPublishDate;
+                                    continue;
+                                }
+                            }
+
+                            var release = new ReleaseInfo();
+
+                            release.MinimumRatio = 1;
+                            release.MinimumSeedTime = 0;
+
+                            var qDLLink = Row.QuerySelector("a[href^=\"torrents.php?action=download\"]");
+                            var qSize = Row.QuerySelector("td:nth-last-child(4)");
+                            var qSeeders = Row.QuerySelector("td:nth-last-child(3)");
+                            var qLeechers = Row.QuerySelector("td:nth-last-child(2)");
+                            var qFreeLeech = Row.QuerySelector("strong[title=\"Free\"]");
+
+                            if (Row.ClassList.Contains("group_torrent")) // torrents belonging to a group
+                            {
+                                release.Description = qDetailsLink.TextContent;
+                                release.Title = GroupTitle + " " + GroupYearStr;
+                                release.PublishDate = GroupPublishDate.Value;
+                                release.Category = GroupCategory;
+                            }
+                            else if (Row.ClassList.Contains("torrent")) // standalone/un grouped torrents
+                            {
+                                var qDescription = Row.QuerySelector("div.torrent_info");
+                                release.Description = qDescription.TextContent;
+                                release.Title = Title + " " + YearStr;
+                                release.PublishDate = YearPublishDate.Value;
+                                release.Category = Category;
+                            }
+
+                            release.Description = release.Description.Replace(" / Free", ""); // Remove Free Tag
+                            release.Title += " " + release.Description; // add year and Description to the release Title to add some meaning to it
+
+                            // check for previously stripped search terms
+                            if (!query.MatchQueryStringAND(release.Title))
+                                continue;
+
+                            var Size = qSize.TextContent;
+                            release.Size = ReleaseInfo.GetBytes(Size);
+
+                            release.Link = new Uri(SiteLink + qDLLink.GetAttribute("href"));
+                            release.Comments = new Uri(SiteLink + qDetailsLink.GetAttribute("href"));
+                            release.Guid = release.Link;
+
+                            release.Seeders = ParseUtil.CoerceInt(qSeeders.TextContent);
+                            release.Peers = ParseUtil.CoerceInt(qLeechers.TextContent) + release.Seeders;
+
+                            if (qFreeLeech != null)
+                                release.DownloadVolumeFactor = 0;
+                            else
+                                release.DownloadVolumeFactor = 1;
+
+                            release.UploadVolumeFactor = 1;
+
+                            releases.Add(release);
+                        }
+                        catch (Exception ex)
+                        {
+                            logger.Error(string.Format("{0}: Error while parsing row '{1}': {2}", ID, Row.OuterHtml, ex.Message));
+                        }
+                    }
+                }
+                catch (Exception ex)
+                {
+                    OnParseError(results.Content, ex);
+                }
+            }
+
+            return releases;
+        }
+    }
+}
+

src/Jackett/Indexers/BakaBT.cs

@@ -0,0 +1,198 @@
+using CsQuery;
+using Jackett.Models;
+using Jackett.Services;
+using Jackett.Utils;
+using Jackett.Utils.Clients;
+using Newtonsoft.Json.Linq;
+using NLog;
+using System;
+using System.Collections.Generic;
+using System.Globalization;
+using System.Linq;
+using System.Net;
+using System.Text;
+using System.Threading.Tasks;
+using System.Web;
+using Jackett.Models.IndexerConfig;
+
+namespace Jackett.Indexers
+{
+    public class BakaBT : BaseIndexer, IIndexer
+    {
+        public string SearchUrl { get { return SiteLink + "browse.php?only=0&incomplete=1&lossless=1&hd=1&multiaudio=1&bonus=1&reorder=1&q="; } }
+        public string LoginUrl { get { return SiteLink + "login.php"; } }
+        public string id = "bakabt";
+
+        new ConfigurationDataBasicLogin configData
+        {
+            get { return (ConfigurationDataBasicLogin)base.configData; }
+            set { base.configData = value; }
+        }
+
+        public BakaBT(IIndexerManagerService i, IWebClient wc, Logger l, IProtectionService ps)
+            : base(name: "BakaBT",
+                description: "Anime Comunity",
+                link: "https://bakabt.me/",
+                caps: new TorznabCapabilities(TorznabCatType.TVAnime),
+                manager: i,
+                client: wc,
+                logger: l,
+                p: ps,
+                configData: new ConfigurationDataBasicLogin())
+        {
+        }
+
+
+        public async Task<IndexerConfigurationStatus> ApplyConfiguration(JToken configJson)
+        {
+            configData.LoadValuesFromJson(configJson);
+
+            var loginForm = await webclient.GetString(new Utils.Clients.WebRequest()
+                {
+                    Url = LoginUrl,
+                    Type = RequestType.GET
+                });
+
+            var pairs = new Dictionary<string, string> {
+                { "username", configData.Username.Value },
+                { "password", configData.Password.Value },
+                { "returnto", "/index.php" }
+            };
+
+            var response = await RequestLoginAndFollowRedirect(LoginUrl, pairs, loginForm.Cookies, true, null, SiteLink);
+            var responseContent = response.Content;
+            await ConfigureIfOK(response.Cookies, responseContent.Contains("<a href=\"logout.php\">Logout</a>"), () =>
+                {
+                    CQ dom = responseContent;
+                    var messageEl = dom[".error"].First();
+                    var errorMessage = messageEl.Text().Trim();
+                    throw new ExceptionWithConfigData(errorMessage, configData);
+                });
+
+            return IndexerConfigurationStatus.RequiresTesting;
+        }
+
+        public async Task<IEnumerable<ReleaseInfo>> PerformQuery(TorznabQuery query)
+        {
+
+            // This tracker only deals with full seasons so chop off the episode/season number if we have it D:
+            if (!string.IsNullOrWhiteSpace(query.SearchTerm))
+            {
+                var splitindex = query.SearchTerm.LastIndexOf(' ');
+                if (splitindex > -1)
+                    query.SearchTerm = query.SearchTerm.Substring(0, splitindex);
+            }
+
+            var releases = new List<ReleaseInfo>();
+            var searchString = query.SanitizedSearchTerm;
+            var episodeSearchUrl = SearchUrl + HttpUtility.UrlEncode(searchString);
+            var response = await RequestStringWithCookiesAndRetry(episodeSearchUrl);
+
+            try
+            {
+                CQ dom = response.Content;
+                var rows = dom[".torrents tr.torrent"];
+
+                foreach (var row in rows)
+                {
+
+                    var qRow = row.Cq();
+                    var qTitleLink = qRow.Find("a.title").First();
+                    var title = qTitleLink.Text().Trim();
+
+                    // Insert before the release info
+                    var taidx = title.IndexOf('(');
+                    var tbidx = title.IndexOf('[');
+
+                    if (taidx == -1)
+                        taidx = title.Length;
+
+                    if (tbidx == -1)
+                        tbidx = title.Length;
+                    var titleSplit = Math.Min(taidx, tbidx);
+                    var titleSeries = title.Substring(0, titleSplit);
+                    var releaseInfo = title.Substring(titleSplit);
+
+                    // For each over each pipe deliminated name
+                    foreach (var name in titleSeries.Split("|".ToCharArray(), StringSplitOptions.RemoveEmptyEntries))
+                    {
+                        var release = new ReleaseInfo();
+
+                        release.Title = (name + releaseInfo).Trim();
+                        // Ensure the season is defined as this tracker only deals with full seasons
+                        if (release.Title.IndexOf("Season") == -1)
+                        {
+                            // Insert before the release info
+                            var aidx = release.Title.IndexOf('(');
+                            var bidx = release.Title.IndexOf('[');
+
+                            if (aidx == -1)
+                                aidx = release.Title.Length;
+
+                            if (bidx == -1)
+                                bidx = release.Title.Length;
+
+                            var insertPoint = Math.Min(aidx, bidx);
+                            release.Title = release.Title.Substring(0, insertPoint) + "Season 1 " + release.Title.Substring(insertPoint);
+                        }
+
+                        release.Description = release.Title;
+                        release.Guid = new Uri(SiteLink + qTitleLink.Attr("href"));
+                        release.Comments = release.Guid;
+
+                        release.Link = new Uri(SiteLink + qRow.Find(".peers a").First().Attr("href"));
+
+                        release.Seeders = int.Parse(qRow.Find(".peers a").Get(0).InnerText);
+                        release.Peers = release.Seeders + int.Parse(qRow.Find(".peers a").Get(1).InnerText);
+
+                        release.MinimumRatio = 1;
+
+                        var size = qRow.Find(".size").First().Text();
+                        release.Size = ReleaseInfo.GetBytes(size);
+
+                        //22 Jul 15
+                        var dateStr = qRow.Find(".added").First().Text().Replace("'", string.Empty);
+                        if (dateStr.Split(' ')[0].Length == 1)
+                            dateStr = "0" + dateStr;
+
+                        if (string.Equals(dateStr, "yesterday", StringComparison.InvariantCultureIgnoreCase))
+                        {
+                            release.PublishDate = DateTime.Now.AddDays(-1);
+                        }
+                        else if (string.Equals(dateStr, "today", StringComparison.InvariantCultureIgnoreCase))
+                        {
+                            release.PublishDate = DateTime.Now;
+                        }
+                        else
+                        {
+                            release.PublishDate = DateTime.ParseExact(dateStr, "dd MMM yy", CultureInfo.InvariantCulture);
+                        }
+
+                        releases.Add(release);
+                    }
+                }
+            }
+            catch (Exception ex)
+            {
+                OnParseError(response.Content, ex);
+            }
+
+            return releases;
+        }
+
+        public override async Task<byte[]> Download(Uri link)
+        {
+            var downloadPage = await RequestStringWithCookies(link.ToString());
+            CQ dom = downloadPage.Content;
+            var downloadLink = dom.Find(".download_link").First().Attr("href");
+
+            if (string.IsNullOrWhiteSpace(downloadLink))
+            {
+                throw new Exception("Unable to find download link.");
+            }
+
+            var response = await RequestBytesWithCookies(SiteLink + downloadLink);
+            return response.Content;
+        }
+    }
+}

src/Jackett/Indexers/BaseIndexer.cs

@@ -17,13 +17,13 @@ namespace Jackett.Indexers
 {
     public abstract class BaseIndexer
     {
-        public string SiteLink { get; private set; }
-        public string DisplayDescription { get; private set; }
-        public string DisplayName { get; private set; }
+        public string SiteLink { get; protected set; }
+        public string DisplayDescription { get; protected set; }
+        public string DisplayName { get; protected set; }
         public string ID { get { return GetIndexerID(GetType()); } }
 
         public bool IsConfigured { get; protected set; }
-        public TorznabCapabilities TorznabCaps { get; private set; }
+        public TorznabCapabilities TorznabCaps { get; protected set; }
         protected Logger logger;
         protected IIndexerManagerService indexerService;
         protected static List<CachedQueryResult> cache = new List<CachedQueryResult>();
@@ -44,7 +44,9 @@ namespace Jackett.Indexers
 
         private List<CategoryMapping> categoryMapping = new List<CategoryMapping>();
 
+        // standard constructor used by most indexers
         public BaseIndexer(string name, string link, string description, IIndexerManagerService manager, IWebClient client, Logger logger, ConfigurationData configData, IProtectionService p, TorznabCapabilities caps = null, string downloadBase = null)
+            : this(manager, client, logger, p)
         {
             if (!link.EndsWith("/"))
                 throw new Exception("Site link must end with a slash.");
@@ -52,12 +54,7 @@ namespace Jackett.Indexers
             DisplayName = name;
             DisplayDescription = description;
             SiteLink = link;
-            this.logger = logger;
-            indexerService = manager;
-            webclient = client;
-            protectionService = p;
             this.downloadUrlBase = downloadBase;
-
             this.configData = configData;
 
             if (caps == null)
@@ -66,6 +63,15 @@ namespace Jackett.Indexers
 
         }
 
+        // minimal constructor used by e.g. cardigann generic indexer
+        public BaseIndexer(IIndexerManagerService manager, IWebClient client, Logger logger, IProtectionService p)
+        {
+            this.logger = logger;
+            indexerService = manager;
+            webclient = client;
+            protectionService = p;
+        }
+
         public IEnumerable<ReleaseInfo> CleanLinks(IEnumerable<ReleaseInfo> releases)
         {
             if (string.IsNullOrEmpty(downloadUrlBase))
@@ -179,8 +185,8 @@ namespace Jackett.Indexers
 
         private String ResolveCookies(String incomingCookies = "")
         {
-            var redirRequestCookies = (CookieHeader != "" ? CookieHeader + " " : "") + incomingCookies;
-            System.Text.RegularExpressions.Regex expression = new System.Text.RegularExpressions.Regex(@"([^\s]+)=([^=]+)(?:\s|$)");
+            var redirRequestCookies = (CookieHeader != null && CookieHeader != "" ? CookieHeader + " " : "") + incomingCookies;
+            System.Text.RegularExpressions.Regex expression = new System.Text.RegularExpressions.Regex(@"([^\\,;\s]+)=([^=\\,;\s]+)");
             Dictionary<string, string> cookieDIctionary = new Dictionary<string, string>();
             var matches = expression.Match(redirRequestCookies);
             while (matches.Success)
@@ -188,10 +194,23 @@ namespace Jackett.Indexers
                 if (matches.Groups.Count > 2) cookieDIctionary[matches.Groups[1].Value] = matches.Groups[2].Value;
                 matches = matches.NextMatch();
             }
-            return string.Join(" ", cookieDIctionary.Select(kv => kv.Key.ToString() + "=" + kv.Value.ToString()).ToArray());
+            return string.Join("; ", cookieDIctionary.Select(kv => kv.Key.ToString() + "=" + kv.Value.ToString()).ToArray());
             
         }
 
+        // Update CookieHeader with new cookies and save the config if something changed (e.g. a new CloudFlare clearance cookie was issued)
+        protected void UpdateCookieHeader(string newCookies, string cookieOverride = null)
+        {
+            string newCookieHeader = ResolveCookies((cookieOverride != null && cookieOverride != "" ? cookieOverride + " " : "") + newCookies);
+            if (CookieHeader != newCookieHeader)
+            {
+                logger.Debug(string.Format("updating Cookies {0} => {1}", CookieHeader, newCookieHeader));
+                CookieHeader = newCookieHeader;
+                if (IsConfigured)
+                    SaveConfig();
+            }
+        }
+
         private async Task DoFollowIfRedirect(WebClientByteResult incomingResponse, string referrer = null, string overrideRedirectUrl = null, string overrideCookies = null, bool accumulateCookies = false)
         {
             if (incomingResponse.IsRedirect)
@@ -267,6 +286,8 @@ namespace Jackett.Indexers
             var response = await RequestBytesWithCookiesAndRetry(link.ToString());
             if (response.Status != System.Net.HttpStatusCode.OK && response.Status != System.Net.HttpStatusCode.Continue && response.Status != System.Net.HttpStatusCode.PartialContent)
             {
+                if(response.Content != null)
+                    logger.Error("Failed download response:\n" + Encoding.UTF8.GetString(response.Content));
                 throw new Exception($"Remote server returned {response.Status.ToString()}");
             }
 
@@ -306,7 +327,9 @@ namespace Jackett.Indexers
 
             if (cookieOverride != null)
                 request.Cookies = cookieOverride;
-            return await webclient.GetString(request);
+            WebClientStringResult result = await webclient.GetString(request);
+            UpdateCookieHeader(result.Cookies, cookieOverride);
+            return result;
         }
 
         protected async Task<WebClientStringResult> RequestStringWithCookiesAndRetry(string url, string cookieOverride = null, string referer = null, Dictionary<string, string> headers = null)
@@ -358,7 +381,9 @@ namespace Jackett.Indexers
 
             if (emulateBrowser.HasValue)
                 request.EmulateBrowser = emulateBrowser.Value;
-            return await webclient.GetString(request);
+            WebClientStringResult result = await webclient.GetString(request);
+            UpdateCookieHeader(result.Cookies, cookieOverride);
+            return result;
         }
 
         protected async Task<WebClientStringResult> PostDataWithCookiesAndRetry(string url, IEnumerable<KeyValuePair<string, string>> data, string cookieOverride = null, string referer = null, Dictionary<string, string> headers = null, string rawbody = null, bool? emulateBrowser = null)

src/Jackett/Indexers/BestFriends.cs

@@ -0,0 +1,229 @@
+using Jackett.Utils.Clients;
+using NLog;
+using Jackett.Services;
+using Jackett.Utils;
+using Jackett.Models;
+using System.Threading.Tasks;
+using Newtonsoft.Json.Linq;
+using System.Collections.Generic;
+using CsQuery;
+using System;
+using System.Globalization;
+using Jackett.Models.IndexerConfig;
+using System.Collections.Specialized;
+
+namespace Jackett.Indexers
+{
+    public class BestFriends : BaseIndexer, IIndexer
+    {
+        string LoginUrl { get { return SiteLink + "login.php"; } }
+        string TakeLoginUrl { get { return SiteLink + "takelogin.php"; } }
+        string BrowseUrl { get { return SiteLink + "browse.php"; } }
+
+        new ConfigurationDataCaptchaLogin configData
+        {
+            get { return (ConfigurationDataCaptchaLogin)base.configData; }
+            set { base.configData = value; }
+        }
+
+        public BestFriends(IIndexerManagerService i, IWebClient wc, Logger l, IProtectionService ps)
+            : base(name: "Best Friends",
+                   description: "A German general tracker.",
+                   link: "http://bf.mine.nu/",
+                   caps: TorznabUtil.CreateDefaultTorznabTVCaps(),
+                   manager: i,
+                   client: wc,
+                   logger: l,
+                   p: ps,
+                   configData: new ConfigurationDataCaptchaLogin())
+        {
+            AddCategoryMapping(18, TorznabCatType.TVAnime); // Anime
+            AddCategoryMapping(8,  TorznabCatType.PCMac); // Appz MAC
+            AddCategoryMapping(9,  TorznabCatType.PC); // Appz other
+            AddCategoryMapping(7,  TorznabCatType.PC); // Appz Windows
+            AddCategoryMapping(23, TorznabCatType.TVDocumentary); // Dokumentationen
+            AddCategoryMapping(32, TorznabCatType.Movies3D); // DVD 3D
+            AddCategoryMapping(15, TorznabCatType.Books); // eBooks
+            AddCategoryMapping(12, TorznabCatType.PCGames); // Games PC
+            AddCategoryMapping(37, TorznabCatType.PCPhoneOther); // Handy_Mobile 
+            AddCategoryMapping(24, TorznabCatType.TVHD); // HDTV
+            AddCategoryMapping(22, TorznabCatType.AudioAudiobook); // Hörbücher
+            AddCategoryMapping(1,  TorznabCatType.MoviesHD); // Movies 1080p/1080i
+            AddCategoryMapping(31, TorznabCatType.Movies3D); // Movies 3D
+            AddCategoryMapping(2,  TorznabCatType.MoviesHD); // Movies 720p/720i
+            AddCategoryMapping(21, TorznabCatType.MoviesBluRay); // Movies BluRay
+            AddCategoryMapping(5,  TorznabCatType.MoviesDVD); // Movies DVD/HDDVD
+            AddCategoryMapping(6,  TorznabCatType.MoviesSD); // Movies M/SVCD/Other
+            AddCategoryMapping(4,  TorznabCatType.MoviesSD); // Movies XVID/DIVX/h.264
+            AddCategoryMapping(10, TorznabCatType.Audio); // Music
+            AddCategoryMapping(25, TorznabCatType.AudioVideo); // Musikvideo
+            AddCategoryMapping(29, TorznabCatType.ConsoleNDS); // Nintendo DS
+            AddCategoryMapping(16, TorznabCatType.Other); // other
+            AddCategoryMapping(13, TorznabCatType.ConsolePS4); // Playstation
+            AddCategoryMapping(28, TorznabCatType.TVHD); // Serien HD
+            AddCategoryMapping(11, TorznabCatType.TVSD); // Serien XviD
+            AddCategoryMapping(33, TorznabCatType.Other); // Specials
+            AddCategoryMapping(30, TorznabCatType.TVSport); // Sport
+            AddCategoryMapping(19, TorznabCatType.TVOTHER); // TVRip
+            AddCategoryMapping(38, TorznabCatType.TVDocumentary); // US Dokus
+            AddCategoryMapping(20, TorznabCatType.MoviesForeign); // US Movies
+            AddCategoryMapping(14, TorznabCatType.TVFOREIGN); // US Serien
+            AddCategoryMapping(36, TorznabCatType.Other); // Wallpaper 
+            AddCategoryMapping(26, TorznabCatType.ConsoleWii); // Wii
+            AddCategoryMapping(27, TorznabCatType.ConsoleXbox360); // Xbox 360
+            AddCategoryMapping(3,  TorznabCatType.XXX); // XXX
+        }
+
+        public override async Task<ConfigurationData> GetConfigurationForSetup()
+        {
+            var loginPage = await RequestStringWithCookies(LoginUrl, string.Empty);
+            CQ dom = loginPage.Content;
+            CQ qCaptchaImg = dom.Find("td.tablea > img").First();
+
+            var CaptchaUrl = SiteLink + qCaptchaImg.Attr("src");
+            var captchaImage = await RequestBytesWithCookies(CaptchaUrl, loginPage.Cookies);
+            configData.CaptchaImage.Value = captchaImage.Content;
+            configData.CaptchaCookie.Value = loginPage.Cookies;
+            return configData;
+        }
+
+        public async Task<IndexerConfigurationStatus> ApplyConfiguration(JToken configJson)
+        {
+            configData.LoadValuesFromJson(configJson);
+
+            var pairs1 = new Dictionary<string, string>
+            {
+                { "proofcode", configData.CaptchaText.Value }
+            };
+            var cookies = configData.CaptchaCookie.Value;
+            var result1 = await RequestLoginAndFollowRedirect(LoginUrl, pairs1, cookies, true, null, LoginUrl, true);
+            if(result1.Content == null || !result1.Content.Contains("takelogin.php"))
+            {
+                CQ dom = result1.Content;
+                var errorMessage = dom["#login_error"].Text().Trim();
+                errorMessage = result1.Content;
+                throw new ExceptionWithConfigData(errorMessage, configData);
+            }
+
+            var pairs2 = new Dictionary<string, string>
+            {
+                { "username", configData.Username.Value },
+                { "password", configData.Password.Value }
+            };
+
+            var result = await RequestLoginAndFollowRedirect(TakeLoginUrl, pairs2, result1.Cookies, true, null, LoginUrl, true);
+            await ConfigureIfOK(result.Cookies, result.Content != null && result.Content.Contains("logout.php"), () =>
+                {
+                    CQ dom = result.Content;
+                    var errorMessage = dom["#login_error"].Text().Trim();
+                    errorMessage = result.Content;
+                    throw new ExceptionWithConfigData(errorMessage, configData);
+                });
+            return IndexerConfigurationStatus.RequiresTesting;
+        }
+
+        public async Task<IEnumerable<ReleaseInfo>> PerformQuery(TorznabQuery query)
+        {
+            TimeZoneInfo.TransitionTime startTransition = TimeZoneInfo.TransitionTime.CreateFloatingDateRule(new DateTime(1, 1, 1, 3, 0, 0), 3, 5, DayOfWeek.Sunday);
+            TimeZoneInfo.TransitionTime endTransition = TimeZoneInfo.TransitionTime.CreateFloatingDateRule(new DateTime(1, 1, 1, 4, 0, 0), 10, 5, DayOfWeek.Sunday);
+            TimeSpan delta = new TimeSpan(1, 0, 0);
+            TimeZoneInfo.AdjustmentRule adjustment = TimeZoneInfo.AdjustmentRule.CreateAdjustmentRule(new DateTime(1999, 10, 1), DateTime.MaxValue.Date, delta, startTransition, endTransition);
+            TimeZoneInfo.AdjustmentRule[] adjustments = { adjustment };
+            TimeZoneInfo germanyTz = TimeZoneInfo.CreateCustomTimeZone("W. Europe Standard Time", new TimeSpan(1, 0, 0), "(GMT+01:00) W. Europe Standard Time", "W. Europe Standard Time", "W. Europe DST Time", adjustments);
+
+            var releases = new List<ReleaseInfo>();
+            
+            var searchString = query.GetQueryString();
+            var searchUrl = BrowseUrl;
+            var queryCollection = new NameValueCollection();
+            queryCollection.Add("showsearch", "1");
+            queryCollection.Add("incldead", "1");
+            queryCollection.Add("blah", "0");
+            queryCollection.Add("orderby", "added");
+            queryCollection.Add("sort", "desc");
+
+            if (!string.IsNullOrWhiteSpace(searchString))
+            {
+                queryCollection.Add("search", searchString);
+            }
+
+            foreach (var cat in MapTorznabCapsToTrackers(query))
+            {
+                queryCollection.Add("c" + cat, "1");
+            }
+            searchUrl += "?" + queryCollection.GetQueryString();
+
+            var response = await RequestStringWithCookiesAndRetry(searchUrl, null, BrowseUrl);
+            var results = response.Content;
+            try
+            {
+                CQ dom = results;
+                var rows = dom["table.tableinborder > tbody > tr:has(td.tableb)"];
+
+                foreach (var row in rows)
+                {
+                    var release = new ReleaseInfo();
+                    release.MinimumRatio = 0.75;
+                    release.MinimumSeedTime = 0;
+                    var qRow = row.Cq();
+
+                    var qDetailsLink = qRow.Find("a[href^=details.php?id=]").First();
+                    release.Title = qDetailsLink.Attr("title");
+
+                    if (!query.MatchQueryStringAND(release.Title))
+                        continue;
+
+                    var qCatLink = qRow.Find("a[href^=browse.php?cat=]").First();
+
+                    // use negative indexes as if a user has "Wartezeit" there's an extra column after the title
+                    var qSeeders = qRow.Find("td:nth-last-child(4)");
+                    var qLeechers = qRow.Find("td:nth-last-child(3)");
+                    var qDateStr = qRow.Find("td:nth-last-child(7)");
+                    var qSize = qRow.Find("td:nth-last-child(6)");
+
+                    var torrentId = qDetailsLink.Attr("href").Replace("&hit=1", "").Split('=')[1];
+
+                    var catStr = qCatLink.Attr("href").Split('=')[1];
+                    release.Category = MapTrackerCatToNewznab(catStr);
+
+                    release.Link = new Uri(SiteLink + "download.php?torrent="+torrentId);
+                    release.Comments = new Uri(SiteLink + qDetailsLink.Attr("href"));
+                    release.Guid = release.Link;
+
+                    var sizeStr = qSize.Text();
+                    release.Size = ReleaseInfo.GetBytes(sizeStr.Replace(",", "."));
+
+                    release.Seeders = ParseUtil.CoerceInt(qSeeders.Text());
+                    release.Peers = ParseUtil.CoerceInt(qLeechers.Text()) + release.Seeders;
+
+                    var dateStr = qDateStr.Text();
+                    var dateGerman = DateTime.SpecifyKind(DateTime.ParseExact(dateStr, "dd.MM.yyyyHH:mm:ss", CultureInfo.InvariantCulture), DateTimeKind.Unspecified);
+                    DateTime pubDateUtc = TimeZoneInfo.ConvertTimeToUtc(dateGerman, germanyTz);
+                    release.PublishDate = pubDateUtc;
+
+                    var files = qRow.Find("td:nth-last-child(9)").Text();
+                    release.Files = ParseUtil.CoerceInt(files);
+
+                    var grabs = qRow.Find("td:nth-last-child(5)").Text();
+                    release.Grabs = ParseUtil.CoerceInt(grabs);
+
+                    if (qRow.Find("font[color=\"red\"]:contains(OnlyUp)").Length >= 1)
+                        release.DownloadVolumeFactor = 0;
+                    else
+                        release.DownloadVolumeFactor = 1;
+
+                    release.UploadVolumeFactor = 1;
+
+                    releases.Add(release);
+                }
+            }
+            catch (Exception ex)
+            {
+                OnParseError(results, ex);
+            }
+
+            return releases;
+        }
+    }
+}
+

src/Jackett/Indexers/BeyondHD.cs

@@ -30,7 +30,7 @@ namespace Jackett.Indexers
         public BeyondHD(IIndexerManagerService i, Logger l, IWebClient w, IProtectionService ps)
             : base(name: "BeyondHD",
                 description: "Without BeyondHD, your HDTV is just a TV",
-                link: "https://beyondhd.me/",
+                link: "https://beyond-hd.me/",
                 caps: new TorznabCapabilities(),
                 manager: i,
                 client: w,
@@ -79,10 +79,11 @@ namespace Jackett.Indexers
         public override async Task<ConfigurationData> GetConfigurationForSetup()
         {
             var loginPage = await RequestStringWithCookies(LoginUrl, string.Empty);
-            string recaptchaSiteKey = new Regex(@"loginwidget', \{[\s]{4,30}'sitekey' : '([0-9A-Za-z]{5,60})',[\s]{4,30}'theme'").Match(loginPage.Content).Groups[1].ToString().Trim();
-            var result = new ConfigurationDataRecaptchaLogin();
+            string recaptchaSiteKey = new Regex(@"loginwidget', \{[\s]{4,30}'sitekey' : '([0-9A-Za-z-]{5,60})',[\s]{4,30}'theme'").Match(loginPage.Content).Groups[1].ToString().Trim();
+            var result = this.configData;
             result.CookieHeader.Value = loginPage.Cookies;
             result.Captcha.SiteKey = recaptchaSiteKey;
+            result.Captcha.Version = "2";
             return result;
         }
 
@@ -187,6 +188,15 @@ namespace Jackett.Indexers
                     release.Seeders = ParseUtil.CoerceInt(row.ChildElements.ElementAt(9).Cq().Text());
                     release.Peers = ParseUtil.CoerceInt(row.ChildElements.ElementAt(10).Cq().Text()) + release.Seeders;
 
+                    var files = qRow.Find("td:nth-child(6)").Text();
+                    release.Files = ParseUtil.CoerceInt(files);
+
+                    var grabs = qRow.Find("td:nth-child(9) > a").Get(0).FirstChild.ToString();
+                    release.Grabs = ParseUtil.CoerceInt(grabs);
+
+                    release.DownloadVolumeFactor = 0; // ratioless
+                    release.UploadVolumeFactor = 1;
+
                     releases.Add(release);
 
                 }

src/Jackett/Indexers/BitCityReloaded.cs

@@ -0,0 +1,199 @@
+using Jackett.Utils.Clients;
+using NLog;
+using Jackett.Services;
+using Jackett.Utils;
+using Jackett.Models;
+using System.Threading.Tasks;
+using Newtonsoft.Json.Linq;
+using System.Collections.Generic;
+using CsQuery;
+using System.Web;
+using System;
+using System.Globalization;
+using Jackett.Models.IndexerConfig;
+using System.Collections.Specialized;
+
+namespace Jackett.Indexers
+{
+    public class BitCityReloaded : BaseIndexer, IIndexer
+    {
+        string LoginUrl { get { return SiteLink + "login.php"; } }
+        string BrowseUrl { get { return SiteLink + "uebersicht.php"; } }
+        TimeZoneInfo germanyTz = TimeZoneInfo.CreateCustomTimeZone("W. Europe Standard Time", new TimeSpan(1, 0, 0), "W. Europe Standard Time", "W. Europe Standard Time");
+
+        new ConfigurationDataBasicLoginWithRSSAndDisplay configData
+        {
+            get { return (ConfigurationDataBasicLoginWithRSSAndDisplay)base.configData; }
+            set { base.configData = value; }
+        }
+
+        public BitCityReloaded(IIndexerManagerService i, IWebClient wc, Logger l, IProtectionService ps)
+            : base(name: "Bit-City Reloaded",
+                   description: "A German general tracker.",
+                   link: "https://bc-reloaded.net/",
+                   caps: TorznabUtil.CreateDefaultTorznabTVCaps(),
+                   manager: i,
+                   client: wc,
+                   logger: l,
+                   p: ps,
+                   configData: new ConfigurationDataBasicLoginWithRSSAndDisplay())
+        {
+            this.configData.DisplayText.Value = "Only the results from the first search result page are shown, adjust your profile settings to show a reasonable amount (it looks like there's no maximum).";
+            this.configData.DisplayText.Name = "Notice";
+
+            AddCategoryMapping(1,  TorznabCatType.Other); // Anderes
+            AddCategoryMapping(2,  TorznabCatType.TVAnime); // Anime
+            AddCategoryMapping(34, TorznabCatType.PC); // Appz/Linux
+            AddCategoryMapping(35, TorznabCatType.PCMac); // Appz/Mac
+            AddCategoryMapping(36, TorznabCatType.PC); // Appz/Other
+            AddCategoryMapping(20, TorznabCatType.PC); // Appz/Win
+            AddCategoryMapping(3,  TorznabCatType.TVDocumentary); // Doku/Alle Formate
+            AddCategoryMapping(4,  TorznabCatType.Books); // EBooks
+            AddCategoryMapping(12, TorznabCatType.ConsolePS4); // Games PS / PSX
+            AddCategoryMapping(11, TorznabCatType.ConsoleNDS); // Games/Nintendo DS
+            AddCategoryMapping(10, TorznabCatType.PCGames); // Games/PC
+            AddCategoryMapping(13, TorznabCatType.ConsoleWii); // Games/Wii
+            AddCategoryMapping(14, TorznabCatType.ConsoleXbox); // Games/Xbox & 360
+            AddCategoryMapping(15, TorznabCatType.PCPhoneOther); // Handy & PDA
+            AddCategoryMapping(16, TorznabCatType.AudioAudiobook); // Hörspiel/Hörbuch
+            AddCategoryMapping(30, TorznabCatType.Other); // International
+            AddCategoryMapping(17, TorznabCatType.Other); // MegaPack
+            AddCategoryMapping(43, TorznabCatType.Movies3D); // Movie/3D
+            AddCategoryMapping(5,  TorznabCatType.MoviesDVD); // Movie/DVD/R
+            AddCategoryMapping(6,  TorznabCatType.MoviesHD); // Movie/HD 1080p
+            AddCategoryMapping(7,  TorznabCatType.MoviesHD); // Movie/HD 720p
+            AddCategoryMapping(32, TorznabCatType.MoviesOther); // Movie/TVRip
+            AddCategoryMapping(9,  TorznabCatType.MoviesOther); // Movie/XviD,DivX,h264
+            AddCategoryMapping(26, TorznabCatType.XXX); // Movie/XXX
+            AddCategoryMapping(41, TorznabCatType.XXXOther); // Movie/XXX/Other
+            AddCategoryMapping(42, TorznabCatType.XXXPacks); // Movie/XXX/Pack
+            AddCategoryMapping(45, TorznabCatType.MoviesHD); // Movies/4K
+            AddCategoryMapping(33, TorznabCatType.MoviesBluRay); // Movies/BluRay
+            AddCategoryMapping(18, TorznabCatType.Audio); // Musik
+            AddCategoryMapping(19, TorznabCatType.AudioVideo); // Musik Videos
+            AddCategoryMapping(44, TorznabCatType.TVOTHER); // Serie/DVD/R
+            AddCategoryMapping(22, TorznabCatType.TVHD); // Serie/HDTV
+            AddCategoryMapping(38, TorznabCatType.TV); // Serie/Pack
+            AddCategoryMapping(23, TorznabCatType.TVOTHER); // Serie/XviD,DivX,h264
+            AddCategoryMapping(25, TorznabCatType.TVSport); // Sport
+        }
+
+        public async Task<IndexerConfigurationStatus> ApplyConfiguration(JToken configJson)
+        {
+            configData.LoadValuesFromJson(configJson);
+
+            var pairs = new Dictionary<string, string>
+            {
+                { "username", configData.Username.Value },
+                { "password", configData.Password.Value }
+            };
+
+            var result = await RequestLoginAndFollowRedirect(LoginUrl, pairs, null, true, null, LoginUrl);
+            await ConfigureIfOK(result.Cookies, result.Content != null && result.Content.Contains("logout.php"), () =>
+                {
+                    CQ dom = result.Content;
+                    var errorMessage = dom["#login_error"].Text().Trim();
+                    throw new ExceptionWithConfigData(errorMessage, configData);
+                });
+            return IndexerConfigurationStatus.RequiresTesting;
+        }
+
+        public async Task<IEnumerable<ReleaseInfo>> PerformQuery(TorznabQuery query)
+        {
+            var releases = new List<ReleaseInfo>();
+            
+            var searchString = query.GetQueryString();
+            var searchUrl = BrowseUrl;
+            var queryCollection = new NameValueCollection();
+            queryCollection.Add("showsearch", "0");
+            queryCollection.Add("incldead", "1");
+            queryCollection.Add("blah", "0");
+            queryCollection.Add("team", "0");
+            queryCollection.Add("orderby", "added");
+            queryCollection.Add("sort", "desc");
+
+            if (!string.IsNullOrWhiteSpace(searchString))
+            {
+                queryCollection.Add("search", searchString);
+            }
+
+            foreach (var cat in MapTorznabCapsToTrackers(query))
+            {
+                queryCollection.Add("c" + cat, "1");
+            }
+            searchUrl += "?" + queryCollection.GetQueryString();
+
+            var response = await RequestStringWithCookiesAndRetry(searchUrl, null, BrowseUrl);
+            var results = response.Content;
+            try
+            {
+                CQ dom = results;
+                var rows = dom["table.tableinborder[cellpadding=0] > tbody > tr"];
+
+                foreach (var row in rows)
+                {
+                    var release = new ReleaseInfo();
+                    release.MinimumRatio = 0.7;
+                    release.MinimumSeedTime = 48 * 60 * 60;
+                    var qRow = row.Cq();
+                    var flagImgs = qRow.Find("table tbody tr: eq(0) td > img");
+                    List<string> flags = new List<string>();
+                    flagImgs.Each(flagImg => {
+                        flags.Add(flagImg.GetAttribute("src").Replace("pic/torrent_", "").Replace(".gif", "").ToUpper());
+                    });
+                        
+                    var titleLink = qRow.Find("table tbody tr:eq(0) td a:has(b)").First();
+                    var DLLink = qRow.Find("td.tableb > a:has(img[title=\"Torrent herunterladen\"])").First();
+                    release.Comments = new Uri(SiteLink + titleLink.Attr("href").Replace("&hit=1", ""));
+                    release.Link = new Uri(SiteLink + DLLink.Attr("href"));
+                    release.Title = titleLink.Text().Trim();
+
+                    if (!query.MatchQueryStringAND(release.Title))
+                        continue;
+
+                    release.Description = String.Join(", ", flags);
+                    release.Guid = release.Link;
+
+                    var dateStr = qRow.Find("table tbody tr:eq(1) td:eq(4)").Html().Replace("&nbsp;", " ").Trim();
+                    var dateGerman = DateTime.SpecifyKind(DateTime.ParseExact(dateStr, "dd.MM.yyyy HH:mm:ss", CultureInfo.InvariantCulture), DateTimeKind.Unspecified);
+                    DateTime pubDateUtc = TimeZoneInfo.ConvertTimeToUtc(dateGerman, germanyTz);
+                    release.PublishDate = pubDateUtc.ToLocalTime();
+
+                    var sizeStr = qRow.Find("table tbody tr:eq(1) td b").First().Text().Trim();
+                    release.Size = ReleaseInfo.GetBytes(sizeStr.Replace(",", "."));
+
+                    release.Seeders = ParseUtil.CoerceInt(qRow.Find("table tbody tr:eq(1) td:eq(1) b:eq(0) font").Text().Trim());
+                    release.Peers = ParseUtil.CoerceInt(qRow.Find("table tbody tr:eq(1) td:eq(1) b:eq(1) font").Text().Trim()) + release.Seeders;
+
+                    var catId = qRow.Find("td:eq(0) a").First().Attr("href").Split('=')[1];
+                    release.Category = MapTrackerCatToNewznab(catId);
+
+                    var cat = qRow.Find("td:eq(0) a").First().Attr("href").Substring(1);
+                    release.Category = MapTrackerCatToNewznab(cat);
+
+                    var files = qRow.Find("td:has(a[href*=\"&filelist=1\"])> b:nth-child(2)").Text();
+                    release.Files = ParseUtil.CoerceInt(files);
+
+                    var grabs = qRow.Find("td:has(a[href*=\"&tosnatchers=1\"])> b:nth-child(1)").Text();
+                    release.Grabs = ParseUtil.CoerceInt(grabs);
+
+                    if (qRow.Find("img[src=\"pic/torrent_ou.gif\"]").Length >= 1)
+                        release.DownloadVolumeFactor = 0;
+                    else
+                        release.DownloadVolumeFactor = 1;
+
+                    release.UploadVolumeFactor = 1;
+
+                    releases.Add(release);
+                }
+            }
+            catch (Exception ex)
+            {
+                OnParseError(results, ex);
+            }
+
+            return releases;
+        }
+    }
+}
+

src/Jackett/Indexers/BitHdtv.cs

@@ -23,7 +23,7 @@ namespace Jackett.Indexers
     {
         private string LoginUrl { get { return SiteLink + "takelogin.php"; } }
         private string SearchUrl { get { return SiteLink + "torrents.php?"; } }
-        private string DownloadUrl { get { return SiteLink + "download.php?/{0}/dl.torrent"; } }
+        private string DownloadUrl { get { return SiteLink + "download.php?id={0}"; } }
 
         new ConfigurationDataBasicLogin configData
         {

src/Jackett/Indexers/BitMeTV.cs

@@ -129,6 +129,15 @@ namespace Jackett.Indexers
                     //if (!release.Title.ToLower().Contains(title.ToLower()))
                     //    continue;
 
+                    var files = row.Cq().Find("td:nth-child(4)").Text();
+                    release.Files = ParseUtil.CoerceInt(files);
+
+                    var grabs = row.Cq().Find("td:nth-child(8)").Get(0).FirstChild.ToString();
+                    release.Grabs = ParseUtil.CoerceInt(grabs);
+
+                    release.DownloadVolumeFactor = 1;
+                    release.UploadVolumeFactor = 1;
+
                     releases.Add(release);
                 }
             }

src/Jackett/Indexers/BlueTigers.cs

@@ -1,257 +0,0 @@
-using CsQuery;
-using Jackett.Models;
-using Jackett.Services;
-using Jackett.Utils;
-using Jackett.Utils.Clients;
-using Newtonsoft.Json.Linq;
-using NLog;
-using System;
-using System.Collections.Generic;
-using System.Collections.Specialized;
-using System.Globalization;
-using System.Linq;
-using System.Text.RegularExpressions;
-using System.Threading.Tasks;
-using Jackett.Models.IndexerConfig.Bespoke;
-
-namespace Jackett.Indexers
-{
-    public class BlueTigers : BaseIndexer, IIndexer
-    {
-        private string LoginUrl => SiteLink + "account-login.php";
-        private string TorrentSearchUrl => SiteLink + "torrents-search.php";
-        private string IndexUrl => SiteLink + "index.php";
-
-        private ConfigurationDataBlueTigers ConfigData
-        {
-            get { return (ConfigurationDataBlueTigers)configData; }
-            set { base.configData = value; }
-        }
-
-        public BlueTigers(IIndexerManagerService i, IWebClient wc, Logger l, IProtectionService ps)
-            : base(name: "BlueTigers",
-                description: "BlueTigers - No Ratio - Private",
-                link: "https://www.bluetigers.ca/",
-                caps: new TorznabCapabilities(),
-                manager: i,
-                client: wc,
-                logger: l,
-                p: ps,
-                configData: new ConfigurationDataBlueTigers(@"BlueTigers can search for one or all languages. 
-                                                            If you select 2 languages below, results will contain all 3 languages.
-                                                            <br> For best results change the torrents per page setting to 50 in your BlueTigers profile."))
-        {
-            AddCategoryMapping("14", TorznabCatType.ConsolePSP);
-            AddCategoryMapping("150", TorznabCatType.ConsoleWii);
-            AddCategoryMapping("150", TorznabCatType.ConsoleWiiwareVC);
-            AddCategoryMapping("150", TorznabCatType.ConsoleWiiU);
-            AddCategoryMapping("52", TorznabCatType.ConsoleXbox);
-            AddCategoryMapping("52", TorznabCatType.ConsoleXbox360);
-            AddCategoryMapping("52", TorznabCatType.ConsoleXBOX360DLC);
-            AddCategoryMapping("52", TorznabCatType.ConsoleXboxOne);
-            AddCategoryMapping("11", TorznabCatType.PCGames);
-            AddCategoryMapping("13", TorznabCatType.ConsolePS4);
-            AddCategoryMapping("13", TorznabCatType.ConsolePS3);
-            AddCategoryMapping("13", TorznabCatType.ConsolePSVita);
-            AddCategoryMapping("12", TorznabCatType.Console3DS);
-            AddCategoryMapping("160", TorznabCatType.PCPhoneIOS);
-            AddCategoryMapping("160", TorznabCatType.PCPhoneAndroid);
-            AddCategoryMapping("1", TorznabCatType.PCPhoneAndroid);
-            AddCategoryMapping("29", TorznabCatType.PCMac);
-            AddCategoryMapping("27", TorznabCatType.PC);
-            AddCategoryMapping("41", TorznabCatType.PC);
-            AddCategoryMapping("50", TorznabCatType.PC);
-            AddCategoryMapping("333", TorznabCatType.BooksMagazines);
-            AddCategoryMapping("38", TorznabCatType.TVDocumentary);
-            AddCategoryMapping("37", TorznabCatType.BooksEbook);
-            AddCategoryMapping("61", TorznabCatType.Movies3D);
-            AddCategoryMapping("45", TorznabCatType.XXX);
-            AddCategoryMapping("59", TorznabCatType.MoviesHD);
-            AddCategoryMapping("222", TorznabCatType.MoviesHD);
-            AddCategoryMapping("22", TorznabCatType.MoviesHD);
-            AddCategoryMapping("60", TorznabCatType.MoviesHD);
-            AddCategoryMapping("56", TorznabCatType.MoviesBluRay);
-            AddCategoryMapping("23", TorznabCatType.MoviesOther);
-            AddCategoryMapping("15", TorznabCatType.MoviesOther);
-            AddCategoryMapping("43", TorznabCatType.MoviesDVD);
-            AddCategoryMapping("24", TorznabCatType.MoviesDVD);
-            AddCategoryMapping("25", TorznabCatType.MoviesOther);
-            AddCategoryMapping("21", TorznabCatType.MoviesOther);
-            AddCategoryMapping("20", TorznabCatType.MoviesDVD);
-            AddCategoryMapping("9", TorznabCatType.TVAnime);
-            AddCategoryMapping("34", TorznabCatType.Other);
-            AddCategoryMapping("35", TorznabCatType.Audio);
-            AddCategoryMapping("36", TorznabCatType.AudioVideo);
-            AddCategoryMapping("31", TorznabCatType.AudioVideo);
-            AddCategoryMapping("2", TorznabCatType.TVOTHER);
-            AddCategoryMapping("16", TorznabCatType.TVHD);
-            AddCategoryMapping("130", TorznabCatType.TVHD);
-            AddCategoryMapping("10", TorznabCatType.TVSD);
-            AddCategoryMapping("131", TorznabCatType.TV);
-            AddCategoryMapping("17", TorznabCatType.TV);
-            AddCategoryMapping("18", TorznabCatType.TV);
-            AddCategoryMapping("19", TorznabCatType.TV);
-            AddCategoryMapping("58", TorznabCatType.TVSport);
-            AddCategoryMapping("33", TorznabCatType.TVOTHER);
-            AddCategoryMapping("34", TorznabCatType.Other);
-        }
-
-        public async Task<IndexerConfigurationStatus> ApplyConfiguration(JToken configJson)
-        {
-            ConfigData.LoadValuesFromJson(configJson);
-
-            if (ConfigData.French.Value == false && ConfigData.English.Value == false && ConfigData.Spanish.Value == false)
-                throw new ExceptionWithConfigData("Please select at least one language.", ConfigData);
-
-            await RequestStringWithCookies(LoginUrl, string.Empty);
-            var pairs = new Dictionary<string, string> {
-                { "username", ConfigData.Username.Value },
-                { "password", ConfigData.Password.Value },
-                { "take_login", "1" }
-            };
-
-            var result = await RequestLoginAndFollowRedirect(LoginUrl, pairs, null, true, IndexUrl, SiteLink);
-            Regex rgx = new Regex(@"uid=[0-9]{1,10}; pass=[a-z0-9]{1,40};");
-            await ConfigureIfOK(result.Cookies, rgx.IsMatch(result.Cookies), () =>
-            {
-                var errorMessage = "Error while trying to login.";
-                throw new ExceptionWithConfigData(errorMessage, ConfigData);
-            });
-
-            return IndexerConfigurationStatus.RequiresTesting;
-        }
-
-        public async Task<IEnumerable<ReleaseInfo>> PerformQuery(TorznabQuery query)
-        {
-            List<ReleaseInfo> releases = new List<ReleaseInfo>();
-
-            NameValueCollection qParams = new NameValueCollection();
-            if (ConfigData.French.Value && !ConfigData.English.Value && !ConfigData.Spanish.Value)
-            {
-                qParams.Add("lang", "1");
-            }
-            else
-            {
-                if (!ConfigData.French.Value && ConfigData.English.Value && !ConfigData.Spanish.Value)
-                {
-                    qParams.Add("lang", "2");
-                }
-                else
-                {
-                    if (!ConfigData.French.Value && !ConfigData.English.Value && ConfigData.Spanish.Value)
-                    {
-                        qParams.Add("lang", "3");
-                    }
-                    else
-                    {
-                        qParams.Add("lang", "0");
-                    }
-                }
-                     
-            } 
-                
-            List<string> catList = MapTorznabCapsToTrackers(query);
-            foreach (string cat in catList)
-            {
-                qParams.Add("c" + cat, "1");
-            }
-
-            if (!string.IsNullOrEmpty(query.SanitizedSearchTerm))
-            {
-                qParams.Add("search", query.GetQueryString());
-            }
-
-            string queryStr = qParams.GetQueryString();
-            string searchUrl = $"{TorrentSearchUrl}?incldead=0&freeleech=0&sort=id&order=ascdesc&{queryStr}";
-
-            List<CQ> torrentRowList = new List<CQ>();
-
-            var results = await RequestStringWithCookiesAndRetry(searchUrl);
-            try
-            {
-                CQ fDom = results.Content;
-                var firstPageRows = fDom["table[class='ttable_headinner'] > tbody > tr:not(:First-child)"];
-                torrentRowList.AddRange(firstPageRows.Select(fRow => fRow.Cq()));
-
-                //If a search term is used, follow upto the first 4 pages (initial plus 3 more)
-                if (!string.IsNullOrWhiteSpace(query.GetQueryString()) && fDom["a[class='boutons']"].Filter("a[href*=&page=]").Length > 0)
-                {
-                    int pageLinkCount;
-                    int.TryParse(fDom["a[class='boutons']"].Filter("a[href*=&page=]").Last().Attr("href").Split(new[] { "&page=" }, StringSplitOptions.None).LastOrDefault(), out pageLinkCount);
-                    for (int i = 1; i < Math.Min(4, pageLinkCount + 1); i++)
-                    {
-                        var sResults = await RequestStringWithCookiesAndRetry($"{searchUrl}&page={i}");
-                        CQ sDom = sResults.Content;
-                        var additionalPageRows = sDom["table[class='ttable_headinner'] > tbody > tr:not(:First-child)"];
-                        torrentRowList.AddRange(additionalPageRows.Select(sRow => sRow.Cq()));
-                    }
-                }
-
-                foreach (CQ tRow in torrentRowList)
-                {
-                    long torrentId = 0;
-                    string idTarget = "bookmarks.php?torrent=";
-                    string id = tRow.Find("a[href*=" + idTarget + "]").First().Attr("href").Trim();
-                    if (!string.IsNullOrEmpty(id) && id.Contains(idTarget))
-                    {
-                        long.TryParse(id.Substring(id.LastIndexOf(idTarget, StringComparison.Ordinal) + idTarget.Length), out torrentId);
-                    }
-
-                    if (torrentId <= 0) continue;
-
-                    long category = 0;
-                    string catTarget = "torrents.php?cat=";
-                    string cat = tRow.Find("a[href*=" + catTarget + "]").First().Attr("href").Trim();
-                    if (!string.IsNullOrEmpty(cat) && cat.Contains(catTarget))
-                    {
-                        long.TryParse(cat.Substring(cat.LastIndexOf(catTarget, StringComparison.Ordinal) + catTarget.Length), out category);
-                    }
-
-                    Uri guid = new Uri($"{SiteLink}torrents-details.php?hit=1&id={torrentId}");
-                    Uri link = new Uri($"{SiteLink}download.php?hit=1&id={torrentId}");
-                    Uri comments = new Uri($"{SiteLink}comments.php?type=torrent&id={torrentId}");
-                    string title = tRow.Find("a[href*=torrents-details.php?id=]").First().Text().Trim();
-                    string stats = tRow.Find("div[id=kt" + torrentId.ToString() + "]").First().Text();
-                    string sizeStr = new Regex("Taille:(.*)Vitesse:").Match(stats).Groups[1].ToString().Trim();
-                    string pubDateStr = new Regex("Ajout.:(.*)Compl.t.s").Match(stats).Groups[1].ToString().Trim();
-                    DateTime pubDate = DateTime.ParseExact(pubDateStr, "yyyy-MM-dd HH:mm:ss", CultureInfo.InvariantCulture, DateTimeStyles.AssumeUniversal).ToLocalTime();
-
-                    string statistics = tRow.Find("a[href*=torrents-details.php?id=]").First().RenderSelection().Trim();
-                    string startTag = "<table ";
-                    string endTag = "</table>";
-                    CQ statsCq = startTag + new Regex(startTag + "(.*)" + endTag).Match(statistics).Groups[1].ToString().Trim() + endTag;
-                    int seeders;
-                    int leechers;
-                    int.TryParse(statsCq.Find("font[color=#05FC09]").First().Text(), out seeders);
-                    int.TryParse(statsCq.Find("font[color=red]").First().Text(), out leechers);
-
-                    var release = new ReleaseInfo();
-
-                    release.Title = title;
-                    release.Guid = guid;
-                    release.Link = link;
-                    release.PublishDate = pubDate;
-                    release.Size = ReleaseInfo.GetBytes(sizeStr);
-                    release.Description = title;
-                    release.Seeders = seeders;
-                    release.Peers = leechers + seeders;
-                    release.MinimumRatio = 1;
-                    release.MinimumSeedTime = 172800;
-                    release.Category = MapTrackerCatToNewznab(category.ToString());
-                    release.Comments = comments;
-
-                    releases.Add(release);
-                }
-
-            }
-            catch (Exception ex)
-            {
-                OnParseError(results.Content, ex);
-            }
-
-            return releases;
-        }
-
-    }
-
-}

src/Jackett/Indexers/CardigannIndexer.cs

@@ -0,0 +1,810 @@
+using Jackett.Utils.Clients;
+using NLog;
+using Jackett.Services;
+using Jackett.Utils;
+using Jackett.Models;
+using System.Threading.Tasks;
+using Newtonsoft.Json.Linq;
+using System.Collections.Generic;
+using System;
+using Jackett.Models.IndexerConfig;
+using System.Collections.Specialized;
+using System.Text;
+using YamlDotNet.Serialization;
+using YamlDotNet.Serialization.NamingConventions;
+using static Jackett.Models.IndexerConfig.ConfigurationData;
+using AngleSharp.Parser.Html;
+using System.Text.RegularExpressions;
+using System.Web;
+using AngleSharp.Dom;
+
+namespace Jackett.Indexers
+{
+    public class CardigannIndexer : BaseIndexer, IIndexer
+    {
+        protected IndexerDefinition Definition;
+        public new string ID { get { return (Definition != null ? Definition.Site : GetIndexerID(GetType())); } }
+
+        new ConfigurationData configData
+        {
+            get { return (ConfigurationData)base.configData; }
+            set { base.configData = value; }
+        }
+
+        // Cardigann yaml classes
+        public class IndexerDefinition {
+            public string Site { get; set; }
+            public List<settingsField> Settings { get; set; }
+            public string Name { get; set; }
+            public string Description { get; set; }
+            public string Language { get; set; }
+            public List<string> Links { get; set; }
+            public capabilitiesBlock Caps { get; set; }
+            public loginBlock Login { get; set; }
+            public ratioBlock Ratio { get; set; }
+            public searchBlock Search { get; set; }
+            // IndexerDefinitionStats not needed/implemented
+        }
+        public class settingsField
+        {
+            public string Name { get; set; }
+            public string Type { get; set; }
+            public string Label { get; set; }
+        }
+
+        public class capabilitiesBlock
+        {
+            public Dictionary<string, string> Categories  { get; set; }
+            public Dictionary<string, List<string>> Modes { get; set; }
+        }
+
+        public class loginBlock
+        {
+            public string Path { get; set; }
+            public string Method { get; set; }
+            public string Form { get; set; }
+            public Dictionary<string, string> Inputs { get; set; }
+            public List<errorBlock> Error { get; set; }
+            public pageTestBlock Test { get; set; }
+        }
+
+        public class errorBlock
+        {
+            public string Path { get; set; }
+            public string Selector { get; set; }
+            public selectorBlock Message { get; set; }
+        }
+
+        public class selectorBlock
+        {
+            public string Selector { get; set; }
+            public string Text { get; set; }
+            public string Attribute { get; set; }
+            public string Remove { get; set; }
+            public List<filterBlock> Filters { get; set; }
+            public Dictionary<string, string> Case { get; set; }
+        }
+
+        public class filterBlock
+        {
+            public string Name { get; set; }
+            public dynamic Args { get; set; }
+        }
+
+        public class pageTestBlock
+        {
+            public string Path { get; set; }
+            public string Selector { get; set; }
+        }
+
+        public class ratioBlock : selectorBlock
+        {
+            public string Path { get; set; }
+        }
+
+        public class searchBlock
+        {
+            public string Path { get; set; }
+            public Dictionary<string, string> Inputs { get; set; }
+            public rowsBlock Rows { get; set; }
+            public Dictionary<string, selectorBlock> Fields { get; set; }
+        }
+
+        public class rowsBlock : selectorBlock
+        {
+            public int After { get; set; }
+            //public string Remove { get; set; } // already inherited
+            public selectorBlock Dateheaders { get; set; }
+        }
+
+        public CardigannIndexer(IIndexerManagerService i, IWebClient wc, Logger l, IProtectionService ps)
+            : base(manager: i,
+                   client: wc,
+                   logger: l,
+                   p: ps)
+        {
+        }
+
+        public CardigannIndexer(IIndexerManagerService i, IWebClient wc, Logger l, IProtectionService ps, string DefinitionString)
+            : base(manager: i,
+                   client: wc,
+                   logger: l,
+                   p: ps)
+        {
+            Init(DefinitionString);
+        }
+
+        protected void Init(string DefinitionString)
+        {
+            var deserializer = new DeserializerBuilder()
+                .WithNamingConvention(new CamelCaseNamingConvention())
+                .IgnoreUnmatchedProperties()
+                .Build();
+            Definition = deserializer.Deserialize<IndexerDefinition>(DefinitionString);
+
+            // Add default data if necessary
+            if (Definition.Settings == null)
+                Definition.Settings = new List<settingsField>();
+
+            if (Definition.Settings.Count == 0)
+            {
+                Definition.Settings.Add(new settingsField { Name = "username", Label = "Username", Type = "text" });
+                Definition.Settings.Add(new settingsField { Name = "password", Label = "Password", Type = "password" });
+            }
+
+            // init missing mandatory attributes
+            DisplayName = Definition.Name;
+            DisplayDescription = Definition.Description;
+            SiteLink = Definition.Links[0]; // TODO: implement alternative links
+            if (!SiteLink.EndsWith("/"))
+                SiteLink += "/";
+            TorznabCaps = TorznabUtil.CreateDefaultTorznabTVCaps(); // TODO implement caps
+
+            // init config Data
+            configData = new ConfigurationData();
+            foreach (var Setting in Definition.Settings)
+            {
+                configData.AddDynamic(Setting.Name, new StringItem { Name = Setting.Label });
+            }
+
+            foreach (var Category in Definition.Caps.Categories)
+            {
+                var cat = TorznabCatType.GetCatByName(Category.Value);
+                if (cat == null)
+                {
+                    logger.Error(string.Format("CardigannIndexer ({0}): Can't find a category for {1}", ID, Category.Value));
+                    continue;
+                }
+                AddCategoryMapping(Category.Key, TorznabCatType.GetCatByName(Category.Value));
+                
+            }
+        }
+
+        protected Dictionary<string, object> getTemplateVariablesFromConfigData()
+        {
+            Dictionary<string, object> variables = new Dictionary<string, object>();
+
+            foreach (settingsField Setting in Definition.Settings)
+            {
+                variables[".Config."+Setting.Name] = ((StringItem)configData.GetDynamic(Setting.Name)).Value;
+            }
+            return variables;
+        }
+
+        // A very bad implementation of the golang template/text templating engine.
+        // But it should work for most basic constucts used by Cardigann definitions.
+        protected string applyGoTemplateText(string template, Dictionary<string, object> variables = null)
+        {
+            if (variables == null)
+            {
+                variables = getTemplateVariablesFromConfigData();
+            }
+
+            // handle if ... else ... expression
+            Regex IfElseRegex = new Regex(@"{{if\s*(.+?)\s*}}(.*?){{\s*else\s*}}(.*?){{\s*end\s*}}");
+            var IfElseRegexMatches = IfElseRegex.Match(template);
+
+            while (IfElseRegexMatches.Success)
+            {
+                string conditionResult = null;
+
+                string all = IfElseRegexMatches.Groups[0].Value;
+                string condition = IfElseRegexMatches.Groups[1].Value;
+                string onTrue = IfElseRegexMatches.Groups[2].Value;
+                string onFalse = IfElseRegexMatches.Groups[3].Value;
+
+                if (condition.StartsWith("."))
+                {
+                    string value = (string)variables[condition];
+                    if (!string.IsNullOrWhiteSpace(value))
+                    {
+                        conditionResult = onTrue;
+                    }
+                    else
+                    {
+                        conditionResult = onFalse;
+                    }
+                }
+                else
+                {
+                    throw new NotImplementedException("CardigannIndexer: Condition operation '" + condition + "' not implemented");
+                }
+                template = template.Replace(all, conditionResult);
+                IfElseRegexMatches = IfElseRegexMatches.NextMatch();
+            }
+
+            // handle range expression
+            Regex RangeRegex = new Regex(@"{{\s*range\s*(.+?)\s*}}(.*?){{\.}}(.*?){{end}}");
+            var RangeRegexMatches = RangeRegex.Match(template);
+
+            while (RangeRegexMatches.Success)
+            {
+                string expanded = string.Empty;
+
+                string all = RangeRegexMatches.Groups[0].Value;
+                string variable = RangeRegexMatches.Groups[1].Value;
+                string prefix = RangeRegexMatches.Groups[2].Value;
+                string postfix = RangeRegexMatches.Groups[3].Value;
+
+                foreach (string value in (List<string>)variables[variable])
+                {
+                    expanded += prefix + value + postfix;
+                }
+                template = template.Replace(all, expanded);
+                RangeRegexMatches = RangeRegexMatches.NextMatch();
+            }
+
+            // handle simple variables
+            Regex VariablesRegEx = new Regex(@"{{\s*(\..+?)\s*}}");
+            var VariablesRegExMatches = VariablesRegEx.Match(template);
+
+            while (VariablesRegExMatches.Success)
+            {
+                string expanded = string.Empty;
+
+                string all = VariablesRegExMatches.Groups[0].Value;
+                string variable = VariablesRegExMatches.Groups[1].Value;
+
+                string value = (string)variables[variable];
+                template = template.Replace(all, value);
+                VariablesRegExMatches = VariablesRegExMatches.NextMatch();
+            }
+
+            return template;
+        }
+
+        protected bool checkForLoginError(WebClientStringResult loginResult)
+        {
+            var ErrorBlocks = Definition.Login.Error;
+
+            if (ErrorBlocks == null)
+                return true; // no error
+
+                var loginResultParser = new HtmlParser();
+            var loginResultDocument = loginResultParser.Parse(loginResult.Content);
+            foreach (errorBlock error in ErrorBlocks)
+            {
+                var selection = loginResultDocument.QuerySelector(error.Selector);
+                if (selection != null)
+                {
+                    string errorMessage = selection.TextContent;
+                    if (error.Message != null)
+                    {
+                        var errorSubMessage = loginResultDocument.QuerySelector(error.Message.Selector);
+                        errorMessage = errorSubMessage.TextContent;
+                    }
+                    throw new ExceptionWithConfigData(string.Format("Login failed: {0}", errorMessage.Trim()), configData);
+                }
+            }
+            return true; // no error
+        }
+
+        protected async Task<bool> DoLogin()
+        {
+            var Login = Definition.Login;
+
+            if (Login == null)
+                return false;
+
+            if (Login.Method == null || Login.Method == "post")
+            {
+                var pairs = new Dictionary<string, string>();
+                foreach (var Input in Definition.Login.Inputs)
+                {
+                    var value = applyGoTemplateText(Input.Value);
+                    pairs.Add(Input.Key, value);
+                }
+
+                var LoginUrl = SiteLink + Login.Path;
+                configData.CookieHeader.Value = null;
+                var loginResult = await RequestLoginAndFollowRedirect(LoginUrl, pairs, null, true, null, SiteLink, true);
+                configData.CookieHeader.Value = loginResult.Cookies;
+
+                checkForLoginError(loginResult);
+            }
+            else if (Login.Method == "form")
+            {
+                var LoginUrl = SiteLink + Login.Path;
+
+                var FormSelector = Login.Form;
+                if (FormSelector == null)
+                    FormSelector = "form";
+
+                var pairs = new Dictionary<string, string>();
+
+                configData.CookieHeader.Value = null;
+                var landingResult = await RequestStringWithCookies(LoginUrl, null, SiteLink);
+
+                var htmlParser = new HtmlParser();
+                var landingResultDocument = htmlParser.Parse(landingResult.Content);
+
+                var form = landingResultDocument.QuerySelector(FormSelector);
+                if (form == null)
+                {
+                    throw new ExceptionWithConfigData(string.Format("Login failed: No form found on {0} using form selector {1}", LoginUrl, FormSelector), configData);
+                }
+
+                var inputs = form.QuerySelectorAll("input");
+                if (inputs == null)
+                {
+                    throw new ExceptionWithConfigData(string.Format("Login failed: No inputs found on {0} using form selector {1}", LoginUrl, FormSelector), configData);
+                }
+
+                var submitUrl = resolvePath(form.GetAttribute("action"));
+                
+                foreach (var input in inputs)
+                {
+                    var name = input.GetAttribute("name");
+                    if (name == null)
+                        continue;
+
+                    var value = input.GetAttribute("value");
+                    if (value == null)
+                        value = "";
+
+                    pairs[name] = value;
+                }
+   
+                foreach (var Input in Definition.Login.Inputs)
+                {
+                    var value = applyGoTemplateText(Input.Value);
+                    pairs[Input.Key] = value;
+                }
+
+                // automatically solve simpleCaptchas, if used
+                var simpleCaptchaPresent = landingResultDocument.QuerySelector("script[src*=\"simpleCaptcha\"]");
+                if(simpleCaptchaPresent != null)
+                {
+                    var captchaUrl = resolvePath("simpleCaptcha.php?numImages=1");
+                    var simpleCaptchaResult = await RequestStringWithCookies(captchaUrl.ToString(), landingResult.Cookies, LoginUrl);
+                    var simpleCaptchaJSON = JObject.Parse(simpleCaptchaResult.Content);
+                    var captchaSelection = simpleCaptchaJSON["images"][0]["hash"].ToString();
+                    pairs["captchaSelection"] = captchaSelection;
+                    pairs["submitme"] = "X";
+                }
+
+                var loginResult = await RequestLoginAndFollowRedirect(submitUrl.ToString(), pairs, landingResult.Cookies, true, null, SiteLink, true);
+                configData.CookieHeader.Value = loginResult.Cookies;
+
+                checkForLoginError(loginResult);
+            }
+            else if (Login.Method == "cookie")
+            {
+                configData.CookieHeader.Value = ((StringItem)configData.GetDynamic("cookie")).Value;
+            }
+            else
+            {
+                throw new NotImplementedException("Login method " + Definition.Login.Method + " not implemented");
+            }
+            return true;
+        }
+
+        protected async Task<bool> TestLogin()
+        {
+            var Login = Definition.Login;
+
+            if (Login == null || Login.Test == null)
+                return false;
+
+            // test if login was successful
+            var LoginTestUrl = SiteLink + Login.Test.Path;
+            var testResult = await RequestStringWithCookies(LoginTestUrl);
+
+            if (testResult.IsRedirect)
+            {
+                throw new ExceptionWithConfigData("Login Failed, got redirected", configData);
+            }
+
+            if (Login.Test.Selector != null)
+            {
+                var testResultParser = new HtmlParser();
+                var testResultDocument = testResultParser.Parse(testResult.Content);
+                var selection = testResultDocument.QuerySelectorAll(Login.Test.Selector);
+                if (selection.Length == 0)
+                {
+                    throw new ExceptionWithConfigData(string.Format("Login failed: Selector \"{0}\" didn't match", Login.Test.Selector), configData);
+                }
+            }
+            return true;
+        }
+
+        public async Task<IndexerConfigurationStatus> ApplyConfiguration(JToken configJson)
+        {
+            configData.LoadValuesFromJson(configJson);
+
+            await DoLogin();
+            await TestLogin();
+
+            SaveConfig();
+            IsConfigured = true;
+            return IndexerConfigurationStatus.Completed;
+        }
+
+        protected string applyFilters(string Data, List<filterBlock> Filters)
+        {
+            if (Filters == null)
+                return Data;
+
+            foreach(filterBlock Filter in Filters)
+            {
+                switch (Filter.Name)
+                {
+                    case "querystring":
+                        var param = (string)Filter.Args;
+                        var qsStr = Data.Split(new char[] { '?' }, 2)[1];
+                        qsStr = qsStr.Split(new char[] { '#' }, 2)[0];
+                        var qs = HttpUtility.ParseQueryString(qsStr);
+                        Data = qs.Get(param);
+                        break;
+                    case "timeparse":
+                    case "dateparse":
+                        var layout = (string)Filter.Args;
+                        try
+                        {
+                            var Date = DateTimeUtil.ParseDateTimeGoLang(Data, layout);
+                            Data = Date.ToString(DateTimeUtil.RFC1123ZPattern);
+                        }
+                        catch (Exception ex)
+                        {
+                            logger.Debug(ex.ToString());
+                        }
+                    break;
+                    case "regexp":
+                        var pattern = (string)Filter.Args;
+                        var Regexp = new Regex(pattern);
+                        var Match = Regexp.Match(Data);
+                        Data = Match.Groups[1].Value;
+                        break;
+                    case "split":
+                        var sep = (string)Filter.Args[0];
+                        var pos = (string)Filter.Args[1];
+                        var posInt = int.Parse(pos);
+                        var strParts = Data.Split(sep[0]);
+                        if (posInt < 0)
+                        {
+                            posInt += strParts.Length;
+                        }
+                        Data = strParts[posInt];
+                        break;
+                    case "replace":
+                        var from = (string)Filter.Args[0];
+                        var to = (string)Filter.Args[1];
+                        Data = Data.Replace(from, to);
+                        break;
+                    case "trim":
+                        var cutset = (string)Filter.Args;
+                        Data = Data.Trim(cutset[0]);
+                        break;
+                    case "append":
+                        var str = (string)Filter.Args;
+                        Data += str;
+                        break;
+                    case "timeago":
+                    case "fuzzytime":
+                    case "reltime":
+                        var timestr = (string)Filter.Args;
+                        Data = DateTimeUtil.FromUnknown(timestr).ToString(DateTimeUtil.RFC1123ZPattern);
+                        break;
+                    default:
+                        break;
+                }
+            }
+            return Data;
+        }
+
+        protected string handleSelector(selectorBlock Selector, IElement Dom)
+        {
+            if (Selector.Text != null)
+            {
+                return applyFilters(Selector.Text, Selector.Filters);
+            }
+
+            IElement selection = Dom;
+            string value = null;
+
+            if (Selector.Selector != null)
+            {
+                selection = Dom.QuerySelector(Selector.Selector);
+                if (selection == null)
+                {
+                    throw new Exception(string.Format("Selector \"{0}\" didn't match {1}", Selector.Selector, Dom.OuterHtml));
+                }
+            }
+
+            if (Selector.Remove != null)
+            {
+                foreach(var i in selection.QuerySelectorAll(Selector.Remove))
+                {
+                    i.Remove();
+                }
+            }
+
+            if (Selector.Case != null)
+            {
+                foreach(var Case in Selector.Case)
+                {
+                    if (selection.Matches(Case.Key) || selection.QuerySelector(Case.Key) != null)
+                    {
+                        value = Case.Value;
+                        break;
+                    }
+                }
+                if(value == null)
+                    throw new Exception(string.Format("None of the case selectors \"{0}\" matched {1}", string.Join(",", Selector.Case), selection.OuterHtml));
+            }
+            else if (Selector.Attribute != null)
+            {
+                value = selection.GetAttribute(Selector.Attribute);
+            }
+            else
+            {
+                value = selection.TextContent;
+            }
+                
+            return applyFilters(value, Selector.Filters); ;
+        }
+
+        protected Uri resolvePath(string path)
+        {
+            if(path.StartsWith("http"))
+            {
+                return new Uri(path);
+            }
+            else
+            {
+                return new Uri(SiteLink + path);
+            }
+            
+        }
+
+        public async Task<IEnumerable<ReleaseInfo>> PerformQuery(TorznabQuery query)
+        {
+            var releases = new List<ReleaseInfo>();
+
+            searchBlock Search = Definition.Search;
+
+            // init template context
+            var variables = getTemplateVariablesFromConfigData();
+
+            variables[".Query.Type"] = query.QueryType;
+            variables[".Query.Q"] = query.SearchTerm;
+            variables[".Query.Series"] = null;
+            variables[".Query.Ep"] = query.Episode;
+            variables[".Query.Season"] = query.Season;
+            variables[".Query.Movie"] = null;
+            variables[".Query.Year"] = null;
+            variables[".Query.Limit"] = query.Limit;
+            variables[".Query.Offset"] = query.Offset;
+            variables[".Query.Extended"] = query.Extended;
+            variables[".Query.Categories"] = query.Categories;
+            variables[".Query.APIKey"] = query.ApiKey;
+            variables[".Query.TVDBID"] = null;
+            variables[".Query.TVRageID"] = query.RageID;
+            variables[".Query.IMDBID"] = query.ImdbID;
+            variables[".Query.TVMazeID"] = null;
+            variables[".Query.TraktID"] = null;
+
+            variables[".Query.Episode"] = query.GetEpisodeSearchString();
+            variables[".Categories"] = MapTorznabCapsToTrackers(query);
+
+            var KeywordTokens = new List<string>();
+            var KeywordTokenKeys = new List<string> { "Q", "Series", "Movie", "Year" };
+            foreach (var key in KeywordTokenKeys)
+            {
+                var Value = (string)variables[".Query." + key];
+                if (!string.IsNullOrWhiteSpace(Value))
+                    KeywordTokens.Add(Value);
+            }
+
+            if (!string.IsNullOrWhiteSpace((string)variables[".Query.Episode"]))
+                KeywordTokens.Add((string)variables[".Query.Episode"]);
+            variables[".Query.Keywords"] = string.Join(" ", KeywordTokens);
+            variables[".Keywords"] = variables[".Query.Keywords"];
+
+            // build search URL
+            var searchUrl = SiteLink + applyGoTemplateText(Search.Path, variables) + "?";
+            var queryCollection = new NameValueCollection();
+            if (Search.Inputs != null)
+            { 
+                foreach (var Input in Search.Inputs)
+                {
+                    var value = applyGoTemplateText(Input.Value, variables);
+                    if (Input.Key == "$raw")
+                        searchUrl += value;
+                    else
+                        queryCollection.Add(Input.Key, value);
+                }
+            }
+            searchUrl += "&" + queryCollection.GetQueryString();
+
+            // send HTTP request
+            var response = await RequestBytesWithCookies(searchUrl);
+            var results = Encoding.GetEncoding("iso-8859-1").GetString(response.Content);
+            try
+            {
+                var SearchResultParser = new HtmlParser();
+                var SearchResultDocument = SearchResultParser.Parse(results);
+                
+                var RowsDom = SearchResultDocument.QuerySelectorAll(Search.Rows.Selector);
+                List<IElement> Rows = new List<IElement>();
+                foreach (var RowDom in RowsDom)
+                {
+                    Rows.Add(RowDom);
+                }
+
+                // merge following rows for After selector
+                var After = Definition.Search.Rows.After;
+                if (After > 0)
+                {
+                    for (int i = 0; i < Rows.Count; i += 1)
+                    {
+                        var CurrentRow = Rows[i];
+                        for (int j = 0; j < After; j += 1)
+                        {
+                            var MergeRowIndex = i + j + 1;
+                            var MergeRow = Rows[MergeRowIndex];
+                            List<INode> MergeNodes = new List<INode>();
+                            foreach (var node in MergeRow.QuerySelectorAll("td"))
+                            {
+                                MergeNodes.Add(node);
+                            }
+                            CurrentRow.Append(MergeNodes.ToArray());
+                        }
+                        Rows.RemoveRange(i + 1, After);
+                    }
+                }
+
+                foreach (var Row in Rows)
+                {
+                    try
+                    {
+                        var release = new ReleaseInfo();
+                        release.MinimumRatio = 1;
+                        release.MinimumSeedTime = 48 * 60 * 60;
+
+                        // Parse fields
+                        foreach (var Field in Search.Fields)
+                        {
+                            string value = handleSelector(Field.Value, Row);
+                            value = ParseUtil.NormalizeSpace(value);
+                            try
+                            {
+                                switch (Field.Key)
+                                {
+                                    case "download":
+                                        if (value.StartsWith("magnet:"))
+                                        {
+                                            release.MagnetUri = new Uri(value);
+                                            release.Link = release.MagnetUri;
+                                        }
+                                        else
+                                        {
+                                            release.Link = resolvePath(value);
+                                        }
+                                        break;
+                                    case "details":
+                                        var url = resolvePath(value);
+                                        release.Guid = url;
+                                        if (release.Comments == null)
+                                            release.Comments = url;
+                                        break;
+                                    case "comments":
+                                        var CommentsUrl = resolvePath(value);
+                                        release.Comments = CommentsUrl;
+                                        if (release.Guid == null)
+                                            release.Guid = CommentsUrl;
+                                        break;
+                                    case "title":
+                                        release.Title = value;
+                                        break;
+                                    case "description":
+                                        release.Description = value;
+                                        break;
+                                    case "category":
+                                        release.Category = MapTrackerCatToNewznab(value);
+                                        break;
+                                    case "size":
+                                        release.Size = ReleaseInfo.GetBytes(value);
+                                        break;
+                                    case "leechers":
+                                        if (release.Peers == null)
+                                            release.Peers = ParseUtil.CoerceInt(value);
+                                        else
+                                            release.Peers += ParseUtil.CoerceInt(value);
+                                        break;
+                                    case "seeders":
+                                        release.Seeders = ParseUtil.CoerceInt(value);
+                                        if (release.Peers == null)
+                                            release.Peers = release.Seeders;
+                                        else
+                                            release.Peers += release.Seeders;
+                                        break;
+                                    case "date":
+                                        release.PublishDate = DateTimeUtil.FromUnknown(value);
+                                        break;
+                                    case "files":
+                                        release.Files = ParseUtil.CoerceLong(value);
+                                        break;
+                                    case "grabs":
+                                        release.Grabs = ParseUtil.CoerceLong(value);
+                                        break;
+                                    case "downloadvolumefactor":
+                                        release.DownloadVolumeFactor = ParseUtil.CoerceDouble(value);
+                                        break;
+                                    case "uploadvolumefactor":
+                                        release.UploadVolumeFactor = ParseUtil.CoerceDouble(value);
+                                        break;
+                                    default:
+                                        break;
+                                }
+                            }
+                            catch (Exception ex)
+                            {
+                                throw new Exception(string.Format("Error while parsing field={0}, selector={1}, value={2}: {3}", Field.Key, Field.Value.Selector, value, ex.Message));
+                            }
+                        }
+
+                        // if DateHeaders is set go through the previous rows and look for the header selector
+                        var DateHeaders = Definition.Search.Rows.Dateheaders;
+                        if (DateHeaders != null)
+                        {
+                            var PrevRow = Row.PreviousElementSibling;
+                            string value = null;
+                            while (PrevRow != null)
+                            {
+                                try
+                                {
+                                    value = handleSelector(DateHeaders, PrevRow);
+                                    break;
+                                }
+                                catch (Exception ex)
+                                {
+                                    // do nothing
+                                }
+                                PrevRow = PrevRow.PreviousElementSibling;
+                            }
+                            
+                            if (value == null)
+                                throw new Exception(string.Format("No date header row found for {0}", release.ToString()));
+
+                            release.PublishDate = DateTimeUtil.FromUnknown(value);
+                        }
+
+                        releases.Add(release);
+                    }
+                    catch (Exception ex)
+                    {
+                        logger.Error(string.Format("CardigannIndexer ({0}): Error while parsing row '{1}': {2}", ID, Row.OuterHtml, ex));
+                    }
+                }
+            }
+            catch (Exception ex)
+            {
+                OnParseError(results, ex);
+            }
+
+            return releases;
+        }
+    }
+}
+

src/Jackett/Indexers/CinemaZ.cs

@@ -18,12 +18,12 @@ using Jackett.Models.IndexerConfig;
 
 namespace Jackett.Indexers
 {
-    public class EuTorrents : AvistazTracker, IIndexer
+    public class CinemaZ : AvistazTracker, IIndexer
     {
-        public EuTorrents(IIndexerManagerService indexerManager, IWebClient webClient, Logger logger, IProtectionService protectionService)
-            : base(name: "EuTorrents",
+        public CinemaZ(IIndexerManagerService indexerManager, IWebClient webClient, Logger logger, IProtectionService protectionService)
+            : base(name: "CinemaZ",
                 desc: "Part of the Avistaz network.",
-                link: "https://eutorrents.to/",
+                link: "https://cinemaz.to/",
                 indexerManager: indexerManager,
                 logger: logger,
                 protectionService: protectionService,

src/Jackett/Indexers/Demonoid.cs

@@ -1,141 +1,147 @@
-using CsQuery;
-using Jackett.Models;
-using Jackett.Models.IndexerConfig;
-using Jackett.Services;
-using Jackett.Utils;
-using Jackett.Utils.Clients;
-using Newtonsoft.Json.Linq;
-using NLog;
-using System;
-using System.Collections.Generic;
-using System.Globalization;
-using System.Linq;
-using System.Text;
-using System.Threading.Tasks;
-using System.Web;
-
-namespace Jackett.Indexers
-{
-    public class Demonoid : BaseIndexer, IIndexer
-    {
-        private string LoginUrl { get { return SiteLink + "account_handler.php"; } }
-        private string SearchUrl { get { return SiteLink + "files/?category={0}&subcategory=All&quality=All&seeded=0&to=1&query={1}"; } }
-
-        new ConfigurationDataBasicLogin configData
-        {
-            get { return (ConfigurationDataBasicLogin)base.configData; }
-            set { base.configData = value; }
-        }
-
-        public Demonoid(IIndexerManagerService i, Logger l, IWebClient wc, IProtectionService ps)
-            : base(name: "Demonoid",
-                description: "Demonoid",
-                link: "http://www.demonoid.pw/",
-                caps: TorznabUtil.CreateDefaultTorznabTVCaps(),
-                manager: i,
-                client: wc,
-                logger: l,
-                p: ps,
-                configData: new ConfigurationDataBasicLogin())
-        {
-            AddCategoryMapping(3, TorznabCatType.TV);
+using CsQuery;
+using Jackett.Models;
+using Jackett.Models.IndexerConfig;
+using Jackett.Services;
+using Jackett.Utils;
+using Jackett.Utils.Clients;
+using Newtonsoft.Json.Linq;
+using NLog;
+using System;
+using System.Collections.Generic;
+using System.Globalization;
+using System.Linq;
+using System.Text;
+using System.Threading.Tasks;
+using System.Web;
+
+namespace Jackett.Indexers
+{
+    public class Demonoid : BaseIndexer, IIndexer
+    {
+        private string LoginUrl { get { return SiteLink + "account_handler.php"; } }
+        private string SearchUrl { get { return SiteLink + "files/?category={0}&subcategory=All&quality=All&seeded=0&to=1&query={1}"; } }
+
+        new ConfigurationDataBasicLogin configData
+        {
+            get { return (ConfigurationDataBasicLogin)base.configData; }
+            set { base.configData = value; }
+        }
+
+        public Demonoid(IIndexerManagerService i, Logger l, IWebClient wc, IProtectionService ps)
+            : base(name: "Demonoid",
+                description: "Demonoid",
+                link: "http://www.dnoid.me/",
+                caps: TorznabUtil.CreateDefaultTorznabTVCaps(),
+                manager: i,
+                client: wc,
+                logger: l,
+                p: ps,
+                configData: new ConfigurationDataBasicLogin())
+        {
+            AddCategoryMapping(3, TorznabCatType.TV);
             AddCategoryMapping(3, TorznabCatType.TVSD);
             AddCategoryMapping(3, TorznabCatType.TVHD);
-            AddCategoryMapping(1, TorznabCatType.Movies);
-        }
-
-        public async Task<IndexerConfigurationStatus> ApplyConfiguration(JToken configJson)
-        {
-            configData.LoadValuesFromJson(configJson);
-            var pairs = new Dictionary<string, string> {
-                { "nickname", configData.Username.Value },
-                { "password", configData.Password.Value },
-                { "returnpath", "/" },
-                { "withq", "0" },
-                { "Submit", "Submit" }
-            };
-
-            var result = await RequestLoginAndFollowRedirect(LoginUrl, pairs, null, true, null, SiteLink);
-            await ConfigureIfOK(result.Cookies, result.Content != null && result.Content.Contains("user_control_panel.php"), () =>
-            {
-                CQ dom = result.Content;
-                var errorMessage = dom[".red"].ElementAt(1).Cq().Text().Trim();
-                throw new ExceptionWithConfigData(errorMessage, configData);
-            });
-            return IndexerConfigurationStatus.RequiresTesting;
-        }
-
-        public async Task<IEnumerable<ReleaseInfo>> PerformQuery(TorznabQuery query)
-        {
-            var releases = new List<ReleaseInfo>();
-            var trackerCats = MapTorznabCapsToTrackers(query);
-            var cat = (trackerCats.Count == 1 ? trackerCats.ElementAt(0) : "0");
+            AddCategoryMapping(1, TorznabCatType.Movies);
+        }
+
+        public async Task<IndexerConfigurationStatus> ApplyConfiguration(JToken configJson)
+        {
+            configData.LoadValuesFromJson(configJson);
+            var pairs = new Dictionary<string, string> {
+                { "nickname", configData.Username.Value },
+                { "password", configData.Password.Value },
+                { "returnpath", "/" },
+                { "withq", "0" },
+                { "Submit", "Submit" }
+            };
+
+            var result = await RequestLoginAndFollowRedirect(LoginUrl, pairs, null, true, SiteLink, SiteLink);
+            await ConfigureIfOK(result.Cookies, result.Content != null && result.Cookies.Contains("uid="), () =>
+            {
+                CQ dom = result.Content;
+                string errorMessage = dom["form[id='bb_code_form']"].Parent().Find("font[class='red']").Text();
+                throw new ExceptionWithConfigData(errorMessage, configData);
+            });
+            return IndexerConfigurationStatus.RequiresTesting;
+        }
+
+        public async Task<IEnumerable<ReleaseInfo>> PerformQuery(TorznabQuery query)
+        {
+            var releases = new List<ReleaseInfo>();
+            var trackerCats = MapTorznabCapsToTrackers(query);
+            var cat = (trackerCats.Count == 1 ? trackerCats.ElementAt(0) : "0");
             var episodeSearchUrl = string.Format(SearchUrl, cat, HttpUtility.UrlEncode(query.GetQueryString()));
-            var results = await RequestStringWithCookiesAndRetry(episodeSearchUrl);
-
+            var results = await RequestStringWithCookiesAndRetry(episodeSearchUrl);
+
             
-            if (results.Content.Contains("No torrents found"))
-            {
-                return releases;
-            }
-
-            try
-            {
-                CQ dom = results.Content;
-                var rows = dom[".ctable_content_no_pad > table > tbody > tr"].ToArray();
-                DateTime lastDateTime = default(DateTime);
-                for (var i = 0; i < rows.Length; i++)
-                {
-                    var rowA = rows[i];
-                    var rAlign = rowA.Attributes["align"];
-                    if (rAlign == "right" || rAlign == "center")
-                        continue;
-                    if (rAlign == "left")
-                    {
-                        // ex: "Monday, Jun 01, 2015", "Monday, Aug 03, 2015"
-                        var dateStr = rowA.Cq().Text().Trim().Replace("Added on ", "");
-                        if (dateStr.ToLowerInvariant().Contains("today"))
-                            lastDateTime = DateTime.Now;
-                        else
-                            lastDateTime = DateTime.SpecifyKind(DateTime.ParseExact(dateStr, "dddd, MMM dd, yyyy", CultureInfo.InvariantCulture), DateTimeKind.Utc).ToLocalTime();
-                        continue;
-                    }
-                    if (rowA.ChildElements.Count() < 2)
-                        continue;
-
-                    var rowB = rows[++i];
-
-                    var release = new ReleaseInfo();
-                    release.MinimumRatio = 1;
-                    release.MinimumSeedTime = 172800;
-
-                    release.PublishDate = lastDateTime;
-
-                    var qLink = rowA.ChildElements.ElementAt(1).FirstElementChild.Cq();
-                    release.Title = qLink.Text().Trim();
-                    release.Description = release.Title;
-
-                    release.Comments = new Uri(SiteLink + qLink.Attr("href"));
-                    release.Guid = release.Comments;
-
-                    var qDownload = rowB.ChildElements.ElementAt(2).ChildElements.ElementAt(1).Cq();
-                    release.Link = new Uri(SiteLink + qDownload.Attr("href"));
-
-                    var sizeStr = rowB.ChildElements.ElementAt(3).Cq().Text();
-                    release.Size = ReleaseInfo.GetBytes(sizeStr);
-
-                    release.Seeders = ParseUtil.CoerceInt(rowB.ChildElements.ElementAt(6).Cq().Text());
-                    release.Peers = ParseUtil.CoerceInt(rowB.ChildElements.ElementAt(6).Cq().Text()) + release.Seeders;
-
-                    releases.Add(release);
-                }
-
-            }
-            catch (Exception ex)
-            {
-                OnParseError(results.Content, ex);
-            }
-            return releases;
-        }
-    }
-}
+            if (results.Content.Contains("No torrents found"))
+            {
+                return releases;
+            }
+
+            try
+            {
+                CQ dom = results.Content;
+                var rows = dom[".ctable_content_no_pad > table > tbody > tr"].ToArray();
+                DateTime lastDateTime = default(DateTime);
+                for (var i = 0; i < rows.Length; i++)
+                {
+                    var rowA = rows[i];
+                    var rAlign = rowA.Attributes["align"];
+                    if (rAlign == "right" || rAlign == "center")
+                        continue;
+                    if (rAlign == "left")
+                    {
+                        // ex: "Monday, Jun 01, 2015", "Monday, Aug 03, 2015"
+                        var dateStr = rowA.Cq().Text().Trim().Replace("Added on ", "");
+                        if (dateStr.ToLowerInvariant().Contains("today"))
+                            lastDateTime = DateTime.Now;
+                        else
+                            lastDateTime = DateTime.SpecifyKind(DateTime.ParseExact(dateStr, "dddd, MMM dd, yyyy", CultureInfo.InvariantCulture), DateTimeKind.Utc).ToLocalTime();
+                        continue;
+                    }
+                    if (rowA.ChildElements.Count() < 2)
+                        continue;
+
+                    var rowB = rows[++i];
+
+                    var release = new ReleaseInfo();
+                    release.MinimumRatio = 1;
+                    release.MinimumSeedTime = 172800;
+
+                    release.PublishDate = lastDateTime;
+
+                    var qLink = rowA.ChildElements.ElementAt(1).FirstElementChild.Cq();
+                    release.Title = qLink.Text().Trim();
+                    release.Description = release.Title;
+
+                    release.Comments = new Uri(SiteLink + qLink.Attr("href"));
+                    release.Guid = release.Comments;
+
+                    var qDownload = rowB.ChildElements.ElementAt(2).ChildElements.ElementAt(0).Cq();
+                    release.Link = new Uri(SiteLink + qDownload.Attr("href"));
+
+                    var sizeStr = rowB.ChildElements.ElementAt(3).Cq().Text();
+                    release.Size = ReleaseInfo.GetBytes(sizeStr);
+
+                    release.Seeders = ParseUtil.CoerceInt(rowB.ChildElements.ElementAt(6).Cq().Text());
+                    release.Peers = ParseUtil.CoerceInt(rowB.ChildElements.ElementAt(6).Cq().Text()) + release.Seeders;
+
+                    var grabs = rowB.Cq().Find("td:nth-child(6)").Text();
+                    release.Grabs = ParseUtil.CoerceInt(grabs);
+
+                    release.DownloadVolumeFactor = 0; // ratioless
+                    release.UploadVolumeFactor = 1;
+
+                    releases.Add(release);
+                }
+
+            }
+            catch (Exception ex)
+            {
+                OnParseError(results.Content, ex);
+            }
+            return releases;
+        }
+    }
+}

src/Jackett/Indexers/DigitalHive.cs

@@ -0,0 +1,219 @@
+using CsQuery;
+using Jackett.Models;
+using Jackett.Services;
+using Jackett.Utils;
+using Jackett.Utils.Clients;
+using Newtonsoft.Json.Linq;
+using NLog;
+using System;
+using System.Collections.Generic;
+using System.Linq;
+using System.Threading.Tasks;
+using Jackett.Models.IndexerConfig;
+using System.Collections.Specialized;
+using System.Text.RegularExpressions;
+
+namespace Jackett.Indexers
+{
+    public class DigitalHive : BaseIndexer, IIndexer
+    {
+        private string SearchUrl { get { return SiteLink + "browse.php"; } }
+        private string LoginUrl { get { return SiteLink + "login.php?returnto=%2F"; } }
+        private string AjaxLoginUrl { get { return SiteLink + "takelogin.php"; } }
+
+        new ConfigurationDataRecaptchaLogin configData
+        {
+            get { return (ConfigurationDataRecaptchaLogin)base.configData; }
+            set { base.configData = value; }
+        }
+
+        public DigitalHive(IIndexerManagerService i, Logger l, IWebClient w, IProtectionService ps)
+            : base(name: "DigitalHive",
+                description: "DigitalHive is one of the oldest general trackers",
+                link: "https://www.digitalhive.org/",
+                caps: new TorznabCapabilities(),
+                manager: i,
+                client: w,
+                logger: l,
+                p: ps,
+                configData: new ConfigurationDataRecaptchaLogin())
+        {
+            AddCategoryMapping(0, TorznabCatType.Other);
+            AddCategoryMapping(48, TorznabCatType.Other); // 0Day
+            AddCategoryMapping(56, TorznabCatType.XXXImageset); // 0Day-Imagesets
+            AddCategoryMapping(6, TorznabCatType.Audio); // 0Day-Music
+            AddCategoryMapping(51, TorznabCatType.XXX); // 0Day-XXX
+            AddCategoryMapping(2, TorznabCatType.TVAnime); // Anime
+            AddCategoryMapping(59, TorznabCatType.MoviesBluRay); // BluRay
+            AddCategoryMapping(40, TorznabCatType.TVDocumentary); // Documentary
+            AddCategoryMapping(20, TorznabCatType.MoviesDVD); // DVD-R
+            AddCategoryMapping(25, TorznabCatType.BooksEbook); // Ebooks
+            AddCategoryMapping(38, TorznabCatType.PCPhoneIOS); // HandHeld
+            AddCategoryMapping(38, TorznabCatType.PCPhoneAndroid); // HandHeld
+            AddCategoryMapping(38, TorznabCatType.PCPhoneOther); // HandHeld
+            AddCategoryMapping(37, TorznabCatType.Other); // Kids Stuff
+            AddCategoryMapping(23, TorznabCatType.PC); // Linux
+            AddCategoryMapping(24, TorznabCatType.PCMac); // Mac
+            AddCategoryMapping(22, TorznabCatType.OtherMisc); // Misc
+            AddCategoryMapping(35, TorznabCatType.MoviesOther); // Movie Pack
+            AddCategoryMapping(36, TorznabCatType.MoviesHD); // Movie-HD
+            AddCategoryMapping(19, TorznabCatType.MoviesSD); // Movie-SD
+            AddCategoryMapping(50, TorznabCatType.Audio); // Music
+            AddCategoryMapping(53, TorznabCatType.AudioLossless); // Music-FLAC
+            AddCategoryMapping(49, TorznabCatType.AudioVideo); // MVID
+            AddCategoryMapping(1, TorznabCatType.PC); // PC Apps
+            AddCategoryMapping(4, TorznabCatType.PCGames); // PC Games
+            AddCategoryMapping(17, TorznabCatType.ConsolePS3); // Playstation
+            AddCategoryMapping(17, TorznabCatType.ConsolePS4); // Playstation
+            AddCategoryMapping(17, TorznabCatType.ConsolePSVita); // Playstation
+            AddCategoryMapping(17, TorznabCatType.ConsolePSP); // Playstation
+            AddCategoryMapping(28, TorznabCatType.ConsolePSP); // PSP
+            AddCategoryMapping(34, TorznabCatType.TVOTHER); // TV Pack
+            AddCategoryMapping(32, TorznabCatType.TVHD); // TV-HD
+            AddCategoryMapping(55, TorznabCatType.TVOTHER); // TV-HDRip
+            AddCategoryMapping(7, TorznabCatType.TVSD); // TV-SD
+            AddCategoryMapping(57, TorznabCatType.TVOTHER); // TV-SDRip
+            AddCategoryMapping(33, TorznabCatType.ConsoleWii); // WII
+            AddCategoryMapping(33, TorznabCatType.ConsoleWiiU); // WII
+            AddCategoryMapping(45, TorznabCatType.ConsoleXbox); // XBox
+            AddCategoryMapping(45, TorznabCatType.ConsoleXbox360); // XBox
+            AddCategoryMapping(45, TorznabCatType.ConsoleXBOX360DLC); // XBox
+            AddCategoryMapping(45, TorznabCatType.ConsoleXboxOne); // XBox
+            AddCategoryMapping(9, TorznabCatType.XXX); // XXX
+            AddCategoryMapping(52, TorznabCatType.XXXOther); // XXX-ISO
+        }
+
+        public override async Task<ConfigurationData> GetConfigurationForSetup()
+        {
+            var loginPage = await RequestStringWithCookies(LoginUrl, configData.CookieHeader.Value);
+            CQ cq = loginPage.Content;
+            string recaptchaSiteKey = cq.Find(".g-recaptcha").Attr("data-sitekey");
+            var result = this.configData;
+            result.CookieHeader.Value = loginPage.Cookies;
+            result.Captcha.SiteKey = recaptchaSiteKey;
+            result.Captcha.Version = "2";
+            return result;
+        }
+
+        public async Task<IndexerConfigurationStatus> ApplyConfiguration(JToken configJson)
+        {
+            configData.LoadValuesFromJson(configJson);
+            var pairs = new Dictionary<string, string> {
+                { "returnto" , "/" },
+                { "username", configData.Username.Value },
+                { "password", configData.Password.Value },
+                { "g-recaptcha-response", configData.Captcha.Value }
+            };
+
+            if (!string.IsNullOrWhiteSpace(configData.Captcha.Cookie))
+            {
+                // Cookie was manually supplied
+                CookieHeader = configData.Captcha.Cookie;
+                try
+                {
+                    var results = await PerformQuery(new TorznabQuery());
+                    if (!results.Any())
+                    {
+                        throw new Exception("Your cookie did not work");
+                    }
+
+                    SaveConfig();
+                    IsConfigured = true;
+                    return IndexerConfigurationStatus.Completed;
+                }
+                catch (Exception e)
+                {
+                    IsConfigured = false;
+                    throw new Exception("Your cookie did not work: " + e.Message);
+                }
+            }
+
+            var result = await RequestLoginAndFollowRedirect(AjaxLoginUrl, pairs, configData.CookieHeader.Value, true, SiteLink, LoginUrl);
+
+            await ConfigureIfOK(result.Cookies, result.Content.Contains("logout.php"), () =>
+            {
+                var errorMessage = result.Content;
+                throw new ExceptionWithConfigData(errorMessage, configData);
+            });
+
+            return IndexerConfigurationStatus.RequiresTesting;
+        }
+
+        public async Task<IEnumerable<ReleaseInfo>> PerformQuery(TorznabQuery query)
+        {
+            List<ReleaseInfo> releases = new List<ReleaseInfo>();
+
+            var queryCollection = new NameValueCollection();
+            var searchString = query.GetQueryString();
+            var searchUrl = SearchUrl;
+
+            foreach (var cat in MapTorznabCapsToTrackers(query))
+            {
+                queryCollection.Add("c" + cat, "1");
+            }
+            
+            if (!string.IsNullOrWhiteSpace(searchString))
+            {
+                queryCollection.Add("search", searchString);
+            }
+
+            queryCollection.Add("blah", "0");
+
+            var results = await RequestStringWithCookiesAndRetry(searchUrl + "?" + queryCollection.GetQueryString());
+            try
+            {
+                releases.AddRange(contentToReleaseInfos(results.Content));
+            }
+            catch (Exception ex)
+            {
+                OnParseError(results.Content, ex);
+            }
+
+            return releases;
+        }
+
+        private IEnumerable<ReleaseInfo> contentToReleaseInfos(CQ dom)
+        {
+            List<ReleaseInfo> releases = new List<ReleaseInfo>();
+
+            // Doesn't handle pagination yet...
+            var rows = dom["div.panel-body > table.table > tbody > tr"];
+            foreach (var row in rows)
+            {
+                var release = new ReleaseInfo();
+                release.MinimumRatio = 1;
+                release.MinimumSeedTime = 259200;
+
+                var qRow = row.Cq();
+                release.Title = qRow.Find("td:nth-child(2) > a").First().Text().Trim();
+                release.Description = release.Title;
+                release.Guid = new Uri(SiteLink + qRow.Find("td:nth-child(2) > a").First().Attr("href"));
+                release.Comments = release.Guid;
+                release.Link = new Uri(SiteLink + qRow.Find("td:nth-child(3) > a").First().Attr("href"));
+                var pubDate = qRow.Find("td:nth-child(2) > span").First().Text().Trim().Replace("Added: ", "");
+                release.PublishDate = DateTime.Parse(pubDate).ToLocalTime();
+                release.Category = MapTrackerCatToNewznab(qRow.Find("td:nth-child(1) > a").First().Attr("href").Split('=')[1]);
+                release.Size = ReleaseInfo.GetBytes(qRow.Find("td:nth-child(7)").First().Text());
+                release.Seeders = ParseUtil.CoerceInt(qRow.Find("td:nth-child(9)").First().Text());
+                release.Peers = ParseUtil.CoerceInt(qRow.Find("td:nth-child(10)").First().Text()) + release.Seeders;
+
+                var files = row.Cq().Find("td:nth-child(5)").Text();
+                release.Files = ParseUtil.CoerceInt(files);
+
+                var grabs = row.Cq().Find("td:nth-child(8)").Text();
+                release.Grabs = ParseUtil.CoerceInt(grabs);
+
+                if (row.Cq().Find("i.fa-star").Any())
+                    release.DownloadVolumeFactor = 0;
+                else
+                    release.DownloadVolumeFactor = 1;
+
+                release.UploadVolumeFactor = 1;
+
+                releases.Add(release);
+            }
+
+            return releases;
+        }
+    }
+}

src/Jackett/Indexers/FrenchADN.cs

@@ -1,987 +0,0 @@
-using System;
-using System.Collections.Generic;
-using System.Collections.Specialized;
-using System.Globalization;
-using System.Linq;
-using System.Reflection;
-using System.Text.RegularExpressions;
-using System.Threading.Tasks;
-using CsQuery;
-using Jackett.Models;
-using Jackett.Models.IndexerConfig.Bespoke;
-using Jackett.Services;
-using Jackett.Utils;
-using Jackett.Utils.Clients;
-using Newtonsoft.Json;
-using Newtonsoft.Json.Linq;
-using NLog;
-
-namespace Jackett.Indexers
-{
-    /// <summary>
-    /// Provider for French-ADN Private Tracker
-    /// </summary>
-    public class FrenchAdn : BaseIndexer, IIndexer
-    {
-        private string LoginUrl => SiteLink + "login.php?";
-        private string LoginCheckUrl => SiteLink + "takelogin.php";
-        private string SearchUrl => SiteLink + "browse.php";
-        private string TorrentCommentUrl => SiteLink + "details.php?id={id}#comments";
-        private string TorrentDescriptionUrl => SiteLink + "details.php?id={id}";
-        private string TorrentDownloadUrl => SiteLink + "download.php?id={id}";
-        private string TorrentThanksUrl => SiteLink + "takethanks.php";
-        private bool Latency => ConfigData.Latency.Value;
-        private bool DevMode => ConfigData.DevMode.Value;
-        private bool CacheMode => ConfigData.HardDriveCache.Value;
-        private static string Directory => System.IO.Path.GetTempPath() + "Jackett\\" + MethodBase.GetCurrentMethod().DeclaringType?.Name + "\\";
-
-        private readonly Dictionary<string, string> _emulatedBrowserHeaders = new Dictionary<string, string>();
-        private CQ _fDom;
-        private ConfigurationDataFrenchAdn ConfigData => (ConfigurationDataFrenchAdn)configData;
-
-        public FrenchAdn(IIndexerManagerService i, IWebClient w, Logger l, IProtectionService ps)
-            : base(
-                name: "French-ADN",
-                description: "Your French Family Provider",
-                link: "https://french-adn.com/",
-                caps: new TorznabCapabilities(),
-                manager: i,
-                client: w,
-                logger: l,
-                p: ps,
-                downloadBase: "https://french-adn.com/download.php?id=",
-                configData: new ConfigurationDataFrenchAdn())
-        {
-            // Clean capabilities
-            TorznabCaps.Categories.Clear();
-
-            // Movies
-            AddCategoryMapping("15", TorznabCatType.Movies);            // ALL
-            AddCategoryMapping("108", TorznabCatType.MoviesSD);         // TS CAM
-            AddCategoryMapping("25", TorznabCatType.MoviesSD);          // BDRIP
-            AddCategoryMapping("56", TorznabCatType.MoviesSD);          // BRRIP
-            AddCategoryMapping("16", TorznabCatType.MoviesSD);          // DVDRIP
-            AddCategoryMapping("49", TorznabCatType.MoviesDVD);         // TVRIP
-            AddCategoryMapping("102", TorznabCatType.MoviesWEBDL);      // WEBRIP
-            AddCategoryMapping("105", TorznabCatType.MoviesHD);         // 1080P
-            AddCategoryMapping("104", TorznabCatType.MoviesHD);         // 720P
-            AddCategoryMapping("17", TorznabCatType.MoviesDVD);         // DVD R
-            AddCategoryMapping("21", TorznabCatType.MoviesDVD);         // DVD R5
-            AddCategoryMapping("112", TorznabCatType.MoviesDVD);        // DVD REMUX
-            AddCategoryMapping("107", TorznabCatType.Movies3D);         // 3D
-            AddCategoryMapping("113", TorznabCatType.MoviesBluRay);     // BLURAY
-            AddCategoryMapping("118", TorznabCatType.MoviesHD);         // MHD
-
-            // Series
-            AddCategoryMapping("41", TorznabCatType.TV);                // ALL
-            AddCategoryMapping("43", TorznabCatType.TV);                // VF
-            AddCategoryMapping("44", TorznabCatType.TV);                // VOSTFR
-            AddCategoryMapping("42", TorznabCatType.TV);                // PACK
-
-            // TV
-            AddCategoryMapping("110", TorznabCatType.TV);               // SHOWS
-
-            // Anime
-            AddCategoryMapping("109", TorznabCatType.TVAnime);          // ANIME
-
-            // Manga
-            AddCategoryMapping("119", TorznabCatType.TVAnime);          // MANGA
-
-            // Documentaries
-            AddCategoryMapping("114", TorznabCatType.TVDocumentary);    // DOCUMENTARY
-
-            // Music
-            AddCategoryMapping("22", TorznabCatType.Audio);             // ALL
-            AddCategoryMapping("24", TorznabCatType.AudioLossless);     // FLAC
-            AddCategoryMapping("23", TorznabCatType.AudioMP3);          // MP3
-
-            // Games
-            AddCategoryMapping("33", TorznabCatType.PCGames);           // ALL
-            AddCategoryMapping("45", TorznabCatType.PCGames);           // PC GAMES
-            AddCategoryMapping("93", TorznabCatType.Console3DS);        // 3DS
-            AddCategoryMapping("94", TorznabCatType.Console);           // PS2
-            AddCategoryMapping("93", TorznabCatType.ConsolePS3);        // PS3
-            AddCategoryMapping("95", TorznabCatType.ConsolePSP);        // PSP
-            AddCategoryMapping("35", TorznabCatType.ConsolePS3);        // WII
-
-            // Applications
-            AddCategoryMapping("11", TorznabCatType.PC);                // ALL
-            AddCategoryMapping("12", TorznabCatType.PC);                // APPS WINDOWS
-            AddCategoryMapping("97", TorznabCatType.PCMac);             // APPS MAC
-            AddCategoryMapping("98", TorznabCatType.PC);                // APPS LINUX
-
-            // Books
-            AddCategoryMapping("115", TorznabCatType.BooksEbook);       // EBOOK
-            AddCategoryMapping("114", TorznabCatType.BooksComics);      // COMICS
-
-            // Other
-            AddCategoryMapping("103", TorznabCatType.Other);            // STAFF
-        }
-
-        /// <summary>
-        /// Configure our FADN Provider
-        /// </summary>
-        /// <param name="configJson">Our params in Json</param>
-        /// <returns>Configuration state</returns>
-        public async Task<IndexerConfigurationStatus> ApplyConfiguration(JToken configJson)
-        {
-            // Retrieve config values set by Jackett's user
-            ConfigData.LoadValuesFromJson(configJson);
-
-            // Check & Validate Config
-            ValidateConfig();
-
-            // Setting our data for a better emulated browser (maximum security)
-            // TODO: Encoded Content not supported by Jackett at this time
-            // emulatedBrowserHeaders.Add("Accept-Encoding", "gzip, deflate");
-
-            // If we want to simulate a browser
-            if (ConfigData.Browser.Value)
-            {
-
-                // Clean headers
-                _emulatedBrowserHeaders.Clear();
-
-                // Inject headers
-                _emulatedBrowserHeaders.Add("Accept", ConfigData.HeaderAccept.Value);
-                _emulatedBrowserHeaders.Add("Accept-Language", ConfigData.HeaderAcceptLang.Value);
-                _emulatedBrowserHeaders.Add("DNT", Convert.ToInt32(ConfigData.HeaderDnt.Value).ToString());
-                _emulatedBrowserHeaders.Add("Upgrade-Insecure-Requests", Convert.ToInt32(ConfigData.HeaderUpgradeInsecure.Value).ToString());
-                _emulatedBrowserHeaders.Add("User-Agent", ConfigData.HeaderUserAgent.Value);
-            }
-
-            await DoLogin();
-
-            return IndexerConfigurationStatus.RequiresTesting;
-        }
-
-        /// <summary>
-        /// Perform login to racker
-        /// </summary>
-        /// <returns></returns>
-        private async Task DoLogin()
-        {
-            // Build WebRequest for index
-            var myIndexRequest = new WebRequest()
-            {
-                Type = RequestType.GET,
-                Url = SiteLink,
-                Headers = _emulatedBrowserHeaders
-            };
-
-            // Get index page for cookies
-            Output("\nGetting index page (for cookies).. with " + SiteLink);
-            var indexPage = await webclient.GetString(myIndexRequest);
-
-            // Building login form data
-            var pairs = new Dictionary<string, string> {
-                { "username", ConfigData.Username.Value },
-                { "password", ConfigData.Password.Value }
-            };
-
-            // Build WebRequest for login
-            var myRequestLogin = new WebRequest()
-            {
-                Type = RequestType.GET,
-                Url = LoginUrl,
-                Headers = _emulatedBrowserHeaders,
-                Cookies = indexPage.Cookies,
-                Referer = SiteLink
-            };
-
-            // Get login page -- (not used, but simulation needed by tracker security's checks)
-            LatencyNow();
-            Output("\nGetting login page (user simulation).. with " + LoginUrl);
-            await webclient.GetString(myRequestLogin);
-
-            // Build WebRequest for submitting authentification
-            var request = new WebRequest()
-            {
-                PostData = pairs,
-                Referer = LoginUrl,
-                Type = RequestType.POST,
-                Url = LoginCheckUrl,
-                Headers = _emulatedBrowserHeaders,
-                Cookies = indexPage.Cookies,
-
-            };
-
-            // Perform loggin
-            LatencyNow();
-            Output("\nPerform loggin.. with " + LoginCheckUrl);
-            var response = await webclient.GetString(request);
-
-            // Test if we are logged in
-            await ConfigureIfOK(response.Cookies, !string.IsNullOrEmpty(response.Cookies) && !response.IsRedirect, () =>
-            {
-                // Default error message
-                var message = "Error during attempt !";
-
-                // Parse redirect header
-                var redirectTo = response.RedirectingTo;
-
-                // Analyzer error code
-                if (redirectTo.Contains("login.php?error=4"))
-                {
-                    // Set message
-                    message = "Wrong username or password !";
-                }
-
-                // Oops, unable to login
-                Output("-> Login failed: " + message, "error");
-                throw new ExceptionWithConfigData("Login failed: " + message, configData);
-            });
-
-            Output("\nCookies saved for future uses...");
-            ConfigData.CookieHeader.Value = indexPage.Cookies + " " + response.Cookies + " ts_username=" + ConfigData.Username.Value;
-
-            Output("\n-> Login Success\n");
-        }
-
-        /// <summary>
-        /// Check logged-in state for provider
-        /// </summary>
-        /// <returns></returns>
-        private async Task CheckLogin()
-        {
-            // Checking ...
-            Output("\n-> Checking logged-in state....");
-            var loggedInCheck = await RequestStringWithCookies(SearchUrl);
-            if (!loggedInCheck.Content.Contains("/logout.php"))
-            {
-                // Cookie expired, renew session on provider
-                Output("-> Not logged, login now...\n");
-                await DoLogin();
-            }
-            else
-            {
-                // Already logged, session active
-                Output("-> Already logged, continue...\n");
-            }
-        }
-
-        /// <summary>
-        /// Execute our search query
-        /// </summary>
-        /// <param name="query">Query</param>
-        /// <returns>Releases</returns>
-        public async Task<IEnumerable<ReleaseInfo>> PerformQuery(TorznabQuery query)
-        {
-            var releases = new List<ReleaseInfo>();
-            var torrentRowList = new List<CQ>();
-            var searchTerm = query.GetQueryString();
-            var searchUrl = SearchUrl;
-
-            // Check login before performing a query
-           await CheckLogin();
-
-            // Check cache first so we don't query the server (if search term used or not in dev mode)
-            if (!DevMode && !string.IsNullOrEmpty(searchTerm))
-            {
-                lock (cache)
-                {
-                    // Remove old cache items
-                    CleanCache();
-
-                    // Search in cache
-                    var cachedResult = cache.FirstOrDefault(i => i.Query == searchTerm);
-                    if (cachedResult != null)
-                        return cachedResult.Results.Select(s => (ReleaseInfo)s.Clone()).ToArray();
-                }
-            }
-
-            // Build our query
-            var request = BuildQuery(searchTerm, query, searchUrl);
-
-            // Getting results & Store content
-            var results = await QueryExec(request);
-            _fDom = results.Content;
-
-            try
-            {
-                // Find torrent rows
-                var firstPageRows = FindTorrentRows();
-
-                // Add them to torrents list
-                torrentRowList.AddRange(firstPageRows.Select(fRow => fRow.Cq()));
-
-                // Check if there are pagination links at bottom
-                var pagination = (_fDom["#quicknavpage_menu"].Length != 0);
-
-                // If pagination available
-                int nbResults;
-                int pageLinkCount;
-                if (pagination)
-                {
-                    // Retrieve available pages (3 pages shown max)
-                    pageLinkCount = _fDom["#navcontainer_f:first > ul"].Find("a").Not(".smalltext").Not("#quicknavpage").Length;
-
-                    // Last button ? (So more than 3 page are available)
-                    var more = _fDom["#navcontainer_f:first > ul"].Find("a.smalltext").Length > 1;
-
-                    // More page than 3 pages ?
-                    if (more)
-                    {
-                        // Get total page count from last link
-                        pageLinkCount = ParseUtil.CoerceInt(Regex.Match(_fDom["#navcontainer_f:first > ul"].Find("a:eq(4)").Attr("href"), @"\d+").Value);
-                    }
-
-                    // Calculate average number of results (based on torrents rows lenght on first page)
-                    nbResults = firstPageRows.Count() * pageLinkCount;
-                }
-                else {
-                    nbResults = 1;
-                    pageLinkCount = 1;
-
-                    // Check if we have a minimum of one result
-                    if (firstPageRows.Length > 1)
-                    {
-                        // Retrieve total count on our alone page
-                        nbResults = firstPageRows.Count();
-                    }
-                    else
-                    {
-                        // Check if no result
-                        if(torrentRowList.First().Find("td").Length == 1)
-                        {
-                            // No results found
-                            Output("\nNo result found for your query, please try another search term ...\n", "info");
-
-                            // No result found for this query
-                            return releases;
-                        }
-                    }
-                }
-                Output("\nFound " + nbResults + " result(s) (+/- " + firstPageRows.Length + ") in " + pageLinkCount + " page(s) for this query !");
-                Output("\nThere are " + firstPageRows.Length + " results on the first page !");
-
-                // If we have a term used for search and pagination result superior to one
-                if (!string.IsNullOrWhiteSpace(query.GetQueryString()) && pageLinkCount > 1)
-                {
-                    // Starting with page #2
-                    for (var i = 2; i <= Math.Min(int.Parse(ConfigData.Pages.Value), pageLinkCount); i++)
-                    {
-                        Output("\nProcessing page #" + i);
-
-                        // Request our page
-                        LatencyNow();
-
-                        // Build our query -- Minus 1 to page due to strange pagination number on tracker side, starting with page 0...
-                        var pageRequest = BuildQuery(searchTerm, query, searchUrl, i);
-
-                        // Getting results & Store content
-                        WebClientStringResult pageResults = await QueryExec(pageRequest);
-
-                        // Assign response
-                        _fDom = pageResults.Content;
-
-                        // Process page results
-                        var additionalPageRows = FindTorrentRows();
-
-                        // Add them to torrents list
-                        torrentRowList.AddRange(additionalPageRows.Select(fRow => fRow.Cq()));
-                    }
-                }
-
-                // Loop on results
-                foreach (var tRow in torrentRowList)
-                {
-                    Output("\n=>> Torrent #" + (releases.Count + 1));
-
-                    // ID
-                    var id = ParseUtil.CoerceInt(Regex.Match(tRow.Find("td:eq(1) > div:first > a").Attr("name"), @"\d+").Value);
-                    Output("ID: " + id);
-
-                    // Check if torrent is not nuked by tracker or rulez, can't download it
-                    if (tRow.Find("td:eq(2) > a").Length == 0)
-                    {
-                        // Next item
-                        Output("Torrent is nuked, we can't download it, going to next torrent...");
-                        continue;
-                    }
-
-                    // Release Name
-                    var name = tRow.Find("td:eq(2) > a").Attr("title").Substring(24).Trim();
-                    Output("Release: " + name);
-
-                    // Category
-                    var categoryId = ParseUtil.CoerceInt(Regex.Match(tRow.Find("td:eq(0) > a").Attr("href"), @"\d+").Value);
-                    var categoryName = tRow.Find("td:eq(0) > a > img").Attr("title").Split(new[] { ':' }, 2)[1].Trim();
-                    Output("Category: " + MapTrackerCatToNewznab(categoryId.ToString()) + " (" + categoryId + " - " + categoryName + ")");
-
-                    // Seeders
-                    var seeders = ParseUtil.CoerceInt(Regex.Match(tRow.Find("td:eq(5) > div > font").Select(s => Regex.Replace(s.ToString(), "<.*?>", string.Empty)).ToString(), @"\d+").Value);
-                    Output("Seeders: " + seeders);
-
-                    // Leechers
-                    var leechers = ParseUtil.CoerceInt(Regex.Match(tRow.Find("td:eq(6) > div > font").Text(), @"\d+").Value);
-                    Output("Leechers: " + leechers);
-
-                    // Completed
-                    var completed = ParseUtil.CoerceInt(Regex.Match(tRow.Find("td:eq(4)").Text(), @"\d+").Value);
-                    Output("Completed: " + completed);
-
-                    // Files
-                    var files = 1;
-                    if (tRow.Find("td:eq(3) > a").Length == 1)
-                    {
-                        files = ParseUtil.CoerceInt(Regex.Match(tRow.Find("td:eq(3) > a").Text(), @"\d+").Value);
-                    }
-                    Output("Files: " + files);
-
-                    // Health
-                    var percent = ParseUtil.CoerceInt(Regex.Match(tRow.Find("td:eq(7) > img").Attr("src"), @"\d+").Value) * 10;
-                    Output("Health: " + percent + "%");
-
-                    // Size
-                    var humanSize = tRow.Find("td:eq(8)").Text().ToLowerInvariant();
-                    var size = ReleaseInfo.GetBytes(humanSize);
-                    Output("Size: " + humanSize + " (" + size + " bytes)");
-
-                    // Date & IMDB & Genre
-                    var infosData = tRow.Find("td:eq(1) > div:last").Text();
-                    var infosList = Regex.Split(infosData, "\\|").ToList();
-                    var infosTorrent = infosList.Select(s => s.Split(new[] { ':' }, 2)[1].Trim()).ToList();
-
-                    // --> Date
-                    var date = FormatDate(infosTorrent.First());
-                    Output("Released on: " + date.ToLocalTime());
-
-                    // --> Genre
-                    var genre = infosTorrent.Last();
-                    Output("Genre: " + genre);
-
-                    // Torrent Details URL
-                    var detailsLink = new Uri(TorrentDescriptionUrl.Replace("{id}", id.ToString()));
-                    Output("Details: " + detailsLink.AbsoluteUri);
-
-                    // Torrent Comments URL
-                    var commentsLink = new Uri(TorrentCommentUrl.Replace("{id}", id.ToString()));
-                    Output("Comments Link: " + commentsLink.AbsoluteUri);
-
-                    // Torrent Download URL
-                    var downloadLink = new Uri(TorrentDownloadUrl.Replace("{id}", id.ToString()));
-                    Output("Download Link: " + downloadLink.AbsoluteUri);
-
-                    // Building release infos
-                    var release = new ReleaseInfo
-                    {
-                        Category = MapTrackerCatToNewznab(categoryId.ToString()),
-                        Title = name,
-                        Seeders = seeders,
-                        Peers = seeders + leechers,
-                        MinimumRatio = 1,
-                        MinimumSeedTime = 172800,
-                        PublishDate = date,
-                        Size = size,
-                        Guid = detailsLink,
-                        Comments = commentsLink,
-                        Link = downloadLink
-                    };
-                    releases.Add(release);
-                }
-
-            }
-            catch (Exception ex)
-            {
-                OnParseError("Error, unable to parse result \n" + ex.StackTrace, ex);
-            }
-
-            // Return found releases
-            return releases;
-        }
-
-        /// <summary>
-        /// Build query to process
-        /// </summary>
-        /// <param name="term">Term to search</param>
-        /// <param name="query">Torznab Query for categories mapping</param>
-        /// <param name="url">Search url for provider</param>
-        /// <param name="page">Page number to request</param>
-        /// <returns>URL to query for parsing and processing results</returns>
-        private string BuildQuery(string term, TorznabQuery query, string url, int page = 0)
-        {
-            var parameters = new NameValueCollection();
-            var categoriesList = MapTorznabCapsToTrackers(query);
-
-            // Building our tracker query
-            parameters.Add("do", "search");
-
-            // If search term provided
-            if (!string.IsNullOrWhiteSpace(term))
-            {
-                // Add search term ~~ Strange search engine, need to replace space with dot for results !
-                parameters.Add("keywords", term.Replace(' ', '.'));
-            }
-            else
-            {
-                // Showing all torrents (just for output function)
-                parameters.Add("keywords", "");
-                term = "all";
-            }
-
-            // Adding requested categories
-            parameters.Add("category", categoriesList.Count > 0 ? string.Join(",", categoriesList) : "");
-
-            // Building our tracker query
-            parameters.Add("search_type", "t_name");
-
-            // Check if we are processing a new page
-            if (page > 1)
-            {
-                // Adding page number to query
-                parameters.Add("page", page.ToString());
-            }
-
-            // Building our query
-            url += "?" + parameters.GetQueryString();
-
-            Output("\nBuilded query for \"" + term + "\"... " + url);
-
-            // Return our search url
-            return url;
-        }
-
-        /// <summary>
-        /// Switch Method for Querying
-        /// </summary>
-        /// <param name="request">URL created by Query Builder</param>
-        /// <returns>Results from query</returns>
-        private async Task<WebClientStringResult> QueryExec(string request)
-        {
-            WebClientStringResult results;
-
-            // Switch in we are in DEV mode with Hard Drive Cache or not
-            if (DevMode && CacheMode)
-            {
-                // Check Cache before querying and load previous results if available
-                results = await QueryCache(request);
-            }
-            else
-            {
-                // Querying tracker directly
-                results = await QueryTracker(request);
-            }
-            return results;
-        }
-
-        /// <summary>
-        /// Get Torrents Page from Cache by Query Provided
-        /// </summary>
-        /// <param name="request">URL created by Query Builder</param>
-        /// <returns>Results from query</returns>
-        private async Task<WebClientStringResult> QueryCache(string request)
-        {
-            WebClientStringResult results;
-
-            // Create Directory if not exist
-            System.IO.Directory.CreateDirectory(Directory);
-
-            // Clean Storage Provider Directory from outdated cached queries
-            CleanCacheStorage();
-
-            // Create fingerprint for request
-            var file = Directory + request.GetHashCode() + ".json";
-
-            // Checking modes states
-            if (System.IO.File.Exists(file))
-            {
-                // File exist... loading it right now !
-                Output("Loading results from hard drive cache ..." + request.GetHashCode() + ".json");
-                results = JsonConvert.DeserializeObject<WebClientStringResult>(System.IO.File.ReadAllText(file));
-            }
-            else
-            {
-                // No cached file found, querying tracker directly
-                results = await QueryTracker(request);
-
-                // Cached file didn't exist for our query, writing it right now !
-                Output("Writing results to hard drive cache ..." + request.GetHashCode() + ".json");
-                System.IO.File.WriteAllText(file, JsonConvert.SerializeObject(results));
-            }
-            return results;
-        }
-
-        /// <summary>
-        /// Get Torrents Page from Tracker by Query Provided
-        /// </summary>
-        /// <param name="request">URL created by Query Builder</param>
-        /// <returns>Results from query</returns>
-        private async Task<WebClientStringResult> QueryTracker(string request)
-        {
-            // Cache mode not enabled or cached file didn't exist for our query
-            Output("\nQuerying tracker for results....");
-
-            // Request our first page
-            LatencyNow();
-            var results = await RequestStringWithCookiesAndRetry(request, ConfigData.CookieHeader.Value, SearchUrl, _emulatedBrowserHeaders);
-
-            // Return results from tracker
-            return results;
-        }
-
-        /// <summary>
-        /// Clean Hard Drive Cache Storage
-        /// </summary>
-        /// <param name="force">Force Provider Folder deletion</param>
-        private void CleanCacheStorage(bool force = false)
-        {
-            // Check cleaning method
-            if (force)
-            {
-                // Deleting Provider Storage folder and all files recursively
-                Output("\nDeleting Provider Storage folder and all files recursively ...");
-
-                // Check if directory exist
-                if (System.IO.Directory.Exists(Directory))
-                {
-                    // Delete storage directory of provider
-                    System.IO.Directory.Delete(Directory, true);
-                    Output("-> Storage folder deleted successfully.");
-                }
-                else
-                {
-                    // No directory, so nothing to do
-                    Output("-> No Storage folder found for this provider !");
-                }
-            }
-            else
-            {
-                var i = 0;
-                // Check if there is file older than ... and delete them
-                Output("\nCleaning Provider Storage folder... in progress.");
-                System.IO.Directory.GetFiles(Directory)
-                .Select(f => new System.IO.FileInfo(f))
-                .Where(f => f.LastAccessTime < DateTime.Now.AddMilliseconds(-Convert.ToInt32(ConfigData.HardDriveCacheKeepTime.Value)))
-                .ToList()
-                .ForEach(f => {
-                    Output("Deleting cached file << " + f.Name + " >> ... done.");
-                    f.Delete();
-                    i++;
-                });
-
-                // Inform on what was cleaned during process
-                if (i > 0)
-                {
-                    Output("-> Deleted " + i + " cached files during cleaning.");
-                }
-                else {
-                    Output("-> Nothing deleted during cleaning.");
-                }
-            }
-        }
-
-        /// <summary>
-        /// Generate a random fake latency to avoid detection on tracker side
-        /// </summary>
-        private void LatencyNow()
-        {
-            // Need latency ?
-            if (Latency)
-            {
-                var random = new Random(DateTime.Now.Millisecond);
-                var waiting = random.Next(Convert.ToInt32(ConfigData.LatencyStart.Value),
-                    Convert.ToInt32(ConfigData.LatencyEnd.Value));
-                Output("\nLatency Faker => Sleeping for " + waiting + " ms...");
-
-                // Sleep now...
-                System.Threading.Thread.Sleep(waiting);
-            }
-            // Generate a random value in our range
-        }
-
-        /// <summary>
-        /// Find torrent rows in search pages
-        /// </summary>
-        /// <returns>JQuery Object</returns>
-        private CQ FindTorrentRows()
-        {
-            // Return all occurencis of torrents found
-            return _fDom["#showcontents > table > tbody > tr:not(:first)"];
-        }
-
-        /// <summary>
-        /// Format Date to DateTime
-        /// </summary>
-        /// <param name="clock"></param>
-        /// <returns>A DateTime</returns>
-        private static DateTime FormatDate(string clock)
-        {
-            DateTime date;
-
-            // Switch from date format
-            if(clock.Contains("Aujourd'hui") || clock.Contains("Hier"))
-            {
-                // Get hours & minutes
-                IList<int> infosClock = clock.Split(':').Select(s => ParseUtil.CoerceInt(Regex.Match(s, @"\d+").Value)).ToList();
-
-                // Ago date with today
-                date = new DateTime(DateTime.Now.Year, DateTime.Now.Month, DateTime.Now.Day, Convert.ToInt32(infosClock[0]), Convert.ToInt32(infosClock[1]), DateTime.Now.Second);
-
-                // Set yesterday if necessary
-                if (clock.Contains("Hier"))
-                {
-                    // Remove one day from date
-                    // ReSharper disable once ReturnValueOfPureMethodIsNotUsed
-                    date.AddDays(-1);
-                }
-            }
-            else
-            {
-                // Parse Date if full
-                date = DateTime.ParseExact(clock, "MM-dd-yyyy HH:mm", CultureInfo.GetCultureInfo("fr-FR"), DateTimeStyles.AssumeLocal);
-            }
-
-            return date.ToUniversalTime();
-        }
-
-        /// <summary>
-        /// Download torrent file from tracker
-        /// </summary>
-        /// <param name="link">URL string</param>
-        /// <returns></returns>
-        public override async Task<byte[]> Download(Uri link)
-        {
-            // This tracker need to thanks Uploader before getting torrent file...
-            Output("\nThis tracker needs you to thank uploader before downloading torrent!");
-
-            // Retrieving ID from link provided
-            var id = ParseUtil.CoerceInt(Regex.Match(link.AbsoluteUri, @"\d+").Value);
-            Output("Torrent Requested ID: " + id);
-
-            // Building login form data
-            var pairs = new Dictionary<string, string> {
-                { "torrentid", id.ToString() },
-                { "_", string.Empty } // ~~ Strange, blank param...
-            };
-
-            // Add emulated XHR request
-            _emulatedBrowserHeaders.Add("X-Prototype-Version", "1.6.0.3");
-            _emulatedBrowserHeaders.Add("X-Requested-With", "XMLHttpRequest");
-
-            // Build WebRequest for thanks
-            var myRequestThanks = new WebRequest()
-            {
-                Type = RequestType.POST,
-                PostData = pairs,
-                Url = TorrentThanksUrl,
-                Headers = _emulatedBrowserHeaders,
-                Cookies = ConfigData.CookieHeader.Value,
-                Referer = TorrentDescriptionUrl.Replace("{id}", id.ToString())
-            };
-
-            // Get thanks page -- (not used, just for doing a request)
-            LatencyNow();
-            Output("Thanks user, to get download link for our torrent.. with " + TorrentThanksUrl);
-            await webclient.GetString(myRequestThanks);
-
-            // Get torrent file now
-            Output("Getting torrent file now....");
-            var response = await base.Download(link);
-
-            // Remove our XHR request header
-            _emulatedBrowserHeaders.Remove("X-Prototype-Version");
-            _emulatedBrowserHeaders.Remove("X-Requested-With");
-
-            // Return content
-            return response;
-        }
-
-        /// <summary>
-        /// Output message for logging or developpment (console)
-        /// </summary>
-        /// <param name="message">Message to output</param>
-        /// <param name="level">Level for Logger</param>
-        private void Output(string message, string level = "debug")
-        {
-            // Check if we are in dev mode
-            if (DevMode)
-            {
-                // Output message to console
-                Console.WriteLine(message);
-            }
-            else
-            {
-                // Send message to logger with level
-                switch (level)
-                {
-                    default:
-                        goto case "debug";
-                    case "debug":
-                        // Only if Debug Level Enabled on Jackett
-                        if (Engine.Logger.IsDebugEnabled)
-                        {
-                            logger.Debug(message);
-                        }
-                        break;
-                    case "info":
-                        logger.Info(message);
-                        break;
-                    case "error":
-                        logger.Error(message);
-                        break;
-                }
-            }
-        }
-
-        /// <summary>
-        /// Validate Config entered by user on Jackett
-        /// </summary>
-        private void ValidateConfig()
-        {
-            Output("\nValidating Settings ... \n");
-
-            // Check Username Setting
-            if (string.IsNullOrEmpty(ConfigData.Username.Value))
-            {
-                throw new ExceptionWithConfigData("You must provide a username for this tracker to login !", ConfigData);
-            }
-            else
-            {
-                Output("Validated Setting -- Username (auth) => " + ConfigData.Username.Value);
-            }
-
-            // Check Password Setting
-            if (string.IsNullOrEmpty(ConfigData.Password.Value))
-            {
-                throw new ExceptionWithConfigData("You must provide a password with your username for this tracker to login !", ConfigData);
-            }
-            else
-            {
-                Output("Validated Setting -- Password (auth) => " + ConfigData.Password.Value);
-            }
-
-            // Check Max Page Setting
-            if (!string.IsNullOrEmpty(ConfigData.Pages.Value))
-            {
-                try
-                {
-                    Output("Validated Setting -- Max Pages => " + Convert.ToInt32(ConfigData.Pages.Value));
-                }
-                catch (Exception)
-                {
-                    throw new ExceptionWithConfigData("Please enter a numeric maximum number of pages to crawl !", ConfigData);
-                }
-            }
-            else
-            {
-                throw new ExceptionWithConfigData("Please enter a maximum number of pages to crawl !", ConfigData);
-            }
-
-            // Check Latency Setting
-            if (ConfigData.Latency.Value)
-            {
-                Output("\nValidated Setting -- Latency Simulation enabled");
-
-                // Check Latency Start Setting
-                if (!string.IsNullOrEmpty(ConfigData.LatencyStart.Value))
-                {
-                    try
-                    {
-                        Output("Validated Setting -- Latency Start => " + Convert.ToInt32(ConfigData.LatencyStart.Value));
-                    }
-                    catch (Exception)
-                    {
-                        throw new ExceptionWithConfigData("Please enter a numeric latency start in ms !", ConfigData);
-                    }
-                }
-                else
-                {
-                    throw new ExceptionWithConfigData("Latency Simulation enabled, Please enter a start latency !", ConfigData);
-                }
-
-                // Check Latency End Setting
-                if (!string.IsNullOrEmpty(ConfigData.LatencyEnd.Value))
-                {
-                    try
-                    {
-                        Output("Validated Setting -- Latency End => " + Convert.ToInt32(ConfigData.LatencyEnd.Value));
-                    }
-                    catch (Exception)
-                    {
-                        throw new ExceptionWithConfigData("Please enter a numeric latency end in ms !", ConfigData);
-                    }
-                }
-                else
-                {
-                    throw new ExceptionWithConfigData("Latency Simulation enabled, Please enter a end latency !", ConfigData);
-                }
-            }
-
-            // Check Browser Setting
-            if (ConfigData.Browser.Value)
-            {
-                Output("\nValidated Setting -- Browser Simulation enabled");
-
-                // Check ACCEPT header Setting
-                if (string.IsNullOrEmpty(ConfigData.HeaderAccept.Value))
-                {
-                    throw new ExceptionWithConfigData("Browser Simulation enabled, Please enter an ACCEPT header !", ConfigData);
-                }
-                else
-                {
-                    Output("Validated Setting -- ACCEPT (header) => " + ConfigData.HeaderAccept.Value);
-                }
-
-                // Check ACCEPT-LANG header Setting
-                if (string.IsNullOrEmpty(ConfigData.HeaderAcceptLang.Value))
-                {
-                    throw new ExceptionWithConfigData("Browser Simulation enabled, Please enter an ACCEPT-LANG header !", ConfigData);
-                }
-                else
-                {
-                    Output("Validated Setting -- ACCEPT-LANG (header) => " + ConfigData.HeaderAcceptLang.Value);
-                }
-
-                // Check USER-AGENT header Setting
-                if (string.IsNullOrEmpty(ConfigData.HeaderUserAgent.Value))
-                {
-                    throw new ExceptionWithConfigData("Browser Simulation enabled, Please enter an USER-AGENT header !", ConfigData);
-                }
-                else
-                {
-                    Output("Validated Setting -- USER-AGENT (header) => " + ConfigData.HeaderUserAgent.Value);
-                }
-            }
-            else
-            {
-                // Browser simulation must be enabled (otherwhise, this provider will not work due to tracker's security)
-                throw new ExceptionWithConfigData("Browser Simulation must be enabled for this provider to work, please enable it !", ConfigData);
-            }
-
-            // Check Dev Cache Settings
-            if (ConfigData.HardDriveCache.Value)
-            {
-                Output("\nValidated Setting -- DEV Hard Drive Cache enabled");
-
-                // Check if Dev Mode enabled !
-                if (!ConfigData.DevMode.Value)
-                {
-                    throw new ExceptionWithConfigData("Hard Drive is enabled but not in DEV MODE, Please enable DEV MODE !", ConfigData);
-                }
-
-                // Check Cache Keep Time Setting
-                if (!string.IsNullOrEmpty(ConfigData.HardDriveCacheKeepTime.Value))
-                {
-                    try
-                    {
-                        Output("Validated Setting -- Cache Keep Time (ms) => " + Convert.ToInt32(ConfigData.HardDriveCacheKeepTime.Value));
-                    }
-                    catch (Exception)
-                    {
-                        throw new ExceptionWithConfigData("Please enter a numeric hard drive keep time in ms !", ConfigData);
-                    }
-                }
-                else
-                {
-                    throw new ExceptionWithConfigData("Hard Drive Cache enabled, Please enter a maximum keep time for cache !", ConfigData);
-                }
-            }
-            else
-            {
-                // Delete cache if previously existed
-                CleanCacheStorage(true);
-            }
-        }
-    }
-}

src/Jackett/Indexers/Freshon.cs

@@ -104,10 +104,13 @@ namespace Jackett.Indexers
                     release.MinimumRatio = 1;
                     release.MinimumSeedTime = 172800;
                     release.Title = qLink.Attr("title");
+                    if (!query.MatchQueryStringAND(release.Title))
+                        continue;
+
                     release.Description = release.Title;
-                    release.Guid = new Uri(SiteLink + qLink.Attr("href"));
+                    release.Guid = new Uri(SiteLink + qLink.Attr("href").TrimStart('/'));
                     release.Comments = release.Guid;
-                    release.Link = new Uri(SiteLink + qRow.Find("td.table_links > a").First().Attr("href"));
+                    release.Link = new Uri(SiteLink + qRow.Find("td.table_links > a").First().Attr("href").TrimStart('/'));
                     release.Category = TvCategoryParser.ParseTvShowQuality(release.Title);
 
                     release.Seeders = ParseUtil.CoerceInt(qRow.Find("td.table_seeders").Text().Trim());
@@ -128,6 +131,18 @@ namespace Jackett.Indexers
                     DateTime pubDateUtc = TimeZoneInfo.ConvertTimeToUtc(pubDateRomania, romaniaTz);
                     release.PublishDate = pubDateUtc.ToLocalTime();
 
+                    var grabs = row.Cq().Find("td.table_snatch").Get(0).FirstChild.ToString();
+                    release.Grabs = ParseUtil.CoerceInt(grabs);
+
+                    if (row.Cq().Find("img[alt=\"100% Free\"]").Any())
+                        release.DownloadVolumeFactor = 0;
+                    else if (row.Cq().Find("img[alt=\"50% Free\"]").Any())
+                        release.DownloadVolumeFactor = 0.5;
+                    else
+                        release.DownloadVolumeFactor = 1;
+
+                    release.UploadVolumeFactor = 1;
+
                     releases.Add(release);
                 }
             }

src/Jackett/Indexers/FunFile.cs

@@ -0,0 +1,154 @@
+using CsQuery;
+using Jackett.Models;
+using Jackett.Services;
+using Jackett.Utils;
+using Jackett.Utils.Clients;
+using Newtonsoft.Json.Linq;
+using NLog;
+using System;
+using System.Collections.Generic;
+using System.Threading.Tasks;
+using Jackett.Models.IndexerConfig;
+using System.Collections.Specialized;
+
+namespace Jackett.Indexers
+{
+    public class FunFile : BaseIndexer, IIndexer
+    {
+        private string SearchUrl { get { return SiteLink + "browse.php"; } }
+        private string LoginUrl { get { return SiteLink + "takelogin.php"; } }
+
+        new ConfigurationDataBasicLoginWithRSSAndDisplay configData
+        {
+            get { return (ConfigurationDataBasicLoginWithRSSAndDisplay)base.configData; }
+            set { base.configData = value; }
+        }
+
+        public FunFile(IIndexerManagerService i, Logger l, IWebClient w, IProtectionService ps)
+            : base(name: "FunFile",
+                description: "A general tracker",
+                link: "https://www.funfile.org/",
+                caps: new TorznabCapabilities(),
+                manager: i,
+                client: w,
+                logger: l,
+                p: ps,
+                configData: new ConfigurationDataBasicLoginWithRSSAndDisplay())
+        {
+            AddCategoryMapping(44, TorznabCatType.TVAnime); // Anime
+            AddCategoryMapping(22, TorznabCatType.PC); // Applications
+            AddCategoryMapping(43, TorznabCatType.AudioAudiobook); // Audio Books
+            AddCategoryMapping(27, TorznabCatType.Books); // Ebook
+            AddCategoryMapping(4,  TorznabCatType.PCGames); // Games
+            AddCategoryMapping(40, TorznabCatType.OtherMisc); // Miscellaneous
+            AddCategoryMapping(19, TorznabCatType.Movies); // Movies
+            AddCategoryMapping(6,  TorznabCatType.Audio); // Music
+            AddCategoryMapping(31, TorznabCatType.PCPhoneOther); // Portable
+            AddCategoryMapping(7,  TorznabCatType.TV); // TV
+        }
+
+        public async Task<IndexerConfigurationStatus> ApplyConfiguration(JToken configJson)
+        {
+            configData.LoadValuesFromJson(configJson);
+            var pairs = new Dictionary<string, string> {
+                { "username", configData.Username.Value },
+                { "password", configData.Password.Value },
+                { "login", "Login" },
+            };
+
+            var result = await RequestLoginAndFollowRedirect(LoginUrl, pairs, null, true, null, LoginUrl);
+            await ConfigureIfOK(result.Cookies, result.Content.Contains("logout.php"), () =>
+            {
+                CQ dom = result.Content;
+                var errorMessage = dom["td.mf_content"].Html();
+                throw new ExceptionWithConfigData(errorMessage, configData);
+            });
+            return IndexerConfigurationStatus.RequiresTesting;
+        }
+
+        public async Task<IEnumerable<ReleaseInfo>> PerformQuery(TorznabQuery query)
+        {
+            List<ReleaseInfo> releases = new List<ReleaseInfo>();
+
+            var searchString = query.GetQueryString();
+            var searchUrl = SearchUrl;
+            var queryCollection = new NameValueCollection();
+            queryCollection.Add("incldead", "1");
+            queryCollection.Add("showspam", "1");
+
+            if (!string.IsNullOrWhiteSpace(searchString))
+            {
+                queryCollection.Add("search", searchString);
+            }
+
+            var cats = MapTorznabCapsToTrackers(query);
+            string cat = "0";
+            if (cats.Count == 1)
+            {
+                cat = cats[0];
+            }
+            queryCollection.Add("cat", cat);
+
+            searchUrl += "?" + queryCollection.GetQueryString();
+
+            var results = await RequestStringWithCookiesAndRetry(searchUrl);
+            try
+            {
+                CQ dom = results.Content;
+                var rows = dom["table[cellpadding=2] > tbody > tr:has(td.row3)"];
+                foreach (var row in rows)
+                {
+                    var release = new ReleaseInfo();
+                    release.MinimumRatio = 1;
+                    release.MinimumSeedTime = 48 * 60 * 60;
+                    
+                    var qRow = row.Cq();
+                    var qCatLink = qRow.Find("a[href^=browse.php?cat=]").First();
+                    var qDetailsLink = qRow.Find("a[href^=details.php?id=]").First();
+                    var qSeeders = qRow.Find("td:eq(9)");
+                    var qLeechers = qRow.Find("td:eq(10)");
+                    var qDownloadLink = qRow.Find("a[href^=download.php]").First();
+                    var qTimeAgo = qRow.Find("td:eq(5)");
+                    var qSize = qRow.Find("td:eq(7)");
+
+                    var catStr = qCatLink.Attr("href").Split('=')[1].Split('&')[0];
+                    release.Category = MapTrackerCatToNewznab(catStr);
+
+                    release.Link = new Uri(SiteLink + qDownloadLink.Attr("href"));
+                    release.Title = qDetailsLink.Attr("title").Trim();
+                    release.Comments = new Uri(SiteLink + qDetailsLink.Attr("href"));
+                    release.Guid = release.Link;
+
+                    var sizeStr = qSize.Text();
+                    release.Size = ReleaseInfo.GetBytes(sizeStr);
+
+                    release.Seeders = ParseUtil.CoerceInt(qSeeders.Text());
+                    release.Peers = ParseUtil.CoerceInt(qLeechers.Text()) + release.Seeders;
+
+                    var dateStr = qTimeAgo.Text();
+                    release.PublishDate = DateTimeUtil.FromTimeAgo(dateStr);
+
+                    var files = qRow.Find("td:nth-child(4)").Text();
+                    release.Files = ParseUtil.CoerceInt(files);
+
+                    var grabs = qRow.Find("td:nth-child(9)").Text();
+                    release.Grabs = ParseUtil.CoerceInt(grabs);
+
+                    var ka = qRow.Next();
+                    var DLFactor = ka.Find("table > tbody > tr:nth-child(3) > td:nth-child(2)").Text().Replace("X", "");
+                    var ULFactor = ka.Find("table > tbody > tr:nth-child(3) > td:nth-child(1)").Text().Replace("X", "");
+                    release.DownloadVolumeFactor = ParseUtil.CoerceDouble(DLFactor);
+                    release.UploadVolumeFactor = ParseUtil.CoerceDouble(ULFactor);
+
+                    releases.Add(release);
+                }
+            }
+            catch (Exception ex)
+            {
+                OnParseError(results.Content, ex);
+            }
+
+            return releases;
+        }
+    }
+}

src/Jackett/Indexers/Fuzer.cs

@@ -241,6 +241,17 @@ namespace Jackett.Indexers
                     }
 
                     release.Category = MapTrackerCatToNewznab(category);
+
+                    var grabs = qRow.Find("td:nth-child(6)").Text();
+                    release.Grabs = ParseUtil.CoerceInt(grabs);
+
+                    if (qRow.Find("img[src=\"/images/FL.png\"]").Length >= 1)
+                        release.DownloadVolumeFactor = 0;
+                    else
+                        release.DownloadVolumeFactor = 1;
+
+                    release.UploadVolumeFactor = 1;
+
                     releases.Add(release);
                 }
             }

src/Jackett/Indexers/GFTracker.cs

@@ -63,16 +63,19 @@ namespace Jackett.Indexers
         {
             var loginPage = await RequestStringWithCookies(StartPageUrl, string.Empty);
             CQ cq = loginPage.Content;
-            var result = new ConfigurationDataRecaptchaLogin();
+            var result = this.configData;
             CQ recaptcha = cq.Find(".g-recaptcha").Attr("data-sitekey");
             if(recaptcha.Length != 0)   // recaptcha not always present in login form, perhaps based on cloudflare uid or just phase of the moon
             {
                 result.CookieHeader.Value = loginPage.Cookies;
                 result.Captcha.SiteKey = cq.Find(".g-recaptcha").Attr("data-sitekey");
+                result.Captcha.Version = "2";
                 return result;
             } else
             {
                 var stdResult = new ConfigurationDataBasicLogin();
+                stdResult.Username.Value = configData.Username.Value;
+                stdResult.Password.Value = configData.Password.Value;
                 stdResult.CookieHeader.Value = loginPage.Cookies;
                 return stdResult;
             }           

src/Jackett/Indexers/GhostCity.cs

@@ -0,0 +1,179 @@
+using CsQuery;
+using Jackett.Models;
+using Jackett.Services;
+using Jackett.Utils;
+using Jackett.Utils.Clients;
+using Newtonsoft.Json.Linq;
+using NLog;
+using System;
+using System.Collections.Generic;
+using System.Collections.Specialized;
+using System.Globalization;
+using System.Linq;
+using System.Threading.Tasks;
+using Jackett.Models.IndexerConfig;
+
+namespace Jackett.Indexers
+{
+    public class GhostCity : BaseIndexer, IIndexer
+    {
+        string LoginUrl { get { return SiteLink + "takelogin.php"; } }
+        string BrowsePage { get { return SiteLink + "browse.php"; } }
+
+        new ConfigurationDataBasicLoginWithRSSAndDisplay configData
+        {
+            get { return (ConfigurationDataBasicLoginWithRSSAndDisplay)base.configData; }
+            set { base.configData = value; }
+        }
+
+        public GhostCity(IIndexerManagerService i, IWebClient wc, Logger l, IProtectionService ps)
+                : base(name: "Ghost City",
+                description: "A German general tracker",
+                link: "http://ghostcity.dyndns.info/",
+                caps: TorznabUtil.CreateDefaultTorznabTVCaps(),
+                manager: i,
+                client: wc,
+                logger: l,
+                p: ps,
+                configData: new ConfigurationDataBasicLoginWithRSSAndDisplay())
+        {
+            this.configData.DisplayText.Value = "Only the results from the first search result page are shown, adjust your profile settings to show the maximum.";
+            this.configData.DisplayText.Name = "Notice";
+            AddMultiCategoryMapping(TorznabCatType.TVAnime, 8, 34, 35, 36);
+            AddMultiCategoryMapping(TorznabCatType.TVDocumentary, 12, 44, 106, 45, 46, 47);
+            AddMultiCategoryMapping(TorznabCatType.Console, 92, 93, 95, 96, 97);
+            AddMultiCategoryMapping(TorznabCatType.ConsoleNDS, 92);
+            AddMultiCategoryMapping(TorznabCatType.ConsolePS3, 95);
+            AddMultiCategoryMapping(TorznabCatType.ConsolePS4, 95);
+            AddMultiCategoryMapping(TorznabCatType.ConsolePS4, 95);
+            AddMultiCategoryMapping(TorznabCatType.ConsolePSP, 95);
+            AddMultiCategoryMapping(TorznabCatType.ConsoleXbox, 97);
+            AddMultiCategoryMapping(TorznabCatType.ConsoleXbox360, 97);
+            AddMultiCategoryMapping(TorznabCatType.ConsoleXBOX360DLC, 97);
+            AddMultiCategoryMapping(TorznabCatType.ConsoleXboxOne, 97);
+            AddMultiCategoryMapping(TorznabCatType.ConsoleWii, 96);
+            AddMultiCategoryMapping(TorznabCatType.PC, 20, 94, 40);
+            AddMultiCategoryMapping(TorznabCatType.PCGames, 94);
+            AddMultiCategoryMapping(TorznabCatType.PCMac, 39);
+            AddMultiCategoryMapping(TorznabCatType.PCPhoneOther, 37, 38);
+            AddMultiCategoryMapping(TorznabCatType.TVSport, 22, 98, 99, 100, 101);
+            AddMultiCategoryMapping(TorznabCatType.Movies, 68, 69, 70, 102, 104, 103, 72, 71, 73, 74, 75, 77, 78, 79);
+            AddMultiCategoryMapping(TorznabCatType.MoviesSD, 68, 69, 102, 104, 103, 72, 71, 73, 74);
+            AddMultiCategoryMapping(TorznabCatType.MoviesHD, 75, 76, 77, 78, 79);
+            AddMultiCategoryMapping(TorznabCatType.MoviesOther, 73);
+            AddMultiCategoryMapping(TorznabCatType.MoviesBluRay, 70);
+            AddMultiCategoryMapping(TorznabCatType.MoviesDVD, 102, 104, 103, 72, 71);
+            AddMultiCategoryMapping(TorznabCatType.Movies3D, 69);
+            AddMultiCategoryMapping(TorznabCatType.AudioVideo, 109);
+            AddMultiCategoryMapping(TorznabCatType.TV, 8, 34, 35, 36, 23, 90, 88, 107, 89);
+            AddMultiCategoryMapping(TorznabCatType.TVHD, 107);
+            AddMultiCategoryMapping(TorznabCatType.TVSD, 89);
+            AddMultiCategoryMapping(TorznabCatType.XXX, 25);
+            AddMultiCategoryMapping(TorznabCatType.TVDocumentary, 88);
+            AddMultiCategoryMapping(TorznabCatType.AudioAudiobook, 84);
+            AddMultiCategoryMapping(TorznabCatType.BooksEbook, 83);
+            AddMultiCategoryMapping(TorznabCatType.BooksMagazines, 85);
+            AddMultiCategoryMapping(TorznabCatType.BooksOther, 108);
+            AddMultiCategoryMapping(TorznabCatType.Other, 3, 93, 24);
+        }
+
+        public async Task<IndexerConfigurationStatus> ApplyConfiguration(JToken configJson)
+        {
+            configData.LoadValuesFromJson(configJson);
+
+            var pairs = new Dictionary<string, string> {
+                { "username", configData.Username.Value },
+                { "password", configData.Password.Value }
+            };
+
+            var result1 = await RequestLoginAndFollowRedirect(LoginUrl, pairs, null, true, null, SiteLink);
+            CQ result1Dom = result1.Content;
+            var link = result1Dom[".trow2 a"].First();
+            var result2 = await RequestStringWithCookies(link.Attr("href"), result1.Cookies);
+            CQ result2Dom = result2.Content;
+
+            await ConfigureIfOK(result1.Cookies, result2.Content.Contains("/logout.php"), () =>
+            {
+                var errorMessage = "Login failed.";
+                throw new ExceptionWithConfigData(errorMessage, configData);
+            });
+            return IndexerConfigurationStatus.RequiresTesting;
+        }
+
+        public async Task<IEnumerable<ReleaseInfo>> PerformQuery(TorznabQuery query)
+        {
+            var releases = new List<ReleaseInfo>();
+            var searchString = query.GetQueryString();
+            var searchUrl = BrowsePage;
+            var queryCollection = new NameValueCollection();
+
+            queryCollection.Add("do", "search");
+
+            if (!string.IsNullOrWhiteSpace(searchString))
+            {
+                queryCollection.Add("keywords", searchString);
+            }
+
+            queryCollection.Add("search_type", "t_name");
+
+            // FIXME: Tracker doesn't support multi category search
+            foreach (var cat in MapTorznabCapsToTrackers(query))
+            {
+                queryCollection.Add("category", cat);
+            }
+
+            if (queryCollection.Count > 0)
+            {
+                searchUrl += "?" + queryCollection.GetQueryString();
+            }
+
+            var results = await RequestStringWithCookiesAndRetry(searchUrl);
+
+            try
+            {
+                CQ dom = results.Content;
+
+                var rows = dom["#sortabletable tr"];
+                foreach (var row in rows.Skip(1))
+                {
+                    var release = new ReleaseInfo();
+                    var qRow = row.Cq();
+                    release.Title = qRow.Find(".tooltip-content div").First().Text();
+                    if (string.IsNullOrWhiteSpace(release.Title))
+                        continue;
+                    release.Description = qRow.Find(".tooltip-content div").Get(1).InnerText.Trim();
+
+                    var qLink = row.Cq().Find("td:eq(2) a:eq(0)");
+                    release.Link = new Uri(qLink.Attr("href"));
+                    release.Guid = release.Link;
+                    release.Comments = new Uri(qRow.Find(".tooltip-target a").First().Attr("href"));
+
+                    var dateString = qRow.Find("td:eq(1) div").Last().Children().Remove().End().Text().Trim();
+                    release.PublishDate = DateTime.ParseExact(dateString, "dd-MM-yyyy HH:mm", CultureInfo.InvariantCulture);
+
+                    var sizeStr = qRow.Find("td:eq(4)").Text().Trim();
+                    release.Size = ReleaseInfo.GetBytes(sizeStr);
+
+                    release.Seeders = ParseUtil.CoerceInt(qRow.Find("td:eq(6)").Text().Trim());
+                    release.Peers = ParseUtil.CoerceInt(qRow.Find("td:eq(7)").Text().Trim()) + release.Seeders;
+
+                    var catLink = row.Cq().Find("td:eq(0) a").First().Attr("href");
+                    var catSplit = catLink.IndexOf("category=");
+                    if (catSplit > -1)
+                    {
+                        catLink = catLink.Substring(catSplit + 9);
+                    }
+
+                    release.Category = MapTrackerCatToNewznab(catLink);
+                    releases.Add(release);
+                }
+            }
+            catch (Exception ex)
+            {
+                OnParseError(results.Content, ex);
+            }
+
+            return releases;
+        }
+    }
+}

src/Jackett/Indexers/HD4Free.cs

@@ -0,0 +1,349 @@
+using CsQuery;
+using Jackett.Models;
+using Jackett.Services;
+using Jackett.Utils;
+using Jackett.Utils.Clients;
+using Newtonsoft.Json.Linq;
+using NLog;
+using System;
+using System.Collections.Generic;
+using System.Linq;
+using System.Threading.Tasks;
+using Jackett.Models.IndexerConfig;
+
+using AngleSharp.Parser.Html;
+
+namespace Jackett.Indexers
+{
+    public class HD4Free : BaseIndexer, IIndexer
+    {
+        private string SearchUrl { get { return SiteLink + "ajax/initial_recall.php"; } }
+        private string LoginUrl { get { return SiteLink + "login.php"; } }
+        private string TakeLoginUrl { get { return SiteLink + "takelogin.php"; } }
+
+        new ConfigurationDataRecaptchaLogin configData
+        {
+            get { return (ConfigurationDataRecaptchaLogin)base.configData; }
+            set { base.configData = value; }
+        }
+
+        public HD4Free(IIndexerManagerService i, Logger l, IWebClient w, IProtectionService ps)
+            : base(name: "HD4Free",
+                description: "A HD trackers",
+                link: "https://hd4free.xyz/",
+                caps: new TorznabCapabilities(),
+                manager: i,
+                client: w,
+                logger: l,
+                p: ps,
+                configData: new ConfigurationDataRecaptchaLogin())
+        {
+            AddCategoryMapping(42, TorznabCatType.MoviesSD); // LEGi0N 480p
+            AddCategoryMapping(17, TorznabCatType.MoviesHD); // LEGi0N  720p 
+            AddCategoryMapping(16, TorznabCatType.MoviesHD); // LEGi0N  1080p 
+            AddCategoryMapping(84, TorznabCatType.Movies3D); // LEGi0N 3D 1080p
+            AddCategoryMapping(31, TorznabCatType.MoviesOther); // LEGi0N  REMUX
+            AddCategoryMapping(70, TorznabCatType.MoviesBluRay); // LEGi0N BD BD25 & BD50
+            AddCategoryMapping(55, TorznabCatType.Movies); // LEGi0N  Movie/TV PACKS
+            AddCategoryMapping(60, TorznabCatType.Other); // shadz shack
+            AddCategoryMapping(85, TorznabCatType.MoviesHD); // MarGe 720p
+            AddCategoryMapping(86, TorznabCatType.MoviesHD); // MarGe 1080p
+            AddCategoryMapping(73, TorznabCatType.MoviesBluRay); // GF44 BD-50
+            AddCategoryMapping(74, TorznabCatType.MoviesBluRay); // GF44 BD-25
+            AddCategoryMapping(88, TorznabCatType.MoviesBluRay); // taterzero BD50
+            AddCategoryMapping(89, TorznabCatType.MoviesBluRay); // taterzero BD25
+            AddCategoryMapping(90, TorznabCatType.Movies3D); // taterzero 3D BD
+            AddCategoryMapping(39, TorznabCatType.MoviesBluRay); // Bluray REMUX
+            AddCategoryMapping(38, TorznabCatType.MoviesBluRay); // Bluray 
+            AddCategoryMapping(75, TorznabCatType.MoviesBluRay); // Bluray 25
+            AddCategoryMapping(36, TorznabCatType.MoviesHD); // Encodes 720p
+            AddCategoryMapping(35, TorznabCatType.MoviesHD); // Encodes 1080p
+            AddCategoryMapping(45, TorznabCatType.Movies3D); // 1080p 3D Encodes
+            AddCategoryMapping(77, TorznabCatType.MoviesHD); // WEB-DL 720p
+            AddCategoryMapping(78, TorznabCatType.MoviesHD); // WEB-DL 1080p
+            AddCategoryMapping(83, TorznabCatType.MoviesDVD); // DVD 5/9's
+            AddCategoryMapping(47, TorznabCatType.Movies); // General x264
+            AddCategoryMapping(58, TorznabCatType.Movies); // General XViD
+            AddCategoryMapping(66, TorznabCatType.Movies); // x265 HEVC
+            AddCategoryMapping(34, TorznabCatType.MoviesHD); // 4K
+            AddCategoryMapping(61, TorznabCatType.Movies); // MOViE PACKS
+            AddCategoryMapping(44, TorznabCatType.TVHD); // HDTV 720p
+            AddCategoryMapping(43, TorznabCatType.TVHD); // HDTV 1080p
+            AddCategoryMapping(41, TorznabCatType.TVHD); // WEB-DL 720p TV
+            AddCategoryMapping(40, TorznabCatType.TVHD); // WEB-DL 1080p TV
+            AddCategoryMapping(52, TorznabCatType.TVHD); // 720p TV BluRay
+            AddCategoryMapping(53, TorznabCatType.TVHD); // 1080p TV BluRay
+            AddCategoryMapping(62, TorznabCatType.TVHD); // HDTV Packs
+            AddCategoryMapping(82, TorznabCatType.TVSD); // SDTV TV PACKS
+            AddCategoryMapping(63, TorznabCatType.PC0day); // Apps Windows
+            AddCategoryMapping(57, TorznabCatType.PCMac); // Appz Mac
+            AddCategoryMapping(72, TorznabCatType.AudioAudiobook); // Audio Books
+            AddCategoryMapping(71, TorznabCatType.Books); // Ebooks
+            AddCategoryMapping(46, TorznabCatType.AudioLossless); // FLAC/Lossless
+            AddCategoryMapping(81, TorznabCatType.AudioMP3); // MP3 Music
+            AddCategoryMapping(87, TorznabCatType.AudioVideo); // HD MUSiC ViDEOS
+            AddCategoryMapping(32, TorznabCatType.Other); // Covers And Artwork
+            AddCategoryMapping(50, TorznabCatType.XXX); // Porn XXX
+        }
+
+        public override async Task<ConfigurationData> GetConfigurationForSetup()
+        {
+            var loginPage = await RequestStringWithCookies(LoginUrl, configData.CookieHeader.Value);
+            CQ cq = loginPage.Content;
+            string recaptchaSiteKey = cq.Find(".g-recaptcha").Attr("data-sitekey");
+            var result = this.configData;
+            result.CookieHeader.Value = loginPage.Cookies;
+            result.Captcha.SiteKey = recaptchaSiteKey;
+            result.Captcha.Version = "2";
+            return result;
+        }
+
+        public async Task<IndexerConfigurationStatus> ApplyConfiguration(JToken configJson)
+        {
+            configData.LoadValuesFromJson(configJson);
+            var pairs = new Dictionary<string, string> {
+                { "returnto" , "/" },
+                { "username", configData.Username.Value },
+                { "password", configData.Password.Value },
+                { "g-recaptcha-response", configData.Captcha.Value },
+                { "submitme", "Login" }
+            };
+
+            if (!string.IsNullOrWhiteSpace(configData.Captcha.Cookie))
+            {
+                // Cookie was manually supplied
+                CookieHeader = configData.Captcha.Cookie;
+                try
+                {
+                    var results = await PerformQuery(new TorznabQuery());
+                    if (!results.Any())
+                    {
+                        throw new Exception("Your cookie did not work");
+                    }
+
+                    SaveConfig();
+                    IsConfigured = true;
+                    return IndexerConfigurationStatus.Completed;
+                }
+                catch (Exception e)
+                {
+                    IsConfigured = false;
+                    throw new Exception("Your cookie did not work: " + e.Message);
+                }
+            }
+
+            var result = await RequestLoginAndFollowRedirect(TakeLoginUrl, pairs, null, true, SiteLink, LoginUrl);
+
+            await ConfigureIfOK(result.Cookies, result.Content.Contains("logout.php"), () =>
+            {
+                CQ dom = result.Content;
+                var messageEl = dom["table.main > tbody > tr > td > table > tbody > tr > td"];
+                var errorMessage = messageEl.Text().Trim();
+                if (string.IsNullOrWhiteSpace(errorMessage))
+                    errorMessage = result.Content;
+                throw new ExceptionWithConfigData(errorMessage, configData);
+            });
+            
+            return IndexerConfigurationStatus.RequiresTesting;
+        }
+
+        public async Task<IEnumerable<ReleaseInfo>> PerformQuery(TorznabQuery query)
+        {
+            List<ReleaseInfo> releases = new List<ReleaseInfo>();
+
+            var pairs = new Dictionary<string, string>();
+            var searchString = query.GetQueryString();
+            var searchUrl = SearchUrl;
+
+            pairs.Add("draw", "1");
+            pairs.Add("columns[0][data]", "");
+            pairs.Add("columns[0][name]", "");
+            pairs.Add("columns[0][searchable]", "false");
+            pairs.Add("columns[0][orderable]", "false");
+            pairs.Add("columns[0][search][value]", "");
+            pairs.Add("columns[0][search][regex]", "false");
+            pairs.Add("columns[1][data]", "id");
+            pairs.Add("columns[1][name]", "");
+            pairs.Add("columns[1][searchable]", "false");
+            pairs.Add("columns[1][orderable]", "true");
+            pairs.Add("columns[1][search][value]", "");
+            pairs.Add("columns[1][search][regex]", "false");
+            pairs.Add("columns[2][data]", "cat");
+            pairs.Add("columns[2][name]", "");
+            pairs.Add("columns[2][searchable]", "false");
+            pairs.Add("columns[2][orderable]", "true");
+            pairs.Add("columns[2][search][value]", "");
+            pairs.Add("columns[2][search][regex]", "false");
+            pairs.Add("columns[3][data]", "name");
+            pairs.Add("columns[3][name]", "");
+            pairs.Add("columns[3][searchable]", "true");
+            pairs.Add("columns[3][orderable]", "true");
+            pairs.Add("columns[3][search][value]", "");
+            pairs.Add("columns[3][search][regex]", "false");
+            pairs.Add("columns[4][data]", "username");
+            pairs.Add("columns[4][name]", "");
+            pairs.Add("columns[4][searchable]", "true");
+            pairs.Add("columns[4][orderable]", "true");
+            pairs.Add("columns[4][search][value]", "");
+            pairs.Add("columns[4][search][regex]", "false");
+            pairs.Add("columns[5][data]", "cat-image");
+            pairs.Add("columns[5][name]", "");
+            pairs.Add("columns[5][searchable]", "false");
+            pairs.Add("columns[5][orderable]", "true");
+            pairs.Add("columns[5][search][value]", "");
+            pairs.Add("columns[5][search][regex]", "false");
+            pairs.Add("columns[6][data]", "cat-name");
+            pairs.Add("columns[6][name]", "");
+            pairs.Add("columns[6][searchable]", "false");
+            pairs.Add("columns[6][orderable]", "true");
+            pairs.Add("columns[6][search][value]", "");
+            pairs.Add("columns[6][search][regex]", "false");
+            pairs.Add("columns[7][data]", "imdbid");
+            pairs.Add("columns[7][name]", "");
+            pairs.Add("columns[7][searchable]", "true");
+            pairs.Add("columns[7][orderable]", "true");
+            pairs.Add("columns[7][search][value]", "");
+            pairs.Add("columns[7][search][regex]", "false");
+            pairs.Add("columns[8][data]", "genre");
+            pairs.Add("columns[8][name]", "");
+            pairs.Add("columns[8][searchable]", "false");
+            pairs.Add("columns[8][orderable]", "true");
+            pairs.Add("columns[8][search][value]", "");
+            pairs.Add("columns[8][search][regex]", "false");
+            pairs.Add("columns[9][data]", "added");
+            pairs.Add("columns[9][name]", "");
+            pairs.Add("columns[9][searchable]", "false");
+            pairs.Add("columns[9][orderable]", "true");
+            pairs.Add("columns[9][search][value]", "");
+            pairs.Add("columns[9][search][regex]", "false");
+            pairs.Add("columns[10][data]", "size");
+            pairs.Add("columns[10][name]", "");
+            pairs.Add("columns[10][searchable]", "false");
+            pairs.Add("columns[10][orderable]", "true");
+            pairs.Add("columns[10][search][value]", "");
+            pairs.Add("columns[10][search][regex]", "false");
+            pairs.Add("columns[11][data]", "rating");
+            pairs.Add("columns[11][name]", "");
+            pairs.Add("columns[11][searchable]", "false");
+            pairs.Add("columns[11][orderable]", "true");
+            pairs.Add("columns[11][search][value]", "");
+            pairs.Add("columns[11][search][regex]", "false");
+            pairs.Add("columns[12][data]", "comments");
+            pairs.Add("columns[12][name]", "");
+            pairs.Add("columns[12][searchable]", "false");
+            pairs.Add("columns[12][orderable]", "true");
+            pairs.Add("columns[12][search][value]", "");
+            pairs.Add("columns[12][search][regex]", "false");
+            pairs.Add("columns[13][data]", "numfiles");
+            pairs.Add("columns[13][name]", "");
+            pairs.Add("columns[13][searchable]", "false");
+            pairs.Add("columns[13][orderable]", "true");
+            pairs.Add("columns[13][search][value]", "");
+            pairs.Add("columns[13][search][regex]", "false");
+            pairs.Add("columns[14][data]", "seeders");
+            pairs.Add("columns[14][name]", "");
+            pairs.Add("columns[14][searchable]", "false");
+            pairs.Add("columns[14][orderable]", "true");
+            pairs.Add("columns[14][search][value]", "");
+            pairs.Add("columns[14][search][regex]", "false");
+            pairs.Add("columns[15][data]", "leechers");
+            pairs.Add("columns[15][name]", "");
+            pairs.Add("columns[15][searchable]", "false");
+            pairs.Add("columns[15][orderable]", "true");
+            pairs.Add("columns[15][search][value]", "");
+            pairs.Add("columns[15][search][regex]", "false");
+            pairs.Add("columns[16][data]", "to_go");
+            pairs.Add("columns[16][name]", "");
+            pairs.Add("columns[16][searchable]", "false");
+            pairs.Add("columns[16][orderable]", "true");
+            pairs.Add("columns[16][search][value]", "");
+            pairs.Add("columns[16][search][regex]", "false");
+            pairs.Add("columns[17][data]", "genre");
+            pairs.Add("columns[17][name]", "");
+            pairs.Add("columns[17][searchable]", "true");
+            pairs.Add("columns[17][orderable]", "true");
+            pairs.Add("columns[17][search][value]", "");
+            pairs.Add("columns[17][search][regex]", "false");
+            pairs.Add("order[0][column]", "9");
+            pairs.Add("order[0][dir]", "desc");
+            pairs.Add("start", "0");
+            pairs.Add("length", "100");
+            pairs.Add("visible", "1");
+            pairs.Add("uid", "-1");
+            pairs.Add("genre", "");
+            
+            pairs.Add("cats", string.Join(",+", MapTorznabCapsToTrackers(query)));
+
+            if (!string.IsNullOrWhiteSpace(searchString))
+            {
+                pairs.Add("search[value]", searchString);
+                pairs.Add("search[regex]", "false");
+            }
+
+            var results = await PostDataWithCookiesAndRetry(searchUrl, pairs);
+
+            try
+            {
+                var json = JObject.Parse(results.Content);
+                foreach (var row in json["data"])
+                {
+                    var release = new ReleaseInfo();
+                    release.MinimumRatio = 1;
+                    release.MinimumSeedTime = 72 * 24 * 60 * 60;
+
+                    var hParser = new HtmlParser();
+                    var hName = hParser.Parse(row["name"].ToString());
+                    var hComments = hParser.Parse(row["comments"].ToString());
+                    var hNumfiles = hParser.Parse(row["numfiles"].ToString());
+                    var hSeeders = hParser.Parse(row["seeders"].ToString());
+                    var hLeechers = hParser.Parse(row["leechers"].ToString());
+
+                    var hDetailsLink = hName.QuerySelector("a[href^=\"details.php?id=\"]");
+                    var hCommentsLink = hComments.QuerySelector("a");
+                    var hDownloadLink = hName.QuerySelector("a[title=\"Download Torrent\"]");
+
+                    release.Title = hDetailsLink.TextContent;
+                    release.Comments = new Uri(SiteLink + hCommentsLink.GetAttribute("href"));
+                    release.Link = new Uri(SiteLink + hDownloadLink.GetAttribute("href"));
+                    release.Guid = release.Link;
+
+                    release.Description = row["genre"].ToString();
+
+                    var poster = row["poster"].ToString();
+                    if(!string.IsNullOrWhiteSpace(poster))
+                    {
+                        release.BannerUrl = new Uri(SiteLink + poster);
+                    }
+
+                    release.Size = ReleaseInfo.GetBytes(row["size"].ToString());
+                    var imdbId = row["imdbid"].ToString();
+                    if (imdbId.StartsWith("tt"))
+                        release.Imdb = ParseUtil.CoerceLong(imdbId.Substring(2));
+
+                    var added = row["added"].ToString().Replace("<br>", " ");
+                    release.PublishDate = DateTimeUtil.FromUnknown(added);
+
+                    var catid = row["catid"].ToString();
+                    release.Category = MapTrackerCatToNewznab(catid);
+
+                    release.Seeders = ParseUtil.CoerceInt(hSeeders.QuerySelector("a").TextContent);
+                    release.Peers = ParseUtil.CoerceInt(hLeechers.QuerySelector("a").TextContent) + release.Seeders;
+
+                    release.Files = ParseUtil.CoerceInt(hNumfiles.QuerySelector("a").TextContent);
+
+                    release.DownloadVolumeFactor = 1;
+                    release.UploadVolumeFactor = 1;
+
+                    releases.Add(release);
+                    
+                }
+            }
+            catch (Exception ex)
+            {
+                OnParseError(results.Content, ex);
+            }
+
+            return releases;
+        }
+    }
+}