Added simple (non-auth) proxy support including processing the second set of headers (server) vs first (proxy). New command line option (-j 127.0.0.1:8888) to set the proxy and port. unfortunatly both -p and -x were already taken

extended refresh header handling to libcurl and safecurl
also some minor tweaks needed to have Curl 'behave' with certain servers by adding accept-language header to Curl
Added ability/template to allow the user to select their own SiteURL (see BitSoup implementation) with a minor update to ConfigurationData class to prevent the "display text" from disappearing on errors.
XSpeeds indexer - updated to handle a case where the returned XML occasionally is null terminated resulting in XML parsing exceptions
BitSoup indexer - added user customizable url selection, including some url validation
BaseIndexer - cleaned up some of my earlier implementation for accumulating cookies.
This commit is contained in:
garreth.jeremiah@gmail.com
2015-12-24 22:26:39 -05:00
parent f6f27e604a
commit f4129dc4a0
11 changed files with 304 additions and 146 deletions

View File

@@ -104,6 +104,28 @@ namespace Jackett.Utils.Clients
case "location":
result.RedirectingTo = header[1];
break;
case "refresh":
if (response.Status == System.Net.HttpStatusCode.ServiceUnavailable)
{
//"Refresh: 8;URL=/cdn-cgi/l/chk_jschl?pass=1451000679.092-1vJFUJLb9R"
var redirval = "";
var value = header[1];
var start = value.IndexOf("=");
var end = value.IndexOf(";");
var len = value.Length;
if (start > -1)
{
redirval = value.Substring(start + 1);
result.RedirectingTo = redirval;
// normally we don't want a serviceunavailable (503) to be a redirect, but that's the nature
// of this cloudflare approach..don't want to alter BaseWebResult.IsRedirect because normally
// it shoudln't include service unavailable..only if we have this redirect header.
result.Status = System.Net.HttpStatusCode.Redirect;
var redirtime = Int32.Parse(value.Substring(0, end));
System.Threading.Thread.Sleep(redirtime * 1000);
}
}
break;
}
}
}