Watch, Follow, &
Connect with Us

For forums, blogs and more please visit our
Developer Tools Community.


Welcome, Guest
Guest Settings
Help

Thread: IdHTTP's Head 403 Forbidden in Bitbuckret / Amazonaws


This question is not answered. Helpful answers available: 2. Correct answers available: 1.


Permlink Replies: 1 - Last Post: Jul 2, 2016 4:06 PM Last Post By: Remy Lebeau (Te...
Michal Mainski

Posts: 3
Registered: 7/26/16
IdHTTP's Head 403 Forbidden in Bitbuckret / Amazonaws  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 2, 2016 4:26 AM
Hello,

I'm trying to get the filename and content length from :

https://bitbucket.org/easyphp/easyphp-webserver/downloads/EasyPHP-Webserver-14.1b2-setup.exe

The url will automatically redirect to:

https://bbuseruploads.s3.amazonaws.com/easyphp/easyphp-webserver/ ........

However, I got HTTP/1.1 403 Forbidden error. I know that 403 means I don't have permission to access the requested URL. But the url doesn't requires a username/password.

The IdHTTP has been set up with the following properties.

AllowCookies = true;
HandleRedirects = true;
MaxAuthRetries = 3;
RedirectMaximum = 15;
IOHandler = IdSSL;
SSLOptions->Mode = sslmClient;
SSLOptions->Method = sslvSSLv23;
SSLOptions->VerifyDepth = 0;
HTTPOptions = hoForceEncodeParams;
ProtocolVersion = pv1_1;
Request Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";
Request Connection = "Keep-Alive";
GIdDefaultUserAgent = "Mozilla/5.0 (Windows NT 6.3; rv:41.0) Gecko/20100101 Firefox/43.0";

The code I'm using is as follows:

String __fastcall GetRemoteFileName(const String URI)
{
    String result;
    try
    {
        TIdHTTP* HTTP = new TIdHTTP(NULL);
        try
        {
            HTTP->Head(URI);
            result = HTTP->Response->RawHeaders->Params["Content-Disposition"]["filename"];
            if (result.IsEmpty())
            {
                result = HTTP->Response->RawHeaders->Params["Content-Type"]["name"];
                if (result.IsEmpty())
                    result = HTTP->URL->Document;
            }
        }
        __finally
        {
            delete HTTP;
        }
    }
    catch(const Exception &ex)
    {
        ShowMessage(const_cast<Exception&>(ex).ToString());
    }
 
    return result;
}


Fyi, the code doesn't produced a 403 Forbidden error when it's using the GET request method. But as you know that by using the HTTP GET request then the file is automatically downloaded. Also, there's NO any error when the file is downloading via web browser, FireFox for example.

So what should I do to eliminate the 403 Forbidden while the code is still using the HEAD method ?.

I need your help to handle this situation.

Thank you very much.

Regards,
Michal
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: IdHTTP's Head 403 Forbidden in Bitbuckret / Amazonaws [Edit]  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Jul 2, 2016 4:06 PM   in response to: Michal Mainski in response to: Michal Mainski
Michal wrote:

However, I got HTTP/1.1 403 Forbidden error. I know that 403 means I
don't have permission to access the requested URL. But the url doesn't
requires a username/password.

No, but the bitbucket.org URL in question does redirect to an AWS URL that
has an AWS signature in it, and an HTTP verb (GET, HEAD, etc) is part of
the signature calculation. The redirect from bitbucket.org is using the
same signature regardless of whether HEAD or GET is used, but the signature
is only valid for GET. That is why HEAD is failing. And since you are using
HEAD, you can't get the error XML that AWS's 403 response would normally
send to explain why the error occured.

Fyi, the code doesn't produced a 403 Forbidden error when it's using
the GET request method.

Yes, because the signature in question is meant for GET.

But as you know that by using the HTTP GET request then the file is
automatically downloaded.

Yes. However, you can tell TIdHTTP.Get() to read and discard the file by
setting the AResponseContent paramter to nil. Your EXE file will still take
a few seconds to download, but at least the request will not fail, and you
will still get the response headers you are looking for.

The only other option I can think of is to either:

1. contact bitbucket.org and report that they are sending bad AWS signatures
for HEAD requests and ask that they fix it.

2. use the TIdHTTP.OnRedirect event to detect a signed redirect to AWS, extract
the relevant values from the redirect, re-calculate a signature that is suitable
for HEAD, and then modify the redirected URL to use that signature instead
of the original. But I'm not sure if you will have access to enough values
to actually do that re-calculation, though.

Also, there's NO any error when the file is downloading via web browser,
FireFox for example.

When you enter a URL in a browser's address bar, the browser sends a GET
request.

--
Remy Lebeau (TeamB)
Legend
Helpful Answer (5 pts)
Correct Answer (10 pts)

Server Response from: ETNAJIVE02