RCurl: HTTP Authentication When Site Responds With HTTP 401 Code Without WWW-Authenticate
I'm implementing an R wrapper around PiCloud's REST API using the RCurl package to make HTTP(S) requests to the API server. The API uses Basic HTTP authentication to verify that users have sufficient permissions. The PiCloud documentation gives an example of using the api and authenticating with curl:
$ curl -u 'key:secret_key' https://api.picloud.com/job/?jids=12
This works perfectly. Translating this to an equivalent RCurl's command:
getURL("https://api.picloud.com/job/?jids=12", userpwd="key:secret")
Executing this function I receive the following error message:
[1] "{\"error\": {\"msg\": \"No HTTP Authorization information present\", \"code\": 995, \"retry\": false}}"
Exploring the issue in more depth I found that the HTTP requests made by the curl command included the Authorization field in the first GET command.
RCurl does not do this. Instead it first sends a GET request without the authorization field set. If it receives a 401 error code AND a response with a WWW-Authenticate field it sends another GET request with the Authorization field.
Although the HTTP spec requires messages that return with a 401 error code to include the WWW-Authenticate field PiCloud API messages do not. Thus when calling getURL even with the userpwd option set RCurl will never send a GET request with the authorization field set. As a result authentication will always fail.
Is there any way to force RCurl to set the Authorization field in the first GET message it sends? If not are there any other R packages that I could look into using?
Answer
I've resolved the problem with the help of the author of RCurl, Duncan Lang. The solution is to explicitly set the httpauth option which sets the authentication method to try initially. This works:
getURL("https://api.picloud.com/job/?jids=12", userpwd="key:secret", httpauth = 1L)
httpauth is a bitmask specifying which authentication methods to use. See the HTTP Authentication section of the libcurl tutorial for more details.