How to get request digest value from provider hosted app?

Question

I am developing SharePoint 2013 Provider hosted app using javascript REST Api. In order to perform create (POST), or update (MERGE) operations on sharepoint items I need to set the 'X-RequestDigest' header with the request.

When in SharePoint-hosted apps I was able to use the http://contoso.sharepoint.com/SharePointHostedApp/_api/contextinfo service to retrieve the request digest value; however, I am having trouble getting that value when in a provider hosted app.

The first difference of provider hosted app is that now we need to make a cross-domain request since we are not running in a sharepoint site, but in a different domain hosted on a different server. To be clear: instead of

$.ajax({
    url: appWebUrl + '/_api/contextinfo',
    method: "POST",
    headers: { "Accept": "application/json; odata=verbose" }
})

I assumed we need to use the SP.RequestExecutor to execute a cross domain request. When I construct the request it looks like the following (I've changed the actual urls to something fake, but basically we're telling the proxy to use the host web has the target and get the /_api/contextinfo endpoint):

https://contoso-6f921c6addc19f.sharepoint.com/ProviderHostedApp/_api/SP.AppContextSite(@target)/contextinfo?@target=%27https://contoso.sharepoint.com%27

However, I receive this error: Cannot find resource for the request contextinfo. meaning that the endpoint does not exist.

I made sure to use the POST method with the correct application/json;odata=verbose headers with an empty body.

How do I get the request digest value from the /_api/contextinfo service to the provider hosted app?

Based on what I've researched:

• We can't use $('#__REQUESTDIGEST').val(); because that is not available to a provider hosted app.
• We need to use some from of cross-domain request since I'm running outside of sharepoint.
• I have tried setting the target of the cross-domain request to both the hostWebUrl and the appWebUrl and both give the same error.

There must be some way to get this value, otherwise we would only be limited to read operations when using JavaScript. Has anyone else solved this using javascript?

Technically I could try to implement the needed services using the CSOM on server and exposing them using WebAPI or WCF but it seem unreasonable to have to implement that.

UPDATE:

I went ahead and tried adding a WebAPI controller which exposes a service that retrieves the request digest value. This actually does retrieve a request digest value; however, when attempting to use this in the header of future calls I receive the error: "The security validation for this page is invalid and might be corrupted. Please use your web browser's Back button to try your operation again." I'm guessing that the request digest value has some referer header information in it which indicates it was requested by the server; however, the future requests made with it are from the browser, and this mismatch might be an acceptable reason for it be invalid.

Few more notes on the attempt at adding the webAPI controller. I based my code off of this example: http://code.msdn.microsoft.com/SharePoint-2013-Perform-335d925b but converted it to use the newer HttpClient. I overloaded the Page_Load method, stored the contextTokenString in a variable that could be accessed by the WebAPI controller then parsed/used it when requesting the contextinfo.

Does anyone know if this is a correct diagnosis of that error? Is there something encoded in the request digest value that would prevent it from be able to be retrieved like I suggested?

I have also opened a related question on the MSDN forums since I'm desperate to find an answer: http://social.msdn.microsoft.com/Forums/sharepoint/en-US/f601fddd-3747-4152-b2d1-4e89f0a771c4/question-about-limitation-of-providerhosted-apps-is-it-possible-to-make-rest-calls-with-javascript?forum=sharepointdevelopmentprevious

I find it very hard to believe this could be a limitation of provider hosted applications, but given all testing I've done, I'm starting to doubt the viability of provider-hosted applications when you want to write in javascript.

Begging for Help!

Answer 1

I realize you've already answered your own question within the context of a provider-hosted app, but for developers like myself who need to access the REST API from a language not based in the .NET framework, (and who cannot write their project as a web app) I'd like to expand on the subject a bit more. I was tasked with writing an iPad app recently that required this functionality, and ended up reverse-engineering the following:

Step 1 - Authentication

Not going to actually cover this, as there are plenty of examples online that demonstrate the more common methods. The Microsoft.SharePoint.Client libraries mostly seem to use claims-based authentication when working with SharePoint Online, with the token being requested through the endpoint found at: https://login.microsoftonline.com/RST2.srf

Step 2 - Acquiring the Request Digest (Dumb Approach)

If you're feeling lazy, you can always take your authenticated cookies, make a GET request to the homepage of the target web, and use a regular expression like:

/(<input (?:[^>]*?)name="?__REQUESTDIGEST"?(?:[^>]*?)\/>)/i

to scrape the HTML from the response. From there, it'd just be a matter of extracting the value attribute for your digest.

Step 2 - Acquiring the Request Digest (SOAP Approach)

The CSOM libraries currently use a SOAP endpoint when acquiring the request digest it uses for its API calls. You can do the same by making a SOAP request to the $(SPWebUrl)/_vti_bin/sites.asmx web service similar to the following:

POST $(SPWebUrl)/_vti_bin/sites.asmx HTTP/1.1
Content-Type: text/xml
SOAPAction: http://schemas.microsoft.com/sharepoint/soap/GetUpdatedFormDigestInformation
X-RequestForceAuthentication: true
Host: $(SPSiteHostname)
Expect: 100-continue
Accept-Encoding: gzip, deflate
Cookie: $(Authenticated Cookies - Either "FedAuth=...; rtFa=..." or "SPOIDCRL=...")
Content-Length: $(Whatever)

<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
    <soap:Body>
        <GetUpdatedFormDigestInformation xmlns="http://schemas.microsoft.com/sharepoint/soap/" />
    </soap:Body>
</soap:Envelope>

When executed successfully, the response body will look something like:

<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
    <soap:Body>
        <GetUpdatedFormDigestInformationResponse xmlns="http://schemas.microsoft.com/sharepoint/soap/">
            <GetUpdatedFormDigestInformationResult>
                <DigestValue>0x1122334455 ... FF,27 Jul 2015 03:06:54 -0000</DigestValue>
                <TimeoutSeconds>1800</TimeoutSeconds>
                <WebFullUrl>$(SPWebUrl)</WebFullUrl>
                <LibraryVersion>16.0.3208.1222</LibraryVersion>
                <SupportedSchemaVersions>14.0.0.0,15.0.0.0</SupportedSchemaVersions>
            </GetUpdatedFormDigestInformationResult>
        </GetUpdatedFormDigestInformationResponse>
    </soap:Body>
</soap:Envelope>

At that point, you can just extract your request digest from the DigestValue block.

Step 2 - Acquiring the Request Digest (REST Approach)

The last approach I'm aware of uses an OData request made to the $(SPWebUrl)/_api/contextinfo endpoint:

POST $(SPWebUrl)/_api/contextinfo HTTP/1.1
Host: $(SPSiteHostname)
DataServiceVersion: 3.0
Accept: application/json; odata=nometadata
Content-Type: application/json; odata=verbose
Cookie: $(Authenticated Cookies)
Content-Length: 2

{}

When executed successfully, the response body will look like the following:

{
    "FormDigestTimeoutSeconds" : 1800,
    "FormDigestValue" : "0x1122334455 ... FF,27 Jul 2015 03:06:54 -0000",
    "LibraryVersion" : "16.0.4230.1217",
    "SiteFullUrl" : "$(SPSiteUrl)",
    "SupportedSchemaVersions" : ["14.0.0.0", "15.0.0.0"],
    "WebFullUrl" : "$(SPWebUrl)"
}

The request digest can then be extracted from the FormDigestValue property.

Step 2 - Acquiring the Request Digest (CSOM Approach)

If you're using CSOM, you have functionality for dealing with this built-in. (probably JSOM, too, unless it uses the __REQUESTDIGEST input) Microsoft.SharePoint.Client.ClientContext uses the SOAP approach internally to manage its request digest and publicly exposes this functionality through its GetFormDigestDirect method.

ClientContext clientContext = new ClientContext(webUrl);
// ...
FormDigestInfo formDigest = clientContext.GetFormDigestDirect();

// X-RequestDigest header value
string headerValue = formDigest.DigestValue;

// Digest expiration
DateTime expirationDate = formDigest.Expiration;

Usage Notes: While ClientContext maintains and reuses a cached form digest for its requests, this method does not give you access to that cached value. Instead, this method requests a brand new form digest with each call, so you'll want to setup your own caching mechanism in order to re-use unexpired digests across multiple requests.

Step 2 - Acquiring the Request Digest (JSOM Approach)

If you're using the JSOM API and don't have access to a __REQUESTDIGEST input value, you can access the ClientContext's cached digest with the following extensions. (Thanks to bdimag for pointing out the cache)

Step 3 - Acquiring New Request Digests

Assuming you use the request digest before the TimeoutSeconds have elapsed, a valid REST request made like the following:

POST $(SPWebUrl)/_api/web/lists/getByTitle('MyList')/getchanges HTTP/1.1
Host: $(SPSiteHostname)
DataServiceVersion: 3.0
Accept: application/json; odata=nometadata
Content-Type: application/json; odata=verbose
X-RequestDigest: $(Request Digest)
Cookie: $(Authenticated Cookies)
Content-Length: 140

{
    "query" : {
        "__metadata" : {
            "type" : "SP.ChangeQuery"
        },
        "Add" : "True",
        "Item" : "True",
        "Update" : "True"
    }
}

should result in a successful response. If you inspect the headers of that response, you'll find something like:

HTTP/1.1 200 OK
Cache-Control: private, max-age=0
Content-Type: application/json;odata=fullmetadata;streaming=true;charset=utf-8
...
X-RequestDigest: 0xAABBCC...00,03 Sep 2014 18:09:34 -0000
...

Extracting the X-RequestDigest response header will allow you to use it in a subsequent call. (I'm guessing that the timeout starts over from the time of your new response + $(TimeoutSeconds) from the original digest request, but I've yet to confirm)

Unfortunately, the X-RequestDigest header is only returned by REST requests that actually require a request digest. You will not receive the header for requests where a request digest is unrequired, such as: $(SPWebUrl)/_api/web/lists/getByTitle('MyList')/items. Should you find yourself needing a new digest after the original has timed out, you'll need to make another request to the $(SPWebUrl)/_vti_bin/sites.asmx web service.

Step ??? - Handling Errors

A few example responses from when our requests fail:

The following response comes from a REST request made to the $(SPWebUrl)/_api/contextinfo endpoint. (no authentication cookies specified)

HTTP/1.1 403 Forbidden
Cache-Control: private, max-age=0
Content-Type: application/json;odata=nometadata;charset=utf-8
...
Server: Microsoft-IIS/8.5
X-SharePointHealthScore: 0
X-Forms_Based_Auth_Required: $(SPRootSiteUrl)/_forms/default.aspx?ReturnUrl=/_layouts/15/error.aspx&Source=%2f_vti_bin%2fclient.svc%2fcontextinfo
X-Forms_Based_Auth_Return_Url: $(SPRootSiteUrl)/_layouts/15/error.aspx
X-MSDAVEXT_Error: 917656; Access+denied.+Before+opening+files+in+this+location%2c+you+must+first+browse+to+the+web+site+and+select+the+option+to+login+automatically.
DATASERVICEVERSION: 3.0
X-AspNet-Version: 4.0.30319
X-IDCRL_AUTH_PARAMS_V1: IDCRL Type="BPOSIDCRL", EndPoint="$(SiteRelativeUrl)/_vti_bin/idcrl.svc/", RootDomain="sharepoint.com", Policy="MBI"
...
Date: Wed, 12 Aug 2015 02:27:35 GMT
Content-Length: 201

{
    "odata.error" : {
        "code" : "-2147024891, System.UnauthorizedAccessException",
        "message" : {
            "lang" : "en-US",
            "value" : "Access denied. You do not have permission to perform this action or access this resource."
        }
    }
}

Next, a response originating from a REST request made with an expired request digest (Note the X-RequestDigest header specified in the response.. Not sure if that's usable, but it's worth a shot):

HTTP/1.1 403 FORBIDDEN
Cache-Control: private, max-age=0
Content-Type: application/json;odata=fullmetadata;charset=utf-8
...
Server: Microsoft-IIS/8.5
Set-Cookie: rtFa=$(RtfaAuthCookie)
Set-Cookie: FedAuth=$(FedAuth)
X-SharePointHealthScore: 0
X-RequestDigest: 0x19EFFF80617AB2E48B0A9FF0ABA1440B5301E7445F3859177771BF6A39C7E4A74643108D862505A2C99350B0EDB871EF3DDE960BB68060601268818027F04956,12 Aug 2015 02:39:22 -0000
DATASERVICEVERSION: 3.0
X-AspNet-Version: 4.0.30319
...
Date: Wed, 12 Aug 2015 02:39:22 GMT
Content-Length: 253

{
    "odata.error" : {
        "code" : "-2130575251, Microsoft.SharePoint.SPException",
        "message" : {
            "lang" : "en-US",
            "value" : "The security validation for this page is invalid and might be corrupted. Please use your web browser's Back button to try your operation again."
        }
    }
}