GitHub v3 API: Get full commit list for large comparison

Question

I'm trying to use the GitHub v3 API to get the full list of commits between two SHAs, using the comparison API (/repos/:owner/:repo/compare/:base...:head), but it only returns the first 250 commits and I need to get all of them.

I found the API pagination docs, but the compare API doesn't appear to support either the page or per_page parameters, either with counts or SHAs (EDIT: the last_sha parameter doesn't work either). And unlike the commits API, the compare API doesn't seem to return a Link HTTP header.

Is there any way to either increase the commit count limit on the compare API or to fetch a second page of commits?

Answer 1

Try using the parameter sha, for example:

https://api.github.com/repos/junit-team/junit/commits?sha=XXX, where the XXX is the SHA of the last returned commit in the current round of the query. Then iterate this process until you reach the ending SHA.

Sample python code:

startSHA = ''
endSHA = ''
while True:
    url = 'https://api.github.com/repos/junit-team/junit/commits?sha=' + startSHA
    r = requests.get(url)
    data = json.loads(r.text)
    for i in range(len(data)):
        commit = data[i]['sha']
        if commit == endSHA:
            #reach the ending SHA, stop here
        startSHA = commit