Git bisect with merged commits

Question

I have a history that looks like this:

* 3830e61 Add data escaping.              (Bad)
* 0f5e148 Improve function for getting page template.
*   aaf8dc5 Merge branch 'navigation'
|\
| * 3e667f8 Add icons.
| * 43a07b1 Add menu styles.              (Breaks)
| * 107ca95 Add Responsive Nav.           (Good)
* | ea3d736 Add ‘Admin’ notice.
* | 17ca0bb Update placeholder text.
|/
* f52cc34 Add featured image.
* 2abd954 Style placeholders.

I am trying to learn more and git bisect, but am having trouble with this history. I know that 107ca95 is good and 3830e61 is bad. When I run a git bisect, commits 107ca95..3e667f8 are ignored. I happen to know that 43a07b1 is the commit that introduced a regression, but it is never evaluated.

Here is roughly what I've done:

git checkout master
git bisect start
git bisect bad
git bisect good 107ca95
git bisect bad (multiple times)

No matter what I do, 107ca95..3e667f8 are never checked out to be tested.

Is there any way that I can essentially "flatten" the history during the bisect to test those commits? I know I can use an interactive rebase to flatten the history, but I do not want to have to do that.

Answer 1

This is a very old but unanswered question. I decided to investigate, and found that I could show that the behavior of Git is different to what the question says it is. One explanation is that Git improved the algorithm for bisect, or that the questioner made a mistake in marking commits.

I am trying to learn more and git bisect, but am having trouble with this history. I know that 107ca95 is good and 3830e61 is bad. When I run a git bisect, commits 107ca95..3e667f8 are ignored. I happen to know that 43a07b1 is the commit that introduced a regression, but it is never evaluated.

I wrote some code to check whether it is evaluated or not. My test shows that it is evaluated. Run the code below and verify that a commit with message Add menu styles. appears.

Further comments:

• "commits 107ca95..3e667f8 are ignored": Please note, that the commit you marked as "good" will not be evaluated because git already knows it to be good.
• Please read the section "Bisection Algorithm" in this article by Christian Couder. Also the section "Checking merge bases" might be relevant.
• As mentioned above, the question was certainly using a different version then the one I used (Question is from 2013, Git 2.11 is from 2016).

Bisect run output

• Notice that first 'Add Admin notice' is checked (line 4) because that provides the most information. (Read "Checking merge bases" from article mentioned above.)
• From then on, it bisects the linear history as would be expected.

# bad: [d7761d6f146eaca1d886f793ced4315539326866] Add data escaping. (Bad)
# good: [f555d9063a25a20a6ec7c3b0c0504ffe0a997e98] Add Responsive Nav. (Good)
git bisect start 'd7761d6f146eaca1d886f793ced4315539326866' 'f555d9063a25a20a6ec7c3b0c0504ffe0a997e98'
# good: [1b3b7f4952732fec0c68a37d5f313d6f4219e4ae] Add ‘Admin’ notice. (Good)
git bisect good 1b3b7f4952732fec0c68a37d5f313d6f4219e4ae
# bad: [f9a65fe9e6cde4358e5b8ef7569332abfb07675e] Add icons. (Bad)
git bisect bad f9a65fe9e6cde4358e5b8ef7569332abfb07675e
# bad: [165b8a6e5137c40ce8b90911e59d7ec8eec30f46] Add menu styles. (Bad)
git bisect bad 165b8a6e5137c40ce8b90911e59d7ec8eec30f46
# first bad commit: [165b8a6e5137c40ce8b90911e59d7ec8eec30f46] Add menu styles. (Bad)

Code

Run in Python 3, with Git 2.11.0. Command to run: python3 script.py

""" The following code creates a git repository in '/tmp/git-repo' and populates
it with the following commit graph. Each commit has a test.sh which can be used
as input to a git-bisect-run.

The code then tries to find the breaking change automatically.
And prints out the git bisect log.

Written in response to http://stackoverflow.com/questions/17267816/git-bisect-with-merged-commits
to test the claim that '107ca95..3e667f8 are never checked out'.

Needs Python 3!
"""


from itertools import chain
import os.path
import os
import sh

repo = {
0x3830e61:  {'message': "Add data escaping.", 'parents': [    0x0f5e148    ], 'test': False} , # Last:    (Bad)
0x0f5e148: {'message': "Improve function for getting page template.", 'parents': [ 0xaaf8dc5], 'test': False},
0xaaf8dc5: {'message': "Merge branch 'navigation'", 'parents': [ 0x3e667f8, 0xea3d736], 'test': False},
    0x3e667f8: {'message': "Add icons.", 'parents': [  0x43a07b1], 'test': False},
    0x43a07b1: {'message': "Add menu styles.", 'parents': [    0x107ca95], 'test': False}  , # First:       (Breaks)
    0x107ca95: {'message': "Add Responsive Nav.", 'parents': [   0xf52cc34], 'test': True}, # First:        (Good)
  0xea3d736: {'message': "Add ‘Admin’ notice.", 'parents': [ 0x17ca0bb], 'test': True},
  0x17ca0bb: {'message': "Update placeholder text.", 'parents': [  0xf52cc34], 'test': True},
0xf52cc34: {'message': "Add featured image.", 'parents': [  0x2abd954], 'test': True},
0x2abd954: {'message': "Style placeholders.", 'parents': [], 'test': True},
}

bad = 0x3830e61
good = 0x107ca95


def generate_queue(_dag, parents):
    for prev in parents:
        yield prev
        yield from generate_queue(_dag, _dag[prev]['parents'])

def make_queue(_dag, inits):
    """ Converts repo (a DAG) into a queue """
    q = list(generate_queue(_dag, inits))
    q.reverse()
    seen = set()
    r = [x for x in q if not (x in seen or seen.add(x))]
    return r

if __name__ == '__main__':
    pwd = '/tmp/git-repo'
    sh.rm('-r', pwd)
    sh.mkdir('-p', pwd)
    g = sh.git.bake(_cwd=pwd)
    g.init()

    parents = set(chain.from_iterable((repo[c]['parents'] for c in repo)))

    commits = set(repo)
    inits = list(commits - parents)
    queue = make_queue(repo, inits)

    assert len(queue) == len(repo), "queue {} vs repo {}".format(len(queue), len(repo))

    commit_ids = {}
    # Create commits
    for c in queue:
        # Set up repo
        parents = repo[c]['parents']
        if len(parents) > 0:
            g.checkout(commit_ids[parents[0]])
        if len(parents) > 1:
            if len(parents) > 2: raise NotImplementedError('Octopus merges not support yet.')
            g.merge('--no-commit', '-s', 'ours', commit_ids[parents[1]])  # just force to use 'ours' strategy.

        # Make changes
        with open(os.path.join(pwd, 'test.sh'), 'w') as f:
            f.write('exit {:d}\n'.format(0 if repo[c]['test'] else 1))
        os.chmod(os.path.join(pwd, 'test.sh'), 0o0755)
        with open(os.path.join(pwd, 'message'), 'w') as f:
            f.write(repo[c]['message'])
        g.add('test.sh', 'message')
        g.commit('-m', '{msg} ({test})'.format(msg=repo[c]['message'], test='Good' if repo[c]['test'] else 'Bad'))
        commit_ids[c] = g('rev-parse', 'HEAD').strip()

    # Run git-bisect
    g.bisect('start', commit_ids[bad], commit_ids[good])
    g.bisect('run', './test.sh')
    print(g.bisect('log'))