“This thing is broken!” - your QA engineer
“It worked last week!” - you, probably
“Can you get it working then?” - everyone else
Sound familiar? Any of you who’ve been in software engineering for more than a few months have undoubtedly been told this by a user, someone on your QA team, or even a coworker. You sit down at your desk, run through the steps - and sure enough! That thing that was working last week no longer works. But why not?
You may be able to clearly see an issue. If not, you could consider using the
history of changes as given by your version control system. Commands like
git log are useful for figuring out why and when things changed (you are using
good commit messages, right?) but as you know things are not always that simple.
Sometimes a change from another part of the code impacts your feature in an
unexpected way and breaks things. So, having spent some time looking at
the history of the file with the new bug, you move up and pull up the history
of your whole project. Hundreds of commits. Ugh.
Good Commit Messages
Your task will be much simpler if your team is making good use of commit messages. Hopefully the list of commits you’re looking at includes details about what is changing, and not things like “Fixed the bug”. “Fixed the login button on the fingerprint login popup” would be much better, and it’s easy to see how this would be more useful for someone looking for a potential change sometime in the future. Hopefully your project also builds at each commit - this will be useful later on while we’re searching through them trying to find the point at which things broke.
There has to be a better way than blindly looking through commit messages though - and there is!
Let’s take a look at
$ git bisect start HEAD 134237c40d6f79777a4def9a361cf12730bc5ddd $ <test your code> $ git bisect bad $ <test your code> $ git bisect good $ (repeat)
Bisect is like running a binary search on your codebase for the commit that
introduced a change. It all starts by running
git bisect start. You can pass
in two commits when you
start to indicate the range which you’d like to search.
In the sample above, we pass HEAD, which is the current code you know is broken,
and 134237c4…5ddd which is a commit that we know is
good. You can also pass a tag name instead of a commit SHA (Secure Hash Algorithm),
if that’s easier.
Bisecting: 4 revisions left to test after this (roughly 2 steps) [48ac08bdb0576b326c6fd85c1df47e5726ca077f] Revert "Renamed hello() method to hi()"
Git will inform you that you’re bisecting and give you an indication of how many
more commits you’ll need to test before you’ve found your issue. Run your code
to see if you can reproduce your bug. If you can, use
git bisect bad to tell
git that the commit is broken. If you can’t reproduce the bug use
git bisect good
to give that information to git. Each time, you’ll be left on a new commit to test
with an indication of about how many more steps there are. If you need to skip a
commit for any reason, you can do that with
git bisect skip.
As I mentioned earlier, you’ll want to make sure that every commit that people are
adding to your project will compile on its own. This means that every commit you land
on while using
git bisect will be able to be compiled, which you’ll need to do by
hand. You’re doing the same thing each time - compiling your project and testing it.
There’s a way to speed that up!
~/P/BisectDemo $ git bisect start HEAD 134237c40d6f79777a4def9a361cf12730bc5ddd Bisecting: 4 revisions left to test after this (roughly 2 steps) [48ac08bdb0576b326c6fd85c1df47e5726ca077f] Revert "Renamed hello() method to hi()" ~/P/BisectDemo $ git bisect run ./test-for-args.sh running ./test-for-args.sh Building at 48ac08bdb0576b326c6fd85c1df47e5726ca077f Bisecting: 2 revisions left to test after this (roughly 1 step) [7634a2192d8dcad223dccb2c0adcc5f0aa720373] Added kotlin native version to printout running ./test-for-args.sh Building at 7634a2192d8dcad223dccb2c0adcc5f0aa720373 Bisecting: 0 revisions left to test after this (roughly 0 steps) [25bd37d3125caaa3177be7cbd752856e8c2cc706] Added ability to print out and additional arguments which are passed on the CLI running ./test-for-args.sh Building at 25bd37d3125caaa3177be7cbd752856e8c2cc706 25bd37d3125caaa3177be7cbd752856e8c2cc706 is the first bad commit commit 25bd37d3125caaa3177be7cbd752856e8c2cc706 Author: Matthew S. Runo <email@example.com> Date: Mon Dec 3 15:40:46 2018 -0800 Added ability to print out any additional arguments which are passed on the CLI :040000 040000 6dd3965194bf3e7665e87429bc35e57b3643edd4 56e99d23cbd924d73beb48e42dae090f11f5af38 M src bisect run success ~/P/BisectDemo $
Bisect allows you to run a command after each new checkout of the code, totally
automating the process of finding your bug. You can provide a script that uses exit
codes to communicate good or bad to git. A script that exits with code
0 will indicate
to git that the code is good. Your script can exit with just about any other exit code
(1 to 127, except 125) to indicate the code is bad. If you need to skip a commit, you
can return exit code 125 (but all your commits compile, right?).
In the above sample, my script was as simple as the following. The bug I was trying to find is that command line arguments are being printed out on standard output - maybe someone left in some debugging println statements.
#!/bin/bash echo "Building at $(git rev-parse HEAD)" ./gradlew build &> /dev/null output=$(./build/bin/macos/main/release/executable/BisectDemo.kexe test) if [[ $output == *"test"* ]]; then exit 1 fi exit 0
All I did to find the offending commit was run the following two commands.
# Start the bisect passing in HEAD and the last known good commit git bisect start HEAD 134237c40d6f79777a4def9a361cf12730bc5ddd # Tell git to use my script. git bisect run ./test-for-args.sh
After building the project a few times, I was left on the commit that introduced the code that added the bug. I can look at the diff for the commit, and sure enough - it added some println statements that are printing out the command line arguments. Oops! Looks like they forgot to revert this before opening a pull request for their code.
$ git bisect bad 25bd37d3125caaa3177be7cbd752856e8c2cc706 is the first bad commit commit 25bd37d3125caaa3177be7cbd752856e8c2cc706 Author: Matthew S. Runo <firstname.lastname@example.org> Date: Mon Dec 3 15:40:46 2018 -0800 Added ability to print out any additional arguments which are passed on the CLI :040000 040000 6dd3965194bf3e7665e87429bc35e57b3643edd4 56e99d23cbd924d73beb48e42dae090f11f5af38 M src bisect run success
Eventually you’ll be finished with the search of commits that could cause your issue, and git will give you the exact commit that introduced your bug along with the author and date when it was committed. You can take this information and use it to shame your teammate - or, even better - you can look at the files changed in the commit and see which change is relevant to your bug and resolve the issue. This is another reason why small commits are better - imagine if you’re left on a commit that changes 100 different files! Good luck!
$ git bisect reset Previous HEAD position was a1a43d1070... bugfix - if session timed out, log user out Switched to branch 'master' Your branch is up-to-date with 'origin/master'.
Once you’re ready
git bisect reset will get you back to HEAD so you can fix the issue.
Git bisect is a great tool when you only know that something was working last week
but the actual cause of a bug isn’t immediately clear. By helping git do a binary
bad you’ll soon be at the commit that introduced your bug.
You can speed up the process and get some time for coffee by using
run and letting
a script do the work for you.