Sharing Cocoa

Learn from my mistakes.

Debug EXC_BAD_ADDRESS in Unit Tests

Some nightmares are worse than others. For me the worst kind, the ones that wake me up at night are problems with no (obvious) solutions. Among this category are random crashes in places where I cannot use my debugging tools.

I have sometimes detected random crashes in my unit tests, some of them were due to problems in the tools, or at least that is what I always said to myself. Clean and rebuild to test, seemed to do the work, and all the tests passed flawlessly, as always.

Well, maybe not! Since I migrated to Xcode 6, my unit tests were crashing far more often than usual (4 out of 5 times). Sometimes even cleaning the build (and I mean Cleaning the build folder) didn't solve the problem. But still in some cases the tests passed without any problems.

The crashes were in different lines and test methods, but they had the following similarities:

  • The reason for crashing was (most times) EXC_BAD_ADDRESS.
  • Most times the test in progress was using OCMock.

At the beginning of this random crashes, I blamed OCMock. After all, the increase in the number of random crashes coincided more or less in time with OCMock version bump to 3.x. As you will see, if you continue reading, some of the problems weren't OCMock's fault, but mine. But it is human nature, to start blaming somebody else before considering your more likely responsibility on the problem. As for the remaining ones, it is yet to be proved.

What does EXC_BAD_ADDRESS mean?

It means that you are trying to do something with a memory position that doesn't contain what is expected. Probably you are messaging an object that isn't where expected (anymore).

Let me give you an example. When you create an object, memory is reserved for holding all its attributes and other relevant data (like the ISA pointer that tells the runtime the class that object belongs to, and thus the messages that it can respond to). If you hold an unsafe_unretained pointer to an object that has been released and send a message to it, the object is accessed via its position in memory (i.e. your now invalid pointer). Since the address that you are using may have been overwritten, it might be pointing to a memory region where the contents don't match the expected structure (ISA pointer among others). Then the runtime determines that this is an unexpeced problem and crashes the app.

So if you have a problem like this what you usually use is the Zombie detection tool. But the tests are compiled in a different bundle and, as far as I know, this tool cannot be used with the test bundle. Too bad!

Using other Xcode functionality

OK. NSZombieEnabled is ignored in the bundle test, so what else could I do? The only solution for a desperate person without tools would be to try to find this behavior by reading the code and figuring out where the mistake is being made. But, come on, we are civilized people. Aren't we? I rather use something more methodic and deterministic to find out where the bug was.

The easiest way to solve this problem is to reduce the number of potential culprits as much as possible. I could have commented out the most suspicious tests. But if you have some more than 400 tests, commenting out tests seems to cumbersome and boring. Instead, a more effective solution is to edit the scheme that I am using and disable whole test modules.

Disable tests with Xcode

So I started by disabling all the tests modules and enabling them one by one. Every time a new module was enabled, I cleaned the build and run the tests three times. My first satisfaction came when I added the tests module for one of the view controllers (Yes, I am one of those who tests his view controllers) and the crashes started to appear.

I removed every other module, and tried again. Yep, the crashes were still there. Still appearing in different lines, but, there nonetheless. Finally some consistency, just the one that I needed to get rid of this awful bug.

My next step was to disable every test method, using the same scheme editor. Then I started to enable each of those tests one by one. As previously, I cleaned the build after each change and the tests were run three times. Some loops later I finally found the (first) culprit. It was the following test method.

- (void) testShowItemsSegueInitializesItemsViewControllerWithSelectedChecklist {
    id tableViewMock = OCMPartialMock(sut.tableView);
    OCMStub([tableViewMock indexPathForSelectedRow]).andReturn([NSIndexPath indexPathForRow:0 inSection:1]);
    sut.tableView = tableViewMock;
    id userDefaultsMock = OCMClassMock([NSUserDefaults class]);
    [sut setValue:userDefaultsMock forKey:@"userDefaults"];
    id itemsViewControllerMock = OCMClassMock([ItemsViewController class]);
    OCMExpect([itemsViewControllerMock setChecklist:OCMOCK_ANY]);
    UIStoryboardSegue *storyboardSegue = [[UIStoryboardSegue alloc] initWithIdentifier:segueShowItems
                                                                                source:sut
                                                                           destination:itemsViewControllerMock];

    [sut prepareForSegue:storyboardSegue sender:nil];

    XCTAssertNoThrow(OCMVerifyAll(itemsViewControllerMock),
                     @"ItemsViewController must be initialized with the selected checklist.");
}

Now that the fog has been removed (and the line highlighted), it might seem obvious to you. It still took me another minute to realize that the value that was being returned by the table view stub wasn't being retained anywhere. The solution couldn't be easier:

- (void) testShowItemsSegueInitializesItemsViewControllerWithSelectedChecklist {
    NSIndexPath *selectedIndexPath = [NSIndexPath indexPathForRow:0 inSection:1];
    id tableViewMock = OCMPartialMock(sut.tableView);
    OCMStub([tableViewMock indexPathForSelectedRow]).andReturn(selectedIndexPath);
    sut.tableView = tableViewMock;
    id userDefaultsMock = OCMClassMock([NSUserDefaults class]);
    [sut setValue:userDefaultsMock forKey:@"userDefaults"];
    id itemsViewControllerMock = OCMClassMock([ItemsViewController class]);
    OCMExpect([itemsViewControllerMock setChecklist:OCMOCK_ANY]);
    UIStoryboardSegue *storyboardSegue = [[UIStoryboardSegue alloc] initWithIdentifier:segueShowItems
                                                                                source:sut
                                                                           destination:itemsViewControllerMock];

    [sut prepareForSegue:storyboardSegue sender:nil];

    XCTAssertNoThrow(OCMVerifyAll(itemsViewControllerMock), @"ItemsViewController must be initialized with the selected checklist.");
}

That ain't all, folks!

After solving that sound mistake, and similar ones in other parts of the code, I thought that my tests were finally fixed. So I run them some more times and discovered that I still got some random crashes with the same error. After some more work, I found that the crashes were appearing in one of two places, either when assigning the mock view to the tableView property of UITableViewController or when the autorelease pool for the test that did that was released. I also found out that the same problem occurs, although less often (approximately 1 out of 20), when inserting a mock in the view property of a plain UIViewController. Let me show you an example of the first case.


- (void) testTableIsNotifiedOfUpdatesBeginningWhenFRCChangesContents {
    id tableViewMock = OCMClassMock([UITableView class]);
    OCMExpect([tableViewMock beginUpdates]);
    sut.tableView = tableViewMock;

    [sut controllerWillChangeContent:nil];

    XCTAssertNoThrow(OCMVerifyAll(tableViewMock),
                     @"Changes in FRC must notify the table to begin upadtes.");
}

Assuming TDD, this test is meant to generate code in a view controller like this:

- (void) controllerWillChangeContent:(NSFetchedResultsController *)controller {
    [self.tableView beginUpdates];
}

Conclusions

Well, first and foremost, enable or disable tests at will is very useful to find out which ones are responsible for the EXC_BAD_ADDRESS crashes. For suspicious tests, run them several times (at least 5 times, 10 for the final tests, 20 for the most occasional ones) before marking them off as solved. Decoupling the tests from the implementation helps in some cases, because the views don't have to be mocked. And if they have you can always use a subclass of the class to be mocked. Still, something odd is happening when assigning a mocked view to the view/tableView property of a view controller. Since it doesn't happen always, I assume there is some kind of race condition that happens when using OCMock.