The Long Weekend Website

Xcode4: Running Application Tests From The Command Line in iOS

By Mark on April 17th, 2011 in Technobabble
Tags: dev, ios4, unit testing, xcode

While researching the “is GHUnit still relevant vis-a-vis Xcode 4” article, I discovered this Carbon Five post by Jonah: Running Xcode 4 unit tests from the command line.

In short, I found a way to run iOS application unit tests from the command line; this article describes that process.

Part of the research is about automation – how well GHUnit and/or OCUnit test suites can be automated.

In my book, command line = automation. Last summer I installed Hudson CI on Paul’s Mac Mini and started automated builds. Why not add unit tests to the mix?

Jonah’s post explains how to make a unit test bundle testable from the command line. Unfortunately, and he notes it as well, it only works for logic tests (e.g. no application tests). His approach uses schemes in Xcode 4.

Jonah’s command:

xcodebuild -workspace workspaceName.xcworkspace -scheme logicTestSchemeName -sdk iphonesimulator4.3 -configuration Debug clean build

I’m still not sure I fully understand schemes/workspaces yet, as they are new to Xcode 4, but this is definitely one case where good ol’ targets appear to work better.

By using the target paradigm, we can build the bundle without creating a new scheme. Instead, just use the test bundle’s name as your build target:

xcodebuild -target unitTestBundleTargetName -configuration Debug -sdk iphonesimulator4.3 clean build

This improved upon the first half of his post – removing the need to create a separate scheme. His post ends with a challenge for application tests:

In case it is of any help to other developers struggling with the difference between command line and Xcode builds.

The “Run Script” build phase of a unit test build target just runs… (edit: references some scripts, omitted for brevity)

Hopefully it will prove possible to use these scripts to match the behavior seen when running tests from within Xcode to automatically run application tests.

So, I included an application test in my unit test bundle, and this was the output:

/Developer/Platforms/iPhoneSimulator.platform/Developer/Tools/RunPlatformUnitTests:94: warning: Skipping tests; the iPhoneSimulator platform does not currently support application-hosted tests (TEST_HOST set).

Isn’t it supported? We’ve seen it working in Xcode 4. Maybe it really is supported, but Apple hasn’t updated the testing script because they were so busy just trying to get Xcode 4 in a semi-releasable state (highly likely). Or, it hasn’t been “officially” said to work; e.g. they don’t want to support it yet, so they’re not letting us do it.

After snaking through all the script calls, I found the culprit.

Change line 95 of /Developer/Platforms/iPhoneSimulator.platform/Developer/Tools/RunPlatformUnitTests from

Warning ${LINENO} "Skipping tests; the iPhoneSimulator platform does not currently support application-hosted tests (TEST_HOST set)."

to these 2 lines:

export OTHER_TEST_FLAGS="-RegisterForSystemEvents"
RunTestsForApplication "${TEST_HOST}" "${TEST_BUNDLE_PATH}"

The first line of code seems to make event handling work with Springboard (see comments below). The second line of code I stole from /Developer/Tools/RunPlatformUnitTests. Seems like it’s the default behavior for other platforms, just not the simulator SDK.

After inserting those lines, your tests should run (I’ve included a sample project with the set up to show the code, it is the “navigation based app” template using Core Data).

Also, it’s important to note that the Simulator will not run 2 instances. You’ll need to shut the Simulator if you have it open.

DISCLAIMER:Yes, I’m asking you to change your /Developer directory. You probably should back everything up, blah blah blah, and I don’t want to be blamed if you have to reinstall Xcode 4.

Also, I originally had Core Data as part of the project, but I was getting an NSCocoaErrorDomain error from my application on the console. After I removed Core Data from the project, the problem went away.

Deniz Demir helped me out there – a little Google brought up his post from late March, where he explained his troubles with Core Data and unit testing. Applying his fix inre: the documents directory makes my code work. Glad to see everyone work at the coal face of the problem!

Best of luck!