81

I was seeking for test frameworks for Android UI automation and I stumbled upon UI Automator and Espresso and this is the part I am confused about -

  1. Both are maintained by Google
  2. Both are used for functional UI testing
  3. Both are Android only frameworks

So my questions/doubts here are -

  • What is the major difference between UI Automator and Espresso?
  • Will the source code have to be modified to integrate any of these frameworks?
  • Does one framework hold any advantage over the other? If yes, then which framework is the most suitable for UI automation of an Android app?
Shikhar
  • 1,411
  • 1
  • 11
  • 17

7 Answers7

48

Actually, you don't need to choose. UIAutomator and Espresso use the same instrumentation runner, so you can use commands from both of them in a single test. Espresso has variety of matchers, assertions and actions on UI, that automator doesn't have:

Espresso Cheat Sheet

Espresso is running in separate thread and it is really fast comparing to other test frameworks.

as Summary: use both of them in your tests, Espresso is main and UIAutomator as an additional tool for OS integration.

Jeff Tulley
  • 3
  • 2
Dmitry Gr
  • 783
  • 6
  • 11
  • 4
    +1 For mentioning that you can use both. Even if you don't use both UIAutomator and Espresso in the same test case, you can certainly use them both in separate test cases in the same suite. – Code-Apprentice Feb 04 '16 at 18:27
46

UIAutomator – is powerful and has good external OS system integration e.g. can turn WiFi on and off and access other settings during test, but lacks backward compatibility as it requires Jelly Bean or higher. But, also lacks detailed view access so one could say it may be more of a pure black-box test. Where as Espresso has access to view internals (see below). This is recommended on developer.android.com for "Testing UI for Multiple Apps"

Espresso - is a bit more light weight compared to ui automator and supports 2.2 Froyo and up it also has a fluent api with powerful hamcrest(https://github.com/hamcrest) integration making code more readable and extensible (it is newer than Ui automator). It does not have access to system integration tests but has access to view internals e.g. can test a webview (useful for Hybrid app testing, or webview heavy testing). Slightly more grey-box testing compared to UI Automator. This is recommended on developer.android.com for "Testing UI for a Single App". As of Android Studio 2.2 this now offers UI test recording (like UIAutomator)

Steven Mark Ford
  • 3,372
  • 21
  • 32
  • 1
    Minimum API level support is not a problem but assuming I don't require any system integration during testing, can't the same functionalities be achieved using `UIAutomator` itself? – Shikhar Jun 27 '15 at 17:45
  • 2
    No. If you not testing system/cross-app I would personally go for Espresso. e.g. espresso-web 2.2 was just released which allows for testing webviews which is not supported by UI Automator. Espresso seems to be getting alot more feature attention. See https://code.google.com/p/android-test-kit/wiki/ReleaseNotes – Steven Mark Ford Jun 27 '15 at 19:59
17

If you are testing only one application, then Espresso.

If you are testing more than one application or its integration with other applications or system, then UiAutomator.

Diego Torres Milano
  • 65,697
  • 9
  • 111
  • 134
10

I've found an interesting article, which talks about why you should use them both. Take a look at:

http://qathread.blogspot.com/2015/05/espresso-uiautomator-perfect-tandem.html

Already Espresso is enough for me, but for some reasons like testing app notifications I would in just a few weeks learn uiautomator.

Spend some time to

  • check Google's examples of using these frameworks:

https://github.com/googlesamples/android-testing/tree/master/ui

  • read a documentation of these frameworks:

http://developer.android.com/training/testing/ui-testing/espresso-testing.html

http://developer.android.com/training/testing/ui-testing/uiautomator-testing.html

piotrek1543
  • 19,130
  • 7
  • 81
  • 94
7

To get a quick notion how both things work let's give an example. Let's try to find and click a button with title "Start" on Lollipop using UIAutomator and Espresso:

  • UIAutomator: You have to search for uppercase "START" because on Lollipop buttons are rendered uppercase. device.findObject(new UiSelector().text("START")).click();
  • Espresso: You would just use R.string.start and wouldn't need to care how the string is actually rendered by the platform. You don't care if the view has textAllCaps=true or it is ellipsized. onView(withText(R.string.start)).perform(click());

TL:DR;

UIAutomator searches views in lower-level style than Espresso - via the Instrumentation mechanism and traversing the AccessibilityNodeInfo tree of the view hierarchy. Espresso on its turn typically traverses the view hierarchy itself.

WindRider
  • 11,958
  • 6
  • 50
  • 57
4

When using UIAutomator you must use an 18+ API while with Espresso, you can go as low as API 8.

As the Android documentation states, multi-application tests should be done with UIAutomator while if you only want the tests to run inside of your own app package you can use Espresso.

At the bottom of this documentation's page, you can find the introduction, as well as examples to both of these testing styles.

Simas
  • 43,548
  • 10
  • 88
  • 116
  • Minimum API level support is not a problem but is it a safe assumption that in case of `Espresso`, the source code of the app under test has to be modified? – Shikhar Jun 27 '15 at 17:42
3

A key benefit of using Espresso is that it provides automatic synchronization of test actions with the UI of the app you are testing. Espresso detects when the main thread is idle, so it is able to run your test commands at the appropriate time, improving the reliability of your tests. This capability also relieves you from having to add any timing workarounds, such as Thread.sleep() in your test code.

The Espresso testing framework is an instrumentation-based API and works with the AndroidJUnitRunner test runner.

Source: https://developer.android.com/training/testing/ui-testing/espresso-testing.html

phq
  • 976
  • 1
  • 6
  • 3