Android Platform NimbleApp Guide

Android platform documentation

This document describes the Android-specific parts of NimbleApp. For the parts shared with iOS, please see the main User Guide.

Contents

Customization

Cold Startup measurement exit point

Our profiler is capable of automatically analyzing your app’s cold startup to determine where startup completes. Although this handles many users’ needs, you might be interested in ending profiling before or after our default endpoint. We have provided a specialized API method to do just that. By placing the following statement somewhere in your app’s Cold Startup code path, you can explicitly define where the profiler should end profiling for the Cold Startup scenario:

Log.i(“NimbleDroidV1”, “ColdStartup.end”);

Example 

// Activity onCreate method in the app code
@Override
protected void onCreate(Bundle savedInstanceState ) {
    // ...misc app code
    // ...misc app code

    Log.i("NimbleDroidV1", "ColdStartup.end"); //stop profiling Cold Startup here

    // ...misc app code
    // ...misc app code

}

Custom Scenarios

NimbleApp’s Espresso Scenarios feature allows you to specify custom user flows for profiling. You use standard Espresso instructions to control user actions in the app, and annotate your code with a few log statements to specify which parts of code should be profiled.

To use this feature NimbleApp you will need the following:

  1. The compiled App APK “begin” and/or “end” Log statements in any relevant code to be profiled (optional)
  2. The compiled Espresso APK
  3. At least one log statement to indicate which test cases to run Log.i("NimbleDroidV1", "Scenario.profile");
  4. “begin” and/or “end” Log statements in any relevant code to be profiled (optional)

Additional Requirements:

  • A test case should not make an assumptions whether app is freshly installed as we run your app in a variety of different conditions
  • Scenarios should generally be 5s or less, with a max of 30s
  • Test classes should be written using the Android Testing Support Library and ActivityTestRule. ActivityInstrumentationTestCase2 is deprecated and is therefore not supported.
  • Your app’s build.gradle should be using the latest versions of the Espresso dependencies.

Our code relies on the app developer using a logcat message to register an Espresso test case for profiling and then subsequently using logcat messages to place at least one pair of bookends with IDs around relevant test or app code.

Registering a test case

It is mandatory to register each Espresso test case you intend to profile by placing the following logcat message inline exactly as seen here:

Log.i("NimbleDroidV1", "Scenario.profile");

Since customer Espresso APKs often have functional tests not intended for profiling by NimbleApp, this is how we identify which test cases are intended for profiling. We will ignore any test case method that does not use the above statement as shown. It cannot be included via method call, only inline.

Bookending code

To generate a new Scenario the app developer needs to place a pair of logcat messages known as bookends with a matching bookend-id parameter anywhere along the code path of a registered test case. The pair of bookends must look like the following:

Log.i("NimbleDroidV1", "Scenario.begin bookend-id");
Log.i("NimbleDroidV1", "Scenario.end bookend-id");

Here, bookend-id is a string with length at most 127 characters. These logcat messages do not need to be placed inline and each one can be in either your app code or your test code as long as they will be executed along the code path of your registered test cases. The bookend-id for a matching pair of bookends must be identical or the Scenario will not be recognized.

Choosing where to add bookends

In order to get the most precise and relevant results, you should ideally define your Scenario bookends to encompass the smallest section of code as possible. This would be about 5 seconds of action or smaller. In general it is better to create two discrete bookend sets than to wrap a large amount of action inside a single bookend set.

For example, rather than creating one Scenario that encompasses entering text on a screen, clicking a submit button to a new screen, then clicking a checkbox and another submit button, you’re better off bookending each form submit separately. If you do choose to create large Scenarios, it must run within 30 seconds to avoid a timeout. This is necessary because of the enormous amount of data profiling generates and increased result variability with long Scenarios.

Examples

The source code for the example app APK and test APKs can be found here: https://github.com/NimbleDroid/BasicSample

Example 1: begin and end in app code

The most precise profiling can be achieved by placing both ends of the Scenario log statements directly in the app code. We will create a Scenario with bookend-id clickActivity.

app-debug-androidTest.apk

// Espresso test case method
@Test
public void appCodeTest() {
    Log.i("NimbleDroidV1", "Scenario.profile"); //this registration logcat must be placed inline exactly as seen here
    // ...

}

app-debug.apk

// MainActivity onClick method in the app code
@Override
public void onClick(View view ) {
    Log.i("NimbleDroidV1", "Scenario.begin clickActivity"); //start profiling clickActivity here
    // ...misc app code

}

...

// ShowTextActivity onCreate method in the app code
@Override
protected void onCreate(Bundle savedInstanceState ) {
    // ...misc app code
    ((TextView)findViewById(R.id.show_text_view)).setText(message);

    Log.i("NimbleDroidV1", "Scenario.end clickActivity"); //stop profiling clickActivity here

}

Note: Although the bookend-id clickActivity was created for this example, it will actually generate a unique Scenario for every test case whose code path flows through this pair of bookends. In essence, multiple test cases can share a single bookend-id. This means you should expect to generate Scenarios with name appCodeTest (clickActivity) in addition to hybridTest (clickActivity) and espressoCodeTest (clickActivity).

Example 2: Bookends in test case

The easiest use case involves placing the logcat messages directly in your Espresso test case method. We will create a Scenario with bookend-id clickTest.

app-debug-androidTest.apk

// Espresso test case method 
@Test
 public void espressoCodeTest() {
    Log.i("NimbleDroidV1", "Scenario.profile"); //this registration logcat must be placed inline exactly as seen here

    onView(withId(R.id.editTextUserInput)).perform(typeText(STRING_TO_BE_TYPED), closeSoftKeyboard());

    Log.i("NimbleDroidV1", "Scenario.begin clickTest"); //start profiling clickTest here

    onView(withId(R.id.activityChangeTextBtn)).perform(click());
    onView(withId(R.id.show_text_view)).check(matches(withText(STRING_TO_BE_TYPED)));

    Log.i("NimbleDroidV1", "Scenario.end clickTest"); //stop profiling clickTest here

}

Example 3: begin profile in test case, end in app code

The next use case is to place the begin log in your Espresso APK test case method and the corresponding end log in your app APK’s code. As long as the test case follows a flow that causes both log statements to be executed, the Scenario will be identified and profiled.

We can create a Scenario with bookend-id testToAppClick where the Scenario starts profiling in the Espresso test code and finishes profiling in the app code:

app-debug-androidTest.apk

// Espresso test case method
@Test
public void hybridTest() {
    Log.i("NimbleDroidV1", "Scenario.profile"); //this registration logcat must be placed inline exactly as seen here

    onView(withId(R.id.editTextUserInput)).perform(typeText(STRING_TO_BE_TYPED), closeSoftKeyboard());

    Log.i("NimbleDroidV1", "Scenario.begin testToAppClick"); //start profiling testToAppClick here

    onView(withId(R.id.activityChangeTextBtn)).perform(click());
    onView(withId(R.id.show_text_view)).check(matches(withText(STRING_TO_BE_TYPED)));

}

app-debug.apk

// Activity onCreate method in the app code
@Override
protected void onCreate(Bundle savedInstanceState ) {
    // …misc app code
    ((TextView)findViewById(R.id.show_text_view)).setText(message);

    Log.i("NimbleDroidV1", "Scenario.end testToAppClick"); //stop profiling testToAppClick here

}

Example 4: begin profile in app code, end in Espresso test code

The reverse is also possible. We will create a Scenario with bookend-id appToTestClick that starts profiling in the app code and stops profiling in the test code:

app-debug.apk

// Activity onClick method in the app code
@Override
public void onClick(View view ) {
    Log.i("NimbleDroidV1", "Scenario.begin appToTestClick"); //start profiling appToTestClick here
    // …misc app code
}

app-debug-androidTest.apk

// Espresso test case method
@Test
public void hybridTest() {
    Log.i("NimbleDroidV1", "Scenario.profile"); //this registration logcat must be placed inline exactly as seen here

    onView(withId(R.id.editTextUserInput)).perform(typeText(STRING_TO_BE_TYPED),
           closeSoftKeyboard());
    onView(withId(R.id.activityChangeTextBtn)).perform(click());
    onView(withId(R.id.show_text_view)).check(matches(withText(STRING_TO_BE_TYPED)));

    Log.i("NimbleDroidV1", "Scenario.end appToTestClick"); //stop profiling appToTestClick here

}

Example 5: multiple bookend pairs per test case

Keep in mind that nesting and interleaving bookends is supported so use them however you’d like as long as the bookend-id’s are unique per test case. We will create three Scenarios with bookend-ids interleaveClick, clickActivity, and startActivity. This will generate three scenarios named nestedInterleaveTest (interleaveClick), nestedInterleaveTest (clickActivity), and nestedInterleaveTest (startActivity) respectively.

app-debug-androidTest.apk

// Espresso test case method 
@Test
 public void nestedInterleaveTest() {
    Log.i("NimbleDroidV1", "Scenario.profile"); //this registration logcat must be placed inline exactly as seen here

    onView(withId(R.id.editTextUserInput)).perform(typeText(STRING_TO_BE_TYPED), closeSoftKeyboard());

    Log.i("NimbleDroidV1", "Scenario.begin interleaveClick"); //start profiling interleaveClick here

    onView(withId(R.id.activityChangeTextBtn)).perform(click());
    onView(withId(R.id.show_text_view)).check(matches(withText(STRING_TO_BE_TYPED)));

}

app-debug.apk

// MainActivity onClick method in the app code
@Override
public void onClick(View view ) {
    Log.i("NimbleDroidV1", "Scenario.begin clickActivity"); //start profiling clickActivity here
    // ...misc app code
    Log.i("NimbleDroidV1", "Scenario.end interleaveClick"); //stop profiling interleaveClick here
    final String text = mEditText.getText().toString();
    switch (view.getId()) {
        case R.id.activityChangeTextBtn:
        Intent intent = ShowTextActivity.newStartIntent(this, text);
        Log.i("NimbleDroidV1", "Scenario.begin startActivity"); //start profiling startActivity here
        startActivity(intent);
        break;
           
    }
}

...

// ShowTextActivity onCreate method in the app code
@Override
protected void onCreate(Bundle savedInstanceState ) {
    super.onCreate(savedInstanceState);

    Log.i("NimbleDroidV1", "Scenario.end startActivity"); //stop profiling startActivity here
    // ...misc app code
    ((TextView)findViewById(R.id.show_text_view)).setText(message);

    Log.i("NimbleDroidV1", "Scenario.end clickActivity"); //stop profiling clickActivity here

}

Advanced Usage

Using the REST API

Endpoint

The base endpoint for the latest API version is https://nimbledroid.com/api/v2/

Authentication

All authentication is done using HTTP Basic auth. Each NimbleApp user has an API key (example format: bfca920e348a0c88af00b60e90ae7d4e) available on their Account page for use with API requests. The API key can be used in either the user or password fields for basic auth, authentication will work so as long as the key is in at least one of those fields.

Uploading

You may want to upload using the REST API instead of the Gradle plugin or the web site for a variety of reasons, for example if you are not using Gradle in your CI process, or have a non-standard setup.

At its simplest, you can use a single HTTP POST to upload your APK for profiling.

Simple example (curl) 

curl -F username="myappuser" -F password='123456' -F apk=@com.myapp.apk -v -u bfca920e348a0c88af00b60e90ae7d4e: https://nimble.app/api/v2/apks

Advanced example (curl) 

curl -F apk=@com.myapp.apk -F username="myappuser" -F password='123456' -F device_config="android5" -F commit="e912f09e819f0e81209e8fh2" -v -u bfca920e348a0c88af00b60e90ae7d4e: https://nimble.app/api/v2/apks

Parameters

Parameter Values Default Notes
auto_login true false true (if username and password present OR app has saved credentials Allows explicit enabling/disabling of the auto login feature for Cold Startup and crawled Sceanarios. Auto login is not available for custom Espresso Scenarios.
auto_scenarios true false true Run the crawler to auto-discover Scenarios
commit String Git commit hash of uploaded APK, for labeling and reference
device_config comma-delim string of android4 android5 android6 android4 Comma list of one or more device configurations under which to run profiles. Listing more than one value will cause profiles to be run for each listed value e.g. "android4,android6"
mapping File ProGuard mapping file for translating obfuscated method names
measurement_cap positive Int Allows a user to run profiling on an upload while lowering the number of measurement iterations. This can be used in the case where the uploader wants to just do a sanity check to make sure an app will profile successfully, without the need for a stable speed measurement. Fastest results will be available by setting this to 1.
password String A valid user account password for using a customer's uploaded app. This is not a NimbleApp credential but a customer's user account for testing with their backend.
scenarios comma-delim String Allows running only specified custom Scenarios instead of all custom Scenarios contained in the test file. Passing this param will also prevent crawling, same as auto_scenarios=false e.g. "mainTests (loadProduct), mainTests (addToCart)" Warning: This feature relies on the passed Scenarios having been identified with an earlier upload. The test class, test method, and bookend name must match.
serial_profiling true false false All profiles will be run serially and not in parallel, in order to prevent "simultaneous credential device use" security locks in customer's backend. Enabling this setting will greatly increase latency of results.
test_apk File Espresso test apk for running custom Scenarios
upload_label String Replaces default upload labeling algorithm with user-provided string
username String A valid user account for using a customer's uploaded app. This is not a NimbleApp credential but a customer's user account for testing with their backend.