How to setup Jacoco for Android project with Java, Kotlin and multiple flavours

Jacoco is a widely used library to measure test code-coverage in JVM-based projects. Setting it up for Android applications has a few quirks and having multiple flavours, using Kotlin and writing (some) tests in Robolectric makes it even tricker. There are already great tutorials in how to set it up, like THIS and THIS one. In this post however I’ll not only give you a ready solution, but share all details how I got to it – this way you’ll be able to adapt it in the best way for your project.

1. Covering unit tests only

Ideally you don’t want to bloat your build.gradle file with random third-party configuration code. To keep things clean – all Jacoco logic will live in it’s own separate file – jacoco.gradle. This file could live next to your main build.gradle file or to keep things even cleaner – I moved it in a separate directory called buildscripts. The project structure now looks like this:


Let’s start implementing the jacoco.gradle file:

apply plugin:  'jacoco'

jacoco {
    toolVersion = "0.8.1"
    // Custom reports directory can be specfied like this:
    // reportsDir = file("$buildDir/customJacocoReportDir")
}

tasks.withType(Test) {
    jacoco.includeNoLocationClasses = true
}

First thing is to make use of Gradle’s Jacoco plugin (line 1). We can then specify two configuration params (the block on line 3): the version of Jacoco (0.8.1 at the time of writing) and optionally where reports will be generated. The second param is commented out, so the default directory – app/build/reports/jacoco/ will be used.

The block on line 9 is the way to correctly set the includeNoLocationClasses property in the latest versions of Jacoco. You’d want to do this if you have Robolectric tests in your suite. Please note that enabling this property was previously done via Android Plugin for Gradle DSL, but this way no longer works!

2. Setting up a Jacoco task

Projects that use Kotlin or have multiple Android flavours need to create a custom Gradle task to generate coverage reports. In this task we’ll tweak a few Jacoco parameters, as otherwise the coverage report won’t be accurate enough (e.g. Kotlin classes will not be included in it). Here’s a snippet to create such task:

project.afterEvaluate {

    android.applicationVariants.all { variant ->
        def variantName = variant.name
        def testTaskName = "test${variantName.capitalize()}UnitTest"

        tasks.create(name: "${testTaskName}Coverage", type: JacocoReport, dependsOn: "$testTaskName") {
            // task implementation here ...
        }
    }
}

Since our project has multiple flavours, we’ll need to create a separate task for each flavour. We iterate over all generated Android variants (line 3). For each one we construct the name of the unit test running task (line 5), which has the format test<YourVariantName>UnitTest. Finally we declare our custom task on line 7. Few things to note here:

  • The name of our task will be test<YourVariantName>UnitTestCoverage. You can pick anything you want here, but as such tasks are created for each variant, it’s a good idea to include the ${variantName} or ${testTaskName} in here.
  • The type of our task is a JacocoReport one, so we can tweak all properties of it (check them in the task documentation).
  • By specifying that our task dependsOn: “$testTaskName”, we guarantee it will always be executed after the unit test are run.
3. Jacoco task implementation

Let’s get to the trickiest bit – implementing the actual task. My solution looks like this:

tasks.create(name: "${testTaskName}Coverage", type: JacocoReport, dependsOn: "$testTaskName") {
	group = "Reporting"
	description = "Generate Jacoco coverage reports for the ${variantName.capitalize()} build."

	reports {
	    html.enabled = true
	    xml.enabled = true
	}

	def excludes = [
	        '**/R.class',
	        '**/R$*.class',
	        '**/BuildConfig.*',
	        '**/Manifest*.*',
	        '**/*Test*.*',
	        'android/**/*.*'
	]
	def javaClasses = fileTree(dir: variant.javaCompiler.destinationDir, excludes: excludes)
	def kotlinClasses = fileTree(dir: "${buildDir}/tmp/kotlin-classes/${variantName}", excludes: excludes)
	classDirectories = files([javaClasses, kotlinClasses])

	sourceDirectories = files([
	        "$project.projectDir/src/main/java",
	        "$project.projectDir/src/${variantName}/java",
	        "$project.projectDir/src/main/kotlin",
	        "$project.projectDir/src/${variantName}/kotlin"
	])

	executionData = files("${project.buildDir}/jacoco/${testTaskName}.exec")
}

A good practice is to always give a group and description when creating custom Gradle tasks (lines 2 and 3). This way they’ll be nicely listed in the output of ./gradlew tasks command for your project:

The next bit is configuring what types of reports we need generated (line 5). The HTML one is more user-friendly and it’s used when inspecting the coverage locally. The XML one is used by the Jenkins Jacoco plugin.

Next we define a list of classes we want to exclude from our coverage reports (line 10). It makes sense to exclude auto-generated code and code that you don’t have control over (e.g. third-party code, etc). If you use Kotlin in your project check out THIS extra read on how to exclude Kotlin-generated code from your reports.

Line 18 defines the path to the compiled java classes. Please note we use the variant.javaCompiler.destinationDir variable that’s provided by Gradle. We’re excluding the classes we want to be ignored from the report using the excludes variable we defined above. Unfortunately the path to the compiled Kotlin classes isn’t provided to us yet, so we need to build it ourselves. At the time of writing this article (Gradle 4.7, Kotlin 1.2.41, Jacoco 0.8.1) it has the format as shown on line 19. I hope Gradle will soon provide a property similar to the Java one, so we won’t need to do this manually.
On line 20 we just set the JacocoReport.classDirectories for our task – e.g. the classes to generate a report for.

Line 22 sets the JacocoReport.sourceDirectories property of our task, where we specify where the source code for the classes above is located. Please note that for multi-flavour projects you can have multiple Java and/or Kotlin source directories. In our example we have two for each language:

  • src/main/java
  • src/<variantName>/java
  • src/main/kotlin
  • src/<variantName>/kotlin

Just list all directories that contain any source code for the specific flavour.

Last thing – on line 29 we set the JacocoReport.executionData property, which links to the .exec file created by Gradle’s Jacoco plugin. This file contains metadata needed to generated the report. Please note the path of the file.

4. Generating Jacoco reports

With the setup above we’re almost ready to generate coverage report for all unit tests (JUnit, Robolectric) for each flavour of the app. The report will correctly cover both Java and Kotlin code.

The last tiny step is to include the file jacoco.gradle as part of your app’s build.gradle file:

... 
apply plugin: 'com.android.application'
apply from: 'buildscripts/jacoco.gradle'
...

And that’s it! You can now generate a report by executing the task we created above:

./gradlew testFreeDebugUnitTestCoverage

Since we didn’t specify an output directory the report is generated in: app/build/reports/jacoco/testFreeDebugUnitTestCoverage.

5. Generating coverage reports for UI tests

In some cases you might want to generate a coverage report for your Instrumentation tests – e.g. if you have a lot of instrumented unit tests (e.g. tests for Activities, Fragments, etc) or even full-blown BDD-style behavioural tests.

To generate such report you can make use of a property available in the Android Gradle Plugin DSL – testCoverageEnabled. Add it in your build.gradle file:

android {
	buildTypes {
        debug {
            ...
	        testCoverageEnabled true
        }
}

Adding this property alone (without any of the work we did above) will automatically add a new Gradle reporting task to the project: createFreeDebugCoverageReport. Executing this task will generate a simple report that includes only your androidTests. By default the report is located in: app/build/reports/coverage/free/debug/.

6. Putting it all together

If you want to generate a single report that covers both unit and UI tests, there’s just a few steps required. If you haven’t already, add the testCoverageEnabled property from step 5. Then we should apply a few changes to the custom task created in steps 1-4:

...
def variantName = variant.name
def testTaskName = "test${variantName.capitalize()}UnitTest"
def uiTestCoverageTaskName = "test${variantName.capitalize()}CoverageReport"

tasks.create(name: "${testTaskName}Coverage", type: JacocoReport, dependsOn: "$testTaskName", "$uiTestCoverageTaskName") {
	...

	executionData = files([
		"${project.buildDir}/jacoco/${testTaskName}.exec",
		"outputs/code-coverage/connected/*coverage.ec"
	]) 
}

Notice how we added another dependency to our custom task – the $uiTestCoverageTaskName task (line 6), which is based on the auto-generated task mentioned in step 5. This dependency generates an .ec coverage report after our androidTests are run. The next change is to include this newly generated file (“outputs/code-coverage/connected/*coverage.ec”) to the executionData configuration of our task (line 11). Please note the wildcard there: *coverage.ec. We use a wildcard, because the name of the file has the name of the actual device the UI tests are run on.

That’s it – running our custom task test<YourFlavourName>UnitTestCoverage will now run both unit and instrumentation tests and will generate report based on both.

Hope you see good numbers when you generate your coverage reports!

17 comments

  • L

    Great tutorial!

    Reply
  • DV

    This task is failed with exception:

    java.lang.NoClassDefFoundError: jdk/internal/reflect/GeneratedSerializationConstructorAccessor1

    Reply
    • veskoiliev

      Which task exactly? For me everything works with Gradle 4.7, JVM: 1.8.0_121 (Oracle Corporation 25.121-b13) on a MacBook Pro.

      Reply
  • Mike

    great post. Thanks

    Reply
  • Ovi Trif

    Finally somebody who knows what he’s doing does a tutorial on this.

    Many thanks!

    I struggled a lot to get it working properly. I’d suggest you write this article on Medium (ProAndroidDev). The android community needs it!

    Reply
  • Ovi Trif

    I had to make a few changes for the UI tests to work, specifically:

    1. Fix uiTestCoverageTaskName variable:
    def uiTestCoverageTaskName = "create${variantName.capitalize()}CoverageReport"

    2. Write dependsOn as an array:
    tasks.create(name: "${testTaskName}Coverage", type: JacocoReport, dependsOn: ["$testTaskName", "$uiTestCoverageTaskName"]) {

    3. Fix uiTestCoverageTaskName variable:
    def uiTestCoverageTaskName = "create${variantName.capitalize()}CoverageReport"

    4. Fix path in executionData:
    executionData = files([
    "$project.buildDir/jacoco/${testTaskName}.exec",
    "$project.buildDir/outputs/code-coverage/connected/flavors/$variant.flavorName/coverage.ec"
    ])

    Reply
  • Shashank

    Hi, your post was really helpful.

    In step 3: Jacoco task implementation, the source directories as mentioned are

    sourceDirectories = files([
    “$project.projectDir/src/main/java”,
    “$project.projectDir/src/${variantName}/java”,
    “$project.projectDir/src/main/kotlin”,
    “$project.projectDir/src/${variantName}/kotlin”
    ])

    I think it should be (${variantName} replaced with ${productFlavorName}) as we dont have directories under src for every build variant. Could anyone please confirm? Thanks 🙂

    sourceDirectories = files([
    “$project.projectDir/src/main/java”,
    “$project.projectDir/src/${productFlavorName}/java”,
    “$project.projectDir/src/main/kotlin”,
    “$project.projectDir/src/${productFlavorName}/kotlin”
    ])

    Reply
    • veskoiliev

      Hello! The idea of step 3 is to list all places where you have some source code.

      To avoid confusion, let me give an example. Imagine your app has 2 flavours – “europe” and “usa”. Let’s say we have the 2 default build types as well – “debug” and “release”. In theory you can have “europe”-specific code in “$project.projectDir/src/europe”. Code specific for the “debug” build type would be in “$project.projectDir/src/debug”.

      If you want to have the most complete coverage report, you’d list all these locations:
      “$project.projectDir/src/main/java”
      “$project.projectDir/src/main/kotlin”
      “$project.projectDir/src/debug/java”
      “$project.projectDir/src/debug/kotlin”
      “$project.projectDir/src/europe/java”
      “$project.projectDir/src/europe/kotlin”
      …..

      Hope that helps!

      Reply
  • Jayesh Thadani

    Hi, thanks for writing this great tutorial.
    I have configured as in this tutorial but facing an issue and seek help if some one have faced this before,
    [ant:jacocoReport] Classes in bundle ‘app’ do no match with execution data. For report generation the same class files must be used as at runtime.

    As we are ensuring that Unit test is executed previous to jacoco report generation, I am not sure why this is occuring.

    Reply
  • Jayesh Thadani

    [ant:jacocoReport] Classes in bundle ‘app’ do no match with execution data. For report generation the same class files must be used as at runtime.

    Getting this error after configuration as mentioned in post, did any one faced this before?

    Reply
    • veskoiliev

      Hello Jayesh,

      Just FYI, there should be no reason to manually run unit tests before running the custom Jacoco task (e.g. testFreeDebugUnitTestCoverage), because it depends on the unit test tasks, so Gradle will do so for you if needed.

      I’ve not faced the issue you mentioned 🙁 Just to confirm – are you experiencing this when running: ./gradlew testFreeDebugUnitTestCoverage? If so, can you please add a bit more of the Gradle errors … perhaps it’ll help see what’s going on.

      P.S. There’s a few Stackoverflow questions about the same issue and they point to having different Java versions when running the unit tests and the jacoco report (https://stackoverflow.com/a/44850229/1759623). Not sure how that’ll be the case if you followed the steps in this guide though 😉

      Reply
  • Mayank Verma

    Hi,

    Thanks for writing this post really Helpful!
    But i am facing issue with the code coverage. I am seeing 0% code coverage for my classes written in Kotlin. Could you please point out what could be wrong in the below code config file.

    apply plugin: ‘jacoco’

    jacoco {

    // This version should be same as the one defined in root project build.gradle file :
    toolVersion = “0.8.3”
    }

    tasks.withType(Test) {
    jacoco.includeNoLocationClasses = true
    }

    project.afterEvaluate {

    android.applicationVariants.all { variant ->

    def testTaskName = “test${variant.name.capitalize()}UnitTest”

    task “${testTaskName}Coverage”(type: JacocoReport, dependsOn: “$testTaskName”) {
    group = “Reporting”
    description = “Generate Jacoco coverage reports on the ${variant.name.capitalize()} build.”

    def excludes = [
    ‘**/R.class’,
    ‘**/R$*.class’,
    ‘**/*$ViewInjector*.*’,
    ‘**/*$ViewBinder*.*’,
    ‘**/BuildConfig.*’,
    ‘**/Manifest*.*’
    ]

    //Tree for all the Java classes
    def javaClasses = fileTree(dir: variant.javaCompiler.destinationDir, excludes: excludes)

    println “Test : ” + javaClasses

    //Tree for all the Kotlin classes
    def kotlinClasses = fileTree(dir: “${buildDir}/tmp/kotlin-classes/${variant.name}”, excludes: excludes)

    //combined source directories
    classDirectories = files([javaClasses, kotlinClasses])

    def coverageSourceDirs = [
    “$project.projectDir/src/main/java”,
    “$project.projectDir/src/${variant.name}/java”,
    “$project.projectDir/src/main/kotlin”,
    “$project.projectDir/src/${variant.name}/kotlin”
    ]
    additionalSourceDirs = files(coverageSourceDirs)
    sourceDirectories = files(coverageSourceDirs)
    executionData = files(“${project.buildDir}/jacoco/${testTaskName}.exec”)
    reports {
    xml.enabled = true
    html.enabled = true
    html.destination = file(“${project.buildDir}/jacoco/”)
    }
    }
    }
    }

    Reply
  • Ahmed AbuQamar

    First thank you for this post and every one help in comments

    Some exceptions happen with me when running the previous code, I found the issue that we need to add .from

    additionalSourceDirs.from = files(coverageSourceDirs)
    sourceDirectories.from = files(coverageSourceDirs)
    executionData.from =

    =======================
    Full Code
    =======================

    apply plugin: ‘jacoco’

    jacoco {

    // This version should be same as the one defined in root project build.gradle file :
    toolVersion = “0.8.6”
    }

    tasks.withType(Test) {
    jacoco.includeNoLocationClasses = true
    }

    project.afterEvaluate {

    android.applicationVariants.all { variant ->

    def testTaskName = “test${variant.name.capitalize()}UnitTest”

    task “${testTaskName}Coverage”(type: JacocoReport, dependsOn: “$testTaskName”) {
    group = “Reporting”
    description = “Generate Jacoco coverage reports on the ${variant.name.capitalize()} build.”

    def excludes = [
    ‘**/R.class’,
    ‘**/R$*.class’,
    ‘**/*$ViewInjector*.*’,
    ‘**/*$ViewBinder*.*’,
    ‘**/BuildConfig.*’,
    ‘**/Manifest*.*’
    ]

    //Tree for all the Java classes
    def javaClasses = fileTree(dir: variant.javaCompiler.destinationDir, excludes: excludes)

    println “Test : ” + javaClasses

    //Tree for all the Kotlin classes
    def kotlinClasses = fileTree(dir: “${buildDir}/tmp/kotlin-classes/${variant.name}”, excludes: excludes)

    //combined source directories
    classDirectories.from = files([javaClasses, kotlinClasses])

    def coverageSourceDirs = [
    “$project.projectDir/src/main/java”,
    “$project.projectDir/src/${variant.name}/java”,
    “$project.projectDir/src/main/kotlin”,
    “$project.projectDir/src/${variant.name}/kotlin”
    ]
    additionalSourceDirs.from = files(coverageSourceDirs)
    sourceDirectories.from = files(coverageSourceDirs)
    executionData.from = files(“${project.buildDir}/jacoco/${testTaskName}.exec”)
    reports {
    xml.enabled = true
    html.enabled = true
    html.destination = file(“${project.buildDir}/jacoco/”)
    }
    }
    }
    }

    Reply
    • Rajesh J

      When adding below lines getting error. can you please suggest how to resolve this issue?
      reports {
      xml.enabled = true
      html.enabled = true
      html.destination = file(“${project.buildDir}/jacoco/”)
      }

      A problem occurred configuring project ‘:app’.
      > Could not create task ‘:app:testDevDebugUnitTestCoverage’.
      > Could not set unknown property ‘enabled’ for Report xml of type org.gradle.api.reporting.internal.TaskGeneratedSingleFileReport.

      Reply
  • Dakota

    Hi there,

    I am following along, and the instructions make sense, but I’m unable to get any reports to actually be generated in the build folder. I am able to run the task and it passes just fine but there is no report that comes from this. If anyone could help that would be great.

    My jacoco.gradle file is laid out as such:

    apply plugin: ‘jacoco’

    jacoco {
    toolVersion = “0.8.7”
    }

    tasks.withType(Test) {
    jacoco.includeNoLocationClasses = true
    jacoco.excludes = [‘jdk.internal.*’]
    }

    project.afterEvaluate {

    android.applicationVariants.all { variant ->
    def variantName = variant.name
    def testTaskName = “test${variantName.capitalize()}UnitTest”

    tasks.create(name: “${testTaskName}Coverage”, type: JacocoReport, dependsOn: “$testTaskName”) {
    group = “Reporting”
    description = “Generate Jacoco coverage reports for the ${variantName.capitalize()} build.”

    reports {
    html.enabled = true
    xml.enabled = true
    }

    def excludes = [
    ‘**/R.class’,
    ‘**/R$*.class’,
    ‘**/BuildConfig.*’,
    ‘**/Manifest*.*’,
    ‘**/*Test*.*’,
    ‘**/test*.*’,
    ‘android/**/*.*’
    ]
    def javaClasses = fileTree(dir: variant.javaCompiler.destinationDir, excludes: excludes)
    def kotlinClasses = fileTree(dir: “${buildDir}/tmp/kotlin-classes/${variantName}”, excludes: excludes)
    classDirectories.from = files([javaClasses, kotlinClasses])

    sourceDirectories.from = files([
    “$project.projectDir/src/main/java”,
    “$project.projectDir/src/${variantName}/java”,
    “$project.projectDir/src/main/kotlin”,
    “$project.projectDir/src/${variantName}/kotlin”
    ])

    executionData.from = files(“${project.buildDir}/jacoco/${testTaskName}.exec”)

    reports {
    xml.enabled = true
    html.enabled = true
    html.destination = file(“${project.buildDir}/reports/jacoco”)
    }
    }
    }
    }

    Reply
    • veskoiliev

      Hey,

      Sorry, it’s been a while since I last ran Jacoco, so can’t provide a specific fix. Still two high-level things to check:
      1. If the Java/Kotlin classes are still generated in the specified folders. Things move every now and then (due to updated versions of Java/Kotlin), so maybe the jacoco task just can’t find your code.

      2. Check the whole `app/build` folders for the generated report, just to ensure it’s not in some arbitrary folder there.

      Good luck!

      Reply
      • Nagara Pambhala

        Hey thank you so much man,got this working
        but how to add finalized jacooc report for this task generate report every time when gradle runs

        Reply

Leave a Comments

Cancel reply
NAME *
EMAIL *
Website