From Punched Cards to Prompts
AndroidIntroduction When computer programming was young, code was punched into cards. That is, holes were punched into a piece of cardboard in a format...
Lint is a static analysis tool that can help identify bugs and problems in a software project. While it’s been around for many years, Lint has remained a timeless tool that’s been ported to a multitude of different platforms. Out of the box, Android Lint has more than 200 different checks, ranging from obscure performance improvements to security issues and even internationalization concerns.
With a bit of searching, you can find the source code for all of the existing Android Lint checks. Peruse at your own risk—it’s a sparsely documented non-final fluctuating API hellscape.
Despite that, there is great power in the Android Lint API, as you can create your own custom Lint rules. You can easily write a rule, then have it be statically checked across some or all of your projects. You can distribute it locally so only you can use it, or you could throw it into continuous integration so your team could use it. This can be useful if you’re in an environment where certain coding standards must be enforced, such as targeting a certain API level or higher. If you’re writing a library and want to check that it’s being accessed correctly, then you could write a custom Lint rule and distribute it with the library.
In this post, we will go through an example of building a custom Lint check that identifies when an Enum is being used (because some folks dislike Enums). We’ll also cover how to unit test your new Lint check, so that we have some level of trust in its veracity. But as a disclaimer before we begin, the Lint API is not public or final—it can and will change, rendering our custom Lint check example code obsolete. However, the principles and topics herein should remain useful to anyone attempting to build custom Lint checks.
There are four main parts to every Lint check: Implementations, Issues, Detectors, and the Registry. We’ll cover each in more depth later on, but here are the basics:
BuiltinIssueRegistry
class. Since we’ll be writing our custom Issues, we’ll also have to provide our own custom Registry that will be included in our final packaged JAR output.Together, these parts will combine to form a full Lint check. Here’s a nice diagram displaying the relationships between all of these individual pieces:
The Registry is just a listing of all of its Issues, but the other pieces can all be interplayed and reused. So harkening back to our example Lint check that searches for Enum usage, we’ll have to create a few parts:
With all of those pieces, we will then unit test that they interact correctly. As a final output, we’ll build a JAR file that can be installed and used alongside all the other default Lint checks, whether locally or in a continuous integration environment.
You should build the custom Lint Detector in its own Android Studio module or even its own project, so the Lint check will be reusable and not tied to a specific project. To start, let’s add a few lines to our build.gradle
dependencies:
dependencies {
compile 'com.android.tools.lint:lint:24.3.1'
compile 'com.android.tools.lint:lint-api:24.3.1'
compile 'com.android.tools.lint:lint-checks:24.3.1'
}
Now we can dive into the details.
The Implementation of our Lint check is part of an Issue, pointing to the Detector class and Scope set. Without the Implementation, the Issue wouldn’t know how to be identified or where to look for the problem.
Here’s an example, from EnumDetector.java
:
private static final Class<? extends Detector> DETECTOR_CLASS = EnumDetector.class;
private static final EnumSet<Scope> DETECTOR_SCOPE = Scope.JAVA_FILE_SCOPE;
private static final Implementation IMPLEMENTATION = new Implementation(
DETECTOR_CLASS,
DETECTOR_SCOPE
);
There are two parameters to take care of:
EnumDetector.java
class, which scours our code for Enums (we’ll look at it more closely later). The type of the Detector parameter is Class<? extends Detector>
, so we can use any custom class that descends from the provided Detector
superclass.EnumSet<Scope>
, which describes the set of file scopes that an Issue can be found in. Possibilities include (but are not limited to) resource files, Java source files, Java class files, Proguard configuration files, Android Manifest files and all files in the project. It’s important to limit the Scope to be as narrow as possible. For instance, if the Issue only appears in the Android Manifest, then don’t use Scope.ALL
but instead Scope.MANIFEST_SCOPE
. If Lint is being used on a per-file basis automatically (such as in an IDE), then limiting the Scope will improve performance by allowing Lint to check for Issues only within that file’s Scope. In our Enum-searching case, we set the Scope to be Scope.JAVA_FILE_SCOPE
, since Enums will only be defined in a Java source file.As a side note, we define the Implementation itself inside of the Detector used in the Implementation, and later we’ll even define the Issue in that same Detector. Defining these three parts of a Lint check in one file is useful if our Detector is searching for one Issue with a single Implementation. In other cases, we will want to break out the various pieces into their own distinct classes.
An Issue represents a problem that Lint should check—it’s what we’re looking for. In our case, our Issue is that an Enum is being used. Issues are nothing but data representations of different situations that occur.
The following code snippet is an example of an Issue, defined inside EnumDetector.java
:
private static final String ISSUE_ID = "Enum";
private static final String ISSUE_DESCRIPTION = "Avoid Using Enums";
private static final String ISSUE_EXPLANATION = "No real Android programmer should ever use enums. EVER.";
private static final Category ISSUE_CATEGORY = Category.PERFORMANCE;
private static final int ISSUE_PRIORITY = 5;
private static final Severity ISSUE_SEVERITY = Severity.WARNING;
public static final Issue ISSUE = Issue.create(
ISSUE_ID,
ISSUE_DESCRIPTION,
ISSUE_EXPLANATION,
ISSUE_CATEGORY,
ISSUE_PRIORITY,
ISSUE_SEVERITY,
IMPLEMENTATION // This was defined in the "Implementations" section
);
As shown above, we create a custom Issue by using the static create()
method with these parameters:
null
. The general convention is to just use a single camel-cased word.The ID, description and explanation are all describable values related to Enums, while the category, priority and severity have predefined values chosen from their respective classes. We chose the Performance Category because Enum usage can have negative effects on an app’s performance, and we gave it a priority of 5 out of 10 because it’s not that important. The Severity of Warning means the build will still pass, but will be flagged as a potential problem. Filling out the builder method is all it takes to create our own Issue!
However, the heart of a Lint check lies elsewhere: in the Detector.
The Detector is responsible for scanning through code, finding individual Issue instances and reporting them. Detectors can find and report multiple Issue types, which can be useful if two distinct Issues might appear in similar circumstances (think about all the different Android Manifest checks).
A Detector implements one of the Scanner interfaces, which give it the ability to scan through code. The three possibilities are XmlScanner
, JavaScanner
and ClassScanner
, used for XML files, Java files and class files, respectively. If we want to detect an Issue in the Android Manifest, then we’d be using the XmlScanner
interface. To find Enums, we’ll be using JavaScanner
.
The various Scanners search through code via the lombok.ast
API, which represents code as an Abstract Syntax Tree, or AST. Instead of lines of code, you get a searchable tree. Lombok provides utilities and hooks for parsing through these trees, allowing you to find specific pieces of code that you care about for your Issues.
Without further ado, here’s the rest of the EnumDetector.java
file, which includes all of the heavy lifting for the Lint check:
public class EnumDetector extends Detector implements Detector.JavaScanner {
... // Implementation and Issue code from above
/**
* Constructs a new {@link EnumDetector} check
*/
public EnumDetector() {
}
@Override
public boolean appliesTo(@NonNull Context context, @NonNull File file) {
return true;
}
@Override
public EnumSet<Scope> getApplicableFiles() {
return Scope.JAVA_FILE_SCOPE;
}
@Override
public List<Class<? extends Node>> getApplicableNodeTypes() {
return Arrays.<Class<? extends Node>>asList(
EnumDeclaration.class
);
}
@Override
public AstVisitor createJavaVisitor(@NonNull JavaContext context) {
return new EnumChecker(context);
}
private static class EnumChecker extends ForwardingAstVisitor {
private final JavaContext mContext;
public EnumChecker(JavaContext context) {
mContext = context;
}
@Override
public boolean visitEnumDeclaration(EnumDeclaration node) {
mContext.report(ISSUE, Location.create(mContext.file), ISSUE.getBriefDescription(TextFormat.TEXT));
return super.visitEnumDeclaration(node);
}
}
}
Let’s break it down. The Detector is instantiated by Lint each time we run a Lint check, similar to how JUnit will tear down and rebuild everything between runs. To allow Lint to automatically instantiate your Detector, we provide a public default constructor. Technically, the Java compiler will provide one for you automatically if you leave it out, but we’re going to define ours explicitly as a reminder that the system is using it.
Here are the methods again:
@Override
public boolean appliesTo(@NonNull Context context, @NonNull File file) {
return true;
}
The appliesTo(...)
method is a hook to determine if a given file is valid and should be scanned, and we return true
to check everything in our given Scope.
@Override
public EnumSet<Scope> getApplicableFiles() {
return Scope.JAVA_FILE_SCOPE;
}
The getApplicableFiles()
method defines the Scope of our Detector, which for this example is all Java files.
@Override
public List<Class<? extends Node>> getApplicableNodeTypes() {
return Arrays.<Class<? extends Node>>asList(
EnumDeclaration.class
);
}
The getApplicableNodeTypes()
method is where things get interesting. A “node” in this sense is a particular segment or piece of code. A node could be a class declaration or a method invocation or even a comment. We care only about the specific case of an Enum being declared, so we return a list of one valid node type: EnumDeclaration.class
.
@Override
public AstVisitor createJavaVisitor(@NonNull JavaContext context) {
return new EnumChecker(context);
}
Now that our Detector knows to apply only to Java files and hook into only Enum declaration nodes, the next step is to traverse our tree and hit the nodes, one by one. The createJavaVisitor(...)
method is our Lombok hook into traversing our Java tree. We create an inner class called EnumChecker
to represent the process of checking this tree for the nodes we care about:
private static class EnumChecker extends ForwardingAstVisitor {
private final JavaContext mContext;
public EnumChecker(JavaContext context) {
mContext = context;
}
@Override
public boolean visitEnumDeclaration(EnumDeclaration node) {
mContext.report(ISSUE, Location.create(mContext.file), ISSUE.getBriefDescription(TextFormat.TEXT));
return super.visitEnumDeclaration(node);
}
}
Since we have only one applicable node type being checked, our inner class just has the one overridden method of visitEnumDeclaration(...)
. For each Enum declaration that is detected while traversing the AST, this method will be called exactly once. All it’s going to do when called is report the Issue being found.
When an Issue is found, we use the report(...)
method. The method parameters are an Issue being reported, a location where the Issue was found, and a brief description of the Issue. There are other versions of the report(...)
method where you can specify exact line numbers and give more detailed information. For our purposes, the simplest report works fine.
Our Detector can now search through Java files, identify Enum declaration nodes and then yell at us by reporting on each of the instances.
An individual Registry is a list of all of the Issues that Lint should care about from a given JAR of Lint rules. By default, Lint pulls from one Registry, the aptly-named BuiltinIssueRegistry
class, which lists more than 200 different Issues. We can include our own custom EnumIssue
in the overall list of Lint Issues by providing our own Registry. The Registry is packaged inside of the final JAR output and will point to all of the fun new Issues that we’ve provided.
The code for a custom Registry extends the abstract IssueRegistry
class and overrides a single method:
public class CustomIssueRegistry extends IssueRegistry {
private List<Issue> mIssues = Arrays.asList(
EnumDetector.ISSUE // Could totally add more here
);
public CustomIssueRegistry() {
}
@Override
public List<Issue> getIssues() {
return mIssues;
}
}
Since the Registry is just a hook for Lint to grab all of the provided Issues, there’s not much excitement here. We override the getIssues()
method so Lint gets our list, and we also provide a default empty constructor (which is required) so that the system can easily instantiate our new Registry.
There’s an additional step for our Registry to be found, however. We have to make some changes to our build.gradle
file, where we’ll add some information about the output JAR that will be built:
jar {
baseName 'com.bignerdranch.linette'
version '1.0'
manifest {
attributes 'Manifest-Version': 1.0
attributes('Lint-Registry': 'com.bignerdranch.linette.registry.CustomIssueRegistry')
}
}
For the basename, version and Manifest Version, use the package name and whatever version you’re on. The important piece is the Lint-Registry
attribute, which needs to be the fully qualified path of our custom Issue Registry. Now, the manifest (i.e., the metadata) of our output JAR will contain the path to our Registry. Lint will use this path to identify all of the new Issues that we introduce.
As far as implementing the custom Lint check itself goes, we’re done! That’s all of the parts: Implementation, Issue, Detector and Registry. However, for ease of use and veracity purposes, we’re going to add some tests before covering how to build the JAR.
While testing on Android is notoriously annoying, testing Lint checks is surprisingly easy. The first thing we need to do is add a few dependencies to our build.gradle
:
dependencies {
testCompile 'junit:junit:4.11'
testCompile 'org.assertj:assertj-core:3.0.0'
testCompile 'org.mockito:mockito-core:1.9.5'
testCompile 'com.android.tools.lint:lint:24.3.1'
testCompile 'com.android.tools.lint:lint-tests:24.3.1'
testCompile 'com.android.tools:testutils:24.3.1'
}
The next step is to specify our source sets explicitly, so that our project will better understand its own structure. We need to do this so our tests will know where to find the test-resources directory.
sourceSets {
main {
java {
srcDirs = ["lint/src/main/java"]
}
}
test {
java {
srcDirs = ["lint/src/test/java"]
}
}
}
With those new build.gradle
additions, we can begin testing.
Let’s start with something simple, like testing our new Registry:
public class CustomIssueRegistryTest {
private CustomIssueRegistry mCustomIssueRegistry;
/**
* Setup for the other test methods
*/
@Before
public void setUp() throws Exception {
mCustomIssueRegistry = new CustomIssueRegistry();
}
/**
* Test that the Issue Registry contains the correct number of Issues
*/
@Test
public void testNumberOfIssues() throws Exception {
int size = mCustomIssueRegistry.getIssues().size();
assertThat(size).isEqualTo(1);
}
/**
* Test that the Issue Registry contains the correct Issues
*/
@Test
public void testGetIssues() throws Exception {
List<Issue> actual = mCustomIssueRegistry.getIssues();
assertThat(actual).contains(EnumDetector.ISSUE);
}
}
We’re just instantiating our custom Registry and checking its size and list of Issues for correctness. There’s not much else to test here.
The more interesting (and useful) tests are for the custom Detector. These tests will pull from external sample files in your test-resources directory and run a Lint check on each of them. In my example code on Github, I abstracted away some of this logic into a superclass for reusability. Here’s a condensed version:
public class EnumDetectorTest extends LintDetectorTest {
private static final String PATH_TEST_RESOURCES = "/lint/src/test/resources/enum/";
private static final String NO_WARNINGS = "No warnings.";
@Override
protected Detector getDetector() {
return new EnumDetector();
}
@Override
protected List<Issue> getIssues() {
return Arrays.asList(EnumDetector.ISSUE);
}
/**
* Test that an empty java file has no warnings.
*/
public void testEmptyCase() throws Exception {
String file = "EmptyTestCase.java";
assertEquals(
NO_WARNINGS,
lintFiles(file)
);
}
/**
* Test that a java file with an enum has a warning.
*/
public void testEnumCase() throws Exception {
String file = "EnumTestCase.java";
String warningMessage = file
+ ": Warning: "
+ EnumDetector.ISSUE.getBriefDescription(TextFormat.TEXT)
+ " ["
+ EnumDetector.ISSUE.getId()
+ "]n"
+ "0 errors, 1 warningsn";
assertEquals(
warningMessage,
lintFiles(file)
);
}
@Override
protected InputStream getTestResource(String relativePath, boolean expectExists) {
String path = (PATH_TEST_RESOURCES + relativePath).replace('/', File.separatorChar);
File file = new File(getTestDataRootDir(), path);
if (file.exists()) {
try {
return new BufferedInputStream(new FileInputStream(file));
} catch (FileNotFoundException e) {
if (expectExists) {
fail("Could not find file " + relativePath);
}
}
}
return null;
}
private File getTestDataRootDir() {
CodeSource source = getClass().getProtectionDomain().getCodeSource();
if (source != null) {
URL location = source.getLocation();
try {
File classesDir = SdkUtils.urlToFile(location);
return classesDir.getParentFile().getAbsoluteFile().getParentFile().getParentFile();
} catch (MalformedURLException e) {
fail(e.getLocalizedMessage());
}
}
return null;
}
}
Phew! That’s a lot of code, so let’s break down the test file piece by piece, starting with two setup methods:
@Override
protected Detector getDetector() {
return new EnumDetector();
}
@Override
protected List<Issue> getIssues() {
return Arrays.asList(EnumDetector.ISSUE);
}
The getDetector()
method provides the Detector that we want to test, while getIssues()
provides the Issues. For these, return EnumDetector
and EnumIssue
, respectively.
Here are the constants from the top of the file:
private static final String PATH_TEST_RESOURCES = "/lint/src/test/resources/enum/";
private static final String NO_WARNINGS = "No warnings.";
The first constant of PATH_TEST_RESOURCES
is our relative path to the test-resources directory, which is used in the getTestResource()
and getTestDataRootDir()
methods. Their purpose is to identify the test resource file to be used in a given test case. The second constant of NO_WARNINGS
is the default Lint message when nothing is wrong, which we’ll need for test comparisons.
Here are the individual test cases again:
public void testEmptyCase() throws Exception {
String file = "EmptyTestCase.java";
assertEquals(
NO_WARNINGS,
lintFiles(file)
);
}
public void testEnumCase() throws Exception {
String file = "EnumTestCase.java";
String warningMessage = file
+ ": Warning: "
+ EnumDetector.ISSUE.getBriefDescription(TextFormat.TEXT)
+ " ["
+ EnumDetector.ISSUE.getId()
+ "]n"
+ "0 errors, 1 warningsn";
assertEquals(
warningMessage,
lintFiles(file)
);
}
We have only two test cases, the empty example and a small Enum example. For each case, we point to the file that we want to test, and then we call lintFiles(String path)
on the path String. If the test-resources directory setup above is correctly found, then the lintFiles()
call will load the test file into memory as an on-the-fly Android project and then run a Lint check on it, returning its output as a String value.
Once we have the String of the finished Lint check, we compare against what we expected. For the empty test case, we expect a value equal to the NO_WARNINGS
constant. For the Enum case, we expect a warning message that is composed of a bunch of different variables we’ve defined throughout this process. Figuring out what the actual warning message will be takes some trial and error, but the simplest of Lint checks (like what we’ve done here) will follow the String concatenation pattern used in the testEnumCase()
method.
When you’re all done, here’s what you have: each test case has a test file, we run a Lint check on it, and then we examine the output to determine if it’s what we expect.
My favorite way to do anything is via terminal. Navigate to the root of our custom Lint check project and run the following command:
./gradlew clean build test
Your output should look something like this:
:clean
:compileJava
:processResources UP-TO-DATE
:classes
:jar
:assemble
:compileTestJava
:processTestResources UP-TO-DATE
:testClasses
:test
:check
:build
BUILD SUCCESSFUL
Total time: 6.373 secs
Hopefully, all is well. I highly recommend a test-driven development approach if you start building custom Lint checks. It’s a perfect use-case—provide a sample file and then write a Detector or Issue to fit the case.
If you’ve made it this far, congratulations. Your hard work is about to pay off. If you recall, the final output of our custom Lint checker is a JAR file. Anytime we want to add Lint checks to our system, we simply add a JAR file to our ~/.android/lint/
directory. The Lint tool will always check there for anything new on each run. By running the assemble
Gradle task, we will generate and assemble the JAR during each build. Now, we could move the output JAR from our build
directory to the Lint directory manually, but I prefer to incorporate it into the build process by adding this handy Gradle task to our build.gradle
file:
defaultTasks 'assemble'
task install(type: Copy) {
from configurations.lintChecks
into System.getProperty('user.home') + '/.android/lint/'
}
Let’s build and install our new Lint check by running the following from a terminal at the root of our custom Lint check project:
./gradlew clean build test install
Assuming everything builds and passes, let’s check to see if our new Lint rule is now available:
lint --show Enum
If everything went smoothly, you should see the following:
Enum
----
Summary: Avoid Using Enums
Priority: 5 / 10
Severity: Warning
Category: Performance
No real Android programmer should ever use enums. EVER.
While having the Lint check available on a system level is great, the final trial is actually running our shiny new Lint check on a real project, so navigate to an actual Android application project of your choice and run the following:
./gradlew lint
You could also use Android Studio’s Analyze->Inspect Code...
menu option, but that’s no fun.
Assuming all is well, the results should be found in build/outputs/lint-results.html
and look something like this:
Ta-da! Finally, our custom Lint check is registered in the system. Now we can use it on any project. It was quite a journey: we built up an Implementation, an Issue, a Detector, a Registry and some test cases, in addition to making some Gradle magic. Lint is an under-utilized tool by many Android developers, but I hope now you can start pushing the boundaries and making the most of static code analysis.
In case you missed it earlier, here is the full source code, which actually includes a few different examples. Enjoy!
Special thanks to my coworker Jason Atwood for all of his research help, and to André Diermann, Jerzy Chalupski, and Cheng Yang for getting me started on the right path.
Introduction When computer programming was young, code was punched into cards. That is, holes were punched into a piece of cardboard in a format...
Jetpack Compose is a declarative framework for building native Android UI recommended by Google. To simplify and accelerate UI development, the framework turns the...
Big Nerd Ranch is chock-full of incredibly talented people. Today, we’re starting a series, Tell Our BNR Story, where folks within our industry share...