In this blog we will construct a simple unmarshaller at compile-time using an annotation processor and measure its performance.
This blog is part 3 of 4, the series consists of the following posts:
- Writing an unmarshaller by hand
- Constructing an unmarshaller using reflection
- Generating an unmarshaller using annotations
- Creating an unmarshaller using bytecode
For a definition of the some of the terminology used here see Phases for generating code.
In part 1 we coded a simple unmarshalled by hand. The obvious disadvantage of this is that, well, you have to code by hand. In part 2 we constructed an unmarshaller automatically using reflection. The initial implemetentation was over 5 times slower, but with some optimization we managed ‘only’ 20% slower.
Ideally we would like to generate the unmarshaller automatically and have the same performance as handwriten code. So in this post we will explore writing an annotation processor to achieve this.
Annotations processors were introduced in Java 6 and allow plugging in custom code into the compiler to extend the compilation process with the generation of additional code. As the name suggests annotation processors are intended to process code, such as classes, which are annotated with a custom annotation. Although they are actually more powerful and can also be used without annotations or (with some hacks) to augment existing code.
We will define a simple annotation called Bean which will be used to mark JavaBean class indicating that we would like an unmarshaller generated. The annotation itself is very simple.
@Retention(RetentionPolicy.SOURCE)
@Target({ElementType.TYPE})
public @interface Bean {
}
The annotation is only present in the source code, not retained in the class file or at run-time. This is excellent for an annotation processer since it deals with source code. The target of the annotation is ‘TYPE’ meaning it can only be used to annotated type elements such as classes and interfaces.
The processor itself is simply a class extending AbstractProcessor.
@AutoService(Processor.class)
@SupportedSourceVersion(SourceVersion.RELEASE_11)
@SupportedAnnotationTypes("dev.sanjuroe.generation.annotation.Bean")
public class BeanProcessor extends AbstractProcessor {
@Override
public boolean process(Set<? extends TypeElement> annotations, RoundEnvironment roundEnv) {
return false;
}
The first annotation AutoService itself triggers an annotation processor which will create the ServiceLoader configuration. It is not necessary to use an annotation processor for this, but, to me, provides a nice feel of recursion of sorts.
The second and third annotation indicate the supported Java version as well as the annotation this processor is triggered by.
Annotation processors very powerful, but Java Compiler API they use is also rather complex. Being able to write unit tests and debug the processor is rather useful, I wrote a seperate post about how to do this, see Debugging annotation processors.
Compilations consists of multiple rounds. This allows an annotation processor to generate source code in one round, which will then be compiled and processed in the next one. The compiler continues adding additional rounds until no more source files are generated. This allows annotation processors to generate source code which is then processed, in the next round, by other annotation processors. Finally the compiler will do a final round to allow annotation processors to wrap up.
Our processor will do the following:
- Get all classes with the Bean annotation that are part of the current round
- Extract the necessary information from these beans
- Generate an unmarshaller for each annotated bean
The final round is skipped. This is reflected in the following main method body:
public boolean process(Set<? extends TypeElement> annotations, RoundEnvironment roundEnv) {
if (roundEnv.processingOver()) {
return false;
}
var elements = getElementsByAnnotations(annotations, roundEnv);
List<BeanInfo> beans = getBeans(elements);
beans.forEach(this::writeUnmarshaller);
return true;
}
The method returns true indicating that this processor ‘claims’ the Bean annotation. If you would like multiple processors to process the same annotation then return false.
Getting the elements to process is straigth forward:
private Set<Element> getElementsByAnnotations(Set<? extends TypeElement> annotations, RoundEnvironment roundEnv) {
return annotations.stream()
.flatMap(a -> roundEnv.getElementsAnnotatedWith(a).stream())
.collect(Collectors.toSet());
}
In order to get the information about the bean we need to go through the syntax tree. The tree itself is rather complex and there are multiple ways of traversing it. Personally I prefer to use the Visitor pattern.
private List<BeanInfo> getBeans(Set<Element> elements) {
return elements.stream()
.map(e -> e.accept(new BeanVisitor(), null))
.collect(Collectors.toList());
}
static class BeanVisitor extends ElementKindVisitor9<BeanInfo, Void> {
@Override
public BeanInfo visitTypeAsClass(TypeElement e, Void unused) {
var pe = (PackageElement) e.getEnclosingElement();
var packageName = pe.getQualifiedName().toString();
var simpleName = e.getSimpleName().toString();
var properties = e.getEnclosedElements().stream()
.flatMap(ee -> ee.accept(new BeanPropertyVisitor(), unused).stream())
.collect(Collectors.toList());
return new BeanInfo(packageName, simpleName, properties);
}
}
Since we are only interested classes we can override just the visitTypeAsClass method. In here we extract basic information such as package and class name. Information about the properties is extracted by iterating over the enclosed elements using another visitor.
static class BeanPropertyVisitor extends ElementKindVisitor9<Optional<PropertyInfo>, Void> {
@Override
public Optional<PropertyInfo> visitVariableAsField(VariableElement e, Void unused) {
var name = e.getSimpleName().toString();
var cls = e.asType().accept(new TypeVisitor(), null);
return Optional.of(new PropertyInfo(name, cls));
}
@Override
protected Optional<PropertyInfo> defaultAction(Element e, Void unused) {
return Optional.empty();
}
}
The second visitor extends the same base class, but this time we are only interested in fields. Extracting the name of the field is simple, however extracting the type is a bit more complex.
It is important to understand the distinction between Elements and Types.
Map<String, Integer>
This defines a type which consist of the element Map parameterized with the types String and Integer. The type String, in turn, consists of the element String with no parameters.
So in order to extract the class we are interested in we use 2 more visitors.
static class TypeVisitor extends TypeKindVisitor9<Class<?>, Void> {
@Override
public Class<?> visitPrimitiveAsBoolean(PrimitiveType t, Void unused) {
return boolean.class;
}
@Override
public Class<?> visitPrimitiveAsInt(PrimitiveType t, Void unused) {
return int.class;
}
@Override
public Class<?> visitDeclared(DeclaredType t, Void unused) {
return t.asElement().accept(new TypeElementVisitor(), null);
}
@Override
protected Class<?> defaultAction(TypeMirror e, Void unused) {
throw new IllegalArgumentException("Unsupported type: " + e);
}
}
static class TypeElementVisitor extends ElementKindVisitor9<Class<?>, Void> {
@Override
public Class<?> visitTypeAsClass(TypeElement e, Void o) {
try {
return Class.forName(e.getQualifiedName().toString());
} catch (ClassNotFoundException cnfe) {
throw new RuntimeException(cnfe);
}
}
}
Now that we have the information we need we can generate the unmarshaller. The best way to do this would probably be to use some sort of templating engine. However for this demonstration we can just use write out a number of Strings.
private void writeUnmarshaller(BeanInfo bean) {
try {
var packageName = bean.getPackageName();
var simpleName = bean.getSimpleName();
var className = simpleName + "Unmarshaller";
var sourceFile = processingEnv.getFiler().createSourceFile(packageName + "." + className);
var w = sourceFile.openWriter();
w.append("package " + packageName + ";\n");
w.append("import dev.sanjuroe.generation.Parser;\n");
w.append("import dev.sanjuroe.generation.Unmarshaller;\n");
w.append("import java.io.IOException;\n");
w.append("public class " + className + " implements Unmarshaller<" + simpleName + "> {\n");
w.append(" public " + simpleName + " read(Parser parser) throws IOException {\n");
w.append(" var bean = new " + simpleName + "();\n");
for (var property : bean.getProperties()) {
var setMethod = ReflectionUtils.determineSetter(property.getName());
var parseMethod = ReflectionUtils.determineParseMethod(property.getType());
w.append(" bean." + setMethod + "(parser." + parseMethod + "());\n");
}
w.append(" return bean;\n");
w.append(" }\n");
w.append("}\n");
w.close();
} catch (IOException e) {
throw new RuntimeException(e);
}
}
This will generate exactly the same (minus whitespace) code as the hand written unmarshaller.
package dev.sanjuroe.generation;
import dev.sanjuroe.generation.Parser;
import dev.sanjuroe.generation.Unmarshaller;
import java.io.IOException;
public class EmployeeUnmarshaller implements Unmarshaller<Employee> {
public Employee read(Parser parser) throws IOException {
var bean = new Employee();
bean.setId(parser.readInteger());
bean.setActive(parser.readBoolean());
bean.setFirstName(parser.readString());
bean.setLastName(parser.readString());
bean.setStartYear(parser.readInteger());
bean.setJobTitle(parser.readString());
return bean;
}
}
The full annotation processor can be found on GitHub. Obviously this code just an example and not anywhere near production quality. However for the our purpose it is sufficient.
In theory both the handwritten and generated code should perform the same. So let’s use JMH again to benchmark:
@State(Scope.Benchmark)
@OutputTimeUnit(TimeUnit.MILLISECONDS)
public class CompileTimeBenchmark {
byte[] input;
Unmarshaller<Employee> unmarshaller;
@Setup
public void setup() throws Exception {
this.input = Data.generateEmployee();
this.unmarshaller = new EmployeeUnmarshaller();
this.unmarshaller.init();
}
@Benchmark
public Employee benchmark() throws Throwable {
var parser = new DataInputParser(new ByteArrayInputStream(input));
return unmarshaller.read(parser);
}
}
And, as expected, the performance it the same (with in the error margins of the benchmark):
Benchmark Mode Cnt Score Error Units
d.s.g.codetime.CodeTimeBenchmark.benchmark thrpt 25 6206,019 ± 24,402 ops/ms
d.s.g.compiletime.CompileTimeBenchmark.benchmark thrpt 25 6215,756 ± 22,693 ops/ms
So we managed to automatically generate an unmarshaller without performance penalty. The downside is that we need the original source code and regenerate the unmarshaller every time the code changes. In the final part of this series we look at a way to lift both restrictions.