Skip to content

lessons

Jump to bottom
Miguel Gamboa edited this page Apr 20, 2023 · 28 revisions

Lessons:

Bibliography - The Well-Grounded Java Developer, Second Edition:

  • 1.1 - The language and the platform
  • 1.2 - The new Java release model
  • 4.1 - Class loading and class objects
  • 4.2 - Class loaders
  • 4.3 - Examining class files
  • 4.5 - Reflection
  • 4.5 - Bytecode
  • 7.1 - Performance terminology: Some basic definitions
  • 7.2 - A pragmatic approach to performance analysis
  • 7.3 - What went wrong? Why do we have to care?
  • 7.4 - Why is Java performance tuning hard?

Lesson 01 - Introduction and Modern VMs, e.g. JVM, Node

  • Bibliography and Lecturing methodology: github and slack.
  • Tools: javac, javap, kotlinc, JDK 17 and gradle.
  • Program outline in 3 parts:
    1. Java Type System and Reflection;
    2. Metaprogramming and Performance;
    3. Iterators versus Sequences (yield). Generics and Reified type parameters.
  • Project in 3 parts according to program outline.
  • Managed Runtime or Execution Environment.
  • Informally virtual machine (VM) or runtime
  • Historic evolution since Java to nowadays
  • Examples of languages targeting JVM: Java, kotlin, Scala, Clojure.
  • Examples of languages targeting Node: JavaScript, TypeScript, Kotlin.
  • Java syntax similar to C and C++.
  • Distinguish between Programming Language <versus> VM

Lesson 02 - Java Type System, Components and Metadata

  • Type System - Set of rules and principles that specify how types are defined and behave.
  • Module .classfor each class definition
  • CLASSPATH
    • e.g. -cp . - local folder
    • e.g. -cp .:JetBrains/IntelliJIdea2022.2/plugins/Kotlin/lib/*
    • (for windows use ; rather than :)
  • Class have Members
  • Members may be: Fields, Constructors or Methods.
  • Constructor is a function with name "<init>" returning void.
  • Using javap -p Student.class to inspect metadata
  • Using javap -c -p AppKt.class to inspect metadata and bytecode definition of data class Point
  • Analyzing kotin properties in Java.
  • There are NO properties in Java Type System.
  • A Kotlin property may generate:
    • Backing field -- accessed with getfield bytecode.
    • Getter, i.e. function get... -- called with invoke... bytecode.
    • Setter, i.e. function set... (if defined with var).
  • Kotlin val <=> Java final.
  • FIELDS => MEM
  • bytecode invokevirtual to call e.g. getNr() and print()
  • Component - Reusable software unit, with:
    • IR - code in intermediate representation (e.g. bytecode, IL, other)
    • Metadata - auto description
    • Ready to use => does not require static compilation.
    • API => conforms to a public interface.
    • Indivisible => 1 module (file)
  • Software development by Components:
    • Reduce Complexity;
    • Promote Reuse.
  • Developer roles:
    • Provider - provides a component with a well-known API (e.g. Point)
    • Client - uses the component from a provider to build an Application, or another component (e.g. App).
  • Managed Runtime:
    • Software components = Metadata + IR (intermediate representation) (e.g. bytecode)
    • Interoperability supported at bytecode and metadata level e.g. Java <> Kotlin
    • Portability - Compile once and run everywhere with VM, e.g. JRE.
    • Jitter - Just-in-time compiler
    • Safety - NO invalid memory accesses
    • GC - Garbage Collector.
    • ClassLoader and CLASSPATH - dynamic and lazy load.

Lesson 03 - Kotlin to JVM

  • Homework 01 resolution
  • Java Type descriptors
  • Java access control
  • Immutability through final fields equivalent to kotlin val
  • Virtual and non-virtual methods (Kotlin open versus Java final )
  • Optional @Override annotation in Java equivalent to kotlin keyword override
  • Java static members
  • Translating kotlin companion objects to Java
  • Translating kotlin extensions to Java

Lesson 04 - Managed versus Unmanaged

  • Unmanaged <versus> Managed
  • Static <versus> Dynamic link
  • Building unmanaged components with static link:
    • NOT a unit, but two parts instead: header + obj
    • Need of a header file that describes the component content
    • Different builds for different architectures
    • Structural modifications (new header) => compilation + link
    • Behavioral modifications => link
  • Demo with an unmanaged App using a Point.
    • Static compilation (gcc -c) and static link (gcc)
  • Demo with a managed App using a Point.
    • Jitter (just-in-time compiler) - compiles IR to native code (e.g. x86, amd64, ppc) at runtime
    • Dynamic link - Point.class on App compilation + link at runtime.
    • Lazy Load - Point.class is loaded only when needed

Lesson 05 - Programming with Metadata

  • Homework 02 resolution
  • Translating kotlin companion objects to Java
  • Java static nested classes e.g. class Person { static class Companion { .. }} => Person$Companion
  • Java static constructor
  • Singleton design pattern
  • Private instance constructor to avoid new objects.
  • Translating kotlin extensions to Java.
  • Reflection object oriented API for metadata
  • KClass is representing a type in Kotlin, whereas Class is representing a type in JVM.
  • KClass --- .java ---> Class
  • Class ---- .kotlin ---> KClass
  • TypeName.class - returns the Class corresponding to the type TypeName.
  • refObject.getClass() - returns the Class of the object referenced by refObject.
  • Java Reflection API (java.lang.reflect): Class, Member, Field, Executable, Constructor, Method and Parameter.
  • Inspecting functions trough Reflection API
  • Method => Class<?> getReturnType(); Parameter[] getParameters();
  • Parameter => Class<?> getType() - the declared type for the parameter represented by this Parameter.
  • Field => Class<?> getType() - the declared type for the field represented by this Field.
  • NOTICE do not mislead getType() with getClass()
  • Invoking methods and creating instances:
  • Method => Object invoke(Object obj, Object... args)
    • Invokes the underlying method represented by this Method object, on the specified object obj with the specified parameters args.
  • Check if a method is static - Modifier.isStatic(m.getModifiers())
  • Constructor => T newInstance(Object... initargs)
    • Uses the constructor represented by this Constructor object to create and initialize a new instance of the constructor's declaring class, with the specified parameters initargs.
  • ClassLoader - class-loading capability exposed to the user programmer.
    • BootstrapClassLoader - used to get the absolute basic system loaded—essentially java.base.
    • AppClassLoader - loads the application classes in CLASSPATH.
    • UrlClassLoader - loads classes and resources from a search path of URLs.
  • Example
static Stream<Class<?>>listClassesInClasspath() throws IOException {
    ClassLoader cl = ClassLoader.getSystemClassLoader();     // An AppClassLoader instance
    return list(cl.getResources("")).stream()
            .filter(url -> url.getProtocol().equals("file")) // Exclude jar files
            .map(url -> new File(toURI(url)))
            .flatMap(f -> f.isDirectory() ? stream(f.listFiles()) : Stream.of(f))
            .filter(f -> f.getName().contains(".class"))
            .map(f -> loadClass(cl, qualifiedName(f.getName())));
}

Lesson 06 - Programming with Metadata

  • Homework 03 resolution
  • Distinguishing between streams of java.io and java.util.stream.
  • java.io streams are related with implementations of InputStream and OutputStream.
  • java.util.stream provides abstractions on sequences and its operations like filter, map, reduce, etc.
  • Type check, e.g. if m is compatible with Method:
    • m instanceof Method => true if m is an instance of the class Method or any inherited class.
    • m.getClass() == Method => true if m has exactly the type Method.
  • get<member> versus getDeclared<member> to get inaccessible members (e.g. private).
  • setAccessible(boolean) => to invoke inaccessible methods or constructors (e.g. private)

Lesson 07 - Logger via reflection

  • Distinguishing the role of App domain ---> Logger utility.
  • Unit tests with sample domain Student and Point
  • Simple Logger ---> Printer
  • Decouple Logger output into a Printer interface.
  • Implement PrinterConsole and PrinterBuffer.
  • Make PrinterConsole singleton.
  • logFields(Object target)
  • logProperties(Object target) - for parameterless methods with get prefix and non void return type.

Lesson 08 - Logger and interface Getter and Annotations

  • Maintain getters on Map<Class<?>, List<Getter>>
  • interface Getter { void getAndPrint(target: Object) }
  • Distinct implementations: loadFieldGetters(), loadPropertyGetters(), ...
  • Annotations
  • isAnnotationPresent(Class<? extends Annotation> annotationClass)
  • getAnnotation(Class<T> annotationClass)
  • Retention
  • @NonLog and AltName(name="<alternate name>"

Lesson 11 - Logger and Annotations with Class parameter

  • Customizing the value printed by Logger.
  • How to pass a function to annotations?
  • Use Class as annotation parameter.

Lesson 14 - Reflection Exercises

  1. Develop the utility Java function Comparator<Object> comparerOrder(Class<?> klass) that returns an instance of Comparator that is able to compare objects of the type represented by klass, according to the properties which are both: Comparable and annotated with Comparison. Notice that you should compare respecting the order specified in annotation. Example:
class Student (
  @Comparison(2) val nr:Int,
  val name: String,
  @Comparison(1) val nationality: String,
)
 
val s1 = Student(12000, "Ana", "pt")
val s2 = Student(14000, "Ana", "pt")
val s3 = Student(11000, "Ana", "en")
val cmp = comparerOrder(Student::class)
assertTrue { cmp.compare(s1, s2) < 0 } // same nationality and 12000 is < 14000
assertTrue { cmp.compare(s2, s3) > 0 } // “pt” is > “en”
  1. Develop the utility Java function Comparator<Object> comparer(Class<?> klass) that returns a new intance of Comparator and it is able to compare instances of type represented by klass, according to the properties which are: Comparable OR annotated with a Comparison that specifies a Comparator for that property, according to the following example:
class Person(
  val id: Int,
  val name: String,
  @Comparison(cmp = AddressByRoad::class) val address: Address,
  @Comparison(cmp = AccountByBalance::class) val account: Account) {
}

class AccountByBalance : Comparator<Account>{
  override fun compare(o1: Account, o2: Account): Int {
    return o1.balance.compareTo(o2.balance);
  }
}

class AddressByRoad : Comparator<Address> {
  override fun compare(o1: Address, o2: Address): Int {
    return o1.road.compareTo(o2.road)
  }
}
 
val p1 = Person(11000, "Ana", Address("Rua Amarela", 24), Account("FD3R", 9900))
val p2 = Person(11000, "Ana", Address("Rua Rosa", 24), Account("8YH5", 9900))
val p3 = Person(11000, "Ana", Address("Rua Rosa", 24), Account("JK2E", 100))
val p4 = Person(11000, "Ana", Address("Rua Rosa", 97), Account("BFR5", 100))
val p5 = Person(17000, "Ana", Address("Rua Rosa", 97), Account("BFR5", 100))
val cmp = Comparer<Person>(Person::class)

assertTrue { cmp.compare(p1, p2) < 0 } // Rua Amarela is < Rua Rosa
assertTrue { cmp.compare(p2, p3) > 0 } // 9900 is > 100
assertEquals(0, cmp.compare(p3, p4))   // All properties are equal
assertTrue { cmp.compare(p4, p5) < 0 } // 11000 is < 17000

Lesson 14 - Bytecode

  • Reference Types are instantiated in bytecode with:
    • new - Allocates storage on Heap, initializes space and the object's header, and returns the reference to newbie object.
    • invokespecial - Call to class <init> method (corresponding to constructor).
  • Instantiating a refence type, e.g. Student(765134, "Ze Manel") may produce in bytecode:
new           #8   // class Student
dup                // duplicates the value on top of the stack
...                // One load (push) for each parameter of <init> (constructor)
invokespecial #14  // Method Student."<init>"
  • invokestatic - no thisrequired
  • invokespecial - static dispatch (i.e. non polymorphic)
  • invokevirtual - dynamic dispatch (i.e. polymorphic)
  • invokeinterface - dynamic dispatch (i.e. polymorphic) for interface methods
  • Constant Pool
  • 16-bit index into the constant pool
  • Load and store opcodes
  • Shortcut opcode forms
  • Arithmetic
  • Execution Flow

Lesson 15 - Benchmarking

  • Benchmark - assess the relative performance
  • Benchmark != Unit Tests
  • A naif approach - direct measurement, e.g.
    • measureTimeMillis { logger.log(Student(...)) }.also { println("logger.log() took $it millis"); }
  • Some Problems:
    1. Mixing domain instantiation (i.e. Student) with operation execution logger.log().
    2. First execution includes Jitter overhead and misses optimizations.
    3. Milliseconds could not be accurate enough.
    4. IO may be orders of magnitude slower than log operation itself
    5. IDE (e.g. InteliJ) may induce other overheads.
    6. Absolute results in milliseconds may vary on different hardware environments.
    7. System.currentTimeMillis() includes a System call with implicit overhead.
    8. Garbage Collector may degrade performance
  • Minimize side effects:
    1. Remove domain instantiation from operation measurement
    2. Include warm-up => Optimizations may improve performance
    3. Measure the total execution of several iterations rather than several measurements of single executions.
    4. Avoid IO => Mocking IO
    5. Avoid extra tools such as IDE (e.g. InteliJ), gradle, or other => run directly on VM (e.g. java)
    6. Baseline => How much can we improve performance?
    7. same as 3.
    8. GC => Run several iterations and discard most divergent results.
  • JMH - Java Microbenchmark Harness.
  • Benchmark tests annotated with @Benchmark.
  • JMH Gradle Plugin
  • gradlew jmhJar
  • java -jar <path to JAR> -f 1 -wi 4 -i 4 -w 2 -r 2 -tu ms:
    • -f - forks
    • -wi - warm-up iterations
    • -i - iterations
    • -w - each warm-up iteration duration
    • -r - each iteration duration
    • -tu - time unit

Lesson 16 - Meta-programming and Cojen Maker

  • Introduction to Metaprograming and dynamic code generation;
  • Cojen Maker API: ClassMaker, MethodMaker, FieldMaker
  • finish(): Class, finishTo(OutputStream)
  • Auxiliary function printBytecodes(bytes: ByteArray) of autorouter test project.

Lesson 17 - LoggerDynamic with Cojen Maker

  • Logger refactoring with a base type AbstractLogger extended by every logger approach: baseline, reflect or dynamic.
  • Implementing a dynamic builder for a logger using Cojen Maker:
    • ClassMaker buildLoggerDynamicForProperties(Class<?> domain)

Lesson 18 - Benchmarking

  • Saving from the dead-code elimination.
  • JMH Blackhole - "consumes" the values, conceiving no information to JIT whether the value is actually used afterwards.
  • JMH API: @Setup, @Param, @State
  • Comparing performance between: baseline, reflect or dynamic.
Clone this wiki locally