Welcome to the Mapbox Maps SDK for Android project. This document describes every step of the development cycle of our project and make sure you have everything you need to get your pull request (PR) landed. Your contributions are appreciated.
The buildsystem of Mapbox Maps SDK for Android is Gradle but there is a Make
file to make some tasks more convenient and discoverable to execute.
Familiarity with make is a plus but not a requirement.
This document will assume you are familiar with
git and will not get into details about git commands.
When not explicitly specified, commands described are assumed to be executed from the root of the project tree.
Clone the git repository:
$ git clone [email protected]:mapbox/mapbox-maps-android.git && cd mapbox-maps-android
The only requirement is to install the latest version of
Android Studio. After installed, run the
sdkmanager command line or
through the GUI interface to make sure the extra requirements to build Android are met.
Note: Android Gradle Plugin v7.0+ is used to build the project. That requires JDK 11 or higher installed
and JAVA_HOME environment variable pointing on it.
$ ${ANDROID_HOME}/tools/bin/sdkmanager \
"platform-tools" \
"platforms;android-33" \
"build-tools;30.0.3" \
"build-tools;33.0.0" \
"extras;android;m2repository" \
"patcher;v4" \
"extras;google;m2repository" \
"extras;m2repository;com;android;support;constraint;constraint-layout;1.0.2" \
"ndk;23.2.8568313"\
"cmake;3.22.1"
Optional : In case of errors related to ndk version, make sure the ndk path is set in Android Studio.
You can set ndk path manually by navigating to File -> Project Structure -> Android NDK Location
or follow the official guide Install and configure NDK
This repository is a regular Android Studio project. Open Android Studio and import this repository’s root mapbox-maps-android folder.
Before building, the project needs to be configured. A couple of configurations need to be in place before a successful build and test run can be made. Either export the following values as environement variables or update them inline in the source code:
The SDK_REGISTRY_TOKEN is a Mapbox access token, used during compile time, with a scope set to DOWNLOADS:READ.
This token allows to download all required Mapbox dependencies from a Mapbox Maven instance.
The DOWNLOADS:READ scope can be set when creating a new access token on https://account.mapbox.com/.
The token configuration can be found in the root build.gradle.kts of the project.
The MAPBOX_ACCESS_TOKEN is a mapbox access token, packaged as part of test application, to load Mapbox tiles and resources.
This token can be configured on https://account.mapbox.com/ and doesn't require any specific scopes.
After first compilation, a gitignored file, is generated in app/src/main/res/values/developer-config.xml.
To verify if everything has been correctly setup so far, you can run the following make command to compile the project:
$ make sdk-build
Before pushing your code and triggering the CI, it is important that all the
checks are passing. Checks are mostly code formatters and code generators. In
order to run the checks you need to execute the check target.
$ make check
This target encapsulates validating:
- ktlint
- lint
- kdoc validation
- code generation
As a convenience, the CI will run the checks again and will let you know if the code is completely formatted and if everything that was supposed to be generated was generated.
This project uses secret-shield to help block secrets such as access tokens from being exposed.
Install secret-shield by entering npm install -g @mapbox/secret-shield.
Install the pre-commit hook by running scripts/install-pre-commit/install-pre-commit.sh
We follow Kotlin Coding Conventions and official Kotlin Style Guide.
All the projects are configured with kotlin.code.style=official in gradle.properties, so you
may autoformat the code in Android Studio by default.
We use ktlint to validate formatting and there's a check on CI preventing PR from merge
if ktlint fails.
Check if format is correct with ktlint or ./gradlew ktlint.
Run ktlintFormat or ./gradlew ktlintFormat or make fix to automatically reformat everything and
fix all the issues.
We use binary-compatibility-validator and Metalava for tracking binary and source compatibility of the APIs we ship.
- When doing code changes that do not imply any changes in public API, no additional actions need to
be performed. The bots named
check-api-*will validate if there's any API breaking changes. - When doing code changes that imply changes in public API, whether it is a new API or adjustments to
the existing one,
check-api-*tasks will start to fail.make update-apishould be executed manually, the resulting diff in.apifile should be verified: only signatures you expected to change should be changed. - Commit the resulting
.apidiff along with code changes.
The Mapbox Maps SDK for Android is a multi module gradle project with Android Libraries and Android applications.
This module is the shared dependency across modules and contains primarily interfaces. This module enables us to independently develop other modules against interfaces and not rely on any concrete implementation. By having this strict decoupling, we makes sure we think well about the contracts a module has with other and what public API it exposes to consumers.
The sdk module is the main module developers integrate in their project. It contains the most important building blocks of the product that couldn't have been decoupled from the underlying c++ implementation at this time.
If a module is prefixed with the name module- it indicates it's a required module and will
be enforced to be available during compilation time. This enforcement is enabled and configured
through the annotation processor of mapbox-base-android.
If a module is prefixed with the name plugin- it indicates it's an optional module that will
be loaded automatically at runtime if it's part of the compilation classpath. Plugins can be
excluded with a gradle exclusion to disable a plugin fully and to save binary size. By
default we package the majority of the plugins found in this repository.
If a module is prefxied with the name extension- it indicates it's a collection of extension
functions that makes it more easy and fluent to integrate with an existing, low-level, API.s The code in those
modules use Kotlin extension functions to
add APIs to existing classes.
The app module is the internal test and development application. It hosts a variety of different example use-cases to test the code from all the other modules. This module can be referenced when building out other examples.
The project contains a large number of code generated files that should not be touched manually.
These can be recognised with a top file comment as // This file is generated.. Code generators
are hosted externally from this repository and generated code requires to be up-to-date when making a
pull request.
Mapbox Maps SDK for Android relies on unit, instrumentation, robo and benchmark test to verify our code correctness. No code is accepted by our CI if any of these tests are failing. When developing a new feature, you are required to write a test and make sure that the test coverage will not go down. A feature will be considered tested if an unit test is provided that verifies the business logic and a instrumentation test is provided that executes the codepath down to the C++ code and back.
When fixing a bug, you are also required to provide a test that fails if the fix for the bug is not applied. The test in this case, triggers the faulty behavior and will make sure the bug is not going to be reintroduced and our code regresses.
Unit tests are the tests that validate the business logic of the SDK. Since unit test on Android are executed on the JVM of the development machine, we aren't able to load native libraries and call into our C++ stack. Any of those interactions will need to be either mocked or shadowed. To achieve this we are using a couple of frameworks as Mockk and Robolectric.
Unit tests, for all modules, can be run locally with:
$ make unit-tests
Mutation testing is a technique used to check quality of unit tests and improve them. The main idea is to change the production code and cause the tests to fail. Each mutated version is called a mutant and tests detect and reject mutants by causing the behavior of the original version to differ from the mutant. Read more about mutation testing. For generating mutants on the JVM we are using Pitest library.
Because of such testing type takes lot of time you can only run it on necessary module:
$ ./gradlew plugin-compass:pitestDebug
Instrumentation tests are integration test that validate the integration with the Android OS and our C++ code. It allows to render the actual Map on an Android device or emulator. It's important that features are accompanied by a instrumentation test as C++ issues will only show at runtime.
Instrumentation tests are run on Firebase TestLab. The CI push a test application and an application with the tests to a physical device on the Firebase device farm. This device is fully allocated to our test at the moment the test is running.
The build step that pushes the app to Firebase will give you an URL that can be used for checking the phone logs in case of a crash or test failure. If you don't have access to Firebase, please ask a member of the Maps team.
In order to run tests on multiple devices, the last commit in a branch must have
multiple_devices message. We also run tests on multiple device automatically
when the release tag is added.
The CircleCI bot will wait for the Firebase test to finish, collect the results, publish them as build artifact and fail/succeed accordingly.
Instrumentation tests are divided across 3 modules:
- sdk: contains instrumentation tests that validate C++ integration of the SDK
- app: contains instrumentation tests that validate specific feature integrations
- extension-style-app: contains instrumentation tests for validating the style integration
Instrumentation tests can be run locally on a device or emulator with:
$ make instrumentation-tests
Next to hand written tests, we generate a large bulk of our tests. Either these are generated as part of a code generator which will generate a test specific to a feature or we generate smoke test that validate integration of a feature within our test application.
Generation of the latter, the sanity smoke tests, can be done with:
$ make generate-sanity-test
The source code of this generator is found under scripts/sanity-test and uses
ejs templates.
Firebase Robo tests are integration tests that involves providing invalid, unexpected, or random data as inputs to the test application. This allows us to find bugs that arise in edge-cases situations. Robo tests are executed as part of a CI run and run for 10 minutes.
Commit messages should read as prose. A commit message should tell a story about the what, how and why a change is happening. All commit messages should be pre fixed with the module their are changing, omitting the type of module.
[logo] Rename visibility feature to enabled.
This is to align plugin vocabulary across the codebase and the plugin system.
Enabled is now made part of the Plugin interface and removed from the Logo specific interface.
All tests and referneces in the SDK and test applications have been updated to resemble this change.
If your change is not trivial, it deserves a changelog entry, so we can tell
our users about it on the next release. Please include a changelog entry
as shown by the PR template. If no changelog needs to be added, please include
skip changelog label.
Kotlin API documentation can be generated with Dokka in build/dokka/htmlCollector with running
$ make dokka-html
Congratulations, you made a pull request. You most certainly added tests, a good PR description and commit message and all the bots are happy about your contribution.
The humans working on the project might have a constructive feedback to your code. You will need the approval of at least one team member of the Maps SDK team to land your patch. It is a rare occasion, except for trivial changes, that a patch will get approved with no comments.
Please take a moment to appreciate the reviewer feedback. Reviewing pull requests is an arduous and time consuming task. You can make the life of the reviewing easier by making small pull requests. It is easier to land a feature one small self contained functionality at a time. It is also OK to disagree with the reviewer or ask for more details on why a change is needed.
Sometimes we will approve with a "nit" or trivial request. One example is, "PR looks good but please fix that typo or let's use a more descriptive name for variable foobar". In this case you can fix your PR, push it again and land it as soon as the bots are green.
A reviewer will always trust the developer. It is the developer responsibility to test the feature end-to-end and make sure we can ship a state of the art map rendering engine to our users.
And as always, everyone is invited to review code.
The SDK, the default extensions and the default plugins use the same versioning, which is defined by the
VERSION_NAME variable in the root project's gradle.properties. However, the project also allows to use
separate versioning for individual subprojects/modules. Subprojects can define the MODULE_VERSION_NAME
in the subproject-level gradle.properties to overwrite the VERSION_NAME defined in the root project.
The CI is setup to publish new releases when new tag that follows the release tag pattern is created.
- To publish a Maps SDK release with all the default plugins/extensions, create a release git tag
v*.
Everyone is welcome to open issues or make feature requests on the Mapbox Maps SDK for Android repository. Make sure, in case of an issue, it can be reproduced with the latest version of Mapbox Maps SDK for Android, otherwise it is very likely it was already fixed. Search our issue tracker for similar issues, if you found a problem, maybe someone else also found the same problem and you may contribute to the issue report. Same goes for feature requests, it might be that someone else already requested the feature you want.
A great bug report will have many of the following:
- Detailed steps to reproduce the issue.
- The version of the SDK causing the problem.
- The hardware in use when the problem was detected (i.e. Samsung Galaxy A71).
- The version of the operating system (i.e. Android KitKat).
- The style JSON or URL used as the map style.
- The camera configuration (coordinates, zoom, bearing and pitch).
- When adding a GeoJSON source to the map, the URL for the GeoJSON.
- If customizations such as overlays and custom layers are in use and how.
- If using runtime styling, a list of changes used to mutate the style.
- A symbolicated backtrace in case of a crash.
- Screenshot or video demonstrating the problem.
- A test reduction, meaning a code example that can reproduce the problem.
The Mapbox Maps SDK for Android uses C++ code under the hood. By default, the used C++ binary will be stripped from symbols which means that stacktraces produced are not readable. To symbolicate C++ traces you can use ndk-stack and a binary that has the symbols attached.
Next to Mapbox Maven distribution mechanism, Mapbox Android binaries can be downloaded as a zipped archive. This zipped version contains the additional shared objects with debug symbols. Replace in following command the SDK_REGISTRY_TOKEN and VERSION with the appropriate values:
$ curl --user mapbox:{SDK_REGISTRY_TOKEN} https://api.mapbox.com/downloads/v2/mobile-maps-android-core/releases/android/{VERSION}/android-core-all.zip --output android-core-all.zip
After unzipping the archive you can run ndk-stack on the unstripped native binary and your stacktrace that was saved as a txt file.
$ ndk-stack -sym obj/arm64-v8a -dump trace.txt
Note that the stacktrace must be formatted properly, otherwise ndk-stack will not be able to symbolicate the stacktrace. Different crash reporting tools often format stacktraces differently, so you might need to reformat the stacktrace. The stacktrace should be formatted as follows:
*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
#00 pc acfed libmapbox-maps.so
...
#12 pc 51f408 libmapbox-maps.so (BuildId: 331a309c6d4e90a8)
e.g. start with asterisks line, followed by stack entries.
Stack entries should start with at least one space, then a hash sign with a stack entry number,
then pc, then the address (without the 0x prefix), then the library name.
The Mapbox Maps SDK for Android publishes snapshot releases to our API downloads infrastructure. These snapshots can be used in downstream project to verify bug fixes or preview new features. However, we do not recommend using snapshot releases in production environment.
By default, each commit to the main branch of the repository will trigger a snapshot release for the sdk, as well as all the plugins/extensions/modules, with an unique version name.
The name of the snapshot is defined in the VERSION_NAME in the project's gradle.properties file,
with the commit hash inserted before the -SNAPSHOT.
An example of snapshot for the sdk is com.mapbox.maps:android:10.0.0-a1d95e7a-SNAPSHOT.
The Mapbox Maps SDK for Android also features manually triggered snapshot releases from a developing branch.
If a last commit in a developing branch has publish_android_snapshot in a title or description,
the build-modules-and-instrumentation-tests CI step will create a snapshot release.
The name of the snapshot release would be:
${LAST_RELEASE_VERSION_BEFORE_THIS_COMMIT}-${BRANCH_NAME}-SNAPSHOT for example:
10.0.0-rc.1-peng-commit-message-based-snapshot-SNAPSHOT
Snapshot releases are distributed from a different Downloads API endpoint, you will need to add
this API endpoint to your project's build.gradle as follows:
allprojects {
repositories {
maven {
url 'https://api.mapbox.com/downloads/v2/snapshots/maven'
authentication {
basic(BasicAuthentication)
}
credentials {
// Do not change the username below.
// This should always be `mapbox` (not your username).
username = 'mapbox'
// Use the secret token you stored in gradle.properties as the password
password = project.properties['MAPBOX_DOWNLOADS_TOKEN'] ?: ""
}
}
}
}Where the password is the secret token and can be configured following the installation guide
And then update the Mapbox Maps SDK's version name to the snapshot version in your app's build.gradle.
Mapbox Maps SDK for Android allows enabling additional Mapbox traces that could be of help troubleshooting performance issues. Working with traces is possible only when app is debuggable:
<application>
...
android:debuggable="true"
...
</application>If the app is not debuggable and MapboxTracing will be used - runtime exception will be thrown.
There are 2 types of traces that could be enabled:
- Native rendering engine traces could be enabled with
MapboxTracing.enableCore(). Those traces include native render calls, style loading, requesting tiles and more. - Android platform traces could be enabled with
MapboxTracing.enablePlatform(). Those traces include Android render thread calls such as preparing / destroying surface.
It is also possible to enable all the traces with MapboxTracing.enableAll(). Tracing could be disabled with MapboxTracing.disableAll().
There are several ways of capturing traces for Android which are described in official documentation. As a reference we will describe the steps of recording traces using Android Studio Profiler:
- Run your app in profile mode e.g. by clicking following icon in Android Studio.
- When app is run, click on CPU in Profiler window and select
System Traceradio-button. When you are ready to actually record the traces, hit theRecordbutton.
- When you are done with recording, press
Stopbutton, wait Android Studio to parse your traces and hitExport system trace recordingto save trace recording locally.
After we exported trace recording we could use Perfetto to investigate the Mapbox traces.
- Open Perfetto and drag-and-drop your local trace recording file there.
- When it is loaded - navigate to your application package and un-squash it.
- Mapbox native render engine traces added by
MapboxTracing.enableCore()could be found at the bottom of your app process in separate groups. Those traces start withmbx:.
- Mapbox platform specific traces will be located inside other Android threads and not in dedicated groups, so you could use global search by
mbx:to locate them.
There are several other ways how to record traces.
Enable Atrace events (other optional probes can be enabled if needed).
After stop recording is pressed, trace will be visible in the UI.
python systrace.py -a "com.mapbox.maps.testapp" -o trace.html -b 20000