Building Android

This guide will asume you follow the official Sony build guide, but add some extra tips.

Initialize your build directory

Create a ~/android/build-env folder, cd into it.

Using the repo tool

Download and install the repo tool, either via your package manager or directly from Google. Instead of copying the file, it’s better to symlink:

mkdir -p ~/.local/bin
# Add ~/.local/bin to PATH
echo "export PATH=$HOME/.local/bin:$PATH" >> ~/.bashrc
# Symlink
ln -sf ~/android/.repo/repo/repo ~/.local/bin/repo
# Check:
readlink $(which repo)
# -> /home/builder/android/.repo/repo/repo

Then, run the repo init command:

repo init -u https://android.googlesource.com/platform/manifest \
  -b android-10.0.0_r39 -g default,-x86,-mips,-darwin,-notdefault

Use the appropriate branch tag instead of _r39. For an overview of currently published tags, see Codenames, Tags and Numbers. Most often, you’ll want to pick the latest Pixel tag, e.g. the Pixel 4.

Ubuntu chroot

If you’re on another system like Fedora or Arch Linux, you can set up a Ubuntu chroot build environment instead.

Another alternative to nspawn is to use docker, see Sony AOSP on docker.

A build chroot takes about 1-1.5GB of space and will save you from headaches, e.g. when Android modules aren’t compiled against your host glibc.

Install debootstrap, then run this (bionic is the ubuntu version codename for 18.04 LTS):

debootstrap --variant=buildd --arch=amd64 bionic ~/android/ubuntu-android

chroot into the newly created builder system, either via chroot or just using systemd-nspawn (Assuming you created your android build environment via repo init in ~/android/build-env):

sudo systemd-nspawn \
  -D /home/<your-username>/android/ubuntu-android
useradd -m -s /bin/bash builder
# -> No need to add a password, nspawn will log you in automatically

For Android Q, the way .img files are mounted during build has changed, e.g. resize2fs fails with a standard nspawn container.

ln -s /proc/self/mounts /etc/mtab

And add this to the options of systemd-nspawn:

    --bind=/dev/loop0:/dev/loop0 \
    --bind=/dev/loop1:/dev/loop1 \
    --bind=/dev/loop2:/dev/loop2 \
    --bind=/dev/loop3:/dev/loop3 \

(So that enough loop devices are available)

Install required packages

apt install software-properties-common python
add-apt-repository universe
apt update
apt install bison g++-multilib git gperf libxml2-utils make \
  zlib1g-dev zip liblz4-tool libncurses5
apt install openjdk-8-jdk
# Might be needed on Android 10:
apt install flex

# The following dependencies are all optional:
##############################################
# Faster recompilation
apt install ccache
apt install rsync libssl-dev aapt
# Kernel recompilation:
apt install bc autoconf automake autopoint autotools-dev bsdmainutils gawk \
  groff-base libarchive-zip-perl libpipeline1 libtimedate-perl libtool kmod
# For selinux:
apt install setools python-networkx policycoreutils

Cross-reference the needed packages with the official Sony build guide should this document be out of date.

If you’re building in a chroot: Exit out of the build system by typing exit (and pressing Ctrl-] three times for nspawn), then re-launch your build system and log in as your builder user. If you want to share your own ccache etc. with your build environment, bind the appropriate directories.

sudo systemd-nspawn \
--bind=/home/<your-username>/android/build-env/:/home/builder/build-env/ \
  --bind=/home/<your-username>/.ccache/:/home/builder/.ccache/ \
  -D /home/<your-username>/android/ubuntu-android \
  --user=builder

Local manifests

Clone the Sony local_manifests git repo as instructed. For legacy devices, meaning loire and tone, you need to checkout the android-10_legacy branch of local_manifests. For recent devices have to use the master branch.

git clone https://github.com/sonyxperiadev/local_manifests .repo/local_manifests
cd .repo/local_manifests
# For recent devices:
git checkout master
# For legacy devices
git checkout android-10_legacy

Applying needed patches

Run repo_update.sh. Your build directory will now be ready for a first build.

Prepare build environment

Run source build/envsetup.sh inside your build environment and then lunch aosp_<product>-userdebug.

To find out the product name of your device for aosp_<product>-userdebug, see the Sony Devices overview. E.g. for a single-SIM (“SS”) Xperia XZ, choose aosp_f8331, for a dual-SIM XZ, choose aosp_f8332.

You can also run lunch without any arguments to get a list of all available build targets. They will look like aosp_f83xx-... where f83xx is the model number of your device. You can choose between eng and userdebug builds, see Choose a target.
For your own usage, userdebug is most likely what you want.

The user target is meant for public releases and doesn’t include root. You’ll need to look into vendorsetup.sh (AndroidProducts.mk for Android Q) - and perhaps add it for your device - if you want to use it.

Build kernels

On Android Q, you need to use a script to build the kernel images. Navigate into kernel/sony/msm-4.9/common-kernel and run build-kernels-gcc.sh.
If building for a Kernel 4.14 device, go into kernel/sony/msm-4.14/common-kernel and use build-kernels-clang.sh instead.

Edit build-kernels-{gcc,clang}.sh and change the list of platforms in the script so that it looks like this:

# Override:
TONE="kagura"
PLATFORMS="tone"

This will save you from building the kernel for all Sony devices and only build for the Xperia XZ (“kagura”).

Build & Flash

For legacy non-A/B devices like the Xperia XZ, run this command:

make bootimage systemimage

Newer devices require building for more partitions, like vendor, dtbo, product, vbmeta and others.

In case your build stops at about 90% with this error, most of the time you should be fine by just restarting the build with -j 1¹.

[ 93% 12745/13686] //frameworks/base:hiddenapi-lists-docs Metalava [common]
FAILED: out/soong/[...]hiddenapi-lists-docs-stubs.srcjar

However, if this still fails, try cherry-picking this patch from LineageOS. See also this reddit post

Then, use fastboot to flash the generated images

cd out/target/product/kagura
fastboot flash boot boot.img
fastboot flash system system.img

For 4.14 devices, also flash the other partitions like vendor, dtbo etc.

Do not forget to flash the appropriate Software Binaries to the oem partition as well!

Optimize the build

A full build will take about two to three hours² on a beefed-out recent system (Core i7 8th gen 4 cores, 16GB RAM, good SSD). That time can however be cut down for successive builds to around an hour by utilizing ccache and parallelizing the build.

It is recommendable to use a ccache size of around 50-100GB, depending on whether you plan to build different ROMS or for multiple devices.

# On Android Pie and before:
_CCACHE_EXEC=prebuilts/misc/linux-x86/ccache/ccache
# On Android Q:
_CCACHE_EXEC=/usr/bin/ccache
# Set ccache size:
$_CCACHE_EXEC -M 50G

Then set the environment variable with export USE_CCACHE=1. This variable needs to be set anew every time you open a new terminal, so it is advisable to put export USE_CCACHE=1 at the end of your .bashrc in your home directory (or /home/builder/.bashrc if you’re in a chroot).

ccache compression might also save a lot of disk space for you. Set export CCACHE_COMPRESS=1 in your .bashrc. This cuts ccache disk usage down to about 5GB per device, but may incur some performance penalty.

To use parallel compilation, you need to find out how many threads your CPU has via nproc. Then use that number for make -j<nr-of-threads>. For most computers it will be the number of CPU cores times 2, e.g. “8” for a 4-core CPU.

You can also set export WITHOUT_CHECK_API=true in your ~/.bashrc to skip the metalava checks which take up a lot of RAM. Obviously, you should run the metalava checks when you change any Java code yourself, but you can save a bit of time if you skip them for routine builds.

Distributing

If you want to share the fruits of your labor, you can create compressed flashable .zip files. Use make otapackage instead of make systemimage [...]. This will create a flashable zip file in out/target/product/<device-codename> named something like aosp_f8331-ota-eng.hostname-2018-10-19. You can flash this file via adb sideload or use a custom recovery like TWRP.

If you intend to share your builds publicly, you should generate and guard your own set of signing keys. See Signing builds for release for more information.

After you have tested your build on your own device, you can share the file freely, given that it contains free/libre software under the GPL and other free licenses. Anything GPL-licensed also demands you have to divulge your build sources. So put your patches into a git repo and share them with the world.

Speedier development

• Use m module.name -j $(nproc) to only rebuild a single module. You can then adb push or even adb sync the changes directly to your device
• Use make bootimage to re-generate kernel and ramdisk only (boot.img)
• Use adb push or adb sync instead of flashing full images. N.b.: This is only possible with disabled vbmeta.

Going further

To show all available build targets, use make modules.

Show commands used live: make showcommands.

Good references:
eLinux: The Android Build System

Edit source on Github