Android Q Changes

sepolicy

sepolicy has new neverallows.

ccache

ccache is no longer supported by default, the prebuilt ccache exec has been removed.

See build/core/ccache.mk:

We no longer provide a ccache prebuilt.

Ours was old, and had a number of issues that triggered non-reproducible results and other failures. Newer ccache versions may fix some of those issues, but at the large scale of our build servers, we weren’t seeing significant performance gains from using ccache – you end up needing very good locality and/or very large caches if you’re building many different configurations.

Local no-change full rebuilds were showing better results, but why not just use incremental builds at that point?

So if you still want to use ccache, continue setting USE_CCACHE, but also set the CCACHE_EXEC environment variable to the path to your ccache executable.

Install ccache and set export CCACHE_EXEC=/usr/bin/ccache in your ~/.bashrc.

Recovery needed for packaging OTAs

FAILED: ninja: unknown target 'otapackage'

To get OTA packaging for legacy devices to work via make otapackage, the target device needs to enable building a recovery image explicitly now.

PRODUCT_BUILD_RECOVERY_IMAGE := true

Since Android Pie/Q, certain OTA parameters are extracted from an intermediate recovery image.

Devices are now supposed to use BOARD_USES_RECOVERY_AS_BOOT and as such the build process - even for legacy devices that do not use recovery-as-boot - is now structured such that it creates the boot image by first contructing a recovery image and patching that.

On Pie, the recovery image was built if it wasn’t disabled explicitly:

# build/make/core/Makefile
ifeq (,$(filter true, $(TARGET_NO_KERNEL) $(TARGET_NO_RECOVERY)))
[... build recoveryimage]

On Q, building a recovery image must now be enabled since the recovery buildvars, including $(recovery_fstab), are guarded by BUILDING_RECOVERY_IMAGE.
See AOSP Gerrit: Add BUILD_RECOVERY_IMAGE and BUILD_BOOT_IMAGE flags.

# build/make/core/board_config.mk
ifeq ($(BOARD_USES_RECOVERY_AS_BOOT),true)
  BUILDING_RECOVERY_IMAGE := true
else ifeq ($(PRODUCT_BUILD_RECOVERY_IMAGE),)
  ifdef BOARD_RECOVERYIMAGE_PARTITION_SIZE
    ifeq (,$(filter true, $(TARGET_NO_KERNEL) $(TARGET_NO_RECOVERY)))
      BUILDING_RECOVERY_IMAGE := true
    endif
  endif
else ifeq ($(PRODUCT_BUILD_RECOVERY_IMAGE),true)
  BUILDING_RECOVERY_IMAGE := true
endif
# build/make/core/Makefile
[...]
ifdef BUILDING_RECOVERY_IMAGE
  [...]
  ifdef TARGET_RECOVERY_FSTAB
    recovery_fstab := $(TARGET_RECOVERY_FSTAB)
  else
    recovery_fstab := $(strip $(wildcard $(TARGET_DEVICE_DIR)/recovery.fstab))
  endif
  [...]
endif # BUILDING_RECOVERY_IMAGE

Since $(recovery_fstab) is needed by build_otatools_package, failing to enable the recovery image leads to the loss of the otapackage target. Simplified code:

# build/make/core/Makefile
[...]
build_ota_package := true
ifeq ($(recovery_fstab),)
  build_ota_package := false
endif
[...]
ifeq ($(build_ota_package),true)
  [...]
  .PHONY: otapackage
  otapackage: $(INTERNAL_OTA_PACKAGE_TARGET)
[...]
endif # build_ota_package

PATH restrictions

If some of your makefiles call make, perl, gcc or some other host-installed tool directly, you will run into an error quite like this one:

Disallowed PATH tool "make" used: \
  []string{"make", "-j4", "-j8", "-C", "/home/[...]/kernel/", \
  "O=../../../../out/target/product/mydevice/obj/kernel/msm-4.9", \
  "ARCH=arm64", "CROSS_COMPILE=/home/[...]/aarch64-linux-android-", \
  "KCFLAGS=-mno-android"}
See https://android.googlesource.com/platform/build/+/master/Changes.md#PATH_Tools for more information.

The stopgap solution is to set export TEMPORARY_DISABLE_PATH_RESTRICTIONS=true in your build environment, but it is worth exploring the reasons behind this.

Beginning to dig further, not even inspecting the PATH works:

builder@android:~/android$ get_build_var PATH

build/make/core/dumpvar.mk:29: error: PATH is obsolete. Do not use PATH directly.
See https://android.googlesource.com/platform/build/+/master/Changes.md#PATH.
22:32:12 dumpvars failed with: exit status 1

One can modify build/make/core/config.mk to temporarily allow using PATH by commenting out this line:

$(KATI_obsolete_var PATH,Do not use PATH directly. See $(CHANGES_URL)#PATH)

So, why all this?

Reasons

With Android Q, Google has restricted the usage of tools in $PATH, ostensibly to cut down on too much variance in build environments. Now, usage of “host tools” ¹ which are not shipped as checked-in prebuilts is heavily discouraged.

The $PATH that is visible to the makefiles inside the Android build system is now set via build/soong/ui/build/path.go and only consists of: prebuilts/build-tools/path/linux-x86:out/.path.

prebuilts/build-tools/path/linux-x86 is made up of symlinks to prebuilt common coreutils etc. such as awk, cat, cp, python and tar, some of which are provided via a host toybox.

.out/path/ is a bit more complicated. It only contains symlinks that all point to a go binary named path_interposer. This path_interposer - source here - checks whether the program the symlink points to is an allowed PATH tool. Which host tools are allowed is specified in soong/build/paths/config.go: They’re either Allowed, Forbidden, or allowed-but-logged(Log) (or some other obscure options).

Measures

As suggested in the Build System Changes advisory, one should stop relying on $PATH in makefiles directly, but if needed, commands can still be invoked with one-off environment variables:

$(OUT)/my-bin: $(MY_INTERMEDIATE)
	PATH=$$(cd device/ti/beagle_x15/hostcc; pwd):$$PATH \
	create-my-out-bin

As for invoking make, especially for kernels, using the full path to a prebuilt make binary instead of relying on the host /usr/bin/make fixes the issue. Google ships a prebuilt make in prebuilts/build-tools/.

# my-kernel.mk
MY_MAKE := \
	PATH="$(PWD)/prebuilts/my-prebuilts/bin:$$PATH" \
	$(PWD)/prebuilts/build-tools/linux-x86/bin/make
[...]
# Stop using $(MAKE) directly. Instead of:
#$(MAKE) -C $(OUT)/my-dir ...
# Use:
$(MY_MAKE) -C $(OUT)/my-dir ...

For more kernel-specific information, including how you can switch to clang, have a look at Android Q: Compiling Kernels in-tree.

More background info

See Google’s Dan Willemsen in answer to some MediaTek guys:

It’s easy to use prebuilt tools from the source directory (or built tools) from your own rules that need them, either just refer to them by name, or you may be able to set up your own directory to be added to PATH inside a rule:
beagle-x15 uboot.mk, beagle-x15 hostcc dir

We’re not going to add an extension that allows modifying the environment of every build command – that causes problems with the attempts to build common system images that aren’t influenced by the device. The common build commands should not need anything except our standard tools – if you’ve modified them, it should be easy enough to also add the dependency to the new tools as well.

On tools:

• We’ve deprecated GCC during Android builds, we recommend clang for compilation.
• We do have prebuilts of make and some other common tools in prebuilts/build-tools/linux-x86/bin
• ccache is not something that we provide anymore – we’ve found it’s generally better to use incremental builds, as ccache can slow down your builds with extra I/O, and incremental builds should have relatively few cache hits (unless you’re sharing the ccache dir; which runs into even more I/O issues).

Perl is not something supported at Google, so we don’t provide that either – we do have built in support for python and the ability to use python packages from the source tree (or checked in as a prebuilt, when using embedded_launcher, it’s a self-contained binary): python_binary_host

Addendum Jan 2021

A great example which lists this page as reference is this adaptation to Q of busybox from the android-x86 project.

GCC deprecation

GCC 4.9 is being ever more aggressively deprecated. A delay is added to every call to the prebuilt GCC and a warning is shown - Google cheekily calls this a “Sleep Constructor” ².

Current plans: See GCC_4_9_DEPRECATION.md

• January 2019
  • Move GCC binary to new file name.
  • Create shell wrapper with old GCC binary name that calls new binary name.
  • Shell wrapper print deprecation warning to stdout, with a link to more info.
• April 2019
  • Move GCC binary name.
  • Add 1s sleep to wrapper.
• July 2019
  • Move GCC binary name.
  • Total 3s sleep.
• October 2019
  • Move GCC binary name.
  • Total 10s sleep.
• January 2020
  • Remove GCC from Android.

New HALs

See: hardware/interfaces, system/hardware/interfaces, hardware/google/interfaces, frameworks/hardware/interfaces.

New build.prop location

See system/core: init: Moving /odm/build.prop to /odm/etc/build.prop There is no more fallback reading of /odm/build.prop anymore.

author Bowgo Tsai <bowgotsai@google.com> Fri May 17 15:40:18 2019 +0800
committer Bowgo Tsai <bowgotsai@google.com> Wed May 22 16:15:44 2019 +0800

Moving /odm/build.prop to /odm/etc/buid.prop

In device root directory, we have the following symlinks:
  - /odm/app -> /vendor/odm/app
  - /odm/bin -> /vendor/odm/bin
  - /odm/etc -> /vendor/odm/etc
  ...

This allows the Generic System Image (GSI) to be used on both devices:
  1) Has a physical odm partition, where those symlink will be hidden
     when /odm is used as the mount point
  2) Has no physical odm partition and fallback to /vendor/odm/.

We can't just have the symlink /odm -> /vendor/odm, because the former
devices won't have /vendor/odm directory, which leads to mount failure
when the mount point /odm is resolved to /vendor/odm.

The existing /vendor/odm/build.prop won't be loaded in the latter
devices, because there is no symlink
    - /odm/build.prop -> /vendor/odm/build.prop.

Note that init blocks reading through direct symlinks (O_NOFOLLOW) so
the above symlink won't work either. This CL moves the odm build.prop
to /odm/etc/build.prop for init to load it (symlinks in earlier
components of the path will still be followed by O_NOFOLLOW).

Bug: 132128501
Test: boot a device and checks /odm/etc/build.prop is loaded
Change-Id: I0733c277baa67c549bb45599abb70aba13fbdbcf
Merged-In: I0733c277baa67c549bb45599abb70aba13fbdbcf
(cherry picked from commit c49655b2a48eca2ab751b853a9a4692f322cafa2)

Published by Felix on 16 Mar, 2019

Edit source on Github