Lately, Android Open Source Project has gone through various changes. For instance, Since JB Google decided to replace bluez bluetooth stack with an open source stack implemented by Broadcom claiming that the later is more optimized for Android devices causing a headache and new bugs for many users and developers, also there are frequent architectural changes (for example : the HAL layer) and so on…
The one fact that proved not to change through Android Open Source project lifetime is that it’s documentation is very poor and that was the main reason for me compiling this document from various blogs, android docs (which are gratefully referred to at the end of the page) and my own experience trying to produce as complete document as possible for the Android Build System including steps to build the source tree , some build tricks and hints!
Also, I tried to answer the famous question : “How do I add a java (or native) application into my android build ?“
I hope this post shall be useful 🙂
Table Of Contents
- ANDROID BUILD SYSTEM
- Building Steps
- Building Tricks
- Build System Architecture
- How to add another component(native or java application) to the build
The build system uses some pre-set environment variables and a series of ‘make’ files in order to build an Android system and prepare it for deployment to a platform.
Android make files end in the extension ‘.mk’ by convention, with the main make file in any particular source directory being named ‘Android.mk’.
There is only one official file named ‘Makefile’, at the top of the source tree for the whole repository. You set some environment variables, then type ‘make’ to build stuff. You can add some options to the make command line (other targets) to turn on verbose output, or perform different actions.
The build output is placed in ‘out/host’ and ‘out/target’ Stuff under ‘out/host’ are things compiled for your host platform (your desktop machine). Stuff under ‘out/target/product/<platform-name>’ eventually makes it’s way to a target device (or emulator).
The directory ‘out/target/product/<platform-name>/obj’ is used for staging “object” files, which are intermediate binary images used for building the final programs. Stuff that actually lands in the file system of the target is stored in the directories root, system, and data, under ‘out/target/product/<platform-name>’. Usually, these are bundled up into image files called system.img, ramdisk.img, and userdata.img.
This matches the separate file system partitions used on most Android devices.
In order to decide what to build, and how to build it, the build system requires that some variables be set. Different products, with different packages and options can be built from the same source tree. The variables to control this can be set via a file with declarations of ‘make’ variables, or can be specified in the environment.
To set up your build environment, you need to load the variables and functions in build/envsetup.sh. Do this by ‘source-ing’ the file into your shell environment, like this:
$ source build/envsetup.sh
$ . build/envsetup.sh
You can type
at this point to see some utility functions that are available to make it easier to work with the source.
Invoke ". build/envsetup.sh" from your shell to add the following functions to your environment: - lunch: lunch <product_name>-<build_variant> - tapas: tapas [<App1> <App2> ...] [arm|x86|mips] [eng|userdebug|user] - croot: Changes directory to the top of the tree. - m: Makes from the top of the tree. - mm: Builds all of the modules in the current directory. - mmm: Builds all of the modules in the supplied directories. - cgrep: Greps on all local C/C++ files. - jgrep: Greps on all local Java files. - resgrep: Greps on all local res/*.xml files. - godir: Go to the directory containing a file. Look at the source to view more functions. The complete list is: addcompletions add_lunch_combo cgrep check_product check_variant choosecombo chooseproduct choosetype choosevariant cproj croot findmakefile gdbclient get_abs_build_var getbugreports get_build_var getlastscreenshot getprebuilt getscreenshotpath getsdcardpath gettargetarch gettop godir hmm isviewserverstarted jgrep key_back key_home key_menu lunch _lunch m mm mmm pid printconfig print_lunch_menu resgrep runhat runtest set_java_home setpaths set_sequence_number set_stuff_for_environment settitle smoketest startviewserver stopviewserver systemstack tapas tracedmdump
To select the set of things you want to build, and what items to build for, you use either the ‘choosecombo’ function or the ‘lunch’ function. ‘choosecombo’ will walk you through the different items you have to select, one-by-one, while ‘lunch’ allows you select some pre-set combinations.
The items that have to be defined for a build are:
the product (‘generic’ or some specific board or platform name)
the build variant (‘user’, ‘userdebug’, or ‘eng’)
whether you’re running on a simulator (‘true’ or ‘false’)
the build type (‘release’ or ‘debug’)
Seeing the actual commands used to build the software
Use the “showcommands” target on your ‘make’ line:
$ make -j4 showcommands
This can be used in conjunction with another make target, to see the commands for that build. That is, ‘showcommands’ is not a target itself, but just a modifier for the specified build.
In the example above, the -j4 is unrelated to the showcommands option, and is used to execute 4 make sessions that run in parallel.
Here is a list of different make targets you can use to build different parts of the system:
make sdk - build the tools that are part of an SDK (adb, fastboot, etc.) make snod - build the system image from the current software binaries make services make runtime make droid - make droid is the normal build. make all - make everything, whether it is included in the product definition or not make clean - remove all built files (prepare for a new build). Same as rm -rf out/<configuration>/ make modules - shows a list of submodules that can be built (List of all LOCAL_MODULE definitions) make <local_module> - make a specific module (note that this is not the same as directory name. It is the LOCAL_MODULE definition in the Android.mk file) make clean-<local_module> - clean a specific module
Speeding up the build
You can use the ‘-j’ option with make, to start multiple threads of make execution concurrently.
You can also specify to use the ‘ccache’ compiler cache, which will speed up things once you have built things a first time. To do this, specify ‘export USE_CCACHE=1’ at your shell command line. (Note that ccache is included in the prebuilt section of the repository, and does not have to be installed on your host separately.)
Building only an individual program or module
If you use build/envsetup.sh, you can use some of the defined functions to build only a part of the tree. Use the ‘mm’ or ‘mmm’ commands to do this.
The ‘mm’ command makes stuff in the current directory (and sub-directories, I believe). With the ‘mmm’ command, you specify a directory or list of directories, and it builds those.
Build helper functions
A whole bunch of build helper functions are defined in the file build/core/definitions.mk
Try grep define build/core/definitions.mk for an exhaustive list.
Here is a snapshot of the file
########################################################### ## Find all of the java files from here. Meant to be used like: ## SRC_FILES := $(call all-subdir-java-files) ########################################################### define all-subdir-java-files $(call all-java-files-under,.) endef ########################################################### ## Find all of the c files under the named directories. ## Meant to be used like: ## SRC_FILES := $(call all-c-files-under,src tests) ########################################################### define all-c-files-under $(patsubst ./%,%, \ $(shell cd $(LOCAL_PATH) ; \ find $(1) -name "*.c" -and -not -name ".*") \ ) endef
Build System Architecture
to be added….
How to add another component to the build
It’s simple to add a new java or native application to your android build , you just
- create the directory and copy your src files to it.
- then you should add the appropriate Android.mk file, the next section will help you to understand Android.mk files and write your own makefile.
- build the image and flash it to your device.
Template Android.mk files
- Executable Template
LOCAL_PATH:= $(call my-dir) # call function my-dir will return the path of Android.mk include $(CLEAR_VARS) # clean all variables mainly started with LOCAL_ LOCAL_SRC_FILES:= foo.c # Source file list LOCAL_MODULE:= foo # The name of executable binary include $(BUILD_EXECUTABLE) # Start to build executable binary
- Shared Library Template
LOCAL_PATH:= $(call my-dir) # call function my-dir will return the path of Android.mk include $(CLEAR_VARS) # clean all variables mainly started with LOCAL_ LOCAL_SRC_FILES:= foo.c bar.c # Source file list LOCAL_MODULE:= libfoo # The name of shared library LOCAL_PRELINK_MODULE := false # Prevent from prelink error include $(BUILD_SHARED_LIBRARY) # Start to build shared library
- Static Library Template
LOCAL_PATH:= $(call my-dir) # call function my-dir will return the path of Android.mk include $(CLEAR_VARS) # clean all variables mainly started with LOCAL_ LOCAL_SRC_FILES:= $(call all-subdir-c-files) # Source file list LOCAL_MODULE:= libbar # The name of static library LOCAL_PRELINK_MODULE := false # Prevent from prelink error include $(BUILD_STATIC_LIBRARY) # Start to build static library
- If you have a tool that generates a source file from an input file using a user specified tool
SRC := $(call my-dir)/include/vnd_generic.txt GEN := $(intermediates)/vnd_buildcfg.h TOOL := $(TOP_DIR)external/bluetooth/bluedroid/tools/gen-buildcfg.sh $(GEN): PRIVATE_PATH := $(call my-dir) $(GEN): PRIVATE_CUSTOM_TOOL = $(TOOL) $< $@ $(GEN): $(SRC) $(TOOL) $(transform-generated-source) LOCAL_GENERATED_SOURCES += $(GEN)
- Adding a prebuilt library
include $(CLEAR_VARS) LOCAL_MODULE := foo-prebuilt LOCAL_SRC_FILES := libfoo.so LOCAL_EXPORT_C_INCLUDES := $(LOCAL_PATH)/include include $(PREBUILT_SHARED_LIBRARY)
The LOCAL_EXPORT_C_INCLUDES definition here ensures that any module that
depends on the prebuilt one will have its LOCAL_C_INCLUDES automatically
prepended with the path to the prebuilt’s include directory, and will thus
be able to find headers inside that.
What is the LOCAL_MODULE_TAGS variable ?
we can add LOCAL_MODULE_TAGS variable to Android.mk file to determine that module to be installed in that source code built. Here are the some defined tags and their meanings
– Default variant
– Installs modules tagged with: eng, debug, user, and/or development
– Installs non-apk modules that have no tags specified.
– Installs APKs according to the product definition files.
– adb is enabled by default.
– Final release
– Installs modules tagged with user
– Installs non-apk modules that have no tags specified
– Install APKs according to the product definition files
– adb is disabled by default.
- userdebug, Same as user except:
– Also installs modules tagged with debug
– adb is enabled by default
How to include a shared library in your component ?
LOCAL_SHARED_LIBRARIES := libfoo
How to include a static library in your component ?
These are the static libraries that you want to include in your module.
Mostly, we use shared libraries, but there are a couple of places, like executables in sbin and host executables where we use static libraries instead.
LOCAL_STATIC_LIBRARIES := libbar
These are the static libraries that you want to include in your module without allowing the linker to remove dead code from them.
This is mostly useful if you want to add a static library to a shared library and have the static library’s content exposed from the shared library.
LOCAL_WHOLE_STATIC_LIBRARIES := libbar
Include path list
LOCAL_C_INCLUDES += \ usr/include \ usr/local/include \ $(LOCAL_PATH)/include
Using c/cpp/cxx/ld flags
LOCAL_CFLAGS += -DONLY_C_NEEDED LOCAL_CXXFLAGS += -DONLY_CXX_NEEDED LOCAL_CPPFLAGS += -DBOTH_C_CXX_NEEDED LOCAL_LDFLAGS += -Wl,--exclude-libs=libgcc_eh.a LOCAL_LDLIBS += -lpthread
Call subdir’s Android.mk
Not recursively, just the directly sudir.
include $(call all-subdir-makefiles)
A full list of Android.mk variables
These are the variables that you’ll commonly see in Android.mk files, listed alphabetically.
But first, a note on variable naming:
LOCAL_ – These variables are set per-module. They are cleared by the include $(CLEAR_VARS) line, so you can rely on them being empty after including that file. Most of the variables you’ll use in most modules are LOCAL_ variables.
PRIVATE_ – These variables are make-target-specific variables. That means they’re only usable within the commands for that module. It also means that they’re unlikely to change behind your back from modules that are included after yours. Check the make documentation to find more about target-specific variables.
INTERNAL_ – These variables are critical to functioning of the build system, so you shouldn’t create variables named like this, and you probably shouldn’t be messing with these variables in your makefiles.
HOST_ and TARGET_ – These contain the directories and definitions that are specific to either the host or the target builds. Do not set variables that start with HOST_ or TARGET_ in your makefiles.
BUILD_ and CLEAR_VARS – These contain the names of well-defined template makefiles to include. Some examples are CLEAR_VARS and BUILD_HOST_PACKAGE.
Any other name is fair-game for you to use in your Android.mk. However, remember that this is a non-recursive build system, so it is possible that your variable will be changed by another Android.mk included later, and be different when the commands for your rule / module are executed.
1 – NDK-provided variables:
These GNU Make variables are defined by the build system before
your Android.mk file is parsed. Note that under certain circumstances
the NDK might parse your Android.mk several times, each with different
definition for some of these variables.
Points to a build script that undefines nearly all LOCAL_XXX variables
listed in the “Module-description” section below. You must include
the script before starting a new module, e.g.:
Points to a build script that collects all the information about the
module you provided in LOCAL_XXX variables and determines how to build
a target shared library from the sources you listed. Note that you
must have LOCAL_MODULE and LOCAL_SRC_FILES defined, at a minimum before
including this file. Example usage:
#Note that this will generate a file named lib$(LOCAL_MODULE).so
A variant of BUILD_SHARED_LIBRARY that is used to build a target static
library instead. Static libraries are not copied into your
project/packages but can be used to build shared libraries (see
LOCAL_STATIC_LIBRARIES and LOCAL_WHOLE_STATIC_LIBRARIES described below).
#Note that this will generate a file named lib$(LOCAL_MODULE).a
Points to a build script used to specify a prebuilt shared library.
Unlike BUILD_SHARED_LIBRARY and BUILD_STATIC_LIBRARY, the value
of LOCAL_SRC_FILES must be a single path to a prebuilt shared
library (e.g. foo/libfoo.so), instead of a source file.
You can reference the prebuilt library in another module using
the LOCAL_PREBUILTS variable (see docs/PREBUILTS.html for more information).
This is the same as PREBUILT_SHARED_LIBRARY, but for a static library
file instead. See docs/PREBUILTS.html for more.
Name of the target CPU architecture as it is specified by the
full Android open-source build. This is ‘arm’ for any ARM-compatible
build, independent of the CPU architecture revision.
Name of the target Android platform when this Android.mk is parsed.
For example, ‘android-3’ correspond to Android 1.5 system images. For
a complete list of platform names and corresponding Android system
images, read docs/STABLE-APIS.html.
refer to /build/core/envsetup.mk for more TARGET_ variables
2 – NDK-provided function macros:
The following are GNU Make ‘function’ macros, and must be evaluated
by using ‘$(call <function>)’. They return textual information.
Returns the path of the last included Makefile, which typically is
the current Android.mk’s directory. This is useful to define
LOCAL_PATH at the start of your Android.mk as with:
LOCAL_PATH := $(call my-dir)
IMPORTANT NOTE: Due to the way GNU Make works, this really returns
the path of the last included Makefile during the parsing of
build scripts. Do not call my-dir after including another file. For example, consider the following example:
LOCAL_PATH := $(call my-dir) ... declare one module include $(LOCAL_PATH)/foo/Android.mk LOCAL_PATH := $(call my-dir) ... declare another module
The problem here is that the second call to ‘my-dir’ will define
LOCAL_PATH to $PATH/foo instead of $PATH, due to the include that
was performed before that. For this reason, it’s better to put additional includes after
everything else in an Android.mk, as in:
LOCAL_PATH := $(call my-dir) ... declare one module LOCAL_PATH := $(call my-dir) ... declare another module # extra includes at the end of the Android.mk include $(LOCAL_PATH)/foo/Android.mk
If this is not convenient, save the value of the first my-dir call
into another variable, for example:
MY_LOCAL_PATH := $(call my-dir) LOCAL_PATH := $(MY_LOCAL_PATH) ... declare one module include $(LOCAL_PATH)/foo/Android.mk LOCAL_PATH := $(MY_LOCAL_PATH) ... declare another module
Returns a list of Android.mk located in all sub-directories of
the current ‘my-dir’ path. For example, consider the following
sources/foo/Android.mk sources/foo/lib1/Android.mk sources/foo/lib2/Android.mk
If sources/foo/Android.mk contains the single line:
include $(call all-subdir-makefiles)
Then it will include automatically sources/foo/lib1/Android.mk and
sources/foo/lib2/Android.mk This function can be used to provide deep-nested source directory
hierarchies to the build system. Note that by default, the NDK
will only look for files in sources/*/Android.mk
Returns the path of the current Makefile (i.e. where the function
Returns the path of the parent Makefile in the inclusion tree,
i.e. the path of the Makefile that included the current one.
A function that allows you to find and include the Android.mk
of another module by name. A typical example is:
And this will look for the module tagged <name> in the list of
directories referenced by your NDK_MODULE_PATH environment
variable, and include its Android.mk automatically for you. Read docs/IMPORT-MODULE.html for more details.
3 – Module-description variables
The following variables are used to describe your module to the build
system. You should define some of them between an ‘include $(CLEAR_VARS)’
and an ‘include $(BUILD_XXXXX)’. As written previously, $(CLEAR_VARS) is
a script that will undefine/clear all of these variables, unless explicitly
noted in their description.
In Android.mk files that include $(BUILD_PACKAGE) set this to the set of files you want built into your app. Usually: LOCAL_ASSET_FILES += $(call find-subdir-assets) This will probably change when we switch to ant for the apps' build system.
If you want to use a different C compiler for this module, set LOCAL_CC to the path to the compiler. If LOCAL_CC is blank, the appropriate default compiler is used.
If you want to use a different C++ compiler for this module, set LOCAL_CXX to the path to the compiler. If LOCAL_CXX is blank, the appropriate default compiler is used.
If you have additional flags to pass into the C or C++ compiler, add them here. For example: LOCAL_CFLAGS += -DLIBUTILS_NATIVE=1
If you have additional flags to pass into only the C++ compiler, add them here. For example: LOCAL_CPPFLAGS += -ffriend-injection LOCAL_CPPFLAGS is guaranteed to be after LOCAL_CFLAGS on the compile line, so you can use it to override flags listed in LOCAL_CFLAGS.
If your C++ files end in something other than ".cpp", you can specify the custom extension here. For example: LOCAL_CPP_EXTENSION := .cc Note that all C++ files for a given module must have the same extension; it is not currently possible to mix different extensions.
Normally, the compile line for C and C++ files includes global include paths and global cflags. If LOCAL_NO_DEFAULT_COMPILER_FLAGS is non-empty, none of the default includes or flags will be used when compiling C and C++ files in this module. LOCAL_C_INCLUDES, LOCAL_CFLAGS, and LOCAL_CPPFLAGS will still be used in this case, as will any DEBUG_CFLAGS that are defined for the module.
Additional directories to instruct the C/C++ compilers to look for header files in. These paths are rooted at the top of the tree. Use LOCAL_PATH if you have subdirectories of your own that you want in the include paths. For example: LOCAL_C_INCLUDES += extlibs/zlib-1.2.3 LOCAL_C_INCLUDES += $(LOCAL_PATH)/src You should not add subdirectories of include to LOCAL_C_INCLUDES, instead you should reference those files in the #include statement with their subdirectories. For example: #include <utils/KeyedVector.h> not #include <KeyedVector.h> There are some components that are doing this wrong, and should be cleaned up.
Set LOCAL_MODULE_TAGS to any number of whitespace-separated tags. If the tag list is empty or contains droid, the module will get installed as part of a make droid. Modules with the tag shell_$(TARGET_SHELL) will also be installed. Otherwise, it will only get installed by running make <your-module> or with the make all pseudotarget.
Set LOCAL_REQUIRED_MODULES to any number of whitespace-separated module names, like "libblah" or "Email". If this module is installed, all of the modules that it requires will be installed as well. This can be used to, e.g., ensure that necessary shared libraries or providers are installed when a given app is installed.
If your executable should be linked statically, set LOCAL_FORCE_STATIC_EXECUTABLE:=true. There is a very short list of libraries that we have in static form (currently only libc). This is really only used for executables in /sbin on the root filesystem.
Files that you add to LOCAL_GENERATED_SOURCES will be automatically generated and then linked in when your module is built. See the Custom Tools template makefile for an example.
If you have additional flags to pass into the javac compiler, add them here. For example: LOCAL_JAVACFLAGS += -Xlint:deprecation
When linking Java apps and libraries, LOCAL_JAVA_LIBRARIES specifies which sets of java classes to include. Currently there are two of these: core and framework. In most cases, it will look like this: LOCAL_JAVA_LIBRARIES := core framework Note that setting LOCAL_JAVA_LIBRARIES is not necessary (and is not allowed) when building an APK with "include $(BUILD_PACKAGE)". The appropriate libraries will be included automatically.
You can pass additional flags to the linker by setting LOCAL_LDFLAGS. Keep in mind that the order of parameters is very important to ld, so test whatever you do on all platforms.
LOCAL_LDLIBS allows you to specify additional libraries that are not part of the build for your executable or library. Specify the libraries you want in -lxxx format; they're passed directly to the link line. However, keep in mind that there will be no dependency generated for these libraries. It's most useful in simulator builds where you want to use a library preinstalled on the host. The linker (ld) is a particularly fussy beast, so it's sometimes necessary to pass other flags here if you're doing something sneaky. Some examples: LOCAL_LDLIBS += -lcurses -lpthread LOCAL_LDLIBS += -Wl,-z,origin
If your package doesn't have a manifest (AndroidManifest.xml), then set LOCAL_NO_MANIFEST:=true. The common resources package does this.
LOCAL_PACKAGE_NAME is the name of an app. For example, Dialer, Contacts, etc. This will probably change or go away when we switch to an ant-based build system for the apps.
The directory your Android.mk file is in. You can set it by putting the following as the first line in your Android.mk: LOCAL_PATH := $(my-dir) The my-dir macro uses the MAKEFILE_LIST variable, so you must call it before you include any other makefiles. Also, consider that any subdirectories you inlcude might reset LOCAL_PATH, so do your own stuff before you include them. This also means that if you try to write several include lines that reference LOCAL_PATH, it won't work, because those included makefiles might reset LOCAL_PATH.
For host executables, you can specify a command to run on the module after it's been linked. You might have to go through some contortions to get variables right because of early or late variable evaluation: module := $(HOST_OUT_EXECUTABLES)/$(LOCAL_MODULE) LOCAL_POST_PROCESS_COMMAND := /Developer/Tools/Rez -d __DARWIN__ -t APPL\ -d __WXMAC__ -o $(module) Carbon.r
When including $(BUILD_PREBUILT) or $(BUILD_HOST_PREBUILT), set these to executables that you want copied. They're located automatically into the right bin directory.
When including $(BUILD_PREBUILT) or $(BUILD_HOST_PREBUILT), set these to libraries that you want copied. They're located automatically into the right lib directory.
These are the libraries you directly link against. You don't need to pass transitively included libraries. Specify the name without the suffix: LOCAL_SHARED_LIBRARIES := \ libutils \ libui \ libaudio \ libexpat \ libsgl
The build system looks at LOCAL_SRC_FILES to know what source files to compile -- .cpp .c .y .l .java. For lex and yacc files, it knows how to correctly do the intermediate .h and .c/.cpp files automatically. If the files are in a subdirectory of the one containing the Android.mk, prefix them with the directory name: LOCAL_SRC_FILES := \ file1.cpp \ dir/file2.cpp
These are the static libraries that you want to include in your module. Mostly, we use shared libraries, but there are a couple of places, like executables in sbin and host executables where we use static libraries instead. LOCAL_STATIC_LIBRARIES := \ libutils \ libtinyxml
LOCAL_MODULE is the name of what's supposed to be generated from your Android.mk. For exmample, for libkjs, the LOCAL_MODULE is "libkjs" (the build system adds the appropriate suffix -- .so .dylib .dll). For app modules, use LOCAL_PACKAGE_NAME instead of LOCAL_MODULE.
Instructs the build system to put the module somewhere other than what's normal for its type. If you override this, make sure you also set LOCAL_UNSTRIPPED_PATH if it's an executable or a shared library so the unstripped binary has somewhere to go. An error will occur if you forget to.
Instructs the build system to put the unstripped version of the module somewhere other than what's normal for its type. Usually, you override this because you overrode LOCAL_MODULE_PATH for an executable or a shared library. If you overrode LOCAL_MODULE_PATH, but not LOCAL_UNSTRIPPED_PATH, an error will occur.
These are the static libraries that you want to include in your module without allowing the linker to remove dead code from them. This is mostly useful if you want to add a static library to a shared library and have the static library's content exposed from the shared library. LOCAL_WHOLE_STATIC_LIBRARIES := \ libsqlite3_android
Any flags to pass to invocations of yacc for your module. A known limitation here is that the flags will be the same for all invocations of YACC for your module. This can be fixed. If you ever need it to be, just ask. LOCAL_YACCFLAGS := -p kjsyy