Related
Update 2010-02-26 Added examples for linking either dynamically or statically (see page 2)
I decided to try and set up a kitchen using the closest thing to a native kernel dev environment that I could set up.
At this point I have:
- cross-compiled the HTC Desire (kernel) sources using the Android 1.6 NDK
- figured out the repo/git stuff to pull the Android sources corresponding to a given Android release point
- cross-compiled the entire Cupcake release to an ARM target using the NDK tools
- built a dynamically linked "hello, world" executable using the NDK arm-eabi-* tools and verified it works on the phone.
Yah, I know - not much yet, but it's a start for a newb.
I have some notes documenting the Ubuntu setup process in case anyone wants it - but it is full of links (URLs), so I can't post anything with links as a new user.
If a mod can "verify" my account, I'll post it up. I promise, I'll be good.
bftb0
OK, I guess I need to make a couple of posts before I can embed links.
I'll post it without the URL protocol prefix (it's gonna look fugly), and then come back and edit it once the restriction is removed. I hope someone finds it useful.
Android kernel dev is not officially supported on Windows; closest match to Google Dev environment seems to be -> Ubuntu 8.04 LTS + JDK 5 (32 bit, not x64).
I realize that this is nothing close to the "kitchens" that the WinMo folks have been using, but figured that (as a newb) I would just end up confusing myself trying to figure out which of those tools could be useful for Android, in addition to needing the native binary cross-compilation tools anyway.
(Note: I briefly tried to set this up in a Ubuntu VMware Player VM, but ran into limitations of available Ubuntu 8.04 LTS player machine file size limits (as well as problems hosting a VM bigger than about 10 Gb on a USB drive - insufficient space on my primary drive)
If anyone wants to follow in my footsteps, here is a thumbnail sketch of useful links. I'll admit, this is not for the faint of heart - you need Linux/Unix skills to work through the kinks.
1) (ubuntu.com/getubuntu/downloadmirrors#mirrors) Ubuntu Download Mirrors list - Find a server near you
2) Download ubuntu-8.04.4-alternate-i386.iso and burn it to 700 Mb CD-R
3) Install Ubuntu 8.04 LTS Desktop Version (gulp!)
4) Use the Ubuntu graphical package manager (System -> Administration -> Synaptic Package Manager) and install the packages (git, flex, bison, etc) mentioned in the "Ubuntu Linux (32-bit x86)" section in this (source.android.com/download)Android Source setup page
5) Note that JDK6 does not work correctly; rather than going to Sun/Oracle for JDK5, install the sun-java5-{bin,jdk,jre,doc,demo} packages (1.5.0-22-0ubuntu) using the Ubuntu package installer mentioned above. Note if you install the docs package, you will also need to go to Sun/Oracle and get jdk-1_5_0-doc.zip towards the end of the package install, and drop it into /tmp for the package manager to finish.
6) Go to the (developer.android.com/sdk/index.html) Android SDK Download page and get android-sdk_r04-linux_86.tgz
7) Unpack ( gunzip -c and...tgz | tar xf - ) to desired location, add tools dir to PATH
Optional Steps (For Java App Devs)
o-8) (eclipse.org/downloads/ Eclipse IDE Download page - chose 'Eclipse Classic 3.5.1 (162 MB)' for Linux, 32 bit (eclipse-SDK-3.5.1-linux-gtk.tar.gz)
o-9) Install by unpacking ( gunzip -c ecli....tar.gz | tar xf - ) to desired location, add eclipse folder to PATH
o-10) Run eclipse, and follow the (developer.android.com/sdk/eclipse-adt.html) Installing and Updating ADT instructions for adding Android functionality to Eclipse 3.5 (Galileo)
o-11) From within Eclipse, launch Window -> Android SDK and AVD Manager, and install all available & compatible Android SDK Tools (all versions). (You might need to use http instead of https... YMMV) Note that this is the same functionality as simply running the command "android" from the command prompt if the Android SDK (steps 6,7) tools directory is in your PATH
12) Familiarize yourself with the command line tool "adb" tool in the SDK; in particular, since Ubuntu needs root privs to access the USB hardware, the best way to start the adb server on your PC is to
Code:
$ sudo /bin/bash
- set up root's environment so the SDK "tools" directory is in root's PATH
- plug your phone into the USB cable
- On the Eris, set the options:
Settings -> Applications -> Development -> USB Debugging (on)
Settings -> Applications -> Development -> Stay awake (on)
Code:
# adb devices
This last command only lists the available USB-connected (real) devices and any emulators you have running, but as a side effect it starts up the adb server on the Ubuntu machine running as root. Thereafter, you can use adb (or ddms) as an unprivileged user.
13) Have a peek at the SDK "ddms" monitor. A variety of things can be done from here, the least of which is screen shots (Select device, then Device-> Screen capture)... and some more twisty things too.
14) Download the (developer.android.com/sdk/ndk/1.6_r1/index.html) Android 1.6 NDK r1 (Native Development Kit) - android-ndk-1.6_r1-linux-x86.zip, and unzip into your desired location. Add the ARM cross-compiler tools to your path (see example below).
15) At this point you might be using something like this to set up your path wherever you please (.profile or elsewhere, according to how you want to manage your environment)
Code:
_INSLOC='/opt/android/'
_JAVAPATH='/usr/lib/jvm/java-1.5.0-sun/bin'
_BASEPATH='/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin'
_ANDTOOLS="${_INSLOC}"'/android-sdk-linux_86/tools'
_ECLPSPATH="${_INSLOC}"'/eclipse-3.5.1'
_ARMXCOMPILE="${_INSLOC}"'/android-ndk-1.5_r1/build/prebuilt/linux-x86/arm-eabi-4.2.1/bin'
PATH="${_JAVAPATH}"':'"${_BASEPATH}"':'"${_ANDTOOLS}"':'"${_ECLPSPATH}"':'"${_ARMXCOMPILE}"
export PATH
16) Download the (member.america.htc.com/download/RomCode/Source_and_Binaries/desirec-be918f7b.tgz) HTC Desire Kernel Source Code and unpack to a location of your choice.
17) Test your enviroment by building the HTC Kernel code:
$ cd v2_6_27/kernel_msm7k; make
I can't remember if I needed to stuff anything into the environment here (e.g. 'CC' definitions, etc.) - but I know that I didn't need to modify any Makefiles. I think the make system will find all the "arm-eabi-*" executables if they are in your path. You will know soon enough.
18) Try to compile a "Hello, world!" program for the phone. Before you get too far along in this (e.g. "where is crt0.o?"), (honeypod.blogspot.com/2007/12/dynamically-linked-hello-world-for.html) read this blog post - steps 3 and 4 only.
You might find a Makefile similar to this useful:
Code:
AR = arm-eabi-ar
AS = arm-eabi-as
CC = arm-eabi-gcc
CXX = arm-eabi-c++
LD = arm-eabi-ld
NDK_KIT = /opt/android/android-ndk-1.5_r1
PLATF_KIT = build/platforms/android-1.5
ARM_INC = $(NDK_KIT)/$(PLATF_KIT)/arch-arm/usr/include
ARM_LIB = $(NDK_KIT)/$(PLATF_KIT)/arch-arm/usr/lib
PLATF_INC = $(NDK_KIT)/$(PLATF_KIT)/common/include
OBJS = hello.o start.o
EXES = hello
hello: hello.o start.o
$(LD) \
--entry=_start \
--dynamic-linker /system/bin/linker -nostdlib \
-rpath /system/lib -rpath $(ARM_LIB) \
-L $(ARM_LIB) -lc -o hello hello.o start.o
hello.o: hello.c
$(CC) -I $(ARM_INC) -I $(PLATF_INC) -c hello.c
start.o: start.c
$(CC) -I $(ARM_INC) -I $(PLATF_INC) -c start.c
clean:
rm -f $(OBJS) $(EXES)
and then
Code:
$ make hello 2>&1 | tee make_hello-log.txt
if all goes well, push to the phone
Code:
$ adb push hello /sqlite_stmt_journals/hello
and then use a shell on the phone to watch the lights gerblinken:
Code:
PC-path$ adb shell
$ cd /sqlite_stmt_journals
$ ./hello
Hello, world!
$ exit
PC-path$
19) Now, download the Android stock kernel sources:
- Make sure you have "git" and "repo" installed; see the (source.android.com/download) Android "Get source" page, starting from "Installing Repo".
- Make a clean directory to initialize "repo" in, and download the code:
Code:
$ mkdir ~/mydroid
$ cd ~/mydroid
$ repo init -u git://android.git.kernel.org/platform/manifest.git -b cupcake
$ repo sync
The sync step above will download the entire source tree corresponding to the Cupcake release (90 minutes on my link).
20) Build the sources
Code:
$ cd ~/mydroid
$ make 2>&1 | tee make_Android-log.txt
On my old PC, this step took about two hours.
Well, that's a little start; everything that comes after this is heavy lifting, I suppose.
bftb0
Thank you for that.
I forgot to add a disk space usage report to this; here it is:
O/S ( /, /boot, /var, /tmp) ~ 4 Gb **
Android SDK, all versions: 1.3 Gb
Android "Cupcake" full sources + full build: 5.5 Gb (2.5 Gb sources, 3.0 Gb build tree)
Android NDK ~0.05 Gb
Eclipse 3.5.1: 0.21 Gb
HTC Desire Kernel Sources + build ~ 0.77 Gb
All up, thats about 12 Gb minimum needed for a full environment. Obviously, you would want some headroom above that.
Note that if you only want to build native binaries, and not the full Android source tree (or develop Java applications), this could be done handily in a virtual machine which is less than 10 Gb in size - for Windows users, there are VMware "Player" versions of Ubuntu 8.0.4 available for that. That eliminates the need for a separate machine, or a dual-boot PC - and the worries of messing up trying to create a dual-boot machine.
All that is really needed for that is the Desktop OS and the NDK. Adding a single version of the SDK so you have 'adb' and 'ddms' available to you would be helpful, but not absolutely necessary if you are using a Windows host machine with those tools already present.
bftb0
** beware that Ubuntu uses a lot of space on /var if you put it in a separate partition - 600+ Mb in my configuration.
bftb0 said:
I forgot to add a disk space usage report to this; here it is:
O/S ( /, /boot, /var, /tmp) ~ 4 Gb **
Android SDK, all versions: 1.3 Gb
Android "Cupcake" full sources + full build: 5.5 Gb (2.5 Gb sources, 3.0 Gb build tree)
Android NDK ~0.05 Gb
Eclipse 3.5.1: 0.21 Gb
HTC Desire Kernel Sources + build ~ 0.77 Gb
All up, thats about 12 Gb minimum needed for a full environment. Obviously, you would want some headroom above that.
Note that if you only want to build native binaries, and not the full Android source tree (or develop Java applications), this could be done handily in a virtual machine which is less than 10 Gb in size - for Windows users, there are VMware "Player" versions of Ubuntu 8.0.4 available for that. That eliminates the need for a separate machine, or a dual-boot PC - and the worries of messing up trying to create a dual-boot machine.
All that is really needed for that is the Desktop OS and the NDK. Adding a single version of the SDK so you have 'adb' and 'ddms' available to you would be helpful, but not absolutely necessary if you are using a Windows host machine with those tools already present.
bftb0
** beware that Ubuntu uses a lot of space on /var if you put it in a separate partition - 600+ Mb in my configuration.
Click to expand...
Click to collapse
great very descriptive thread
Hey bftb0
thanks for posting this guide. I am trying to just play around with someone of the exploit binaries on my eris (from the tattoo/hero rooting threads) , so these guidelines are coming in handy. However for some reason whenever I try and do anything over adb i get this error:
Code:
error: insufficient permissions for device
I cant use adb shell at all, even trying to push to the sdcard gives permissions denied. In the adb devices output the phone is coming up as "?????? no permissions", Did I miss a step ?
My setup so far: Ubuntu 9.10 i386,
Hey bftb0
thanks for posting this guide. I am trying to just play around with someone of the exploit binaries on my eris (from the tattoo/hero rooting threads) , so these guidelines are coming in handy. However for some reason whenever I try and do anything over adb i get this error:
Code:
error: insufficient permissions for device
I cant use adb shell at all, even trying to push to the sdcard gives permissions denied. In the adb devices output the phone is coming up as "?????? no permissions", Did I miss a step ?
My setup so far: Ubuntu 9.10 i386, android-sdk_r04-linux_86, droid eris with most recent OTA
someguy, you have to go to developer.android.com/guide/developing/device.html and install the linux driver (as root) and restart your system.
@someguy
Type "adb kill-server" followed by "sudo adb". You need to run adb as root in unix for it to work.
zifnab06, that was it, I must have not run the first command as root which caused the server to be run without su.
What exactly does this do?
bigcj55 said:
What exactly does this do?
Click to expand...
Click to collapse
Because of the lineage between Linux and Android, the primary platform for compilation of the kernel and other ARM instruction-set binaries ("native" code) has been on Linux. It appears (at the moment...reading between the lines) that Google engineers are using Ubuntu 8.04 LTS as their choice of Linux platform for maintaining Android kernel & utilities code... so, in principle, it is the "best supported" platform for doing "native" development work, if for no other reason than it's what Google has been using. (Not because it is somehow a "better" O/S, but rather because you are likely to run into fewer quirks, and have more resources on the internet who might have experienced the same difficulties that you run into.)
For instance, the folks that have been building kernel exploit programs have been using the Google "NDK". They are not writing/compiling those in Java - they are writing or compiling programs written in "C", and compiling them into native (ARM) object code.
I provided notes from my own installation for anyone considering "dipping their toe" into building Android kernels or native binaries using Linux as a host O/S - most forum readers are not going to be interested in doing that.
Note that Google now releases an NDK for Windows, Linux, and Mac OS/X. The strange thing about doing development on Windows using Cygwin is that if you have sufficient skills to do so - and are capable of resolving problems that inevitably crop up - then you probably already have sufficient skills to be using Unix/Linux in the first place. Sort of depends on your past experience, though. I'd be willing to bet that Devs with a history of doing WinMo development probably would prefer to climb the Android development learning curve by trying their hand with the Windows/Cygwin NDK because of their familiarity with other Windows-based development tools (IDEs, hex editors, assemblers, GUI generators, etc).
That reminds me though - I need to update this thread with something else.
Cheers
bftb0
Static OR Dynamic Linking of Native Binaries
I was a little unsatisfied with the "hack" used in the above examples to circumvent the need to launch programs without linking to C runtime initialization using that strange code shown above, i.e.
Code:
start() { exit( main() ); }
Moreover, for purposes of debugging applications running on the phone (using "gdbserver"), it simply seems easier to link statically - it allows you to step into the syscall library routines and actually see the assembler implementations wrapped around "STI" interrupts.
So, I spent some time experimenting, and came up with a Makefile that allows me to easily toggle between building an ARM executable linked statically or dynamically. Moreover, it lets "argc" and "argv" work they way they are supposed to, and also allows main() to do a return().
Here's an example Makefile, for a program "hello.c". Make sure you read the notes which follow it.
Code:
#
# FIX THESE (to match your installation)
# Top of the Android Source/build tree (retrieved w/ repo)
ANDSRCROOT := /home/user/android/repo-root
# link dynamically against the libraries shipped on the phone!
# e.g.: $cd /opt/android/Eris_libs/ruu1_16_605_1 ; adb pull /system/lib/
PHONELIBS := /opt/android/Eris_libs/ruu1_16_605_1/lib
# Choose one or the other; static is easiest for debugging use
LINKMODE := dynamic
#LINKMODE := static
# tools
CROSS_COMPILE := arm-eabi-
CC := $(CROSS_COMPILE)gcc
LD := $(CROSS_COMPILE)ld
AR := $(CROSS_COMPILE)ar
CXX := $(CROSS_COMPILE)c++
OBJDUMP := $(CROSS_COMPILE)objdump
RANLIB := $(CROSS_COMPILE)ranlib
READELF := $(CROSS_COMPILE)readelf
# Note STATLIBROOT is relative to the Android Source root (ANDSRCROOT)
STATLIBROOT := out/target/product/generic/obj/STATIC_LIBRARIES
INCLUDE := -I $(ANDSRCROOT)/bionic/libc/include \
-I $(ANDSRCROOT)/bionic/libc/arch-arm/include \
-I $(ANDSRCROOT)/bionic/libc/kernel/common \
-I $(ANDSRCROOT)/bionic/libc/kernel/arch-arm
ifeq ($(LINKMODE),dynamic)
LIBDIRS := -L $(PHONELIBS)
else
LIBDIRS := -L $(ANDSRCROOT)/prebuilt/linux-x86/toolchain/arm-eabi-4.2.1/lib/gcc/arm-eabi/4.2.1 \
-L $(ANDSRCROOT)/$(STATLIBROOT)/libc_intermediates \
-L $(ANDSRCROOT)/$(STATLIBROOT)/libc_common_intermediates
endif
ifeq ($(LINKMODE),dynamic)
CRTBEG := $(ANDSRCROOT)/out/target/product/generic/obj/lib/crtbegin_dynamic.o
LDFLAGS := -rpath /system/lib -rpath-link /system/lib --dynamic-linker /system/bin/linker
EXTLIBS := -lc -ldl
else
CRTBEG := $(ANDSRCROOT)/out/target/product/generic/obj/lib/crtbegin_static.o
LDFLAGS := -static
EXTLIBS := -lc -lgcc -lc_common
endif
CRTEND := $(ANDSRCROOT)/out/target/product/generic/obj/lib/crtend_android.o
CFLAGS := -g -Wall
%.o: %.c
$(CC) -c $(INCLUDE) \
$(CFLAGS) $< -o [email protected]
%.s: %.c
$(CC) -S -c $(INCLUDE) \
$(CFLAGS) $< -o [email protected]
all: hello
hello: hello.o
$(LD) $(LDFLAGS) \
-o [email protected] --entry=_start -nostdlib \
$(CRTBEG) \
$^ \
$(CRTEND) \
$(LIBDIRS) $(EXTLIBS)
This makefile has certain assumptions built into it:
(1) Static Linkage
If you are going to link statically, you will need to have already built the entire Android source tree - the Android NDK does not provide any static libraries (In the OP I provided instructions on how to do this download and build the Android "Cupcake" release.)
(2) Dynamic Linking
If you want to compile code which is "as close to the phone as possible" then there is really only one way to do that: compile against the headers that HTC used, and perform linking against the dynamic libraries that you literally pull off the phone ("adb pull /system/lib").
There's only one problem with this strategy: HTC did not release their copy of the bionic (or other library) source code, so you are forced to punt... the second-best choice is to use headers from the Android (Cupcake) source tree. So... you will notice that for both the dynamic-link or static-link cases in the above makefile, the $(INCLUDE) paths used during compilation come from the Android source tree.
bftb0
PS I guess there is a mechanism to configure the top-level "make" of the Android source tree for not only a target type (defaults to "arm"), but also a "machine" dependency. I didn't research that, as I have only built user-space code. I'm doubt there should be any differences at the syscall interface API - excepting perhaps things like ptrace() which have a machine dependency (register structures, etc).
bftb0 said:
Because of the lineage between Linux and Android, the primary platform for compilation of the kernel and other ARM instruction-set binaries ("native" code) has been on Linux. It appears (at the moment...reading between the lines) that Google engineers are using Ubuntu 8.04 LTS as their choice of Linux platform for maintaining Android kernel & utilities code... so, in principle, it is the "best supported" platform for doing "native" development work, if for no other reason than it's what Google has been using. (Not because it is somehow a "better" O/S, but rather because you are likely to run into fewer quirks, and have more resources on the internet who might have experienced the same difficulties that you run into.)
For instance, the folks that have been building kernel exploit programs have been using the Google "NDK". They are not writing/compiling those in Java - they are writing or compiling programs written in "C", and compiling them into native (ARM) object code.
I provided notes from my own installation for anyone considering "dipping their toe" into building Android kernels or native binaries using Linux as a host O/S - most forum readers are not going to be interested in doing that.
Note that Google now releases an NDK for Windows, Linux, and Mac OS/X. The strange thing about doing development on Windows using Cygwin is that if you have sufficient skills to do so - and are capable of resolving problems that inevitably crop up - then you probably already have sufficient skills to be using Unix/Linux in the first place. Sort of depends on your past experience, though. I'd be willing to bet that Devs with a history of doing WinMo development probably would prefer to climb the Android development learning curve by trying their hand with the Windows/Cygwin NDK because of their familiarity with other Windows-based development tools (IDEs, hex editors, assemblers, GUI generators, etc).
That reminds me though - I need to update this thread with something else.
Cheers
bftb0
Click to expand...
Click to collapse
Wow, i was a little off in this case. I was under the assumption y'all were somehow going to compile ubuntu to run on the eris. :O
There are a lot of toturials for building CM9 on ubuntu or CM7 on Mac but I couldn't find a decent toturial for building CM9 on Mac (specially Lion). Development in AOSP/CM land is rapid and guides frequently need updating. I had to spend a little time to figure everything out and I decided to share it here.
This tutorial is for building CM9 (ICS) for Galaxy Nexus GSM (maguro) on Mac OS X Lion 10.7.3 using Xcode 4.3 and homebrew . You can easily make the instructions work for most other cm9 devices, but I wouldn't know anything about that.
DISCLAIMER: I'm not responsible if you blow yourself up, blah blah blah
However, I've tried to make this as noob friendly as possible because, well I'm a noob myself
Instrunctions:
UPADTE (MAY 29TH) : The Xcode 4.3 default compiler (llvm-gcc) used to be incompatible with CM9. Thanks to jocelyn and topprospect, the LLVM compatibility patches from mainline AOSP are now merged into CM9. Therefore, you can now use Xcode 4.3 and its command line tools without installing another compiler. However, since GCC is still the only officially supported compiler, incompatibilites with llvm-gcc could still be introduced with future updates. Therefore, if your build fails, it might be worth it to try installing and compiling with GCC 4.2. See the Troubleshooting section for more info.
Now that we have Xcode 4.3 and Xcode command line tools (CLT) installed, let's continue.
Open Terminal and run
Code:
java
if you don't have Java, you will get a prompt asking you to download and install Java. Go ahead and install it.
If you don't have adb and fastboot working, download the android-sdk from google (version r18 as of now) and put it in /usr/local/ and rename the folder to "android-sdk".
Install the homebrew package manager
Code:
/usr/bin/ruby -e "$(/usr/bin/curl -fsSL https://raw.github.com/mxcl/homebrew/master/Library/Contributions/install_homebrew.rb)"
To make sure that homebrew and android-sdk executables are in $PATH:
Code:
touch ~/.bash_profile && echo "PATH=/usr/local/bin:/usr/local/sbin:$PATH:/usr/local/android-sdk/tools:/usr/local/android-sdk/platform-tools" >> ~/.bash_profile
Relaunch Terminal for the change to take effect.
At this point you can run
Code:
brew doctor
to detect any problems there might be (Homebrew may instruct you to use the xcode-select utility to select the xcode installation path). Hopefully, your system is raring to brew
Now we have to install a bunch of packages:
Code:
brew install git coreutils findutils gnu-sed gnupg pngcrush repo
We now need to create a couple of symlinks so that the gnu versions of 'sed' and 'find' are used rather than the osx provided versions :
Code:
ln -s /usr/local/bin/gfind /usr/local/bin/find && ln -s /usr/local/bin/gsed /usr/local/bin/sed && ln -s /usr/local/bin/gstat /usr/local/bin/stat
It's time to create a case sensitive image which will hold our working directory:
Code:
hdiutil create -type SPARSE -fs "Case-sensitive Journaled HFS+" -size 40g -volname "android" -attach ~/Desktop/Android
Now we have a disk image in ~/Desktop/Android. Mount it if it's not mounted already. (Don't be picky about the size, the image will only take as much as space as its contents).
Now we need to create a working directory inside the mounted volume:
Code:
cd /Volumes/android && mkdir cm9 && cd cm9
We can initialize and download the source now:
Code:
repo init -u git://github.com/CyanogenMod/android.git -b ics && repo sync && say 'finished'
Now we need to get the required proprietry files for our device. We can get these from the device itself. Connect your phone (make sure USB Debugging is enabled) and run the following (for maguro):
Code:
cd /Volumes/android/cm9/device/samsung/maguro/ && ./extract-files.sh
[If you see errors in the output from extract-files.sh, see the Troubleshooting section below]
For Google devices,we can also get them directly from google. For maguro, download the 3 files and extract them to /Volumes/android/cm9. Then,
Code:
cd /Volumes/android/cm9
/Volumes/android/cm9/extract-broadcom-maguro.sh
/Volumes/android/cm9/extract-imgtec-maguro.sh
/Volumes/android/cm9/extract-samsung-maguro.sh
We also need the prebuilts (like ROM manager and Term.apk):
Code:
/Volumes/android/cm9/vendor/cm/get-prebuilts
You can optionally tell the build to use the ccache tool. CCache acts as a compiler cache that can be used to speed-up rebuilds :
Code:
export USE_CCACHE=1 && /Volumes/android/cm9/prebuilt/darwin-x86/ccache/ccache -M 20G
Default is 1GB. Anything between 20GB-50GB should be fine.
Before starting the build, we need to workaround an issue with Lion and compiling the QEMU emulator.
[This step doesn't seem to be needed anymore. QEMU is automatically ignored on OS X/Darwin]
If you build now, you're probably gonna get kernel build errors regarding the missing elf.h header (this error might be device specific). Fortunately, we already have this file downloaded, so we only need to copy it to /usr/local/include:
Code:
cp /Volumes/android/cm9/external/elfutils/libelf/elf.h /usr/local/include
FINALLY, we are ready to build:
Code:
cd /Volumes/android/cm9 && source build/envsetup.sh && brunch
Pick your device from the list and enter the number. For maguro, you could use "brunch maguro" instead and skip the menu. Depending on your system, this will take 30min-4hours.
You should now see a beautiful zip file waiting to be flashed:
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
Troubleshooting:
The extract script for maguro seems to be a little outdated, as it doesn't pull the gps proprietary blob. You can either use the google provided scripts, or add koush's git repository for your device to your local_manifest.xml.
As explained above, the CM9 source is currently compatible with llvm-gcc. In the future, if llvm-gcc fails to build correctly, you should try installing and compiling using GCC4.2 (the Xcode 3 compiler). You can install apple-gcc4.2 from homebrew:
Code:
brew install https://raw.github.com/Homebrew/homebrew-dupes/master/apple-gcc42.rb
This version of gcc can happily coexist with Xcode 4.x .
So now you have GCC 4.2 installed, but it won't be used unless we update the corresposing environment variables:
Code:
export CC=/usr/local/bin/gcc-4.2 && export CXX=/usr/local/bin/g++-4.2
Notice that using the export command is temporary. If you relaunch Terminal, you will need to set these again. However, this is a good thing, because changing these values permanently (by putting them in ~/.bash_profile) can interfere with other builds.
If you use this method, the build might fail while compiling "external/zlib/x86/adler32.c". It appears that a recent change in zlib has introduced an incompatibility with gcc 4.2. you have to revert the following 2 commits:
Code:
cd external/zlib
git revert dd6786cae3f4493faa6661d5f74db587932f15d7
git revert 13bf40af68236c961542bdee1d4b7c0176bf15a0
Alternatively, you can add topprospect's zlib on github (which has those commits reverted) to your local_manifast.xml. Simply run:
Code:
nano /Volumes/android/cm9/.repo/local_manifest.xml
and add the following line
Code:
<project name="dferg/android_external_zlib" path="external/zlib" remote="github" />
If you get a build error, and your error is not covered here, copy the last 20-30 lines of the build output AND the output from the following command into pastebin and post the link. Hopefully me or someone else will help you.
Code:
echo -e "\nENV:\n$(env)\n\nWHICH GCC\n:$(which gcc)\n\nWHICH G++:\n$(which g++)\n\nWHICH CC:\n$(which cc)\n\nWHICH C++:\n$(which c++)\n\nBREW DOCTOR:\n$(brew doctor)\n\nBREW LIST:\n$(brew list)\n\n/USR/BIN:\n$(ls -l /usr/bin | grep gcc)\n\n/USR/LOCAL/BIN:\n$(ls -l /usr/local/bin | grep gcc)\n\n"
Notes/Extras:
To quickly setup your environment, add an alias like the following to ~/.bash_profile:
Code:
alias cm9env="hdiutil attach PATH-TO-DISK-IMAGE -mountpoint /Volumes/android && cd /Volumes/android/cm9 && source ./build/envsetup.sh && export USE_CCACHE=1"
alias cm9build="cm9env && make clobber && reposync && brunch maguro"
Now you can save time by using "cm9env" to get your environment setup or "cm9build" to compile a clean updated build.
To clear your output directory for a new build, run "make clobber". You probably don't need this if you've only changed a few lines of code.
To cherry pick yet-to-be-merged changes from the gerrit instance:
1. Pick an open commit from CM Gerrit
2. Under list of Patch Sets pick the latest and open cherry-pick tab
3. Check what Git repository the url is pointing e.g. http://review.cyanog...frameworks_base
4. In your CM9 working tree go to the corresponding directory, which in this case is something like ~/your-working-directory/frameworks/base/
5. Now simply paste the whole line seen in CM Gerrit cherry-pick tab e.g. "git fetch http://review.cyanog....rameworks_base refs/changes/00/13100/4 && git cherry-pick FETCH_HEAD"
It should be now included in your next compiled build. When doing repo sync again, cherry picks will be lost.[CREDIT Fihlvein from xda]
Click to expand...
Click to collapse
Enjoy your custom built CM9!
I updated OP with some updated info about Xcode 4.3.
Nice work man. Very helpful.
conantroutman said:
Nice work man. Very helpful.
Click to expand...
Click to collapse
thanks! I'll try to keep this topic updated as issues with mac are introduced/resolved.
Nice write up. Thanks.
Before I start again from scratch I have a question. Does this guide apply to previous versions of mac os x (mine is 10.6.8)? I used the official android initializing build environment page & cm7 wiki page for the instructions to setup my build environment.
Also, any tips to switch from macports to homebrew?
In the past I've had to cherry pick to get my OS X build environment set up for CM9. The compile from source fails because I started with macports instead of homebrew (bad idea). I tried to switch to homebrew without success. Any tips to switch from macports to homebrew?
For the sake of keeping this page on topic a pm response is ok if that is what you prefer.
Hi thanks for this !! Helpful one question what do i change so i can do AOSP instead of cm9??
grad061980 said:
Nice write up. Thanks.
Before I start again from scratch I have a question. Does this guide apply to previous versions of mac os x (mine is 10.6.8)? I used the official android initializing build environment page & cm7 wiki page for the instructions to setup my build environment.
Also, any tips to switch from macports to homebrew?
In the past I've had to cherry pick to get my OS X build environment set up for CM9. The compile from source fails because I started with macports instead of homebrew (bad idea). I tried to switch to homebrew without success. Any tips to switch from macports to homebrew?
For the sake of keeping this page on topic a pm response is ok if that is what you prefer.
Click to expand...
Click to collapse
Yes, this guide should work fine on Snow Leopard. It mostly depends on your Xcode version. If you have Xcode 3, you can skip step 1 entirely, since you already have gcc4.2 as part of Xcode.
If you have access to Xcode 4.2 and above, you will need to install gcc4.2 separately, as explained in the guide.
Now regarding Macports, I strongly suggest that you completely uninstall Macports before installing homebrew. Instructions are here: http://guide.macports.org/chunked/installing.macports.uninstalling.html
WonkyYew said:
Hi thanks for this !! Helpful one question what do i change so i can do AOSP instead of cm9??
Click to expand...
Click to collapse
Full instructions are available on android.com : http://source.android.com/source/initializing.html
If you are using this guide, you need to change the repo initialization command to :
Code:
repo init -u https://android.googlesource.com/platform/manifest
and then do repo sync. You can setup ccache as usual. I don't think AOSP has the "brunch command", so you have to use launch and then make.
Run "lunch" and select an option from the menu. You can find more info about the options here: http://source.android.com/source/building.html. For maguro, you should use "full_maguro-userdebug".
To start the build, use
Code:
make -j$(sysctl -n hw.ncpu)
@ArmanUV. Sounds good & thanks for the input. I'll give the link to macports uninstall a go.
Im a real noob, whats the advantage of compiling from source?
Thanks for the help man !
Hi, thanks for your guide, setting up the repo was no problem at all!
But: I'm getting the following error when building.
Code:
external/zlib/x86/adler32.c: In function ‘adler32_MMX’:
external/zlib/x86/adler32.c:747: error: can't find a register in class ‘GENERAL_REGS’ while reloading ‘asm’
external/zlib/x86/adler32.c:747: error: ‘asm’ operand has impossible constraints
make: *** [out/host/darwin-x86/obj/STATIC_LIBRARIES/libz_intermediates/adler32.o] Error 1
make: *** Waiting for unfinished jobs....
Seems to be a problem with the compiler, but I'm on xcode 4.3 and I've installed gcc-4.2 and set the env vars. Any help?
ArmanUV,
Thanks so much for posting this guide. Very helpful!
Are you having any trouble with errors like this?
Code:
external/zlib/x86/adler32.c: In function ‘adler32_MMX’:
external/zlib/x86/adler32.c:747: error: can't find a register in class ‘GENERAL_REGS’ while reloading ‘asm’
external/zlib/x86/adler32.c:747: error: ‘asm’ operand has impossible constraints
Googling for this error implies that the fix is to use a version of GCC > 4.2. But there does not seem to be a GCC 4.4 in Homebrew.
Thanks again for the guide!
EDIT: Sorry for the double post with empyyy. Seems like there is someone else having my same issue!
topprospect said:
ArmanUV,
Thanks so much for posting this guide. Very helpful!
Are you having any trouble with errors like this?
Code:
external/zlib/x86/adler32.c: In function ‘adler32_MMX’:
external/zlib/x86/adler32.c:747: error: can't find a register in class ‘GENERAL_REGS’ while reloading ‘asm’
external/zlib/x86/adler32.c:747: error: ‘asm’ operand has impossible constraints
Googling for this error implies that the fix is to use a version of GCC > 4.2. But there does not seem to be a GCC 4.4 in Homebrew.
Thanks again for the guide!
EDIT: Sorry for the double post with empyyy. Seems like there is someone else having my same issue!
Click to expand...
Click to collapse
empyyy said:
Hi, thanks for your guide, setting up the repo was no problem at all!
But: I'm getting the following error when building.
Code:
external/zlib/x86/adler32.c: In function ‘adler32_MMX’:
external/zlib/x86/adler32.c:747: error: can't find a register in class ‘GENERAL_REGS’ while reloading ‘asm’
external/zlib/x86/adler32.c:747: error: ‘asm’ operand has impossible constraints
make: *** [out/host/darwin-x86/obj/STATIC_LIBRARIES/libz_intermediates/adler32.o] Error 1
make: *** Waiting for unfinished jobs....
Seems to be a problem with the compiler, but I'm on xcode 4.3 and I've installed gcc-4.2 and set the env vars. Any help?
Click to expand...
Click to collapse
You guys seem to have the same issue. What sort of Xcode configuration are you using? Can you post the output from "which gcc","which g++", "gcc -v", "g++ -v" and "cc -v"?
ArmanUV said:
You guys seem to have the same issue. What sort of Xcode configuration are you using? Can you post the output from "which gcc","which g++", "gcc -v", "g++ -v" and "cc -v"?
Click to expand...
Click to collapse
I am on Lion 10.7.4 with Xcode 4.3.2. Here is the output that you asked for:
Code:
# echo -n "which gcc: "; which gcc; echo -n "which g++: "; which g++; echo ""; echo "gcc -v:"; gcc -v; echo ""; echo "g++ -v:"; g++ -v; echo ""; echo "cc -v:"; cc -v
which gcc: /usr/bin/gcc
which g++: /usr/bin/g++
gcc -v:
Using built-in specs.
Target: i686-apple-darwin11
Configured with: /private/var/tmp/llvmgcc42/llvmgcc42-2336.9~22/src/configure --disable-checking --enable-werror --prefix=/Applications/Xcode.app/Contents/Developer/usr/llvm-gcc-4.2 --mandir=/share/man --enable-languages=c,objc,c++,obj-c++ --program-prefix=llvm- --program-transform-name=/^[cg][^.-]*$/s/$/-4.2/ --with-slibdir=/usr/lib --build=i686-apple-darwin11 --enable-llvm=/private/var/tmp/llvmgcc42/llvmgcc42-2336.9~22/dst-llvmCore/Developer/usr/local --program-prefix=i686-apple-darwin11- --host=x86_64-apple-darwin11 --target=i686-apple-darwin11 --with-gxx-include-dir=/usr/include/c++/4.2.1
Thread model: posix
gcc version 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2336.9.00)
g++ -v:
Using built-in specs.
Target: i686-apple-darwin11
Configured with: /private/var/tmp/llvmgcc42/llvmgcc42-2336.9~22/src/configure --disable-checking --enable-werror --prefix=/Applications/Xcode.app/Contents/Developer/usr/llvm-gcc-4.2 --mandir=/share/man --enable-languages=c,objc,c++,obj-c++ --program-prefix=llvm- --program-transform-name=/^[cg][^.-]*$/s/$/-4.2/ --with-slibdir=/usr/lib --build=i686-apple-darwin11 --enable-llvm=/private/var/tmp/llvmgcc42/llvmgcc42-2336.9~22/dst-llvmCore/Developer/usr/local --program-prefix=i686-apple-darwin11- --host=x86_64-apple-darwin11 --target=i686-apple-darwin11 --with-gxx-include-dir=/usr/include/c++/4.2.1
Thread model: posix
gcc version 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2336.9.00)
cc -v:
Apple clang version 3.1 (tags/Apple/clang-318.0.58) (based on LLVM 3.1svn)
Target: x86_64-apple-darwin11.3.0
Thread model: posix
They are pointing to LLVM, but my CC and CXX variables point to:
Code:
env|grep 4.2
CXX=/usr/local/bin/g++-4.2
CC=/usr/local/bin/gcc-4.2
Do I need to add something to PATH? I'm sure I just missed something obvious in your instructions.. Thanks for helping!
topprospect said:
I am on Lion 10.7.4 with Xcode 4.3.2. Here is the output that you asked for:
Do I need to add something to PATH? I'm sure I just missed something obvious in your instructions.. Thanks for helping!
Click to expand...
Click to collapse
Everything checks out. I'm away from my main machine so I can't run a test build, but I suspect that a recent change is causing problems.
Now, regarding the compiler, Setting CC/CXX *should* take care of everything, but I am currently not 100% sure that somewhere in a makefile, these environment variables aren't being ignored. Since I wrote the guide, I noticed a lot of clang warnings in the build, which means that CC/CXX is not honored and /usr/bin/cc and /usr/bin/c++ is being used.
A more robust method of making sure gcc-4.2 is being used is creating symlinks to gcc-4.2 and g++-4.2 :
Code:
ln -s /usr/local/bin/gcc-4.2 /usr/local/bin/gcc
ln -s /usr/local/bin/gcc-4.2 /usr/local/bin/cc
ln -s /usr/local/bin/g++-4.2 /usr/local/bin/c++
ln -s /usr/local/bin/g++-4.2 /usr/local/bin/g++
Obviousely, a systemic method like this has its downsides but it may be the only choice without having to change CM code (especially since I lack the knowledge to do so )
[I recently found out that master aosp is no longer using CC/CXX to find the compiler (see ./build/core/combo/). Instead, it uses "gcc" and "g++" directly, which means that llvm-gcc will be used no matter what env variable you have. Fortunately, unlike cm9, master aosp is supposed to build fine with llvm-gcc (except for qemu, which doesn't matter for device images). ]
ArmanUV said:
Everything checks out. I'm away from my main machine so I can't run a test build, but I suspect that a recent change is causing problems.
Now, regarding the compiler, Setting CC/CXX *should* take care of everything, but I am currently not 100% sure that somewhere in a makefile, these environment variables aren't being ignored. Since I wrote the guide, I noticed a lot of clang warnings in the build, which means that CC/CXX is not honored and /usr/bin/cc and /usr/bin/c++ is being used.
A more robust method of making sure gcc-4.2 is being used is creating symlinks to gcc-4.2 and g++-4.2 :
Code:
ln -s /usr/local/bin/gcc-4.2 /usr/local/bin/gcc
ln -s /usr/local/bin/gcc-4.2 /usr/local/bin/cc
ln -s /usr/local/bin/g++-4.2 /usr/local/bin/c++
ln -s /usr/local/bin/g++-4.2 /usr/local/bin/g++
Click to expand...
Click to collapse
Okay, just tried this. I created a new dir (/Volumes/Android/bin) that simply houses those softlinks you recommended. Then I put /Volumes/Android/bin at the beginning of my PATH. That should fix it without breaking the rest of the system, e.g. homebrew.
The GENERAL_REGS problem still exists though. Pretty sure b/c gcc 4.2.1 doesn't understand this construct properly (need a newer version of gcc).
So I backed out the change that introduced this adler32.c.
https://github.com/CyanogenMod/android_external_zlib/commit/13bf40af68236c961542bdee1d4b7c0176bf15a0
The compile is getting farther now. I have to run to work so I'll post later if it succeeds.
The weird thing is: This change was made back in December. Why would it have worked for you?
topprospect said:
The compile is getting farther now. I have to run to work so I'll post later if it succeeds.
Click to expand...
Click to collapse
Build works (and boots!) with the following:
Code:
cd external/zlib
git revert dd6786cae3f4493faa6661d5f74db587932f15d7
git revert 13bf40af68236c961542bdee1d4b7c0176bf15a0
Note the 1st revert is just to avoid massive conflicts seen when reverting the 2nd one by itself. The 2nd revert is the one that really matters here.
So this isn't really a solution.. Seems like we need to move to a newer version of gcc or figure out a patch to adler32.c that makes it gcc 4.2 compatible.
topprospect said:
Build works (and boots!) with the following:
Code:
cd external/zlib
git revert dd6786cae3f4493faa6661d5f74db587932f15d7
git revert 13bf40af68236c961542bdee1d4b7c0176bf15a0
Click to expand...
Click to collapse
The first revert works fine, however the second one gives me the following error:
Code:
$ git revert 13bf40af68236c961542bdee1d4b7c0176bf15a0
error: could not revert 13bf40a... Implement vectorized adler32 and optimized slhash
hint: after resolving the conflicts, mark the corrected paths
hint: with 'git add <paths>' or 'git rm <paths>'
hint: and commit the result with 'git commit'
error: Could not parse conflict hunks in zlib.h
empyyy said:
The first revert works fine, however the second one gives me the following error:
Code:
$ git revert 13bf40af68236c961542bdee1d4b7c0176bf15a0
error: could not revert 13bf40a... Implement vectorized adler32 and optimized slhash
hint: after resolving the conflicts, mark the corrected paths
hint: with 'git add <paths>' or 'git rm <paths>'
hint: and commit the result with 'git commit'
error: Could not parse conflict hunks in zlib.h
Click to expand...
Click to collapse
Well, run git status. You'll see that the only conflict is in the comments in zlib.h. So, you can just ignore it.
topprospect said:
Okay, just tried this. I created a new dir (/Volumes/Android/bin) that simply houses those softlinks you recommended. Then I put /Volumes/Android/bin at the beginning of my PATH. That should fix it without breaking the rest of the system, e.g. homebrew.
The GENERAL_REGS problem still exists though. Pretty sure b/c gcc 4.2.1 doesn't understand this construct properly (need a newer version of gcc).
So I backed out the change that introduced this adler32.c.
https://github.com/CyanogenMod/android_external_zlib/commit/13bf40af68236c961542bdee1d4b7c0176bf15a0
The compile is getting farther now. I have to run to work so I'll post later if it succeeds.
The weird thing is: This change was made back in December. Why would it have worked for you?
Click to expand...
Click to collapse
topprospect said:
Build works (and boots!) with the following:
Code:
cd external/zlib
git revert dd6786cae3f4493faa6661d5f74db587932f15d7
git revert 13bf40af68236c961542bdee1d4b7c0176bf15a0
Note the 1st revert is just to avoid massive conflicts seen when reverting the 2nd one by itself. The 2nd revert is the one that really matters here.
So this isn't really a solution.. Seems like we need to move to a newer version of gcc or figure out a patch to adler32.c that makes it gcc 4.2 compatible.
Click to expand...
Click to collapse
Nice find. I tried to compile this morning and I ran into the same issue. This is what I don't understand: I did a couple of builds about a week ago without running into this issue. But, the latest commits on zlib are from 2 months ago.
Amazingly, the Xcode 4.3 toolchain (clang and llvm-gcc) builds this external/zlib/adler32.c just fine.
An alternative to this problem is to install an up to date gcc 4.7 :
Code:
brew install https://raw.github.com/Homebrew/homebrew-dupes/master/gcc.rb
and then create symlinks to gcc-4.7/g++-4.7. I have not tested this yet.
Based on frostincredible Script
OS X Build Version 2 - By Ptichalouf
I Will add so Many Team !
So you want to build AOKP, PA, OMNI or CM on OS X?
Well worry no longer my friends,
it is about to get a whole lot easier. Why is that?
Because this script I've written will do pretty
much everything for you, from creating the case sensitive DMG to installing the required files,
to setting up all the folders.
The best part? It also will initialize the AOKP or CM repo (depending
on your choice) and you can also build them straight from the script!
How fantastic is that?!
Simply read on to see how! Make sure to follow all the instructions (there aren't many)!
Follow these steps:
Install required files (MUST install MacPorts first) MacPorts available free at www.macports.org and Xcode : here Run 1 time Xcode !
Go here and download OS X Build (OSX_Setup.zip) - DOWNLOAD
Place the ZIP file on your Desktop.
Extract the ZIP to your Desktop.
Here comes the only tricky part. Open the file located at OSX_Setup/scripts/advanced_scripts/build_cm or build_aokp, whichever you're going to build.
Find this line -
Code:
HTML:
# BELOW IS THE ONLY LINE YOU SHOULD BE EDITING. REPLACE *YOUR DEVICE* WHITH YOUR DEVICE'S CODE NAME. EX - NEXUS 7 = "grouper"
Just below that, edit this line with the instructions in the one I just posted -
Code:
HTML:
cp /Volumes/Android/OUT/target/product/*YOUR DEVICE*/aokp_*YOUR DEVICE*-ota-eng.$ME.zip /Volumes/Android/Builds/AOKP_$ME.$NOW.zip
IT IS IMPORTANT THAT YOU DON'T EDIT ANYTHING ELSE.
Open your favorite Terminal application (the default on is Terminal located at /Applications/Utilities/Terminal).
cd to the OSX_Setup folder on your Desktop.
Code:
HTML:
cd /Users/YOUR NAME/OSX_Setup
HTML:
$ git config --global user.email "[email protected]"
HTML:
$ git config --global user.name "YourName"
Run the following -
Code:
HTML:
./runme
After running this, you will be presented with the following menu -
Code:============================================================
OS X Build Set Up 1 - based on Kyle Frost
By [email protected]
============================================================
---Main Menu---
1 - Create DMG
2 - Install required files (MUST install MacPorts first)
MacPorts available free at www.macports.org
3 - Set up folders
WARNING: DO NOT RUN ADVANCED MENU UNTIL
YOU HAVE RUN ALL 3 ABOVE, OR HAVE DONE
THEM ALL MANUALLY. THANK YOU.
0 - Advanced Menu
x - Exit
Enter option:
As you can see, you need to go here and install MacPorts before proceeding - MacPorts
After installing MacPorts you can go ahead and run everything.
Create DMG will give you the options of creating a 25GB, 30GB, 40GB, 50GB, 60GB or 120GB DMG. 25GB is the lowest recommended by the Android Team.
Install required files will simply do just that, it will install the files needed as specified in the AOSP documents. This can take a while, so go take a nap while it runs.
Set up folders will give you the option of setting up the folders for AOKP or CM10.1. Simply choose which you want and the folders will be created.
Choosing Advanced Menu will give you this -
Code:
============================================================
OS X Build Set Up 1 - based on Kyle Frost
By @ptichalouf
============================================================
---Advanced Menu---
MUST INSTALL XCODE BEFORE BUILDING!
XCODE IS FREE IN THE MAC APP STORE!
1 - Initialize AOKP Repo
2 - Initialize CM11 Repo
3 - Initialize Paranoid Repo
When building, it will give you a list
of devices. Enter the number for your device
configuration and the build will continue.
It will take a while the first time, as the
source must first completely download.
4 - Build AOKP
5 - Build CM11
6 - Build Paranoid
Note: When the build is complete, you can
find the finished ZIP in the Builds
folder on your Android disk image.
0 - Go to Main Menu
Enter option:
There you can see you need to install Xcode, if you haven't already. It is free in the Mac App Store (this script is for OS X, you know).
All the options are pretty self explanatory. Make sure you have run the folder set up for the ROM which you wish to build first. Run the initialize option for your desired ROM before running the build option.
That is it! After the build has done (takes a while the first time you set all this up, much faster the next few times) you will be able to build your own nightly's, or whatever you so desire! Have fun!
NOTE: If you have an error regarding "Xcode" and the "10.6 SDK" when trying to build, download the MacOSX10.6.sdk.zip, decompress it, and place it in "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/".
Disclaimer: This will only work with devices that are officially supported by AOKP PA or CM Team If your device is not supported by their official repos, either wait to see if it is, try to edit my script to get the repo your device's tree is on, or wait until I get around to writing a tutorial on how to add custom repos.
Update :
V2 : Now we have :
AOKP :
Repo KitKat
Repo JB MR2
Repo JB MR1
CM :
Repo CM 11
Repo CM 10.2
Repo CM10.1
Paranoid KitKat
OMNI :
Repo KitKat
Repo 4.3
For me ...
Any return ?
V3 :
echo "============================================================"
echo " OS X Build $version - based on Kyle Frost"
echo " By @ptichalouf"
echo "============================================================"
echo
echo " ---Advanced Menu---"
echo
echo " MUST INSTALL XCODE BEFORE BUILDING!"
echo " XCODE IS FREE IN THE MAC APP STORE!"
echo
echo " 1 - Initialize AOKP Repo"
echo " 2 - Initialize CM Repo"
echo " 3 - Initialize PARANOID Repo"
echo " 4 - Initialize OMNI Repo"
echo " 5 - Initialize LIQUID Repo"
echo " 6 - Initialize CODENAME Repo"
echo " 7 - Initialize CARBON Repo"
echo
echo " 0 - Go to Main Menu"
echo
echo -n "Enter option: "
And Linux version
V3 & Linux
--
[GUIDE]A Noob Guide On Building Your Own Custom Kernel on WIN10 (ARM & ARM64 & MTK)
1. INTRODUCTION:
This is a guide to build your own custom kernel. Although I'm still a "noob" at this,
I've struggled a lot to build one as all the guides which I followed were not very clear.
So I hope this will be clear enough and as noob friendly as possible!
You will learn how to:
- Build a kernel for arm and mediatek devices on windows 10
- Add feature
- Basic use of git
Prerequise :
- Updated windows 10 64bits (falls creators update)
- A decent computer with a decent internet speed
- Space on your HDD The minimum space for a kernel source (and its compiled code) is about 2 GB
- Minimal linux knowledge (Terminal, Commands etc)
- Your Brain
- And finally patience
2.Setting UP ENVIRONMENT:
Installing ubuntu :
1 - Go in Settings -> Update and Security -> For developers and turn on developers mode then
2 - go in Control Panel > Programs > Turn Windows Features On Or Off and enable Windows subsystem for linux
3 - Reboot your computer
4 - launch linux subsystem now and let it download all it need and set up your password (remember it ! you'll need this password later)
5 - Go in microsoft app store and download Ubunutu by canonical group limited
6 - Open ubuntu (a windows with your name and computer name wil appear), congrats you installed ubuntu on windows 10 !
Seting up you environment :
1 - Type "apt-get update" (will update all repo for apps and dependencies)
From here it is nearly the same as my previous guide, but be careful there is some little changes
2 - Type "sudo apt-get install -y build-essential kernel-package libncurses5-dev bzip2" (will install all dependencies to build kernel)
3 - Check if dependencies are correctly installed :
- Then type "gcc"
If "gcc" is already installed, you should see "gcc : fatal error : no input file"
- Then type "make"
If "make" is already installed, you should see "make: *** no target specified and no makefile found. stop."
- Then type "git"
If "git" is already installed, you should see bunch of basic git commands
Now you're almost ready to start building your kernel!
Toolchains:
There are several types of toolchains (GCC, Linaro and few custom made ones)
Warning : Not every single device kernel will boot (or even compiles) with older or newer GCC
- For ARM:
We'll be using GCC 4.7 in this tutorial (link : https://android.googlesource.com/platform/prebuilts/gcc/linux-x86/arm/arm-eabi-4.7/ )
-Open terminal and type: "mkdir kernel"(Type the name you want, I used "kernel")
-Then type "cd kernel" (the name which you used above)
-Then type "git clone https://android.googlesource.com/platform/prebuilts/gcc/linux-x86/arm/arm-eabi-4.7"
-Wait till it finishes.
- For ARM 64:
For ARM 64 you need a 64 bit kernel compiler (there's "aarch64" in the name for telling that's a 64 bit compiler)
Exemple : https://releases.linaro.org/archive/13.07/components/toolchain/binaries/gcc-linaro-aarch64-linux-gnu-4.8-2013.07-1_linux.tar.xz
3.DOWNLOADING SOURCE FILES FOR YOUR DEVICE:
Now you have to find a github that contains your kernel source.
Search on Google or XDA to find a kernel github repo for your device.
A kernel github looks like this: "https://github.com/atxoxx/android_kernel_samsung_msm8974/tree/xenomTW"
On the upper left side you can see branch: completed by a name
These are the different versions of the kernel/project (generally can be for testing, android version etc)
QUICK EXPLANATION OF FILES/FOLDERS:
- /arch/arm/configs : contains the config files for device (where you add option like new governors, features etc)
- /output/arch/arm/boot/ : Where zimage is stored (will explain that later)
- build.sh : Script to make the building much easier (will explain how it works later)
- /arm-cortex-linux-gnueabi-linaro_5.2-2015.11-2 : I put the toolchain in my kernel source making it easier to find (your kernel's toolchain name may be different)
If you don't have your kernel source yet, you need to download it.
Open terminal and make sure that you are in "kernel" folder (the one you previously created)
Then type in terminal : "git clone "URL of the github kernel" -b "name of the branch" "
For Example : "git clone https://github.com/atxoxx/android_kernel_samsung_msm8974 -b xenomTW"
Good! Now you have your kernel source!
4.BUILDING:
For an easier way you can go to the location using your file explorer to : "/home/"name of your session"/kernel"
You'll see two folders (The Toolchain and The Kernel Source)
Go into your kernel source folder.
- For ARM:
Copy paste this:
#!/bin/bash
export ARCH=arm
export CROSS_COMPILE=
mkdir output
make -C $(pwd) O=output "name of defconfig and variant if needed"
make -j4 -C $(pwd) O=output
Click to expand...
Click to collapse
Explaination:
- #!/bin/bash: Tells the script to run in shell command
- export ARCH=arm: Defining which kernel architecture type it is (For example arm64 etc)
- export CROSS_COMPILE= : Locate where the toolchain is, it has to match the exact path to it and the dash ("-") in the end is really important ! (Almost everyone makes an error at this part!!!)
- mkdir output: Create a directory for storing compiled zimage
- make -C $(pwd) O=output : Defining defconfig for guiding kernel compilation (will explain later)
- make -j4 -C $(pwd) O=output: where the building start, "-j4" is how fast it'll compile, you have to setup this number according to your CPU !
- cp output/arch/arm/boot/Image $(pwd)/arch/arm/boot/zImage: This one is for moving image into the second path (thanks @Has.007 for this infromation)
Click to expand...
Click to collapse
Example :
#!/bin/bash
export ARCH=arm
export CROSS_COMPILE=$(pwd)/arm-cortex-linux-gnueabi-linaro_5.2-2015.11-2/bin/arm-cortex-linux-gnueabi-
mkdir output
make -C $(pwd) O=output msm8974_sec_defconfig VARIANT_DEFCONFIG=msm8974_sec_ks01_skt_defconfig SELINUX_DEFCONFIG=selinux_defconfig
make -j4 -C $(pwd) O=output
cp output/arch/arm/boot/Image $(pwd)/arch/arm/boot/zImage
Click to expand...
Click to collapse
- For ARM 64:
Copy paste this:
#!/bin/bash
export ARCH=arm64
export CROSS_COMPILE="path to your toolchain" (it have to end by something like "nameofarch-something-")
mkdir output
make -C $(pwd) O=output "name of defconfig and variant if needed"
make -j4 -C $(pwd) O=output
Click to expand...
Click to collapse
Explaination:
- #!/bin/bash: Tells the script to run in shell command
- export ARCH=arm64: Defining which kernel architecture type it is (For example arm64 etc)
- export CROSS_COMPILE= : Locate where the toolchain is, it has to match the exact path to it and the dash ("-") in the end is really important ! (Almost everyone makes an error at this part!!!)
- mkdir output: Create a directory for storing compiled zimage
- make -C $(pwd) O=output : Defining defconfig for guiding kernel compilation (will explain later)
- make -j4 -C $(pwd) O=output: where the building start, "-j4" is how fast it'll compile, you have to setup this number according to your CPU !
- cp output/arch/arm/boot/Image $(pwd)/arch/arm/boot/zImage: This one is for moving image into the second path (thanks @Has.007 for this infromation)
Click to expand...
Click to collapse
Example :
#!/bin/bash
export ARCH=arm64
export CROSS_COMPILE=$(pwd)gcc-linaro-aarch64-linux-gnu-4.8-2013.07-1_linux\bin\aarch64-linux-gnu-
mkdir output
make -C $(pwd) O=output msm8974_sec_defconfig VARIANT_DEFCONFIG=msm8974_sec_ks01_skt_defconfig SELINUX_DEFCONFIG=selinux_defconfig
make -j4 -C $(pwd) O=output
cp output/arch/arm/boot/Image $(pwd)/arch/arm/boot/zImage
Click to expand...
Click to collapse
- For Mediatek:
Copy paste this:
#!/bin/bash
export CROSS_COMPILE="path to your toolchain" (it have to end by something like "nameofarch-something-")
export ARCH=arm ARCH_MTK_PLATFORM=
make "name of defconfig and variant if needed"
make -j4
Click to expand...
Click to collapse
Explaination:
- #!/bin/bash: Tells the script to run in shell command
- export CROSS_COMPILE= : Locate where the toolchain is, it has to match the exact path to it and the dash ("-") in the end is really important ! (Almost everyone makes an error at this part!!!)
- export ARCH=arm ARCH_MTK_PLATFORM=: Defining which kernel architecture type it is (For example arm64 etc) "ARCH_MTK_PLATFORM=" is for specifying which mediatek platform it is
- make _defconfig : Defining which defconfig to use (will explain later)
- make -j4: where the building starts, "-j4" is how fast it'll compile, you have to setup this number according to your CPU !
Click to expand...
Click to collapse
Example :
#!/bin/bash
export CROSS_COMPILE=$(pwd)/arm-eabi-4.8/bin/arm-eabi-
export ARCH=arm ARCH_MTK_PLATFORM=mt6580
make pixi4_4_8g1g_defconfig
make -j4
Click to expand...
Click to collapse
When these step are done make sure you are in kernel folder in terminal and type "sudo bash build.sh" then type your password you set up in first launch of linux subsytem
(sudo is important, windows 10 ubuntu seems to handle permission differently than native ubuntu)
The compilation have started
If it compiles without any problems:
Wait till it finishes (it'll say something like "zimage is ready")
If you followed arm and arm64:
Then go to "/Output/arch/arm/boot/" to find your zimage.
Click to expand...
Click to collapse
If you followed mediatek:
Then go to "/arch/arm/boot/" to find your zimage.
Click to expand...
Click to collapse
Caution : Not all kernel build Zimage, it can build image or other compressed image
If in case you have any errors:
Check and see what it says, generally it'll tell you where the error is.
If the text is going too fast reduce the -j number as explained above.
For reference I compile with an AMD Phenom X4 3.4GHz,Samsung HDD and 8GB of RAM and it takes around 10min to build
It is recommanded to type in the terminal "make clean" and "make mrproper" before compiling again
5.MAKING THE KERNEL BOOT:
You have 2 solutions here:
1) You can use @osm0sis anykernel method, which is explainded here: "https://forum.xda-developers.com/showthread.php?t=2670512" (A huge shoutout to him!)
OR
2) You can unpack the boot.img (from the same rom (CM, touchwizz,sense etc) and android version) and swap Zimage in it explained here : "https://forum.xda-developers.com/showthread.php?t=2073775" (thanks again to @osm0sis !)
Before flashing the kernel which you've made, backup your "stock" boot.img and Then flash your kernel and see if it boots!
6.HOW TO ADD FEATURE TO KERNEL WORK:
Here starts the most interesting part! Now let's see how it works:
Basically you can add: Governors, IO Schedulers, Overclock the CPU & Many Tweaks...
Checkout the github section (Section 7) to see how to add them properly.
Here's an exemple for adding a governor (this one is called Intellimm) : https://github.com/gugu0das/android...mmit/7186ee60c171b06ea3a027e8383be392d3186bb1
The text in the blue box is the commit description (generally tells you about the changelog, general information and who originally made the commit)
The other text boxes tell you about where and which files have been modified/changed.
Everything in green indicates what has been added.
Everything in red indicates what has been deleted.
We can see in the first 2 text boxes that in "arch/arm/configs/" "msm8974_sec_defconfig" and "cm_msm8974_sec_defconfig" have been modified.
Between the lines 140 and 141 of this files this text has been added : "CONFIG_CPU_FREQ_GOV_INTELLIMM=y"
(This line is for enabling Intellimm when you're compiling your kernel)
Same technique applies to the other text boxes (what has been added and deleted and it's location)
Depending on the features you add, more or less files can be modified, added or deleted.
So to sum it up, a Commit let's you see all the changes which have been made and everything else!
7.GUIDE TO GITHUB:
For this, I'll direct you over to this awsome guide made by @eagleeyetom !
8.GPL (IMPORTANT !!!):
The Rules as they apply on XDA
As XDA has no legal power to uphold the GPL (and frankly we want to stay as far away from doing so as possible), we can’t force any of our users to abide by the GPL. However it is in XDA’s interests as well as the interests of our developer-base to ensure all GPL-derived materials hosted or linked on XDA comply fully with the GPL.
GPL-derived materials that do not come with the complete sources used to compile the GPL components are considered warez, and will be treated as such under forum rule 6 and 9.
If you use GPL components, but do not make any modifications to them whatsoever, you should provide a link to the original source of your GPL code.
Sources accompanying a release should be complete, and contain all the necessary source code for any modules, scripts or definition files. Complete sources will be defined as those which compile correctly and completely against the platform for which the software is distributed, and which contain any and all modifications made to the released General Public Licenced code. The source code supplied should be the exact version for which the source code is being requested, complete with all modifications.
EXAMPLE: Here’s a bit of code that could be used as a template to post your releases
<Kernel Or Author Name> <Kernel Nr>:
<Source>|<ReadMe>|<Credits>|<Other>
The Very Quick Summary of General Public License (GPL)
The text of the GPL Licence itself will be used to reach any final conclusion regarding any disputes over GPL Licenced materials. The above is a summary of what XDA expects of members using GPL code, and the complete text can be read at the GNU website.
The GPL states that anyone who modifies GPL licenced code is required to make available the sources used to compile it. This is to further improve and encourage collaborative work, as well as to ensure that the best code possible is produced, and to encourage peer-review of all work. This benefits both developers and end users in numerous ways, including:
Allowing anyone to verify the code they are trusting with their data, and its authenticity
Encouraging community collaboration to produce faster fixes and updates, and better code
Helping bring new developments from other devices and fields to your own, letting you benefit from new code that wouldn’t have been available without this sharing.
The GPL imparts great freedom for GPL end users. It ensures innovation is never stifled and no project is dependent upon any single developer.
It is in everyone’s interest for the GPL to be adhered to, as it gives us all better ROMs, better transparency, and a better atmosphere for developers to work together to make great code.
Click to expand...
Click to collapse
THANKS :
- @ravish_919 : For testing and correcting this guide
- @karkasss : As my friend and support
- @gugu0das : For helping me a lot when I tried to build my kernel
- @eagleeyetom : For his awsome github guide
- @osm0sis For his aswsome anykernel solution
- @kirito9 : Huge thanks to him for providing mediatek guide !
- @F4uzan : Huge thanks to him for giving me a lot of useful information to fill this guide !
- @sunilpaulmathew : For providing an advanced method to rename your kernel ! (again)
- @nathanchance : For a proper kernel naming method
- @RendyAK and @DroidThug : For correcting me about "#!/bin/bash"
- @ahmed.ismael : For helping me, giving feedback and his huge support !
- Microsoft and canonical for the windows linux subsystem documentation
- All the developers for their hard work !
- XDA and The Community!
TIPS AND TRICKS
1. You can use a copy of a defconfig file with different setup :
Usage : Use a "stock" one and use another one with esperimental feature for testing without altering original defconfig
Exemple : copy "stock" defconfig and in copied one add a governor see if it compile and work
How to do : Create a second build.sh with modified defconfig name !
2. Change kernel name and version :
Simple method :Edit this line "CONFIG_LOCALVERSION="-" after - in your defconfig
Exemple : CONFIG_LOCALVERSION="-XenomTW-3.2.6"
Advanced methods :
Method 1 :
1. Go in Makefile in the root folder of your kernel source
2. Add
CONFIG_LOCALVERSION="nameofyourkernel"
LOCALVERSION="versionofyourkernel"
Click to expand...
Click to collapse
Exemple :
VERSION = 4
PATCHLEVEL = 4
SUBLEVEL = 127
EXTRAVERSION =
CONFIG_LOCALVERSION="-FlashKernel"
export LOCALVERSION="-v1.00"
Click to expand...
Click to collapse
Caution ! Never touch or edit VERSION, PATCHLEVEL, SUBLEVEL, and EXTRAVERSION !
Click to expand...
Click to collapse
Method 2 :
1. Go in "scripts/mkcompile_h"
2. Add
LINUX_COMPILE_BY="nameofyourchoice"
LINUX_COMPILE_HOST="nameofyourchoice"
Exemple
Click to expand...
Click to collapse
3. Solve problem with PATH :
If you encounter "IS YOUR PATH CORRECT" problem try in terminal :
"export PATH="pathtotoolchainlocation"/bin:$PATH"
Exemple : export PATH=/home/3lambda/kernel/M8_Kernel/arm-eabi-4.7/bin:$PATH
4. Access ubuntu folders :
Path location to ubuntu folder is : C:\Users"NAME"\AppData\Local\Packages\CanonicalGroupLimited.UbuntuonWindows_79rhkp1fndgsc\LocalState\rootfs\home
Caution ! Editing files here directly from windows may break permission, you'll have to fix them if so (look on google on how to)
More to come...
Edit and update are coming, I may have forgot things let me know
Feed back also appreciated
highly appreciated
thank you
Good guide although I would say the advanced method for changing the kernel name is making totally unnecessary edits to the source code. There is already a framework in place for configuring the version string however you want. Editing EXTRAVERSION could result in conflicts during stable merges. The version gets generated in the following order:
Code:
$(VERSION).$(PATCHLEVEL).$(SUBLEVEL)$(EXTRAVERSION)$(CONFIG_LOCALVERSION)$(LOCALVERSION)
VERSION, PATCHLEVEL, SUBLEVEL, and EXTRAVERSION are set in the main Makefile, the first three should never be touched. CONFIG_LOCALVERSION and LOCALVERSION should be what the user sets.
For example:
Code:
VERSION = 4
PATCHLEVEL = 4
SUBLEVEL = 127
EXTRAVERSION =
CONFIG_LOCALVERSION="-FlashKernel"
export LOCALVERSION="-v1.00"
would result in the following output:
Code:
4.4.127-FlashKernel-v1.00
nathanchance said:
Good guide although I would say the advanced method for changing the kernel name is making totally unnecessary edits to the source code. There is already a framework in place for configuring the version string however you want. Editing EXTRAVERSION could result in conflicts during stable merges. The version gets generated in the following order:
VERSION, PATCHLEVEL, SUBLEVEL, and EXTRAVERSION are set in the main Makefile, the first three should never be touched. CONFIG_LOCALVERSION and LOCALVERSION should be what the user sets.
For example:
would result in the following output:
Click to expand...
Click to collapse
Thanks for pointing this out
I'll take a look when I'll have time
Nice! Been looking for something like this. XDA feed brought me here.
is it necessary do in win10? or you can simply do it in linux pc without WIN10?
thanks for the post!
tobarreh said:
is it necessary do in win10? or you can simply do it in linux pc without WIN10?
thanks for the post!
Click to expand...
Click to collapse
it should be done on linux
but windows can handle linux as a subsystem now so the guide for people like me who is dumb enough to use windows
seriously skip the ubuntu installation steps on sindows and start building your kernel by following the other steps :good:
Sent from my OnePlus 5 using XDA Labs
nathanchance said:
Good guide although I would say the advanced method for changing the kernel name is making totally unnecessary edits to the source code. There is already a framework in place for configuring the version string however you want. Editing EXTRAVERSION could result in conflicts during stable merges. The version gets generated in the following order:
Code:
$(VERSION).$(PATCHLEVEL).$(SUBLEVEL)$(EXTRAVERSION)$(CONFIG_LOCALVERSION)$(LOCALVERSION)
VERSION, PATCHLEVEL, SUBLEVEL, and EXTRAVERSION are set in the main Makefile, the first three should never be touched. CONFIG_LOCALVERSION and LOCALVERSION should be what the user sets.
For example:
Code:
VERSION = 4
PATCHLEVEL = 4
SUBLEVEL = 127
EXTRAVERSION =
CONFIG_LOCALVERSION="-FlashKernel"
export LOCALVERSION="-v1.00"
would result in the following output:
Code:
4.4.127-FlashKernel-v1.00
Click to expand...
Click to collapse
I personally like to define "LINUX_COMPILE_BY" and "LINUX_COMPILE_HOST" in "scripts/mkcompile_h" just like in this commit by @franciscofranco. By adding this
Code:
LINUX_COMPILE_BY="francisco"
LINUX_COMPILE_HOST="franco"
would display "[email protected]"
3lambda said:
Thanks for pointing this out
I'll take a look when I'll have time
Click to expand...
Click to collapse
Added new kernel naming method by @nathanchance and @sunilpaulmathew, huge thanks to them !
I'll need feedback see if some of these steps aren't clear or if I forgot things
I may also add a video soon and maybe screenshots
Stay tuned
sunilpaulmathew said:
I personally like to define "LINUX_COMPILE_BY" and "LINUX_COMPILE_HOST" in "scripts/mkcompile_h" just like in this commit by @franciscofranco. By adding this
Code:
LINUX_COMPILE_BY="francisco"
LINUX_COMPILE_HOST="franco"
would display "[email protected]"
Click to expand...
Click to collapse
He stopped doing that after I pointed out the same thing can be achieved without any source code edits (since it would change it for everyone building your source)
Code:
export KBUILD_BUILD_USER=francisco
export KBUILD_BUILD_HOST=franco
From: https://github.com/torvalds/linux/blob/master/scripts/mkcompile_h
If I downloaded a zip file of my source code, how do I extract it, since this is no longer just cloning from Git?
https://github.com/gugu0das/android...mmit/7186ee60c171b06ea3a027e8383be392d3186bb1
link is dead . please update this part of the guide as it is the most important part for me . Thanks for this awesome guide. more updates to come
I'm really looking forward to any and all updates on this thread. I'm old and loosing my mind and all my "Re-Memories" too, so I need all the help I can get.... LOL
permission denied
* placed the toolchain in home folder *
when i initiate compilation it says permission denied
how do i fix?
kwshl said:
* placed the toolchain in home folder *
when i initiate compilation it says permission denied
how do i fix?
Click to expand...
Click to collapse
execute with 'sudo' make
otonieru said:
execute with 'sudo' make
Click to expand...
Click to collapse
that doesn't help, i did
kwshl said:
* placed the toolchain in home folder *
when i initiate compilation it says permission denied
how do i fix?
Click to expand...
Click to collapse
did you place it by doing a git clone or do a cp command ?
this will lessen the nuisance i hope
otonieru said:
did you place it by doing a git clone or do a cp command ?
Click to expand...
Click to collapse
[email protected]:/mnt/g/linux/oreo# bash k.sh
make: execvp: /home/kwshl/aarch64-linux-android-4.9/bin/aarch64-linux-android-gcc: Permission denied
HOSTCC scripts/basic/fixdep
HOSTCC scripts/basic/bin2c
Hello, when you using phh's GSIs, you may found bugs on auto brightness, battery usage data, etc (e.g: Xiaomi, Huawei, etc.). For this, you can make an overlay files for your mobiles.
Requirements:
framework-res.apk on your stock rom. (It may on /system/framework/framework-res.apk)
apktool (to extract framework-res.apk)
git (to clone repo)
Linux (to build overlay files)
Steps:
1. Fork phhusson/vendor_hardware_overlay on GitHub.
2. Clone Repository which you forked, and make a new branch for your works.
3. Copy a device which similar for your device, and rename it to your own device.
4. Edit <Your device>/Android.mk (Just modify the contents of the brackets, same as below)
Code:
LOCAL_PATH := $(call my-dir)
include $(CLEAR_VARS)
LOCAL_MODULE_TAGS := optional
LOCAL_PACKAGE_NAME := treble-overlay-[Manufacturer of your device]-[Name of your device]
LOCAL_MODULE_PATH := $(TARGET_OUT)/overlay
LOCAL_IS_RUNTIME_RESOURCE_OVERLAY := true
LOCAL_PRIVATE_PLATFORM_APIS := true
include $(BUILD_PACKAGE)
5. Edit <Your device>/AndroidManifest.xml
Code:
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
package="me.phh.treble.overlay.[Manufacturer of your device].[Name of your device]"
android:versionCode="1"
android:versionName="1.0">
<overlay android:targetPackage="android"
android:requiredSystemPropertyName="ro.vendor.build.fingerprint"
android:requiredSystemPropertyValue="+[write as similar format]*"
android:priority="[Take next number of last used]"
android:isStatic="true" />
</manifest>
If you don't know how to take androidriority, you can execute this commands to list priority which used already:
Code:
cd [Path of repository]
find -name AndroidManifest.xml |while read manifest;do packagename="$(xmlstarlet sel -t -m '//manifest' -v @package -n $manifest)";priority="$(xmlstarlet sel -t -m '//overlay' -v @android:priority -n $manifest)";echo -e "$priority\t$packagename";done|more
(the single line command split here, just for viewing or making a shell script):
Code:
find -name AndroidManifest.xml |while read manifest;do
packagename="$(xmlstarlet sel -t -m '//manifest' -v @package -n $manifest)";
priority="$(xmlstarlet sel -t -m '//overlay' -v @android:priority -n $manifest)";
echo -e "$priority\t$packagename";
done
It will print a list for you (unsorted), and you just take an unused priority.
You can find device fingerprint on /vendor/build.prop, or execute following commands (on your device or adb shell):
Code:
getprop ro.vendor.build.fingerprint
If you can't find ro.vendor.build.fingerprint, you can find ro.vendor.product.name or ro.product.vendor.device, then change android:requiredSystemPropertyName and android:requiredSystemPropertyValue what you found.
Huawei seems to use ro.hw.oemName to detect phone, and others manufacturer maybe have their own detection, too. But it should be unique for other phones with same manufacturer.
(It's NOT recommended unless your device don't have ro.vendor.build.fingerprint.)
6. Extract framework-res.apk with apktool on another side. (to avoid commited by mistake)
7. Replace <Your device>/res/xml/power_profile.xml to which power_profile.xml for your device.
It will on <Extracted framework-res.apk>/res/xml/power_profile.xml.
Or you can find whoever's device tree, it may have this file, too.
8. Edit <Your device>/res/values/config.xml
Find each key on following files what for your device:
Code:
[Extracted framework-res.apk]/res/values/arrays.xml
[Extracted framework-res.apk]/res/values/bools.xml
[Extracted framework-res.apk]/res/values/fractions.xml
[Extracted framework-res.apk]/res/values/integers.xml
And then replace values into <Your device>/res/values/config.xml
9. Add your device into <Path of repository>/overlay.mk
Just write like other lines.
10. Build overlays
Just execute following commands:
Code:
chmod u+x [Path of repository]/build/build.sh
[Path of repository]/build/build.sh
If You get this, Do what it said:
Code:
Please install aapt (apt install aapt should do)
Or if you get this:
Code:
OpenJDK Server VM warning: You have loaded library /root/overlay/vendor_hardware_overlay/build/signapk/libconscrypt_openjdk_jni.so which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
Exception in thread "main" java.lang.UnsatisfiedLinkError: org.conscrypt.NativeCrypto.get_cipher_names(Ljava/lang/String;)[Ljava/lang/String;
at org.conscrypt.NativeCrypto.get_cipher_names(Native Method)
at org.conscrypt.NativeCrypto.<clinit>(NativeCrypto.java:764)
at org.conscrypt.OpenSSLProvider.<init>(OpenSSLProvider.java:56)
at org.conscrypt.OpenSSLProvider.<init>(OpenSSLProvider.java:49)
at com.android.signapk.SignApk.main(SignApk.java:942)
I have no solution, either. Try to build on another computer.
11. Test overlay with tests.sh and yourself
For general checks (symtax, etc.), just execute following command:
Code:
chmod u+x [Path of repository]/tests/tests.sh
[Path of repository]/tests/tests.sh
You should fix errors what reported for your device, and then restart from step 10.
If it reported errors but not for your device, just ignore it.
When check passed, you can find overlay file on <Path of repository>/build/treble-overlay-<Manufacturer of your device>-<Name of your device>.apk, just copy it into your phone's /system/overlay/, and set permission to rw-r--r-- / 0644.
And then reboot your phone to test your overlay.
12. If it works for your device, don't forget to perform a pull request for phhusson/vendor_hardware_overlay, to support his awesome works.
Thanks for the guide @minadzuki
but i'm getting this error after step 10:
(core dumped) aapt package -f -F ${name}-unsigned.apk -M $path/AndroidManifest.xml -S $path/res -I android.jar
even when i try to build for existing devices I got the same error
king1990 said:
Thanks for the guide @minadzuki
but i'm getting this error after step 10:
(core dumped) aapt package -f -F ${name}-unsigned.apk -M $path/AndroidManifest.xml -S $path/res -I android.jar
even when i try to build for existing devices I got the same error
Click to expand...
Click to collapse
Oh...I forgot this, sorry.
Because I haven't encountered this error, so I don't have a way to solve it, either. (I tried to built it on three remote servers only)
And...is it cause segment fault?
Or you can try to install google-android-build-tools instead of aapt. (because I can't find aapt on archlinux but found this)
Very nice guide, thanks!
I'll link it as part of a "how to contribute"
For aapt, there is an aapt included in the git. If it doesn't work for you (like @king1990's segfault), know that the one included in the git has the lowest priority. It will pick up aapt from the system first.
(but then you'll need to have a system-wide aapt).
Or if you have an Android SDK somewhere, you can set your PATH to include it.
Edit: I just realized that what I just described might actually be the issue. If there is a system-wide aapt, it will use the libc++ in the git, which would explain the crash.
I fixed this case in my current git.
If there is a system-wide aapt it will use it.
If for some reason, system-wide aapt isn't suitable, you can run build.sh with --local-aapt to force the use of the in-git aapt (i.e. it will have higher priority than system's so no conflict should happen)
king1990 said:
Thanks for the guide @minadzuki
but i'm getting this error after step 10:
(core dumped) aapt package -f -F ${name}-unsigned.apk -M $path/AndroidManifest.xml -S $path/res -I android.jar
even when i try to build for existing devices I got the same error
Click to expand...
Click to collapse
Could you check again with latest git?
If it still doesn't work, can you try to call build.sh --local-aapt ?
@phhusson Thanks now it working fine and building without faults
I have few questions regarding the AndroidManifest.xml :
1- for android:requiredSystemPropertyValue how do I know which name to use ? is it trial and error ?
2- for androidriority does matter which number I choose ?
3- some vendors have this overlay (framework-res__auto_generated_rro.apk) under "vendor/overlay". Does is affect the device overlay in system/overlay ?
4- to install the overlay is it just as @minadzuki says ?
and thanks again for your great work
king1990 said:
@phhusson Thanks now it working fine and building without faults
I have few questions regarding the AndroidManifest.xml :
1- for android:requiredSystemPropertyValue how do I know which name to use ? is it trial and error ?
2- for androidriority does matter which number I choose ?
3- some vendors have this overlay (framework-res__auto_generated_rro.apk) under "vendor/overlay". Does is affect the device overlay in system/overlay ?
4- to install the overlay is it just as @minadzuki says ?
and thanks again for your great work
Click to expand...
Click to collapse
1. For example, Xiaomi Mi 6X has this fingerprint:
xiaomi/wayne/wayne:8.1.0/OPM1.171019.011/8.7.12:user/release-keys/
So you can take xiaomi/wayne to use, and android:requiredSystemPropertyValue will be: +xiaomi/wayne*
2. it will affect to build images, or affect the detection on System. If you got this:
Code:
F: .xxx/AndroidManifest.xml: priority xx conflicts with another manifest
You just take next number and try again, until now priority 53 has been taken (without pull request), you can take priority since 54.
3. Sorry, I don't know for this.
4. Yes, just simply copy and permission change.
king1990 said:
@phhusson Thanks now it working fine and building without faults
Click to expand...
Click to collapse
Cool
3- some vendors have this overlay (framework-res__auto_generated_rro.apk) under "vendor/overlay". Does is affect the device overlay in system/overlay ?
Click to expand...
Click to collapse
overlays in /system/overlay and /vendor/overlay are the same thing, and on my GSI both are loaded, so yes it affects it.
But if your vendor has a framework-res__auto_generated_rro.apk, you probably don't need an overlay file for your phone, because it's already there.
(Please note that /system/overlay doesn't exist in AOSP, that's a change specific to my ROM, and custom ROMs based on it. AOSP only has /vendor/overlay)
@minadzuki Thanks for answering my questions. @phhusson its clear now , I don't know if this is related to your treble approach or not but some GSI ROMs refuse to go the minimum value in the brightness curve no mater what value I set in the curve, it never achieved.
is this treble or ROM related ?
king1990 said:
@minadzuki Thanks for answering my questions. @phhusson its clear now , I don't know if this is related to your treble approach or not but some GSI ROMs refuse to go the minimum value in the brightness curve no mater what value I set in the curve, it never achieved.
is this treble or ROM related ?
Click to expand...
Click to collapse
Brightness is a bit complex on Android Pie... (well it already was before, but it got even worse)
Do you have a link to your current overlay, so I can take a look at which code path you'd go to?
What's the lowest backlight value you get in /sys/class/backlight/panel0-backlight/brightness or /sys/class/leds/lcd-backlight/brightness?
Also what's your device? Does it have extended brightness range? (what's the value of getprop persist.sys.qcom-brightness // is it a samsung device?)
phhusson said:
Brightness is a bit complex on Android Pie... (well it already was before, but it got even worse)
Do you have a link to your current overlay, so I can take a look at which code path you'd go to?
What's the lowest backlight value you get in /sys/class/backlight/panel0-backlight/brightness or /sys/class/leds/lcd-backlight/brightness?
Also what's your device? Does it have extended brightness range? (what's the value of getprop persist.sys.qcom-brightness // is it a samsung device?)
Click to expand...
Click to collapse
for overlay this the extracted files from the overlay apk (my device is Axon7)
The lowest backlight is 10 (once manually edit it to 1 it goes to really dim brightness)
what I don't understand that I already built two GSI ROMs one is DU13 & other Liquid pie both report the same backlight light value at 10 at minimum value but in DU its very dim and in liquid its still bright ?!
The command did not give back any information so I assume we don't extended brightness range.
Thanks
Edit : it was overlay problem , fixed after modifying it
I tried to create an overlay for my Honor View 10 (Berkeley)
Is there a list of recommended settings to copy over from the stock ROM? I started with copying everything listed in tests/knownKeys and it seems to work
There's a power_profile.xml in the framework-res.apk I dumped from my stock ROM but the battery capacity is set to 1000 instead of 3750. Is that right?
There's also a power_profile_test.xml in there with a much more sane value of 3000 mAh. Should I use this one instead?
Copperhead100 said:
I tried to create an overlay for my Honor View 10 (Berkeley)
Is there a list of recommended settings to copy over from the stock ROM? I started with copying everything listed in tests/knownKeys and it seems to work
There's a power_profile.xml in the framework-res.apk I dumped from my stock ROM but the battery capacity is set to 1000 instead of 3750. Is that right?
There's also a power_profile_test.xml in there with a much more sane value of 3000 mAh. Should I use this one instead?
Click to expand...
Click to collapse
I don't have any Huawei phones, sorry.
Maybe you can referrer /Huawei/kirin970/CLT or /Huawei/kirin970/EML, Honor V10 maybe have its OEM name and called "BKL"
Copperhead100 said:
I tried to create an overlay for my Honor View 10 (Berkeley)
Is there a list of recommended settings to copy over from the stock ROM? I started with copying everything listed in tests/knownKeys and it seems to work
There's a power_profile.xml in the framework-res.apk I dumped from my stock ROM but the battery capacity is set to 1000 instead of 3750. Is that right?
There's also a power_profile_test.xml in there with a much more sane value of 3000 mAh. Should I use this one instead?
Click to expand...
Click to collapse
On Huawei, you'll find real power_profile.xml somewhere in /odm or /product. I'd say /product/etc/power_profile.xml
phhusson said:
On Huawei, you'll find real power_profile.xml somewhere in /odm or /product. I'd say /product/etc/power_profile.xml
Click to expand...
Click to collapse
Thanks. It's actually /product/etc/xml/power_profile.xml
There's a new pull request incoming
is it possible to create an overlay in Widows usingGIT for Windows?
JEANRIVERA said:
is it possible to create an overlay in Widows usingGIT for Windows?
Click to expand...
Click to collapse
Yes, you can write source code everywhere (don't forget to replace CRLF to LF), but maybe you should build it on Linux, or you have a way to build it on Windows (e.g.: WSL, Mingw, etc.)
@minadzuki I can't find a config.xml in my framework-res.apk, am I missing something? or that gets created somehow in the process?
another question
on step 5 you say to edit AndroidManifext.xml, does that means delete everything in it and copy paste what you put on the code box? or just add that at the end of the xml file?
JEANRIVERA said:
@minadzuki I can't find a config.xml in my framework-res.apk, am I missing something? or that gets created somehow in the process?
another question
on step 5 you say to edit AndroidManifext.xml, does that means delete everything in it and copy paste what you put on the code box? or just add that at the end of the xml file?
Click to expand...
Click to collapse
framework-res.apk doesn't contain config.xml, it split into several files: arrays.xml, bools.xml, fraction.xml, integers.xml. just create config.xml or copy from other devices and find values on these files (to replace it).
and step 5, just replace values which for your device, like what other devices wrote.
minadzuki said:
framework-res.apk doesn't contain config.xml, it split into several files: arrays.xml, bools.xml, fraction.xml, integers.xml. just create config.xml or copy from other devices and find values on these files (to replace it).
and step 5, just replace values which for your device, like what other devices wrote.
Click to expand...
Click to collapse
thanks for the response but I am really lost here those 4 xml has way to much data in it, should I just copy everything to config.xml?
to be honest what I am trying to fix are 2 things the battery stats not showing and the auto brightness and brightness slider and if the led light is fixable by an overlay that too, I can provide you all the files needed could you build it for me please? my device is an HTC U12+
JEANRIVERA said:
thanks for the response but I am really lost here those 4 xml has way to much data in it, should I just copy everything to config.xml?
to be honest what I am trying to fix are 2 things the battery stats not showing and the auto brightness and brightness slider and if the led light is fixable by an overlay that too, I can provide you all the files needed could you build it for me please? my device is an HTC U12+
Click to expand...
Click to collapse
you can check like this:
https://github.com/phhusson/vendor_hardware_overlay/blob/master/Xiaomi/Mi8/res/values/config.xml
then take values from those 4 xmls.