Update 2010-02-26 Added examples for linking either dynamically or statically (see page 2)
I decided to try and set up a kitchen using the closest thing to a native kernel dev environment that I could set up.
At this point I have:
- cross-compiled the HTC Desire (kernel) sources using the Android 1.6 NDK
- figured out the repo/git stuff to pull the Android sources corresponding to a given Android release point
- cross-compiled the entire Cupcake release to an ARM target using the NDK tools
- built a dynamically linked "hello, world" executable using the NDK arm-eabi-* tools and verified it works on the phone.
Yah, I know - not much yet, but it's a start for a newb.
I have some notes documenting the Ubuntu setup process in case anyone wants it - but it is full of links (URLs), so I can't post anything with links as a new user.
If a mod can "verify" my account, I'll post it up. I promise, I'll be good.
bftb0
OK, I guess I need to make a couple of posts before I can embed links.
I'll post it without the URL protocol prefix (it's gonna look fugly), and then come back and edit it once the restriction is removed. I hope someone finds it useful.
Android kernel dev is not officially supported on Windows; closest match to Google Dev environment seems to be -> Ubuntu 8.04 LTS + JDK 5 (32 bit, not x64).
I realize that this is nothing close to the "kitchens" that the WinMo folks have been using, but figured that (as a newb) I would just end up confusing myself trying to figure out which of those tools could be useful for Android, in addition to needing the native binary cross-compilation tools anyway.
(Note: I briefly tried to set this up in a Ubuntu VMware Player VM, but ran into limitations of available Ubuntu 8.04 LTS player machine file size limits (as well as problems hosting a VM bigger than about 10 Gb on a USB drive - insufficient space on my primary drive)
If anyone wants to follow in my footsteps, here is a thumbnail sketch of useful links. I'll admit, this is not for the faint of heart - you need Linux/Unix skills to work through the kinks.
1) (ubuntu.com/getubuntu/downloadmirrors#mirrors) Ubuntu Download Mirrors list - Find a server near you
2) Download ubuntu-8.04.4-alternate-i386.iso and burn it to 700 Mb CD-R
3) Install Ubuntu 8.04 LTS Desktop Version (gulp!)
4) Use the Ubuntu graphical package manager (System -> Administration -> Synaptic Package Manager) and install the packages (git, flex, bison, etc) mentioned in the "Ubuntu Linux (32-bit x86)" section in this (source.android.com/download)Android Source setup page
5) Note that JDK6 does not work correctly; rather than going to Sun/Oracle for JDK5, install the sun-java5-{bin,jdk,jre,doc,demo} packages (1.5.0-22-0ubuntu) using the Ubuntu package installer mentioned above. Note if you install the docs package, you will also need to go to Sun/Oracle and get jdk-1_5_0-doc.zip towards the end of the package install, and drop it into /tmp for the package manager to finish.
6) Go to the (developer.android.com/sdk/index.html) Android SDK Download page and get android-sdk_r04-linux_86.tgz
7) Unpack ( gunzip -c and...tgz | tar xf - ) to desired location, add tools dir to PATH
Optional Steps (For Java App Devs)
o-8) (eclipse.org/downloads/ Eclipse IDE Download page - chose 'Eclipse Classic 3.5.1 (162 MB)' for Linux, 32 bit (eclipse-SDK-3.5.1-linux-gtk.tar.gz)
o-9) Install by unpacking ( gunzip -c ecli....tar.gz | tar xf - ) to desired location, add eclipse folder to PATH
o-10) Run eclipse, and follow the (developer.android.com/sdk/eclipse-adt.html) Installing and Updating ADT instructions for adding Android functionality to Eclipse 3.5 (Galileo)
o-11) From within Eclipse, launch Window -> Android SDK and AVD Manager, and install all available & compatible Android SDK Tools (all versions). (You might need to use http instead of https... YMMV) Note that this is the same functionality as simply running the command "android" from the command prompt if the Android SDK (steps 6,7) tools directory is in your PATH
12) Familiarize yourself with the command line tool "adb" tool in the SDK; in particular, since Ubuntu needs root privs to access the USB hardware, the best way to start the adb server on your PC is to
Code:
$ sudo /bin/bash
- set up root's environment so the SDK "tools" directory is in root's PATH
- plug your phone into the USB cable
- On the Eris, set the options:
Settings -> Applications -> Development -> USB Debugging (on)
Settings -> Applications -> Development -> Stay awake (on)
Code:
# adb devices
This last command only lists the available USB-connected (real) devices and any emulators you have running, but as a side effect it starts up the adb server on the Ubuntu machine running as root. Thereafter, you can use adb (or ddms) as an unprivileged user.
13) Have a peek at the SDK "ddms" monitor. A variety of things can be done from here, the least of which is screen shots (Select device, then Device-> Screen capture)... and some more twisty things too.
14) Download the (developer.android.com/sdk/ndk/1.6_r1/index.html) Android 1.6 NDK r1 (Native Development Kit) - android-ndk-1.6_r1-linux-x86.zip, and unzip into your desired location. Add the ARM cross-compiler tools to your path (see example below).
15) At this point you might be using something like this to set up your path wherever you please (.profile or elsewhere, according to how you want to manage your environment)
Code:
_INSLOC='/opt/android/'
_JAVAPATH='/usr/lib/jvm/java-1.5.0-sun/bin'
_BASEPATH='/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin'
_ANDTOOLS="${_INSLOC}"'/android-sdk-linux_86/tools'
_ECLPSPATH="${_INSLOC}"'/eclipse-3.5.1'
_ARMXCOMPILE="${_INSLOC}"'/android-ndk-1.5_r1/build/prebuilt/linux-x86/arm-eabi-4.2.1/bin'
PATH="${_JAVAPATH}"':'"${_BASEPATH}"':'"${_ANDTOOLS}"':'"${_ECLPSPATH}"':'"${_ARMXCOMPILE}"
export PATH
16) Download the (member.america.htc.com/download/RomCode/Source_and_Binaries/desirec-be918f7b.tgz) HTC Desire Kernel Source Code and unpack to a location of your choice.
17) Test your enviroment by building the HTC Kernel code:
$ cd v2_6_27/kernel_msm7k; make
I can't remember if I needed to stuff anything into the environment here (e.g. 'CC' definitions, etc.) - but I know that I didn't need to modify any Makefiles. I think the make system will find all the "arm-eabi-*" executables if they are in your path. You will know soon enough.
18) Try to compile a "Hello, world!" program for the phone. Before you get too far along in this (e.g. "where is crt0.o?"), (honeypod.blogspot.com/2007/12/dynamically-linked-hello-world-for.html) read this blog post - steps 3 and 4 only.
You might find a Makefile similar to this useful:
Code:
AR = arm-eabi-ar
AS = arm-eabi-as
CC = arm-eabi-gcc
CXX = arm-eabi-c++
LD = arm-eabi-ld
NDK_KIT = /opt/android/android-ndk-1.5_r1
PLATF_KIT = build/platforms/android-1.5
ARM_INC = $(NDK_KIT)/$(PLATF_KIT)/arch-arm/usr/include
ARM_LIB = $(NDK_KIT)/$(PLATF_KIT)/arch-arm/usr/lib
PLATF_INC = $(NDK_KIT)/$(PLATF_KIT)/common/include
OBJS = hello.o start.o
EXES = hello
hello: hello.o start.o
$(LD) \
--entry=_start \
--dynamic-linker /system/bin/linker -nostdlib \
-rpath /system/lib -rpath $(ARM_LIB) \
-L $(ARM_LIB) -lc -o hello hello.o start.o
hello.o: hello.c
$(CC) -I $(ARM_INC) -I $(PLATF_INC) -c hello.c
start.o: start.c
$(CC) -I $(ARM_INC) -I $(PLATF_INC) -c start.c
clean:
rm -f $(OBJS) $(EXES)
and then
Code:
$ make hello 2>&1 | tee make_hello-log.txt
if all goes well, push to the phone
Code:
$ adb push hello /sqlite_stmt_journals/hello
and then use a shell on the phone to watch the lights gerblinken:
Code:
PC-path$ adb shell
$ cd /sqlite_stmt_journals
$ ./hello
Hello, world!
$ exit
PC-path$
19) Now, download the Android stock kernel sources:
- Make sure you have "git" and "repo" installed; see the (source.android.com/download) Android "Get source" page, starting from "Installing Repo".
- Make a clean directory to initialize "repo" in, and download the code:
Code:
$ mkdir ~/mydroid
$ cd ~/mydroid
$ repo init -u git://android.git.kernel.org/platform/manifest.git -b cupcake
$ repo sync
The sync step above will download the entire source tree corresponding to the Cupcake release (90 minutes on my link).
20) Build the sources
Code:
$ cd ~/mydroid
$ make 2>&1 | tee make_Android-log.txt
On my old PC, this step took about two hours.
Well, that's a little start; everything that comes after this is heavy lifting, I suppose.
bftb0
Thank you for that.
I forgot to add a disk space usage report to this; here it is:
O/S ( /, /boot, /var, /tmp) ~ 4 Gb **
Android SDK, all versions: 1.3 Gb
Android "Cupcake" full sources + full build: 5.5 Gb (2.5 Gb sources, 3.0 Gb build tree)
Android NDK ~0.05 Gb
Eclipse 3.5.1: 0.21 Gb
HTC Desire Kernel Sources + build ~ 0.77 Gb
All up, thats about 12 Gb minimum needed for a full environment. Obviously, you would want some headroom above that.
Note that if you only want to build native binaries, and not the full Android source tree (or develop Java applications), this could be done handily in a virtual machine which is less than 10 Gb in size - for Windows users, there are VMware "Player" versions of Ubuntu 8.0.4 available for that. That eliminates the need for a separate machine, or a dual-boot PC - and the worries of messing up trying to create a dual-boot machine.
All that is really needed for that is the Desktop OS and the NDK. Adding a single version of the SDK so you have 'adb' and 'ddms' available to you would be helpful, but not absolutely necessary if you are using a Windows host machine with those tools already present.
bftb0
** beware that Ubuntu uses a lot of space on /var if you put it in a separate partition - 600+ Mb in my configuration.
bftb0 said:
I forgot to add a disk space usage report to this; here it is:
O/S ( /, /boot, /var, /tmp) ~ 4 Gb **
Android SDK, all versions: 1.3 Gb
Android "Cupcake" full sources + full build: 5.5 Gb (2.5 Gb sources, 3.0 Gb build tree)
Android NDK ~0.05 Gb
Eclipse 3.5.1: 0.21 Gb
HTC Desire Kernel Sources + build ~ 0.77 Gb
All up, thats about 12 Gb minimum needed for a full environment. Obviously, you would want some headroom above that.
Note that if you only want to build native binaries, and not the full Android source tree (or develop Java applications), this could be done handily in a virtual machine which is less than 10 Gb in size - for Windows users, there are VMware "Player" versions of Ubuntu 8.0.4 available for that. That eliminates the need for a separate machine, or a dual-boot PC - and the worries of messing up trying to create a dual-boot machine.
All that is really needed for that is the Desktop OS and the NDK. Adding a single version of the SDK so you have 'adb' and 'ddms' available to you would be helpful, but not absolutely necessary if you are using a Windows host machine with those tools already present.
bftb0
** beware that Ubuntu uses a lot of space on /var if you put it in a separate partition - 600+ Mb in my configuration.
Click to expand...
Click to collapse
great very descriptive thread
Hey bftb0
thanks for posting this guide. I am trying to just play around with someone of the exploit binaries on my eris (from the tattoo/hero rooting threads) , so these guidelines are coming in handy. However for some reason whenever I try and do anything over adb i get this error:
Code:
error: insufficient permissions for device
I cant use adb shell at all, even trying to push to the sdcard gives permissions denied. In the adb devices output the phone is coming up as "?????? no permissions", Did I miss a step ?
My setup so far: Ubuntu 9.10 i386,
Hey bftb0
thanks for posting this guide. I am trying to just play around with someone of the exploit binaries on my eris (from the tattoo/hero rooting threads) , so these guidelines are coming in handy. However for some reason whenever I try and do anything over adb i get this error:
Code:
error: insufficient permissions for device
I cant use adb shell at all, even trying to push to the sdcard gives permissions denied. In the adb devices output the phone is coming up as "?????? no permissions", Did I miss a step ?
My setup so far: Ubuntu 9.10 i386, android-sdk_r04-linux_86, droid eris with most recent OTA
someguy, you have to go to developer.android.com/guide/developing/device.html and install the linux driver (as root) and restart your system.
@someguy
Type "adb kill-server" followed by "sudo adb". You need to run adb as root in unix for it to work.
zifnab06, that was it, I must have not run the first command as root which caused the server to be run without su.
What exactly does this do?
bigcj55 said:
What exactly does this do?
Click to expand...
Click to collapse
Because of the lineage between Linux and Android, the primary platform for compilation of the kernel and other ARM instruction-set binaries ("native" code) has been on Linux. It appears (at the moment...reading between the lines) that Google engineers are using Ubuntu 8.04 LTS as their choice of Linux platform for maintaining Android kernel & utilities code... so, in principle, it is the "best supported" platform for doing "native" development work, if for no other reason than it's what Google has been using. (Not because it is somehow a "better" O/S, but rather because you are likely to run into fewer quirks, and have more resources on the internet who might have experienced the same difficulties that you run into.)
For instance, the folks that have been building kernel exploit programs have been using the Google "NDK". They are not writing/compiling those in Java - they are writing or compiling programs written in "C", and compiling them into native (ARM) object code.
I provided notes from my own installation for anyone considering "dipping their toe" into building Android kernels or native binaries using Linux as a host O/S - most forum readers are not going to be interested in doing that.
Note that Google now releases an NDK for Windows, Linux, and Mac OS/X. The strange thing about doing development on Windows using Cygwin is that if you have sufficient skills to do so - and are capable of resolving problems that inevitably crop up - then you probably already have sufficient skills to be using Unix/Linux in the first place. Sort of depends on your past experience, though. I'd be willing to bet that Devs with a history of doing WinMo development probably would prefer to climb the Android development learning curve by trying their hand with the Windows/Cygwin NDK because of their familiarity with other Windows-based development tools (IDEs, hex editors, assemblers, GUI generators, etc).
That reminds me though - I need to update this thread with something else.
Cheers
bftb0
Static OR Dynamic Linking of Native Binaries
I was a little unsatisfied with the "hack" used in the above examples to circumvent the need to launch programs without linking to C runtime initialization using that strange code shown above, i.e.
Code:
start() { exit( main() ); }
Moreover, for purposes of debugging applications running on the phone (using "gdbserver"), it simply seems easier to link statically - it allows you to step into the syscall library routines and actually see the assembler implementations wrapped around "STI" interrupts.
So, I spent some time experimenting, and came up with a Makefile that allows me to easily toggle between building an ARM executable linked statically or dynamically. Moreover, it lets "argc" and "argv" work they way they are supposed to, and also allows main() to do a return().
Here's an example Makefile, for a program "hello.c". Make sure you read the notes which follow it.
Code:
#
# FIX THESE (to match your installation)
# Top of the Android Source/build tree (retrieved w/ repo)
ANDSRCROOT := /home/user/android/repo-root
# link dynamically against the libraries shipped on the phone!
# e.g.: $cd /opt/android/Eris_libs/ruu1_16_605_1 ; adb pull /system/lib/
PHONELIBS := /opt/android/Eris_libs/ruu1_16_605_1/lib
# Choose one or the other; static is easiest for debugging use
LINKMODE := dynamic
#LINKMODE := static
# tools
CROSS_COMPILE := arm-eabi-
CC := $(CROSS_COMPILE)gcc
LD := $(CROSS_COMPILE)ld
AR := $(CROSS_COMPILE)ar
CXX := $(CROSS_COMPILE)c++
OBJDUMP := $(CROSS_COMPILE)objdump
RANLIB := $(CROSS_COMPILE)ranlib
READELF := $(CROSS_COMPILE)readelf
# Note STATLIBROOT is relative to the Android Source root (ANDSRCROOT)
STATLIBROOT := out/target/product/generic/obj/STATIC_LIBRARIES
INCLUDE := -I $(ANDSRCROOT)/bionic/libc/include \
-I $(ANDSRCROOT)/bionic/libc/arch-arm/include \
-I $(ANDSRCROOT)/bionic/libc/kernel/common \
-I $(ANDSRCROOT)/bionic/libc/kernel/arch-arm
ifeq ($(LINKMODE),dynamic)
LIBDIRS := -L $(PHONELIBS)
else
LIBDIRS := -L $(ANDSRCROOT)/prebuilt/linux-x86/toolchain/arm-eabi-4.2.1/lib/gcc/arm-eabi/4.2.1 \
-L $(ANDSRCROOT)/$(STATLIBROOT)/libc_intermediates \
-L $(ANDSRCROOT)/$(STATLIBROOT)/libc_common_intermediates
endif
ifeq ($(LINKMODE),dynamic)
CRTBEG := $(ANDSRCROOT)/out/target/product/generic/obj/lib/crtbegin_dynamic.o
LDFLAGS := -rpath /system/lib -rpath-link /system/lib --dynamic-linker /system/bin/linker
EXTLIBS := -lc -ldl
else
CRTBEG := $(ANDSRCROOT)/out/target/product/generic/obj/lib/crtbegin_static.o
LDFLAGS := -static
EXTLIBS := -lc -lgcc -lc_common
endif
CRTEND := $(ANDSRCROOT)/out/target/product/generic/obj/lib/crtend_android.o
CFLAGS := -g -Wall
%.o: %.c
$(CC) -c $(INCLUDE) \
$(CFLAGS) $< -o [email protected]
%.s: %.c
$(CC) -S -c $(INCLUDE) \
$(CFLAGS) $< -o [email protected]
all: hello
hello: hello.o
$(LD) $(LDFLAGS) \
-o [email protected] --entry=_start -nostdlib \
$(CRTBEG) \
$^ \
$(CRTEND) \
$(LIBDIRS) $(EXTLIBS)
This makefile has certain assumptions built into it:
(1) Static Linkage
If you are going to link statically, you will need to have already built the entire Android source tree - the Android NDK does not provide any static libraries (In the OP I provided instructions on how to do this download and build the Android "Cupcake" release.)
(2) Dynamic Linking
If you want to compile code which is "as close to the phone as possible" then there is really only one way to do that: compile against the headers that HTC used, and perform linking against the dynamic libraries that you literally pull off the phone ("adb pull /system/lib").
There's only one problem with this strategy: HTC did not release their copy of the bionic (or other library) source code, so you are forced to punt... the second-best choice is to use headers from the Android (Cupcake) source tree. So... you will notice that for both the dynamic-link or static-link cases in the above makefile, the $(INCLUDE) paths used during compilation come from the Android source tree.
bftb0
PS I guess there is a mechanism to configure the top-level "make" of the Android source tree for not only a target type (defaults to "arm"), but also a "machine" dependency. I didn't research that, as I have only built user-space code. I'm doubt there should be any differences at the syscall interface API - excepting perhaps things like ptrace() which have a machine dependency (register structures, etc).
bftb0 said:
Because of the lineage between Linux and Android, the primary platform for compilation of the kernel and other ARM instruction-set binaries ("native" code) has been on Linux. It appears (at the moment...reading between the lines) that Google engineers are using Ubuntu 8.04 LTS as their choice of Linux platform for maintaining Android kernel & utilities code... so, in principle, it is the "best supported" platform for doing "native" development work, if for no other reason than it's what Google has been using. (Not because it is somehow a "better" O/S, but rather because you are likely to run into fewer quirks, and have more resources on the internet who might have experienced the same difficulties that you run into.)
For instance, the folks that have been building kernel exploit programs have been using the Google "NDK". They are not writing/compiling those in Java - they are writing or compiling programs written in "C", and compiling them into native (ARM) object code.
I provided notes from my own installation for anyone considering "dipping their toe" into building Android kernels or native binaries using Linux as a host O/S - most forum readers are not going to be interested in doing that.
Note that Google now releases an NDK for Windows, Linux, and Mac OS/X. The strange thing about doing development on Windows using Cygwin is that if you have sufficient skills to do so - and are capable of resolving problems that inevitably crop up - then you probably already have sufficient skills to be using Unix/Linux in the first place. Sort of depends on your past experience, though. I'd be willing to bet that Devs with a history of doing WinMo development probably would prefer to climb the Android development learning curve by trying their hand with the Windows/Cygwin NDK because of their familiarity with other Windows-based development tools (IDEs, hex editors, assemblers, GUI generators, etc).
That reminds me though - I need to update this thread with something else.
Cheers
bftb0
Click to expand...
Click to collapse
Wow, i was a little off in this case. I was under the assumption y'all were somehow going to compile ubuntu to run on the eris. :O
Hi all, I'm just starting getting into App development and I want to attach the Android source code into Eclipse so that I can reference it locally. Is there anyone here who has experience doing this on Windows who is willing to walk me through it?
Alternatively, I'll go it on my own if someone could help me get past this one roadblock. Thus far I've installed Python and msysGit (and updated my PATH variable accordingly) so that I can use git and repo. Something isn't working however and repo runs into an error that I can't seem to figure out. I'm contemplating uninstalling all that and starting from scratch with Cygwin.
Here's a tutorial on setting up Eclipse on windows (albeit over a year old so no guarantees) link
personally I would dual boot linux... it was a breeze setting it up and there's plenty of tutorials for that.
You could install ubuntu (i think) side by side with win7 using wubi, as if it were a program or something. That way you're not messing with partitions.
I already have Eclipse set up. This is not the problem. I'm having trouble downloading the Android source code - NOT the SDK - on my Windows PC. Specifically, I always run into errors when I try to use "repo init -u blah blah" as instructed on the android source page. If someone has successfully done this on Windows, I'd like someone to give me some guidance, because either I'm doing something wrong or my setup is not correct.
Right now I'm trying to use Cygwin (basically creates a linux environment inside Windows). The last time I tried using repo, I got this:
EDIT: Code removed. I found a solution, outlined below, for anyone else who wants to get the Android source into Eclipse.
Okay, I got the source code, it was actually pretty easy to do, and I didn't even need repo to do it . For those who may find this useful, here's how.
0. It is assumed you have already installed both Eclipse and the Android SDK. If you don't have those yet, download them and set those up. If you aren't on Windows but still want to know how to get the source and import it into Eclipse, you can skip step 1.
1. Install Cygwin on your computer. During the installation, you will be presented with a package explorer and you can choose specific packages to install. Choose whatever you like, but make sure to include automake, bison, curl, flex, gcc, git, gnupg, python, zip, and an editor of your choice, like nano or vim (you can use the search field at the top to find these quickly).
This will basically give you a Linux shell on Windows whose root directory is something you've designated, like C:\cygwin.
2. A few more things to set up. Fire up Cygwin and make a bin folder in your home directory. Add this directory to your PATH variable, then cd into it.
Code:
mkdir ~/bin
PATH=~/bin:$PATH
cd bin
This blog post has code for a python script that will be used later. Make a new file using an editor of your choice. Copy the code and save the file as anything you want (I named it "pack.py"). When you're done, cd back to your home directory.
3. Downloading source. From your home directory (or a subdirectory if you wish), use git to clone the platform frameworks base into a new directory (here I've called it "android").
Code:
git clone http://android.git.kernel.org/platform/frameworks/base.git android
cd into that directory and examine the git tags.
Code:
cd android
git tag
You'll get a list of tags. Choose the one that matches the API level you want to build your project(s) in. I'm using API level 10, so I chose android-2.3.3_r1. That's what you're going to check out.
Code:
git checkout android-2.3.3_r1
4. Reorganizing the source code. You need to locate all the java source files and restructure everything into a directory structure that matches the package naming. Fortunately, that script from earlier does exactly this and then zips it up into a nice little package for you called "sources.zip".
Code:
python ~/bin/pack.py
5. Move this zip file to the proper SDK platform folder. Since I downloaded sources for API level 10, I want to move this zip file to
Code:
C:\android-sdk-windows\platforms\android-[B]10[/B]
You can do this with your regular old file explorer on Windows. Extract the contents into a subfolder named "sources".
6. Get Eclipse to recognize them. Open up a project (or start a new one) whose target is the API level you just did all that stuff for. In the package explorer, right-click the project root and click "Refresh". Now when you browse the class files under project-root/android 2.3.3/android.jar/whatever, you'll see actual source code instead of that nasty "Source not found" page.
Enjoy ^_^
Hi,
This is a guide on compiling iPerf3 for the Zenfone 2 (x86-64 architecture). iPerf3 is a tool for active measurements of the maximum achievable bandwidth on IP networks. It supports tuning of various parameters related to timing, buffers and protocols (TCP, UDP, SCTP with IPv4 and IPv6). For each test it reports the bandwidth, loss, and other parameters.
Requirements :-
1) A recent Linux distro (I use Ubuntu 14.04.2 LTS) running x86-64.
2) Development tools i.e. C compiler, linker, etc... all the necessary tools to run configure, make, etc..
3) iPerf3 source code - latest as of this post is 3.0.11.
4) Small patch to source code to make it work on Android. iperf-3.0.11-android-patch.
Steps :-
(1) Extract the source code to a directory of its own :-
Code:
tar xzf iperf-3.0.11-source.tar.gz
cd iperf-3.0.11
(2) Patch the source :-
Code:
patch -p1 < ../iperf-3.0.11-android-patch
(3) Run configure :-
Code:
./configure
(4) Run make :-
Code:
make
(5) If all goes well, you will end up with a dynamically linked binary which is not what we want on the Android device. We will have to delete this and build a statically linked binary.
Code:
rm src/iperf3
(6) Edit src/Makefile and look for the line :-
Code:
iperf3_CFLAGS = -g
and replace it with :-
Code:
iperf3_CFLAGS = -g -all-static
(7) Now, run make again for the final time :-
Code:
make
(8) Strip the binary & verify it is a static binary :-
Code:
strip src/iperf3
file src/iperf3
That's it! just copy the iperf3 binary file to your Android device and move it to your /system/xbin or whatever directory in your PATH.
Hope it works for you guys. I'll be happy to help out anyone who needs more details.
Cheers.
UPDATE 2015-08-12 :
Here's a link to a precompiled & stripped binary. https://goo.gl/Q1l1yh
Can you please share binary file. I don't have development enviroment.
harpreet.s said:
Can you please share binary file. I don't have development enviroment.
Click to expand...
Click to collapse
Hi,
I've added a link to a compiled binary. Use at your own risk
Cheers.
Hi,
could you share instruction how to build iperf for arm?
because now it compiled for x86
thanks,
By any chance do you know how to compile a static binary for windows? The instructions you provided worked great I compiled a static binary for MAC and linux and android. now I could use one for windows
Compile 3.1.7, 3.2 and 3.3 for Android
Hi there,
I am new to Android Development. I have a need to compile iPerf 3.1.7, 3.2 and 3.3 for android. I see that a patch file is required in the reading here. Could you elaborate perhaps on what the patch does? I assume i will need to create something similar for 3.1.7, 3.2 and 3.3. Any help would be much appreciated thanks.
Some OEMs violating GPL licenses and won't publish linux kernel sources for their Android devices.
On this post I’ll explain how to port kernel source to any android device.
Maybe you can port kernel sources to other SoCs with similiar ways but I've explained qcom way.
What do we need ?
A Computer which has Linux or Mac installation.
Device to test.
Let’s Start, Shall we ?
First of all we need to take dmesg of stock kernel using adb shell dmesg > dmesg.txt command. (I recommend you to take it while device is booting)
You got dmesg ? Nice. Open it with any text editor and search for “hardware” or “machine” and note the result to somewhere. By doing this we’ll get the name of device-tree-blobs.
Download android image kitchen and extract it to somewhere.
Now hold and move your device’s stock boot.img to unpackimg.(bat/sh)
Now your stock kernel got extracted to split_img/boot.img-zImage. After this line I will referance this file as “stock kernel”
Fetch split-appended-dtb using git clone https://github.com/MacTavishAO/split-appended-dtb-mac command. We’ll use this tool to extract dtb from stock kernel.
Copy stock kernel to split-appended-dtb-mac folder.
Now we will extract dtb(device-tree-blobs) from stock kernel. If you are on Linux use ./split-appended-dtb boot.img-zImage command for Mac use ./split-appended-dtb-mac boot.img-zImage command.
Now we have bunch of files named dtbdump_xx.dtb. We have to find out which one our device uses. Use grep -r <machine name that we found at step 2> . to find out which dtb our device using. (Write different parts of machine name to find out)
Install dtc using apt install device-tree-compiler command.
Let’s say it turns out dtbdump_21.dtb is the right one. We have to decompile dtb file to dts using dtc -I dtb -O dts -o extracted.dts dtbdump_21.dtb command.
Now we have to extract defconfig from stock kernel.
Use wget https://github.com/torvalds/linux/raw/master/scripts/extract-ikconfig && chmod a+x extract-ikconfig command to download necessary tool.
Use ./extract-ikconfig boot.img-zImage > extracted_defconfig to get defconfig from stock kernel.
Now we got what we need, we need to download kernel source to start porting to our device.
After this point I’ll start to explain as porting to Qualcomm device but almost same on other system-on-chips.
Go to Settings -> About Phone -> and note “Kernel version”
We need to download soc manufacturer’s sources. From here find msm-X.X that matching with your phone and copy link of it.
And from here find tag matching your chipset and write it down somewhere.
Use git clone <sources from 2 steps up> -b <tag>
Now enter to msm-X.X folder.
Copy extracted_defconfig to arch/<your device's architecture>/configs/ folder.
If your device is arm64 copy extracted.dts to arch/arm64/boot/dts/qcom/ if this directory doesn’t exist copy to arch/arm/boot/dts/qcom/ folder. (If you have 3.x qcom device skip the arm64 part and copy it directly into arm)
Open “Makefile” file in arch/arm(64)/boot/dts/qcom/ and add extracted.dts to the line matching with your chipset.
It’s time to import drivers. After this step I cannot help you because every devices’ hardware isn’t same but I can give some tactics.
To get driver names you can read dts file or install any device info app from Google Play Store etc.
Search these names and GitHub you can find files and commits which will help you to add drivers.
You did all of these without my help ? Excellent! Now it’s time to build kernel. I suggest you to use @natchanchance 's kernel compilation guide.
After compiling kernel. Copy compiled kernel to split_img folder which mentioned at early of this tutorial. rename it to boot.img-zImage and double click repackimg.(bat.sh) and you have new file named image-new.img.
Boot it using fastboot boot image-new.img command and if everything is working you can use fastboot flash boot image-new.img to use it permanently.
Leave a comment here about your questions. I'll try to reply all of them.
Planned to make a guide video about it but I don't have much time. Keep checking this thread may I post soon.
Reserved
Reserved.
Reserved
Thanks Dude!
This is very useful guide especially fir those who want to port the kernel source.
Thanks man. I am following your guide.
Thanks ?
While looking for matching dtbdump file I got a match in all four files for PMI8996, which file should I use?
"We have to decompile dtb file to dts using dtc" stuck at this step.
Any help on which "App" to use to find driver names..
Thanks
CPUZ / AIDA64
ataberkozen said:
It’s time to import drivers. After this step I cannot help you because every devices’ hardware isn’t same but I can give some tactics.
To get driver names you can read dts file or install any device info app from Google Play Store etc.
Search these names and GitHub you can find files and commits which will help you to add drivers.
Click to expand...
Click to collapse
I've found another way to find hardware info using hwinfo tool inside termux. The trick is to use it with root permissions.
Install hwinfo in termux:
Code:
apt-get install hwinfo
Run it with root permission tee into a text file:
Code:
su -c $(which hwinfo) | tee hwinfo.txt
[NOOB-FRIENDLY] HOW TO BUILD ANDROID FROM SOURCE CODE [2020 EDITION]
Introduction
Hi, my name is Varun, during the times I was building I faced some trouble building, and I thought it would be helpful if I put together a guide that covers every aspect of building. Many roms fundamentally differ so this is a generic guide, but when it comes to modifying device specific files you are going to have to refer to your rom developer or your device tree maintainer.
I see a lot of lead devs referring to older guides on xda, and felt like there should be a good reference thats more recent.
I have heavily depended upon Nathan Chance's Guide which can be found HERE. I feel like this may be a little intimidating for new builders hence this thread. I will slowly upload screenshots here as I find time. But for now this is what I have. You can find my guide HERE. If there are any changes to be made in this thread then it also needs to be changed on gh, so please do comment or make a pr and fix what is requested.
Table of Contents
1) Getting started
2) Prerequisites
3) Understanding the DOC
4) Preparing Linux environment
5) Getting the source
6) Getting device specific files
7) Building
GETTING STARTED
• This guide will help you understand the basics of rom building.
• You must have exprience with linux and terminal
• You must read everything at least once
• If you get stuck at any step, google is your best friend.
If you still struggle, these are places you can get help
1) This thread
2) GitHub
3) Other XDA threads
4) Telegram (Your SOC groups or Android Builders Help)
• Build only if you have time to spare, never rush through the learning stage
• Never think about skipping a step
• If you are planning to modify a ROM, always ask for permission
• If you are planning on including other peoples work in your rom always ask for permission
Prerequisites
• Your system should have at least 8GB RAM and 4 cores (Use Zram for 8gb ram devices)
• Strong internet connection. You will be downloading 50GB+ worth of files
• Ubuntu - this should be installed alongside a primary OS. NOT VM
• Minimum 250GB of free disk space (you can sync 1 ROMs and build for 3 devices at once)
• Git - Read some documentation
• Command Line - get yourself familiar, learn with CodeAcademy
Understanding This Doc
• Notes with background information will be given in italics
•
Code:
is used to represent a command that is run from terminal
PREPARING LINUX ENVIRONMENT FOR BUILDING
• Now the you have finished setting up, a special environment is required for building.
• Preparing the environment is as easy as double clicking .exe in Windows or .dmg in MacOS
thanks to developer Akhil N.
• We need to clone his scripts (installer) from his GitHub
NOTE:
A script is a file which contains multiple commands that run sequentially when executed
Code:
git clone https://github.com/akhilnarang/scripts
• This should copy the scripts to directory /home/username/scripts
• Cd into the directory called setup
Code:
cd scripts/setup
• List out directory
Code:
ls
• Find the file that corresponds to our Linux build. Since we are using Ubuntu it will be android_build_env.sh
For other Distros refer to the readme that has also been cloned
• Run the script
Code:
. android_build_env.sh
• Setup GitHub using on screen instructions
• Congratulations you set up your computer for building
• Go back the previous dir that you were originally on (/home/username)
Code:
cd
NOTE:
cd will always take you to home/username
GETTING THE SOURCE
• This is probably one of the worst steps, you have to wait a while
• GCloud users will be able to download source in 15 minutes
• Other users may have to wait longer (depends on internet connection speed)
• Make a directory for your ROM
Code:
mkdir “ROMNAME”
NOTE:
this will make folder in directory home/username/romname
• Cd in and initiate repository, this basically tells your computer where to download source files from.
The repository initiation command can be found on the GitHub page under “manifest” then scroll down to
repo initialisation and copy command
• It should look like
Code:
repo init -u git://github.com/crdroidandroid/android.git -b 10.0
HINTS
NOTE:
1) Linux users can paste into terminal using Ctrl + Shift + V
2) Linux users can copy from terminal using Ctrl + Shift + C
3) GCloud users can paste into window using Ctrl + V
4) GCloud users can copy from window simply by highlighting
5) GCloud users using ssh from terminal can just follow terminal commands
• Once repo is initialised, you can begin download using
Code:
repo sync -f --force-sync --no-clone-bundle --no-tags -j$(nproc --all)
• Let everything download
GETTING DEVICE SPECIFIC FILES
• As most of you know the kernel acts as the translator between the OS and your actual hardware
• The files mentioned above are android version specific and will only work with that android
version
• The developer for your device aka dt maintainer will have these files on their GitHub and its
usually linked at the end of the op of other rom threads in the source code section
• You have to make sure that the rom version you downloaded matches up
• If the device tree is for an older version then wait until your dev. releases the latest trees, if no
developer is working on your device then it is possible to modify the tree to work with newer
version of android but it may take some effort.
• In case your tree doesnt match with the rom version that you have downloaded, the easiest and best option is
to scrap everything and download the source version that your tree is designed for
• To scrap everything, use
Code:
rm -rf “folder containing rom"
• To modify the init command to download an older version you need to pick out the branch you
need (the branch is basically the version the newer version is added to a new branch)
• Change the init command after -b to include desired branch. For example
repo init -b ten to repo init -b seven
• To get the device specific files, we need to fork them to your GitHub.
Find the stuff (kernel vendor device tree hardware) and fork them
NOTE:
Forking basically copies them to your GitHub
• At this point we need to know if we need to modify the device tree as some ROMs don’t require
you to modify it (CrDroid)(LOS)
• Some roms require only partial modifications, this is rom specific so I wont be going too much into detail
• If modification is required follow these steps , otherwise skip to the end of this section
• After forking we need to modify the files we will do this from the browser for ease of
understanding
•There are three files that need modification and they are located your device tree
• Some tree developers may split your device tree into two parts if there is more than one phone
running the same SOC, so keep an eye out for that. (devices that have the same soc are like the op3/op3t and the op7/op7p)
• The files that we need to modify are under your "device codename" device tree, not "device SOC-common" device tree.
• For example; we need to go to "device_samsung_crownlte" tree not "device_samsung_universal9810-common"
• In here the three files that need modification are
1) Android Products.mk
2) lineage.dependencies
3) lineage_"device codename".mk (example:lineage_crownlte.mk)
• First open AndriodProducts.mk Here replace wherever you see lineage with name of ROM in lowercase only.
• Second, open lineage dependencies.mk and rename file to "romname".dependencies (ecample: aim.dependencies)
• Third, open lineage_"codename".mk and rename to "romname"_"codename".mk and replace
lineage with "romname in the code. (Remember,use lowercase)
NOTE:
If you are building an older version of android make sure you branch out the right branch for your trees
People who do not need to modify can join us here
• Now that all your files are in your GitHub, we need to download them
• There are two easy ways
1) Fast and easy Room Service
2) Slow and boring manual cloning
• Room Service
1) cd into .repo from romsource
Code:
cd .repo
2) Make directory local_manifests
Code:
mkdir local_manifests
3) Make room service file
Code:
nano roomservice.xml
(follow next bullet point on how to make roomservice.xml)
4) Return to ROM folder
Code:
cd && cd "romfolder"
• Making the roomservice.xml is as easy as copy paste.
Visit
https://github.com/vrnsoma/local_manifests/blob/lineage-17.1/roomservice.xml
and copy and paste it into a text editor and edit it then paste it into the terminal window (step 3 in above point)
Hint for Device Tree
Example: link= userabc/repositories/device_samsung/crownlte (location on github)
path= device/samsung/crownlte (Where it should download to)
remote= github (where files are stored)
revision= branchname (which version you want)
• Resync Repo like I mentioned before
Code:
repo sync -f --force-sync --no-clone-bundle --no-tags -j$(nproc --all)
BUILDING
• Run
Code:
. build/envsetup.sh
• Then
Code:
brunch codename
• Make sure to clean build by running this in between builds
Code:
make clobber && make clean
• If you are using a slow system and want to improve build speeds, use the following command
for making partially clean builds for incremental updates
Code:
make installclean
• Building without cleaning is called dirty building and is never advised and do not do this unless
you know what you are doing
KEEPING SETUP UP TO DATE
• Repo sync will keep your rom and device specific files that have been deined in roomservice up to date
CREDITS
NathanChance
All threads on XDA
All guides on Github
All devs found on Telegram
If you want specific credits plz do notify me!
XDA:DevDB Information
BUILDING ANDROID, ROM for all devices (see above for details)
Contributors
vsoma
ROM OS Version: Android 10
ROM Firmware Required: NOOB
Version Information
Status: Stable
Created 2020-10-03
Last Updated 2020-10-03
I need rom porting guide for android 10....can you make thread on it?
I tried the tutorial, after the step repo sync -f --force-sync --no-clone-bundle --no-tags -j$(nproc --all). I get message error: RPC failed; curl 56 GnuTLS recv error (-24): Decryption has failed.
i can't finish compiling because java.lang.outofmemoryerror : heap size but i have already set the heap size to 8gb