Database Users

[MOD] No forced encryption on CM12 stock kernel - once and forever

Force-Encrypt Toggler
Disclaimer: I have no experience in Android/ROM development. This is the first time i decided to share something (that i initially made for myself). Flash at your own risk. I am not responsible for boot loops, unexpectedly encrypted partitions, data loses, etc. Make sure you have a backup first!
Click to expand...
Click to collapse
Now that CyanogenMod developers stated that they are going to follow Google's guidelines about forced encryption in their ROM for Nexus 9, those of us who want their tablets unencrypted will have to flash a modified boot image every time they update CM. I hate doing such things manually every time, so i created a patch that disables forced encryption the right way - flash once and forget forever.
force-encrypt-toggler reads and unpacks your boot image, patches the ramdisk, creates a new boot image and flashes it back to the boot partition. And all that is done right on your tablet. Than it uses CM's addon.d framework, so that each time you flash a new CM zip, force-encrypt-toggler is invoked automatically to patch the newly flashed boot image. Thus, you can just upgrade through CM's built-in updater and everything will be patched automatically every time . Also, the script itself can be used on any Linux computer to patch (probably) any boot.img you have.
Thus far i have used it on my Nexus 9 to stay unencrypted starting from 20150216 through 20150224 nightlies. It seems to be quite stable, i believe . See some notes in the second post.
Download
force-encrypt-toggler-1.1-flounder.zip : https://goo.gl/bw7YDq
force-encrypt-toggler-1.1-flounder-dbg.zip : https://goo.gl/95JN34 - this one creates log files in /cache every time
force-encrypt-toggler-1.1-linux.tar.gz : https://goo.gl/3PF6ru - to be used on a desktop to patch arbitrary boot.img
old versions (for CM12): https://goo.gl/125eey
Sorry, it looks like i can not post clickable links yet. Remove space between "https" and colon.
Compatibility
I have tested this script only on my Nexus 9 Wi-Fi with TWRP recovery and CM stock boot image. However, i believe that it should be possible to make it work on other devices (e.g. Nexus 6) by just changing a few constants at the beginning.
Version 1.1 requires TWRP version >= 2.8.7.1 and a CM13 nightly >= 20160110. Version 1.0 will work for CM12.x (but not for recent CM13 nightlies).
Known issues
The addon.d script uses a dirty hack to trick the recovery. While this never happened to me, if you ever encounter a strange recovery behavior regarding installation or backup of boot images after flashing CM zip over CM with this mod, this might be it. Just reboot and it should be ok. See the second post for more info.
If CM changes something in their updater or if something changes in the recovery, this mod may easily break due to the hack mentioned above. Read the disclaimer.
Each time you flash a CM zip over a CM installation with this mod installed, TWRP recovery will hang for 5-10 seconds after reporting successful completion and before showing buttons at the bottom (or before rebooting in case of open recovery script execution). It is possible to fix this easily, but than this mod might be much easier to break.
Installation
First, you should read the disclaimer above and backup your data. Than you just have to flash the zip you downloaded with TWRP recovery. CyanogenMod must be installed first. If your /data is currently encrypted, you will have to do a full factory reset to decrypt it (backup your data first!).
Your current boot image will be patched during installation process (it should not hurt, if it's already patched). In case if something goes wrong, installation script will tell you. In any case you can get force-encrypt-toggler debug output by something like:
Code:
adb pull /tmp/fet.log
Be sure to do this before you reboot, because that file is created in the RAM.
Removal
In order to remove force-encrypt-toggler you have to delete the following files from your /system partition:
Code:
/system/xbin/mkbootimg (v1.1)
/system/xbin/unpackbootimg (v1.1)
/system/bin/force-encrypt-toggler
/system/addon.d/90-force-encrypt-toggler.sh
/system/bin/mkbootimg (v1.0)
/system/bin/unpackbootimg (v1.0)
And than restore your original boot image. You can also just format /system and flash CM again, but that is such an overkill .
Usage
Normally you will not need to run force-encrypt-toggler yourself, but in case you need, you should be able to run it both in Android and in recovery via adb shell. Just run it with --help option to see what it can do. In case you will have to debug some glitches, this command may be useful:
Code:
force-encrypt-toggler --set-not-forced --debug --dry-run --no-cleanup
If you use it on a Linux computer, this is what you will probably need:
Code:
sudo ./force-encrypt-toggler --set-not-forced [ --input path/to/boot.img --output path/to/new/boot.img ]
Note that Android and computer versions are functionally equivalent, so you can theoretically patch boot images for one Android device on another one...
Changelog
Code:
v1.1
+ use toybox instead of busybox because CM now ships only the later
+ mkbootimg and unpackbootimg are now installed to /system/xbin
+ the --help option can now be used without root privileges
v1.0 - initial release
Credits
mkbootimg is built from AOSP source
unpackbootimg is taken from this GitHub page: https://github.com/Dees-Troy/unpackbootimg
update-binary is taken from a CM zip

There is one problem with patching the new boot image from an addon.d script: CM's updater-script flashes boot image after it invokes all addon.d scripts. Therefore at a time, when the script is called, it is possible to patch only the old boot image, and than it will still be overwritten anyway. In order to overcome this, i used a very dirty hack. In short, i replace the device node with a fifo and let the updater write new boot image into it, and than... Ok, so, i think there must be a cleaner solution, so i will appreciate if a more experienced developer takes a look at my code and proposes a better solution.
As for the 5-10 second hang, it is (unexpectedly) caused by that line with "sleep 15" at the end of addon.d script. If it really annoys you, you can comment it out along with the line, where force-encrypt-toggler is called directly (not through the helper script). Updating will be a bit faster than, but if CM devs ever decide to flash boot image prior to calling addon.d scripts, you /data will be silently encrypted.

Download :: For the lazy
force-encrypt-toggler-1.0-flounder.zip : http://goo.gl/N4rZDk
force-encrypt-toggler-1.0-flounder-dbg.zip : http://goo.gl/4nXmkD - this one creates log files in /cache every time
force-encrypt-toggler-1.0-linux.tar.gz : http://goo.gl/hDFNOY - to be used on a desktop to patch arbitrary boot.img

While there was zero discussion in this thread, goo.gl tells me that there was some downloads, so someone might be actually using my small mod. If so, they might notice that it got broken around one month ago. So i decided to share a fixed version. See updated links in the description (links in the USBhost's post above are for the old version, if you wonder).
If you currently have v1.0 installed, you can just flash v1.1 over it. Note however, that if you was flashing recent nightlies while using v1.0, you /data probably have been silently re-encrypted. Also, if you current boot image is patched by v1.0, the initial patching by v1.1 during zip installation will fail, but it should work during system updates afterward. Flash stock boot image and try again if you want to be sure.
The reason for v1.0 malfunction was that around a month ago CM13 stopped shipping busybox in favor of toybox. New version is only compatible with CM nightlies >= 20160110. Also you need TWRP >= 2.8.7.1.

I never took notice to this thread before until your post in the FED thread. I am going to keep an eye on it as I have been issued lately with the FED patch saying that my device is not supported (Nexus 9 Wi-Fi). Hopefully I will have better luck with this. Thank you for sharing your.

Modify the system partition on Android Nougat?

Hi all,
has anyone been able to do this? Following the guide here, no longer works for Android N. The phone boots, but ignores all changes to system. How do I modify both build.prop and hosts? It seems that there are now possibly two system partitions?
Thanks!

Same issue on Nexus 5X

No answer on this? How is it that nobody else seems to be having this issue?

What I've done
It looks to me like everyone has moved to systemless and the /system partition cannot be adequately modified in this way anymore.
Maybe this will help others:
I was modifying the system directory for two reasons: 1. modify /system/etc/hosts to remove ads and modifying build.prop to increase lcd.density. I found that here are the alternatives for each:
Removing Ads
Using something similar to AdAway_systemless_hosts_v2.zip (google it for a copy) and modifying the hosts file in that zip file to be the one I use (and rezipping, deploying on the Android device). This basically mounts over /system/etc/hosts with a custom hosts file instead of actually modifying the system specific hosts file which is no longer writable.
The alternative is to use Netguard which routes non https network traffic through a private VPN where you can block ads according to a hosts file. This seems to work OK, but I have noticed that websites seem to take longer to load.
Modifying lcd.density
You can use the same trick as AdAway_systemless_hosts_v2.zip uses, but modify it to also mount a modified copy of build.prop. Alternatively just use the Android N Display settings that are small (what I did anyhow).

I have been able to edit build.prop and still maintain systemless root.
Sent from my Nexus 6P using XDA-Developers mobile app

I was able to modify my system partition; by installing busy box to /su/xbin and running "su busybox mount -o rw,remount system" (no quotes) in material terminal with root

ArminasAnarion said:
I was able to modify my system partition; by installing busy box to /su/xbin and running "su busybox mount -o rw,remount system" (no quotes) in material terminal with root
Click to expand...
Click to collapse
Have you been able to do this with simply fastboot boot <twrp-image>, mounting system in rw mode and modifying it? I did that as I didn't want to root the phone, and while it looks like it did the write, it does not affect the system partition that is used by the phone after boot. I think there are two system partitions, and twrp mounts only one in rw mode. It does seem like it may be possible to do what you say using adb though after the phone is fully booted up. I'll try that!

dontblinkwatchout said:
Have you been able to do this with simply fastboot boot <twrp-image>, mounting system in rw mode and modifying it? I did that as I didn't want to root the phone, and while it looks like it did the write, it does not affect the system partition that is used by the phone after boot. I think there are two system partitions, and twrp mounts only one in rw mode. It does seem like it may be possible to do what you say using adb though after the phone is fully booted up. I'll try that!
Click to expand...
Click to collapse
I had the same problem. I don't want to root but I do make a few changes to my /system partition through adb in recovery such as the hosts file and some font files (namely the Emoji font file). I had modified stock boot image to not enforce encryption. I would boot back up into the system and couldn't see any changes made. The only thing I found that worked was installing a custom kernel (I use ElementalX). After that, changes I made to /system in TWRP were reflected in the OS. I don't know enough about kernel development to understand why on (mostly) stock kernel my changes couldn't be seen but on a custom one they were.
I never had this "problem" prior to Nougat.

Same issue here. Something has changed with how this is handled in Nougat.
I don't want to root just to overwrite the hosts file...
I'll keep debugging but my capability in this is definitely limited!

I use a similar approach as described in the OP's linked guide except I use my own recovery image that I compiled as an engineering build from source, and I am also experiencing the same behavior. Modifying the hosts file seems to have no impact on the system though the changes persist. Comparing the host file I installed and the host file from the latest Nexus 5X image with 'ls -lZ' the SELinux info looks to be the same. The only information that appears to differ is the modified date and one additional line in the file itself for testing. I thought I was doing something wrong with my hosts file, even though I have been using this approach since Android 6.0. However, I agree, it appears that changes to system are being ignored. Further, changing the system partition no longer shows the red warning at boot about the system being corrupted.
---------- Post added at 09:58 PM ---------- Previous post was at 09:38 PM ----------
DanRyb;68654939 I would boot back up into the system and couldn't see any changes made.[/QUOTE said:
Oooh. You're right. Neither /etc/hosts or /system/etc/hosts is modified in the booted OS after I modify it from live image, but the change is retained when I reboot into live image and mount system. Hmm, so either:
1) Need to figure out where the the system files are being loaded from and modify them from live image if possible
2) Use a mechanism similar to what dontblinkwatchout described AdAway is using of having a custom mount setup (have to reverse engineer AdAway I guess to see what it's doing)
3) ?
Click to expand...
Click to collapse

There's absolutely no way to modify or mount system partition r+w unless you disable dm-verity
Enviado desde mi Nexus 6P mediante Tapatalk

alexiuss said:
There's absolutely no way to modify or mount system partition r+w unless you disable dm-verity
Enviado desde mi Nexus 6P mediante Tapatalk
Click to expand...
Click to collapse
dm-verity has been around since Android 4.4. Are you saying there is something new around this in Android 7.0?
You can modify the system partition by compiling an engineering build of Android and booting it, then mounting the system partition and modifying it. I've been doing this to update the hosts file since Android 6.0 for every OTA update (since more recently OTA updates bomb out unless you reflash the clean "uncorrupted" system.img first). Changing the system image before Android 7.0 did result in an extra screen with a red warning about a corrupted something or other (I'm sure because dm-verity checking failed). Regardless, you can still change the system partition, the information just no longer seems to be used, which is a bit perplexing to me atm.

crashenx said:
dm-verity has been around since Android 4.4. Are you saying there is something new around this in Android 7.0?
Click to expand...
Click to collapse
Android 7.0 introduced redundant bits for reed solomon forward error correction into the system and vendor partitions and code in the kernel to perform the error correction.
Your changes are being written to emmc but when you boot with 7.0 kernel with dm-verity enabled your changes are being treated as data corruption and on-the-fly error corrected back to original.
You can see your changes if you boot into twrp because it has dm-verity disabled. However if you boot into android with dm-verity enabled it will look like original image again even though your changes are technically still there.
It took me a day to figure out what was really going on because i initially had no idea they added this feature to Android N.
The simple way to disable dm-verity is to install SuperSU, but you can also accomplish the same patching your own kernel, installing pre-patched kernel, installing custom kernel, etc.

sfhub said:
Android 7.0 introduced redundant bits for reed solomon forward error correction into the system and vendor partitions and code in the kernel to perform the error correction.
Your changes are being written to emmc but when you boot with 7.0 kernel with dm-verity enabled your changes are being treated as data corruption and on-the-fly error corrected back to original.
You can see your changes if you boot into twrp because it has dm-verity disabled. However if you boot into android with dm-verity enabled it will look like original image again even though your changes are technically still there.
It took me a day to figure out what was really going on because i initially had no idea they added this feature to Android N.
The simple way to disable dm-verity is to install SuperSU, but you can also accomplish the same patching your own kernel, installing pre-patched kernel, installing custom kernel, etc.
Click to expand...
Click to collapse
That's good info and makes total sense. Thanks! Pretty neat actually, just a bummer for me.
Yeah so SuperSU path is not really one I want to pursue. I could learn how to update the dm-verity shas used for verification. That'd probably be the most secure, but it's gonna be a PITA I bet. I imagine I'd need to compile my own image similar to how I made my live image and update a few things. Might have to deal with encryption which is probably an even bigger headache. Also, I bet it would break OTA and have to reflash to update, though that's true now.
I'm really curious what AdAway is doing. Maybe I should pursue reverse engineering that.
I really appreciate you pointing us in the right direction.

I am glad found this thread..willing to assist here without permanent root..

Ericarthurc said:
I was able to modify my system partition; by installing busy box to /su/xbin and running "su busybox mount -o rw,remount system" (no quotes) in material terminal with root
Click to expand...
Click to collapse
I was trying to create a /system/xbin/post-boot but couldn't remount /system, and so I added busybox to the front of my command. I am not using adb so I cut that part off. Thanks a lot!

Building arm64 kernel for Galaxy S7 Edge (SM-G935F) question.

Hello,
First of all, im new to android modding and linux. I tried to build a kernel from stock G935FXXU1BPH6 source based on their readme file. My problem is, I don't have a zImage file after build that I could put into a flashable zip. I only have Image and Image.gz (I guess thats normal for arm64 kernels?) and I don't know how to make them flashable. I found a youtube video where the dude placed the Image file in a directory called "tools" in his flashable zip, I tried it and TWRP said I installed it successfully but when I start up my phone and check kernel version its still the previous one not mine.
Used this toolchain: aarch64-linux-android-4.9/bin/aarch64-linux-android-
I've set up CROSS_COMPILE path, then I set up configuration:
make ARCH=arm64 exynos8890-hero2lte_defconfig
make menuconfig (to tweak it a little bit more, like custom kernel version string so I can see if it worked)
Then I started build:
make ARCH=arm64
Output when build finished:
>>>>> Time used for generated all hashes is 6 sec
OBJCOPY arch/arm64/boot/Image
GZIP arch/arm64/boot/Image.gz
DTC arch/arm64/boot/dts/exynos8890-smdk8890.dtb
DTC arch/arm64/boot/dts/exynos8890-universal8890.dtb
Anyone knows how could I make it flashable? Thanks.

keezay said:
Hello,
First of all, im new to android modding and linux. I tried to build a kernel from stock G935FXXU1BPH6 source based on their readme file. My problem is, I don't have a zImage file after build that I could put into a flashable zip. I only have Image and Image.gz (I guess thats normal for arm64 kernels?) and I don't know how to make them flashable. I found a youtube video where the dude placed the Image file in a directory called "tools" in his flashable zip, I tried it and TWRP said I installed it successfully but when I start up my phone and check kernel version its still the previous one not mine.
Used this toolchain: aarch64-linux-android-4.9/bin/aarch64-linux-android-
I've set up CROSS_COMPILE path, then I set up configuration:
make ARCH=arm64 exynos8890-hero2lte_defconfig
make menuconfig (to tweak it a little bit more, like custom kernel version string so I can see if it worked)
Then I started build:
make ARCH=arm64
Output when build finished:
>>>>> Time used for generated all hashes is 6 sec
OBJCOPY arch/arm64/boot/Image
GZIP arch/arm64/boot/Image.gz
DTC arch/arm64/boot/dts/exynos8890-smdk8890.dtb
DTC arch/arm64/boot/dts/exynos8890-universal8890.dtb
Anyone knows how could I make it flashable? Thanks.
Click to expand...
Click to collapse
This particular chipset (64-bit Exynos) uses the uncompressed Image and a separate dtb.img file made from combining all the dtb revisions of your device codename and region into a DTBH format.
You can flash them using my LazyFlasher project.
See: https://github.com/jcadduono/lazyflasher
You will want to use the kernel-flasher branch. The kernel-flasher-samsung branch isn't fully ready and adds additional patch files to remove TIMA/Knox. I've yet to find out everything that needs to be changed from stock state to allow a bootable custom kernel without disabling encryption, unfortunately.
You can simply git clone it, then place your Image and optionally dtb.img in the root folder of the repository, then type "make" to build a TWRP flashable zip. They will be dynamically replaced in the current boot image on the device when the zip is flashed. You can check out the README.md for more info.
If you want to generate your own dtb.img to include in the installer, you can use a script I made from my universal8890 kernel sources on GitHub:
https://github.com/jcadduono/android_kernel_samsung_universal8890/blob/stock-6.0/dtbgen.sh
(correct the toolchain location for your build in the script)
It also requires the scripts/dtbTool folder (from the same git linked above) to be present in your repository. It's not the same as the Qualcomm dtbTool, and the sources are included (and fairly clean!) if you're interested in learning the Exynos dtb.img (DTBH) format.
./dtbgen.sh hero2lte xx
Now, there's still quite a bit that needs to be done to make the device actually boot successfully and be stable with a custom kernel. While the kernel is perfectly stable, the Samsung customized Android OS will absolutely freak out. That's a bit beyond me, and the reason I haven't really worked on any custom kernels for it myself.
Have fun!

jcadduono said:
This particular chipset (64-bit Exynos) uses the uncompressed Image and a separate dtb.img file made from combining all the dtb revisions of your device codename and region into a DTBH format.
You can flash them using my LazyFlasher project.
See: https://github.com/jcadduono/lazyflasher
You will want to use the kernel-flasher branch. The kernel-flasher-samsung branch isn't fully ready and adds additional patch files to remove TIMA/Knox. I've yet to find out everything that needs to be changed from stock state to allow a bootable custom kernel without disabling encryption, unfortunately.
You can simply git clone it, then place your Image and optionally dtb.img in the root folder of the repository, then type "make" to build a TWRP flashable zip. They will be dynamically replaced in the current boot image on the device when the zip is flashed. You can check out the README.md for more info.
If you want to generate your own dtb.img to include in the installer, you can use a script I made from my universal8890 kernel sources on GitHub:
https://github.com/jcadduono/android_kernel_samsung_universal8890/blob/stock-6.0/dtbgen.sh
(correct the toolchain location for your build in the script)
It also requires the scripts/dtbTool folder (from the same git linked above) to be present in your repository. It's not the same as the Qualcomm dtbTool, and the sources are included (and fairly clean!) if you're interested in learning the Exynos dtb.img (DTBH) format.
./dtbgen.sh hero2lte xx
Now, there's still quite a bit that needs to be done to make the device actually boot successfully and be stable with a custom kernel. While the kernel is perfectly stable, the Samsung customized Android OS will absolutely freak out. That's a bit beyond me, and the reason I haven't really worked on any custom kernels for it myself.
Have fun!
Click to expand...
Click to collapse
Thank you very much!

@jcadduono couldn't make the kernel boot after packing it with lazyflasher. I built a completely stock kernel from the mentioned source, pasted the "Image" (not the Image.gz) file in lazyflasher root and then used make command. Tried including "exynos8890-smdk8890.dtb" file as well. Same story. Not sure if I need anything else in the package or I made user mistake. Do you have any ideas how could I debug what makes it stuck on that screen?
Thanks!
EDIT: Solved Problem.

keezay said:
@jcadduono couldn't make the kernel boot after packing it with lazyflasher. I built a completely stock kernel from the mentioned source, pasted the "Image" (not the Image.gz) file in lazyflasher root and then used make command. Tried including "exynos8890-smdk8890.dtb" file as well. Same story. Not sure if I need anything else in the package or I made user mistake. Do you have any ideas how could I debug what makes it stuck on that screen?
Thanks!
EDIT: Solved Problem.
Click to expand...
Click to collapse
Can you help me set up an environment to build a kernel, arm64, for s7? I'm on Ubuntu having a heel of a time..

Galaxy S7 Edge Kernel Flashing Issues
I am currently trying to flash a different kernel into a galaxy s7 edge (SM-G935S).
I have gone as far as building a kernel and extracting a Image file from it, but every time I try flashing a boot.img with a replaced kernel image file, it seems to never work.
A mkbootimg tool that I am currently using requires a dtb file, but I cannot find where to get it from.
I have tried using @jcadduono's git code, but the dtb file created from it doesn't seem to work as well.
Can anyone tell me what I should do to flash a kernel successfully?

kernel panic after flashing
Hi @ll,
with this guide I was able to compile my own kernel and also flashing it to my phone. Unfortunatelly I'm getting kernel panic after rebooting the phone. is there any possibility to get the reason for this? Or do you have any hint, what I may have done wrong?
Thanks for your help.
Kind regards
v0ti

New file_contexts.bin format and make_ext4fs

Trying to alter a stock ROM and resign with my own custom key just for myself. This process will allow me to use a device with custom changes (NOT root), but still allow me to use Android Pay.
This process involves using the "make_ext4fs" tool. However, this tool (as far as I can tell) requires using the old plaintext version of "file_contexts". Android 7+ uses a new binary version of this file instead ("file_contexts.bin").
So if anyone can solve either of these questions, I would be very grateful:
1) Is there a tool to convert my device's binary "file_contexts.bin" to the old plaintext "file_contexts"? -OR-
2) Is there a build of "make_ext4fs" that accepts the new binary format "file_contexts.bin"? I tried to build the new nougat version myself from AOSP, but after a successful build of the tool, I can't get the thing to read files on either 64-bit or 32-bit systems. It yields an error "No such file or directory" when trying to point to file_contexts. I am not a great C developer, so I don't know if I built it wrong. For reference, no, I am not missing the lib32stdc++6 libraries. I have attached the binary if you want to see if you can get it to work. -OR-
3) If anyone has the file_contexts text file for the OnePlus 3, ROM version 7.X Nougat, that would work too!​
Any help anyone can offer in any way would be fantastic! Thank you so much for your time to look at this!

A good question

1) yes, see my signature

munjeni said:
1) yes, see my signature
Click to expand...
Click to collapse
I'm trying to do the same with a 7.1.1. rom. I convert the file_contexts.bin to file_contexts, unpack the rom, do the modifications to the rom, repack the rom then replace the original system.new.dat and system.transfer.list with the newly packed ones (original file_contexts.bin is still in the original zip) and when I go to flash it in TWRP it doesn't flash the system and I get a No OS installed Is there something I have to do to the file_contexts.bin after I repack the system.new.dat?

"file_contexts.bin" to the old plaintext "file_contexts" tool: http://blog.cofface.com/archives/2255.html

puargs said:
I tried to build the new nougat version myself from AOSP, but after a successful build of the tool, I can't get the thing to read files on either 64-bit or 32-bit systems. It yields an error "No such file or directory" when trying to point to file_contexts. I am not a great C developer, so I don't know if I built it wrong. For reference, no, I am not missing the lib32stdc++6 libraries.
Click to expand...
Click to collapse
Try building with the pie (Position Independent Executable) flag. Had this exact same problem when trying to compile a CVE for android.
Hopefully this should work for you and sorry for the Necro post

[TOOLS] Necessary tools for building the Pixel 2 (XL) Kernel

THIS IS NOT A "HOW TO BUILD A KERNEL" POST
The Pixel 2, unlike other phones I've worked with, requires two extra tools: dtc (Device Tree Compiler) and mkdtimg.
Yes, dtc is available in the main repos for most linux distros, but the Pixel 2 kernel requires the AOSP modified version of dtc.
First, grab the Pixel 2 + XL Unified kernel source (wahoo) from https://android.googlesource.com/kernel/msm/+/android-msm-wahoo-4.4-oreo-dr1
Then download wahoo-kernel-tools.zip file attatched to this post and unzip it. Make sure that the 2 files in the bin folder are available in your path (possibly by moving them to ~/bin) and symlink or move the libc++.so into an included library directory, such as /lib, /usr/lib, or /usr/local/lib.
These specific binaries were compiled on Arch Linux by me, so they may not work on every system. To build binaries compatible with your system, sync any android oreo source (AOSP, Lineage, Carbon, etc.) and run "make dtc mkdtimage -jX" instead of "make bacon" or your normal command. The binaries will be located in out/host/linux-x86/.
This is tested and working with the stock google 4.9 toolchain and the initial source as of Oct 23rd, 2017.

frap129 said:
THIS IS NOT A "HOW TO BUILD A KERNEL" POST
The Pixel 2, unlike other phones I've worked with, requires two extra tools: dtc (Device Tree Compiler) and mkdtimg.
Yes, dtc is available in the main repos for most linux distros, but the Pixel 2 kernel requires the AOSP modified version of dtc.
First, grab the Pixel 2 + XL Unified kernel source (wahoo) from https://android.googlesource.com/kernel/msm/+/android-msm-wahoo-4.4-oreo-dr1
Then download wahoo-kernel-tools.zip file attatched to this post and unzip it. Make sure that the 2 files in the bin folder are available in your path (possibly by moving them to ~/bin) and symlink or move the libc++.so into an included library directory, such as /lib, /usr/lib, or /usr/local/lib.
These specific binaries were compiled on Arch Linux by me, so they may not work on every system. To build binaries compatible with your system, sync any android oreo source (AOSP, Lineage, Carbon, etc.) and run "make dtc mkdtimage -jX" instead of "make bacon" or your normal command. The binaries will be located in out/host/linux-x86/.
This is tested and working with the stock google 4.9 toolchain and the initial source as of Oct 23rd, 2017.
Click to expand...
Click to collapse
Thanks for the files dude, you saved me some serious time since I don't need to sync AOSP

DespairFactor said:
Thanks for the files dude, you saved me some serious time since I don't need to sync AOSP
Click to expand...
Click to collapse
No problemo, I figured I'd post this since my fastboot is being retarded and I can't do much else

Forgive me for being a noob, but would these new requirements for the Pixel 2 cause existing tools used to pack/unpack the boot image fail or at least partially fail? I'm wondering if this is why we can't get Magisk to work
edit: to be clear, when I unpack the boot.img file all I see are kernel and ramdisk. When I unpack the boot.img from the Pixel 1 XL, I see those same 2 files along with devicetree.dtb and kernelimage

skaforey said:
Forgive me for being a noob, but would these new requirements for the Pixel 2 cause existing tools used to pack/unpack the boot image fail or at least partially fail? I'm wondering if this is why we can't get Magisk to work
edit: to be clear, when I unpack the boot.img file all I see are kernel and ramdisk. When I unpack the boot.img from the Pixel 1 XL, I see those same 2 files along with devicetree.dtb and kernelimage
Click to expand...
Click to collapse
Maybe? It doesn't seem like how the boot image is packed has been changed, just how the actual kernel image is put together.

nathanchance said:
Maybe? It doesn't seem like how the boot image is packed has been changed, just how the actual kernel image is put together.
Click to expand...
Click to collapse
Strange. Thanks for the response though. In the Magisk thread here, someone has posted a boot image that was simply unpacked/repacked (no changes) and we can't get it to boot. I don't want to derail this thread any further though.

skaforey said:
Forgive me for being a noob, but would these new requirements for the Pixel 2 cause existing tools used to pack/unpack the boot image fail or at least partially fail? I'm wondering if this is why we can't get Magisk to work
edit: to be clear, when I unpack the boot.img file all I see are kernel and ramdisk. When I unpack the boot.img from the Pixel 1 XL, I see those same 2 files along with devicetree.dtb and kernelimage
Click to expand...
Click to collapse
skaforey said:
Strange. Thanks for the response though. In the Magisk thread here, someone has posted a boot image that was simply unpacked/repacked (no changes) and we can't get it to boot. I don't want to derail this thread any further though.
Click to expand...
Click to collapse
No they don't, these just build the dtb/dtbo. The magisk issue is with patching sepolicy likely.

frap129 said:
No problemo, I figured I'd post this since my fastboot is being retarded and I can't do much else
Click to expand...
Click to collapse
Did you try to flash a factory image and get stuck in the bootloader? If so, extract 'image-walleye-[build number].zip' and flash each image manually. fastboot -w update doesn't work for some reason. After you flash all the images, do fastboot format userdata, and you should be back to normal. Took me a couple hours to figure this out.

ryanbg said:
Did you try to flash a factory image and get stuck in the bootloader? If so, extract 'image-walleye-[build number].zip' and flash each image manually. fastboot -w update doesn't work for some reason. After you flash each image, do fastboot format userdata, and you should be back to normal. Took me a couple hours to figure this out.
Click to expand...
Click to collapse
Thanks! I don't get stuck in the bootloader, but it won't let me flash boot images. Fastboot either hangs or spits out "remote: error: max size not reported"