How to build Android Marshmallow on Nexus S
The story...
Nexus S (crespo) got its last update in Oct 2012. It was Android 4.1.2 Jelly Bean. Android M (marshmallow) just came out recently. I decided to port M to crespo for fun, and as a demo that old hardware can in fact run new versions of android. In this article you'll read about some of the challenges I faced in making this happen and find the information you need to reproduce my results. And for the lazy, I also have a pre-built AOSP Marshmallow image set to download on the bottom of this page. Since Nexus S was originally a Google-Play-equipped device, you can legally install Google Apps on this image and enjoy a full Google Android 6.0 experience on your Nexus S. That part, however, is up to you to do yourself. I am not offering GApps downloads here.
For fun, to make this work a few very cool hacks had to be done, read further below in the "cool hacks" section about them, if you care.
How to build
#get Android M AOSP into folder called M mkdir M cd M repo init -u https://android.googlesource.com/platform/manifest -b android-6.0.0_r1 repo sync -j4 -c cd .. #get latest J that supported crespo AOSP into folder called J mkdir J cd J repo init -u https://android.googlesource.com/platform/manifest -b android-4.1.2_r1 repo sync -j4 -c cd .. #get blobs cd M wget https://dl.google.com/dl/android/aosp/akm-crespo-jzo54k-41bd82a7.tgz wget https://dl.google.com/dl/android/aosp/broadcom-crespo-jzo54k-3272fe6e.tgz wget https://dl.google.com/dl/android/aosp/imgtec-crespo-jzo54k-c20bd30b.tgz wget https://dl.google.com/dl/android/aosp/nxp-crespo-jzo54k-59170f80.tgz wget https://dl.google.com/dl/android/aosp/samsung-crespo-jzo54k-1248bb36.tgz wget https://dl.google.com/dl/android/aosp/widevine-crespo-jzo54k-22298742.tgz #unpack blobs tar xvfz akm-crespo-jzo54k-41bd82a7.tgz tar xvfz imgtec-crespo-jzo54k-c20bd30b.tgz tar xvfz samsung-crespo-jzo54k-1248bb36.tgz tar xvfz broadcom-crespo-jzo54k-3272fe6e.tgz tar xvfz nxp-crespo-jzo54k-59170f80.tgz tar xvfz widevine-crespo-jzo54k-22298742.tgz dd if=extract-akm-crespo.sh bs=14376 skip=1 | tar xvz dd if=extract-broadcom-crespo.sh bs=14395 skip=1 |tar xvz dd if=extract-imgtec-crespo.sh bs=14413 skip=1 |tar xvz dd if=extract-nxp-crespo.sh bs=14383 skip=1 |tar xvz dd if=extract-samsung-crespo.sh bs=14707 skip=1 |tar xvz dd if=extract-widevine-crespo.sh bs=14377 skip=1 |tar xvz rm extract-*-crespo.sh *-crespo-jzo54k-*.tgz #binary patch samsung's ril: echo -n dmitrygr_helper | dd bs=1 seek=60382 conv=notrunc of=vendor/samsung/crespo/proprietary/libsec-ril.so #copy over the device tree for crespo from J mkdir device/samsung cp -Rvf ../J/device/samsung/crespo device/samsung/crespo #apply source patches to android cd device/samsung/crespo git apply ../../../../patches/device-samsung-crespo_Support_M.patch mv overlay/frameworks/base/core/res/res/drawable-hdpi overlay/frameworks/base/core/res/res/drawable-nodpi cd ../../.. cd bionic git apply ../../patches/bionic_Allow-non-PIE-executables.patch cd ../ cd frameworks/native git apply ../../../patches/frameworks-native_allow-crespo-to-boot.patch cd ../.. cd packages/apps/Nfc git apply ../../../../patches/packages-apps-Nfc_enable_NFC_for_crespo.patch cd ../../.. cd system/bt/ git apply ../../../patches/system-bt_Allow-Bluedroid-to-build-on-systems-with-no-LE-suppot.patch cd ../.. #apply optional patches to android (you WILL need these if you intend to use google play services or facebook) cd frameworks/base git apply ../../../patches/frameworks-base_increase_timeout_to_allow_crespo_to_compile_GMS_core.patch cd ../.. cd art git apply ../../patches/art_increase_timeout_to_allow_the_slow_CPU_to_finish_compiling.patch cd .. #get kernel sources & apply patches cd .. git clone https://android.googlesource.com/kernel/samsung.git cd samsung git checkout remotes/origin/android-samsung-3.0-jb-mr0 git apply ../patches/kernel/0*.patch #optionally apply an overclocking patch (YMMV) git apply ../patches/kernel/OTIONAL-*.patch #build kernel export ARCH=arm export CROSS_COMPILE=`pwd`/../M/prebuilts/gcc/linux-x86/arm/arm-eabi-4.8/bin/arm-eabi- make herring_defconfig make -j4 #place kernel into android build cp arch/arm/boot/zImage ../M/device/samsung/crespo/kernel cd .. #build android cd M source build/envsetup.sh lunch full_crespo-userdebug make -j4
Cool hacks (or: "you wrote your own RIL?!?")
Missing libraries for the original RIL
Nexus S's RIL was made to work with Android J, and no sources are provided. Android M changed lots of things. In the case of the original samsung RIL, what was missing is one function: android::Parcel::writeString16(unsigned short const*, unsigned int). Instead M has android::Parcel::writeString16(char16_t const*, unsigned int). As it is obvious, these really do the same thing, just a param type change (causing a mangled name change). I wrote an interposition library that implements this thunk and it is called libdmitrygr_helper. How does it work?
Curious information: Due to peculiarities of the ELF format, when a binary baz imports function foo() from libbar.so, nowhere in baz's ELF file does it say that foo() must from from libbar. In fact there are two separate records. One that says that libbar is "NEED"ed, and another that says that there is an import of function "foo". What that means is that if the process were to also load libxyz, which also exported foo(), there is no way to be sure which foo() would get called. Why do we care? Well, consider our problems. We can repalce a NEED record in Samsung's RIL with one referencing a library we control, like "libdmitrygr_helper.so", implement the missing function in terms of the really available ones in "libdmitrygr_helper.so". This is in fact what I did. Thus the binary patch to the RIL is minimal (one string). The implementation of "libdmitrygr_helper.so" I am providing under AOSP license as part of this post.
But then why write a RIL?
Android's RIL interface is versioned. Current version is 11, and currently in M, ril-daemon will allegedly support a RIL version as low as 6. This is cool because the RIL in crespo is version 6. I say allegedly because in reality (as we can see with crespo) it does not work. With lidbmitrygr_helper.so the RIL will load, but the phone will say "No Sim" and nothing will work. Why? Well, it looks like java code in android that talks through the ril-daemon to the RIL sends some commands to the RIL that are not appropriate for RIL version 6. Now, the fault is not just with Android. The RIL can respond to an unknown command with an error. Samsung's RIL does not. These two commands it just ignores. This is a problem, as the java code just waits... forever. Well, in theory one could patch the java code to not do this, or simply to not wait for a reply indefinitely, but that ends up being a rather invasive patch. And, even if we did, we're still no closer to solving anything. In addition to this, ril-daemon sends an extra parameter to the RIL, "-c", which Samsung's RIL does not understand, barfs, and refuses to run. Once again, Android's code should have checked the RIL version before sending this new parameter. But, again, Samsung's not faultless. Their RIL should just ignore unknown options instead of aborting.
As can be seen, fixing all this would require a lot of invasive patches, including some to java. Plus, one would have to support these patches in the future. I did not like this idea, so I went a different route. Why, I thought, not write my own RIL? It can sub-load Samsungs RIL. I can then filter out and properly respond to those invalid commands, and remove that pesky "-c" option before it arrives to Samsung's RIL and confuses it. I am happy to that this works very well. My RIL is called dmitry-ril and I am releasing it under the AOSP license here now.
HALs & other Samsung code
Crespo shipped with some very old HAL versions. So old that their version #defines no longer even exist. The audio HAL was also pretty old and unsupported by modern Android. I made some changes to update the audio HAL and the hardware composer code to make them support modern interfaces. Well, at least suported, if not modern. The new hwcomposer is now version 1.0 (from 0.3). Lots of other code had to be patched up to make it build in modern android. Almost every samsung library in /device had to be modified. Luckily I was able to do all that and it all works. Bluetooth also changed a lot from J, and I added a BT config file as is now required. This was fun - AOSP code will actually fail to build when BLE is off. I made a patch to fix this, and even sent it off to be included in AOSP.
Kernel
The latest kernel for crespo is 3.0.31. This is rather ancient, and lacks many important things that Android M needs. I ported many of them back, while also making other improvements (including ability to compile in modern gcc, which it originally did not). A few cool things: some prebuilts and google apps assume that your CPU supports interger divide instructions (SDIV/UDIV). These are optional, and crespo's Hummingbird CPU does not support them. I added support for emulating them in the kernel. It is slow, but will work. Google apps only do this rarely (and only in the keyboard JNI library as far as I can tell) so it helps without too much overhead. M needed new netlink code, so I simply copied that from 3.4 kernel, and then made changes to get it to build and work. Defconfig was updated to reflect proper configs for M as well. I also produced an optional patch to overclock the device (in its entirety) 10%, in case you want some extra oomph.
Optional: ART timeout
ART is the ahead-of-time compiler used to run code in Android M. Sadly, crespo's measly RAM allowance for android (384MB) and puny CPU (1GHz Cortex-A8) mean that compilation takes a while. ART has a timeout, and for large apps like Facebook or Google Apps, this timeout is inadequate in M. I provided two patches to increase it, so that Facebook and GApps can work on crespo. I also, as mentioned above, provided a patch to slightly overclock the device.
Partitioning & Storage
This is where the story turns simply weird. Nexus S has both raw NAND and EMMC storage. NAND stores: bootloader, boot.img, recovery, cache, and radio.img. EMMC stores: system, userdata, and media. "Media?" you might ask. Yes! Nexus S had an "internal SD card" provided by this media partition that was normally formatted FAT32 and could be exposed to a host PC over usb. This was the archaic way of transferring files to and from Android phones. Needless to say we no longer do this. System partition on crespo was 512MB. Not big enough to fit M with dex-preopt, but big enough to fit M without it. Good enough. Userdata partition on Crespo is 1GB, and media as 14.8GB. Initialy I expected to expose media as an actual SD card, and allow users to use M's new "adopt SD card to internal memory" approach, but I found that this was (1) impractical and (2) added needless slowdowns. I considered repartitionning the EMMC to join userdata and media. This was easy to do, but would make it hard to revert to any other ROM. I decided against that thus. So what to do? Android in 1GB of storage sucks. I decided to just format media as ext4 and use it as userdata, and simply abandon and not use the old userdata partition. I considered reusing old userdata as system, and it worked, but this will confuse many tools and TWRP, so I decided against it. So, sure, you only get 14.8GB instead of 15.8GB, but I feel ability to revert to any other ROM is worth it.
Graphics
Sometime in the M timeframe, Android decided that nobody cares about graphics chips that cannot do RGBA8888. This is not good for crespo, because the combination of its GPU & driver will not do that. PowerVR itself can, but the driver provided for crespo prefers BGRA8888 and does not support RGBA8888. Android had to be patched to support BGRA. It was not a large patch. It just reverts to the old method of asking the GPU what it can do instead of assuming RGBA8888 is supported. Recovery also had this issue, resulting in broken colors. Fixed that too.
Work left to do & known problems
- Kernel per-uid CPU accounting needs to be done for better battery blaming. I might get it done later, but it is not urgent. I did do it for N10 and N4 - I am just being lazy now. Not having this will cause the framework to try finding it 10 times, and log the failures, then it will quiet down and life will proceed.
- Newer kernel in general would be cool. I got rather far with a 3.4 port. It booted and ran mostly. But after I got netlink to work in 3.0.31, I decided to use it instead as it is better tested and proven.
- APN settings in M default to IPv6. The kernel does not work well enough with IPv6 to make that happen, so to get cell data and MMS to work, you'll have to go into settings and change APN type from IPv6 to IPv4. Not a big deal. I am sure it is fixable, I just have not dedicated time to it.
Ok, ok, give me the files already!
Download patches if you want to make the build yourself: => [PACKAGE YOU'LL NEED (mirror)] <=If you're not a person who wants to build android yourself from source, here is fully usable userdebug image. Tested: Sensors, NFC, WiFi, Bluetooth, Camera, GPS, Cellular, Recovery.
Fastboot-flashable images here => [FASTBOOT IMAGES (mirror)] <=
Or, if you prefer to flash from TWRP => [TWRP flashable zip (mirror)] (updated) <=
Plase do not re-host this elsewhere and claim this as your work.
