Related
I have a G1 htc dream (european variant = rogers 32a), and I'm slowly losing out. The thing is on CM-6.0, it never had enough memory, and eventually crashed it's way into a mess, and I'm pondering the next move.
What works: All phone features, root terminal, fastboot (r2d2s on skateboards), some programs. I can read/write the sdcards independently. I even have access with the old version of adb. It boots, & reboots.
I have a nandroid backup from late last year on the sdcard, which would do very nicely thank you if I could get it installed. Is there any way to install the backup without access to recovery mode? How are those image files cobbled together?
What doesn't work:
1. Recovery Mode(?) turn-on-with-home-pressed. I can't get off the (opening) blue screen. Remove the battery to restart.
2. /etc/fstab & /etc/mtab are no longer there :-/. /proc/mounts is. I don't know what else is missing, but it implies some essential daemon is awol. Amazingly it writes to the sdcard :-//.
3. Market - It says "starting download....." but it doesn't:-(
4. Memory control. I can start with a bit free, but as uptime continues, the memory vanishes, and anything you open crashes.
I do have linux, and a root terminal on the phone. But without /etc/fstab, all those crazy android device names are not to be seen :-//. ls /dev/block shows me loop0 - loop7; mmcblk0, mmcblk0p1 - mmcblk0p3; mtdblock0 - mtdblock5; vold.
If you have fastboot and a nandroid, you can flash all images separately, i.e.
fastboot flash system <system.img>
fastboot flash userdata <data.img>
...
In the Cyanogenmod Wiki you can find probably some additional hints
With fastboot you can also flash a new recovery ...
AndDiSa said:
If you have fastboot and a nandroid, you can flash all images separately, i.e.
fastboot flash system <system.img>
fastboot flash userdata <data.img>
...
In the Cyanogenmod Wiki you can find probably some additional hints
With fastboot you can also flash a new recovery ...
Click to expand...
Click to collapse
Thanks for the reply.
Image on pc, and fastboot on pc? I'll try that.
Ok. Not out of the woods, but I've made some progress. The obstacle is: recovery isn't mounted, and I only have /proc/mounts. In /dev/block
mtdblock3 is mounted on /system
mtdblock4 is mounted on /cache
mtdblock5 is mounted on /system/xbin
There is mtdblock 0-5. Where the $£%@! is recovery supposed to be mounted? what about the other ones? anyone got an /etc/fstab from a G1??
Actually, I probably have but it's archived. I had figured this on the android by using adb and dump_image, but that's restoring a system while it's running, and there should be laws against that sort of thing.
??? ... recovery is never mounted, it's like a second "mini" os.
Additionally: for using fastboot your phone must be in fastboot mode, i.e. boot with back+power. I suggest you to first read a bit more about fastboot, g1 partitions, etc. ...
Sent from my Gingerbread on Dream using XDA App
That worked after a fashion. I had the backup: system.img, boot.img, cache.img, recovery.img, data.img, & misc.img. Of these, I could flash recovery, cache, system, & boot. The data.img & misc.img threw errors, (me)not knowing where they should be put.
It wasn't enough. I had a trail of processes bailing out and a repeated crash in acore typical of memory being clogged. But I got back recovery mode, and I could apply the latest backup normally. I'm now back to the moments after my successful install of CM-6.0 - before I started civilising it. And I don't really have the ram for it.
Good good good, thank you very much. As I don't have the ram, I'll have to add swap, which I have my sdcard set up for. It's certainly less pain than looking for a lower level rom and going to a smaller OS.
I do have one hitch: I've lost root in the xterm (and I don't know where else).
$ su
permission denied
$
:-O?
EDIT:: I'm a twit. I forgot to run the fix_permissions script. Didn't know I backed up _THAT_ promptly. Permissions were AWOL. So are a few apps, but nothing I'm worried about. Now
This thread is for brainstorming and developing safe ways of booting our OUYAs to use custom ROMs.
I've developed an initial method that allows OUYA to boot CM 10 from a usb thumb drive that does not require you to mess with any of your internal storage at all! This allows you to keep your OUYA in pristine stock condition, while still enjoying CM 10 from an external thumb drive.
Unfortunately it does require you to setup the thumb drive in a particular way to allow it to be used and this may be difficult for some users without Linux experience. I hope some will find it useful and plan to develop easier methods in the future (probably involving a custom recovery image with the ability to setup the thumb drive for you)
I plan to post more details on how I achieved this so that others can use the knowledge and apply it to other roms or develop improved solutions, but I might work on some other things first. Mainly it involved unpacking/repacking the Android boot.img (google it and there are tutorials about how to do this, I also recommend checking out the "abootimg" program from the Ubuntu packages) and modifying a couple of the init/fstab files.
Note that the zip I am going to link to is not a flashable zip, extract it and follow the instructions in the README which I am including below. Also note the the Google apps are not included - so if you want them you will need to add them yourself. Just be aware things you add to the system partition will need correct permission set.
One of my other ideas I'm going to eventually try is swapping the boot/recovery partitions so that the device normally boots up into recovery and then will have an option to reboot into the recovery partition which would actually boot a full rom for regular use. This may still be dangerous on OUYA though, so is not recommended unless you know what you're doing. I have a Notion Ink Adam which I can boot into APX mode (a low-level nVidia recovery mode that can be used to restore the device in the case of bricking, unfortunately we do not have this level of acces on OUYA), so it will be much safer for me to experiment on it. I haven't really developed on it before though (in fact this OUYA bit is my first major Android development apart from doing/tweaking some CM builds from source myself for some of my devices), so it may take me some time to set stuff up to experiment with it.
Eventually I think we need some form of bootloader for the OUYA. I have read about some very interesting kexec hardboot patches that were developed for the original Nexus 7 that it would be very awesome if we could port to OUYA - that would allow us to boot the patched kernel and have it boot us into kernels/roms stored elsewhere.
Please post any feedback or ideas. Sorry the current method isn't that easy yet and I hope the instructions make sense - hopefully others can also help clarify.
If you'd like to donate to me, I'm certainly not going to turn you away, but keep the other devs in mind 'cause I haven't done that much yet! ( PayPal: [email protected] )
~Troop
ouya-sda123-boot by Trooper_Max
Safely boot your OUYA to CM 10 without messing with your internal storage! (fast thumb drive recommended!)
========
Credits:
========
khanning88 for the initial CM10 Experimental ROM - this is just a repackaging/reconfiguration of it.
mybook4, sonofskywalker3, rayman, professorpoptart for their CWM recovery
And of course the CyanogenMod/ClockworkMod team for the basis of everything!
=========
Contents:
=========
ouya-cm10-system.img:
ext4 system image dumped from CM 10 experimental OUYA rom
ouya-sda123-boot.img:
Android boot.img to boot CM 10 from thumb drive partitions (details below)
==============
Prerequisites:
==============
For this current method, you will need to be able to partition the thumb drive into three ext4 partitions. This probably means you need Linux - if you don't have a Linux system, I highly recommend checking out pendrivelinux.com for methods of booting Linux off of a thumb drive (I recommend Ubuntu 12.04 LTS or whatever version you are comfortable with).
I hope to develop more methods in the future that will be easier than this, but this is the initial method.
=============
Instructions:
=============
1. Partition the thumb drive with three ext4 partitions - note that they must be the first three partitions on the thumb drive, and I would likely just dedicate a thumb drive to this. I recommend using Disk Utility or gparted (usually both available from an Ubuntu thumb drive).
* The first partition will be the data partition and should be the largest
* Note that Android will create the virtual sd card at /data/media
* The second partition will be system (OUYA internal system is 512 M)
* The third partition will be cache (OUYA internal cache is 768 M)
* Feel free to adjust the size of the partitions, but I'd recommend sticking close to the stock sizes
* I've tested this on an 8 GB as well as a smaller 4 GB thumb drive, bigger should not be a problem
2. Note the device name of your thumb drive - It will likely be sdX where X is a letter corresonding to the order it was mounted in - I would expect it to be sdd if it happens to be the fourth drive connected to your machine. The system partition will then be sdX2.
3. Write the ouya-cm10-system.img to the second partition of the thumb drive. It is crucial that the files get copied into the partition with the correct permissions.
* The easiest way to ensure this is to use dd to do a byte for byte copy of the system image directly over the partition, but this is also very dangerous if you type it wrong, so be sure you have the write device name for your thumbdrive.
* I recommend looking in the Disk Utility or gparted, or running "mount", "df -h", and "cat /proc/partitions", to make sure you have an understanding of what drives are what device names before continuing
* Once you are certain of the device name, ie sdX2, where X is the letter for the thumb drive and 2 denotes the second partition which we are using as system, run "dd if=ouya-cm10-system.img of=/dev/sdX2 bs=4M" as administrator (on Ubuntu either by sticking the word "sudo" in front of the command or running "sudo su" first to switch to root)
* This command will not show any output until the end and may take a little while.
4. Clean/fix the filesystem and resize the filesystem to fill the partition.
* Run "esfsck -fp /dev/sdX2" as administrator
* Run "resize2fs /dev/sdX2" as administrator
5. Thumb drive is now ready (the data/cache partitions can be empty ext4, Android will fill them in). Connect it to the OUYA. Through a hub is fine - just be sure it is the only thumb drive connected at boot.
6. Boot the OUYA into fastboot mode. The only real way to do this right now is to first boot up the OUYA normally, then use "adb reboot bootloader" to reboot into the bootloader. You should be able to run "fastboot devices" then and see a device listed.
7. Boot ouya-sda123-boot.img using the command "fastboot boot ouya-sda123-boot.img". You should see it download to the OUYA and it should start booting.
8. Wait patiently. Remember that CM boots slow the first time and depending on the speed of your thumb drive may boot even slower. You can however type "adb devices" to see if it has started the adb daemon. If you don't see a boot animation after a while, you can try running "adb shell" and if you get a permission error, it probably means you didn't flash the system partition correctly, but hopefully all goes well. Note though that once it gets farther into the boot sequence it will turn off the adb daemon so you will lose adb access until it boots up and you can re-enable it.
9. Enjoy CM 10 on your OUYA without having messed with your internal storage! Just be sure not to let your OUYA fall asleep, as it may not be able to wake back up! I'm guessing this is because I was using a hub and so when it falls asleep, the thumb drive essentially gets disconnected and it cannot immediately find it again when it tries to resume. I'd recommend using a wakelock application or power toggle to keep the screen on all the time, etc.
========
For Help
========
Look for us on the XDA Developers Forum under the appropriate threads! Keep in mind that using this method to boot CM 10 may introduce new bugs that would not have occurred using CM 10 the regular way, so be sure to report problems in the appropriate place and mention what methods you used!
Click to expand...
Click to collapse
http://troopermax.com/releases/ouya-sda123-boot.zip
(be gentle - it's 145 MB, I use shared hosting, so feel free to mirror)
md5sum: 49c8e16e27b6deb9d1e8e86363b56f2f
Mirror: http://www.mediafire.com/download/hban76kzeys6ybd/ouya-sda123-boot.zip
~Troop
I'm including some rough developer details here for now about how I did it.
Many thanks once again to the OUYA CWM recovery, I found it insanely useful: http://forum.xda-developers.com/showthread.php?t=2295645
The main part was unpacking/modifying/repacking the boot.img (which contains the kernel/ramdisk/etc)
abootimg helped with this greatly:
http://manpages.ubuntu.com/manpages/precise/man1/abootimg.1.html
Then I followed instructions I found via google for unpacking/repacking the ramdisk:
http://android-dls.com/wiki/index.php?title=HOWTO:_Unpack,_Edit,_and_Re-Pack_Boot_Images
The partitions the OUYA uses are mostly defined in fstab.cardhu:
# Android fstab file.
#<src> <mnt_point> <type> <mnt_flags> <fs_mgr_flags>
# The filesystem that contains the filesystem checker binary (typically /system) cannot
# specify MF_CHECK, and must come before any filesystems that do specify MF_CHECK
# We remount because we first mount as rw in order to generate NVSI symlink. See init.rc for details.
/dev/block/platform/sdhci-tegra.3/by-name/APP /system ext4 ro,remount wait
/dev/block/platform/sdhci-tegra.3/by-name/CAC /cache ext4 noatime,nosuid,nodev,journal_async_commit,data=writeback,nodelalloc,errors=panic wait
/dev/block/platform/sdhci-tegra.3/by-name/UDA /data ext4 noatime,nosuid,nodev,journal_async_commit,data=writeback,nodelalloc,errors=panic wait,encryptable=/dev/block/platform/sdhci-tegra.3/by-name/MDA
Click to expand...
Click to collapse
I found that I could use fastboot to boot into the CWM recovery and then use adb pull to dump images of the internal OUYA partitions, using the paths above, ex. "adb pull /dev/block/platform/sdhci-tegra.3/by-name/APP ouya-system.img" to dump the system partition. I did a regular clockwordmod backup first, but adb pull was useful here for pulling raw images.
So after backing up my OUYA stock system, I actually did flash the CM10 rom so that I could dump the partitions to my computer using adb. The main reason I did this was to ensure the system partition gets generated with the correct permissions. I didn't boot into CM10 though, though I was tempted. Instead I restored back to my backup to put my OUYA back in stock condition.
With the system image dumped into a file, I used resize2fs to shrink the file to the minimum size (resize2fs -fM ouya-system.img) This probably wasn't absolutely necessary since it would compress down in the zip, but this allows it to be written to smaller partition sizes.
http://manpages.ubuntu.com/manpages/precise/en/man8/resize2fs.8.html
Here is what I modified the fstab.cardhu to:
# Android fstab file.
#<src> <mnt_point> <type> <mnt_flags> <fs_mgr_flags>
# The filesystem that contains the filesystem checker binary (typically /system) cannot
# specify MF_CHECK, and must come before any filesystems that do specify MF_CHECK
# We remount because we first mount as rw in order to generate NVSI symlink. See init.rc for details.
/dev/block/sda2 /system ext4 rw wait
/dev/block/sda3 /cache ext4 noatime,nosuid,nodev,journal_async_commit,data=writeback,nodelalloc,errors=panic wait
/dev/block/sda1 /data ext4 noatime,nosuid,nodev,journal_async_commit,data=writeback,nodelalloc,errors=panic wait
Click to expand...
Click to collapse
Pretty straightforward, except that I removed the encryptable part on /data (not sure how that bits work and if I might have broken it) and made system read/write.
Also note, that I removed the initial read/write mounting of system in init.cardhu.rc since I was having trouble with it mounting from there (I'm guessing because my usb hub/thumb drive weren't yet available when that tried)
Here's the relevant part of init.cardhu.rc:
Before:
on fs
setprop ro.crypto.tmpfs_options size=128m,mode=0771,uid=1000,gid=1000
setprop ro.crypto.umount_sd false
# PLEASE DO NOT REMOVE NVSI SYMLINK! IF CHANGES ARE NEEDED PLEASE ENSURE THAT NVSI SYMLINK IS ALWAYS CREATED.
# Mount system to allow NVSI symlink
mount ext4 /dev/block/platform/sdhci-tegra.3/by-name/APP /system rw
# Create NVSI filter symlink
symlink /data/data/com.nvidia.NvCPLSvc/files/com.nvidia.nvsiutilv1.xml /system/etc/permissions/com.nvidia.nvsiutilv1.xml
mount_all /fstab.cardhu
#chmod for OUYA parameters
chmod 0644 /dev/block/mmcblk0p5
Click to expand...
Click to collapse
After:
on fs
setprop ro.crypto.tmpfs_options size=128m,mode=0771,uid=1000,gid=1000
setprop ro.crypto.umount_sd false
# PLEASE DO NOT REMOVE NVSI SYMLINK! IF CHANGES ARE NEEDED PLEASE ENSURE THAT NVSI SYMLINK IS ALWAYS CREATED.
# Mount system to allow NVSI symlink
#mount ext4 /dev/block/sda2 /system rw
mount_all /fstab.cardhu
# Create NVSI filter symlink
symlink /data/data/com.nvidia.NvCPLSvc/files/com.nvidia.nvsiutilv1.xml /system/etc/permissions/com.nvidia.nvsiutilv1.xml
#chmod for OUYA parameters
chmod 0644 /dev/block/mmcblk0p5
Click to expand...
Click to collapse
Basically I commented out the mount command - before it was mounting system read/write for a moment to insert the symlink and let the filesystem update itself before remounting read-only in the fstab. The way I do it now it is simply mounted read/write from the fstab and left that way (some may consider this unsafe, but it is good for development... not too hard to remount it read-only if you so desire) I also shifted the symlink command down below the mount all command so it would hopefully still work... not sure what that was for, but they made it sound really important! XD
I also commented out this part about the usb drive since it doesn't really make sense anymore:
# Mount usb drives as /usbdrive. Generally usb drives are formatted with FAT
# filesystem, so we support FAT as of now.
#on device-added-/sys/block/sda
# mount vfat /dev/block/sda /mnt/usbdrive
Click to expand...
Click to collapse
I think those were all the changes I made before repacking it, though it did take me 5 attempts to get a working ramdisk so I may have lost track of something at some point. Those should be the key changes at least.
I also initially used raw dumps of the data and cache partitions on my thumb drive as well to make sure it would work. Then I tested to make sure it would work if they were empty ext4 partitions, which they do - Android fills them in as necessary. So only the system image was necessary to include. (and of course the boot.img for fastbooting)
This was a little bit harder than I had anticipated because I'd previously only looked into this type of thing for Gingerbread devices which aren't set up the same way. I'd kinda been hoping to get a solution where the user could just drop a system.img and data.img onto a fat32 usb thumb drive and the boot.img would loop mount them off the thumb drive. I haven't tried this for Gingerbread, but it seems pretty straightforward. It's not straightforward anymore here since fat32 sdcards aren't used anymore. Being familiar with Linux though, I am happy they are using ext4 everywhere and using the same space alloation for /data and /sdcard so you don't have to worry about which one to make big, but it does make some things more complicated for more average users. It's probably still possible to get the OUYA to do what I wanted, but more complicated and not sure it's worth pursuing over other methods which may be more fruitful.
Hope that's enough details to enlighten some people who may be wondering about the methods used here. My goal is to not only develop new ways of booting the OUYA, but also to share how I accomplished things so that others can learn from it.
~Troop
Is this similar to the Nook HD + CM10 boot?
---------- Post added at 05:07 AM ---------- Previous post was at 04:58 AM ----------
Wait that was blank a second ago. I am downloading now. I will mirro with my mediafir stuff if I can get it to work.
kairnage said:
Is this similar to the Nook HD + CM10 boot?
---------- Post added at 05:07 AM ---------- Previous post was at 04:58 AM ----------
Wait that was blank a second ago. I am downloading now. I will mirro with my mediafir stuff if I can get it to work.
Click to expand...
Click to collapse
Sorry, I was setting up the posts...
I'm not sure this is similar to any solution currently used on other devices, but I do know that other developers have used similar methods to develop their roms.
I don't have a Nook and haven't looked into what they use too much, but we may very well be able to learn from their work.
~Troop
So I dug up an old nvflash package from my Notion Ink Adam and tested it to make sure it still works (with the Adam in APX mode it allows me to directly flash the device with a new partition layout and flash the partitions directly). After verifying I had it setup correctly, I swapped the boot and recovery images and ran it again and sure enough... now when I boot my Adam normally, the recovery rom comes up, and when I boot my Adam into recovery it loads the normal rom - so the theory seems to work.
So I think the next thing I will try is building a new recovery for OUYA from source and adding an option to reboot into "recovery" to the menu, allowing it to be stored on the boot partition and the real boot.img to be stored on the recovery partition, allowing you to boot the OUYA and it comes up in recovery, then you select the new reboot option and it would reboot into "recovery" which would load a full rom. You could then safely flash boot images to the recovery partition without fear of bricking your device.
I think it should work nicely, but not sure how long it will take me since I need to set my build machine back up.
For those who are confused, read more about how these ideas started below!
~Troop
Trooper_Max said:
I think this is a great idea that needs more attention:
mybook4 said:
Devs, I propose the following to get rid of the potential brick risk:
Since we can't get into recovery manually (via HW buttons), let's reverse the role of SOS (recovery) and LNX (kernel). Since LNX is the booting kernel partition, let's flash recovery there and flash the kernel to the recovery partition. I believe we could do this by modifying the fstab and having our updater-scripts flash to the appropriate partitions.
From a cold start, a user will enter recovery (a minor inconvenience for safety). Depending on how we modify the recovery.fstab, getting to the ROM could be as simple as pressing power twice (recovery does a reboot system now and its fstab has the system actually reboot to the recovery partition, which is the ROM's kernel).
Definitely not straightforward, but should prevent bricks. Thoughts?
Sent from my Nexus 7 using xda premium
Click to expand...
Click to collapse
But then I'm definitely biased, because I had the same idea:
Trooper_Max said:
I agree with everyone it would be nice if we had a hardware means to boot into some kind of recovery or flashing mode to flash stock images.
But I was trying to think if there is anything we can do if we don't get that... (guess that makes me a pessimist >_>)
Let me preface this with don't try anything I'm describing here unless you know what you're doing, these are half-baked thoughts of someone who doesn't even have their OUYA yet:
What if we could swap the boot and recovery partitions? (I haven't received my OUYA yet, so I'm not even aware of what partitions it has, so assuming it has them, and yeah, it wouldn't be quite that simple)
Basically what I mean is when the device tries to boot normally, it would boot up a stable recovery rom. (ie the boot partition that normally has the kernel and then loads the rest of the rom instead just has a recovery rom)
Then you have the recovery partition be what boots the full rom. That way you only chance bricking your device when you flash the boot partition, which if we can get a stable recovery rom that works for this, wouldn't be often. We would probably want to modify the recovery to have a 3 second/configurable timer where if you don't do anything it boots into the "recovery" partition which would boot up the full rom.
Basically the boot partition becomes a new recovery rom which gets used like GRUB to boot into the "recovery" partition which boots up your actual rom, or maybe it could also boot from USB or netboot or whatever...
Pretty much what we need is a solid bootloader, sound about right? Let me know if any of this makes sense/doable or if I'm entirely off base here! I don't know if I will have the time to try any of this when I do get my OUYA, but wanted to share some ideas, please proceed at your own risk!
EDIT: Alternatively, the boot partition could be left stock and the recovery partition could be used as a bootloader to boot into USB or other options for loading roms without messing with the stock boot experience and risking bricking the device. ie in this configuration if you boot your device normally it would be stock, if you reboot into "recovery" it would load up a custom rom. Or instead of a custom rom, a custom recovery with bootloader capabilities.
~Troop
Click to expand...
Click to collapse
And I do plan to try this idea out eventually, but I've got some other ideas I intend to try out first.
The first one I hope to have done by the end of this weekend (hopefully sooner though) is to reconfigure this experimental CM build to be usable without touching any of the internal storage space. I plan to have a modified boot.img that will load system/data/sdcard folders off a usb thumb drive. This method would be completely safe, though a little inconvenient because you would need to use adb to reboot the OUYA into bootloader mode so that you can then fastboot it to load the modified boot.img, but after that it could be disconnected and would be running using only external storage. You're essentially using a computer to jumpstart your OUYA into CM10, while leaving everything on the OUYA itself in pristine stock condition. Then I plan to try out arm versions of adb/fastboot so I can use my tablet or possibly phone (USB OTG) to jumpstart the OUYA instead of having to rely on my computer all the time.
I think this will be an extremely safe way of using external ROMs until we can get some kind of special bootloader figured out.
So if anyone is thinking about trying CM10, but leery of messing with their OUYA, just wait another week or so and I should have a safe, non-intrusive solution worked out!
I welcome any input/thoughts on these ideas! And if anyone knows how to do the things I'm describing and wants to beat me to the punch, feel free and run with these ideas - I won't mind as long as you share your work and give me a little credit if any credit is due!
~Troop
Click to expand...
Click to collapse
I just ordered a 64GB PNY off Amazon. Once I get it Tuesday I am going to try this. I am uploading to my mediafire now to mirror.
Trooper_Max said:
So I dug up an old nvflash package from my Notion Ink Adam and tested it to make sure it still works (with the Adam in APX mode it allows me to directly flash the device with a new partition layout and flash the partitions directly). After verifying I had it setup correctly, I swapped the boot and recovery images and ran it again and sure enough... now when I boot my Adam normally, the recovery rom comes up, and when I boot my Adam into recovery it loads the normal rom - so the theory seems to work.
Click to expand...
Click to collapse
After some more playing with my Adam, I noticed that when I reboot into "recovery" which I have booting a CyanogenMod rom and then I reboot and from the reboot menu select "Reboot", it reboots into CyanogenMod again (even though I didn't expect it to). From the reboot menu I tell it to reboot into "recovery" and it boots into CM again, which that part makes sense, since I have the CM boot image stored in the recovery partition. So I got to thinking that maybe when you tell it to reboot into recovery it sets an SOS signal (the recovery partition is sometimes refered to as SOS) and when you boot a normal rom from recovery, it never clears that signal because it isn't normally booting that way. Whereas a true recovery rom knows it needs to clear that signal so you don't get stuck booting into the recovery rom all the time.
I was close - I scanned the recovery source on CM's github and found this bit:
https://github.com/CyanogenMod/android_bootable_recovery/blob/jellybean/recovery.c said:
// Reset to normal system boot so recovery won't cycle indefinitely.
struct bootloader_message boot;
memset(&boot, 0, sizeof(boot));
set_bootloader_message(&boot);
Click to expand...
Click to collapse
So there seems to be a "bootloader message" that tells it whether to boot from the boot or recovery partition. I was going to say we need to be careful how we reboot the device and probably modify the custom rom to be able to reset the boot message, but I just realized we can do better and use this to our advantage. We could set our custom rom to always clear the boot message so that you have to go through the recovery rom each time to boot the system normally, but we could also have the option of selectively resetting the bootloader message - purposefully get ourselves stuck rebooting into "recovery" all the time because that is our normal rom, then from our normal rom have the option of resetting the bootloader message when we want to go into our recovery image stored on the boot partition.
Something to think about - I don't like getting stuck out of my recovery image either, so we gotta be sure we can reset the bootloader message when we need to... I'm going to play with this some more tomorrow, but not sure how far I will get.
EDIT: More info on the bootloader message - seems it is stored on the misc partition. The more I think about this too, the more I'm not sure swapping the boot/recovery is much safer - if we had them swapped, but then the rom gets stuck in a boot loop before it can reset the bootloader message, we'd be stuck always booting into "recovery" and into the boot loop, so we'd be in just as much trouble as if we'd gotten our boot partition to cause boot loops. So whereas normally flashing the boot partition is potentially dangerous, in this scenario flashing the recovery partition would be potentially dangerous - you gotta be sure it's going to be able to reset the bootloader message. We could try to mitigate this by modifying the boot process to reset the bootloader message very early in the boot process (ie before the potential bootloop) so that if we get stuck, the next time we reboot, we'll reboot normally into the recovery rom.
But then we could also do just the opposite of that without swapping the boot/recovery partitions - very early in the boot process, modify the bootloader message to tell the bootloader to boot into recovery - that way if we get stuck the next time we boot up will be into recovery. Then later when the system successfully boots, we could reset the bootloader message so that after a successful boot the next boot will be another normal one instead of into recovery. If we get this right a failed boot would automatically take us to recovery on the next boot, while a successful boot would boot normally on the next boot. So this would be a way to build in some "brick-protection" into the boot.img to make it safer to flash. It'd probably be safest to not reset the bootloader message until the user actually selects the shutdown option - that way any abnormal reboot would cause it to come up into recovery mode.
So I'm not sure swapping boot/recovery partitions really buys us anything anymore, since it also swaps which partition is potentially dangerous to flash. The solution in either case is going to be to cleverly manipulate the bootloader message as described above. So we might just want to not swap them to minimize confusion and focus instead on building "brick-protection" like this into our boot images.
https://github.com/CyanogenMod/android_bootable_recovery/blob/jellybean/bootloader.h said:
/* Bootloader Message
*
* This structure describes the content of a block in flash
* that is used for recovery and the bootloader to talk to
* each other.
*
* The command field is updated by linux when it wants to
* reboot into recovery or to update radio or bootloader firmware.
* It is also updated by the bootloader when firmware update
* is complete (to boot into recovery for any final cleanup)
*
* The status field is written by the bootloader after the
* completion of an "update-radio" or "update-hboot" command.
*
* The recovery field is only written by linux and used
* for the system to send a message to recovery or the
* other way around.
*/
struct bootloader_message {
char command[32];
char status[32];
char recovery[1024];
};
/* Read and write the bootloader command from the "misc" partition.
* These return zero on success.
*/
int get_bootloader_message(struct bootloader_message *out);
int set_bootloader_message(const struct bootloader_message *in);
Click to expand...
Click to collapse
kairnage said:
I just ordered a 64GB PNY off Amazon. Once I get it Tuesday I am going to try this. I am uploading to my mediafire now to mirror.
Click to expand...
Click to collapse
Sounds nice - I'm tempted to get a nice big new thumb drive myself... I don't have any that big >_>
~Troop
I've been following your think tank and I agree with your latest idea 100%
Sending a message for bootloader to always boot into recovery if the system wasn't shut down normally is clean and simple.
I am curious though what about development, with nvflash requiring encryption of commands first mistake will render your ouya unusable.
dexter84 said:
I've been following your think tank and I agree with your latest idea 100%
Sending a message for bootloader to always boot into recovery if the system wasn't shut down normally is clean and simple.
I am curious though what about development, with nvflash requiring encryption of commands first mistake will render your ouya unusable.
Click to expand...
Click to collapse
Glad you agree with my new idea!
As far as development, yeah, that is why I used my Notion Ink Adam for some of the testing that I did.
As far as developing for OUYA, I don't think we are limited by not having nvflash, we just have to be more careful. Personally, if I were developing anything, I would use my above methods to fastboot boot images that load everything else from a thumb drive at least until whatever I'm developing becomes stable enough that I want to use it more permanently. Then I would be very careful about how I flash it.
Heck, if you wanted to be really safe, you could flash recovery images to both the boot and recovery partitions and just use external boot methods
In the long term though, I hope we can develop a bootloader based on the kexec hardboot patches for the original Nexus 7, which would allow booting completely off boot images stored in other locations, loading partitions from other locations. Other locations could even just be the virtual sd card if you don't want to use external media, but at that point we'd have a safe extra bootloader stored on the boot partition.
http://forum.xda-developers.com/showthread.php?t=2104706
Hope that makes sense.
~Troop
I was not aware that any content of RAM can survive a reboot, hard kexec booting custom OS from USB or image is definitely a safe solution, looks like all the pieces are there.
Wish I had something more than ouya alone, since you have your Adam are you going to try to develop something ?
dexter84 said:
I was not aware that any content of RAM can survive a reboot, hard kexec booting custom OS from USB or image is definitely a safe solution, looks like all the pieces are there.
Wish I had something more than ouya alone, since you have your Adam are you going to try to develop something ?
Click to expand...
Click to collapse
I do intend to, but I don't have a great deal of time to devote to Android development, so not sure how long it will take me. I'm trying to find what time I can though. This past weekend/last week I had a bit more than usual.
~Troop
Trooper_Max said:
http://troopermax.com/releases/ouya-sda123-boot.zip
(be gentle - it's 145 MB, I use shared hosting, so feel free to mirror)
md5sum: 49c8e16e27b6deb9d1e8e86363b56f2f
~Troop
Click to expand...
Click to collapse
Here is a mirror for you.
http://www.mediafire.com/download/hban76kzeys6ybd/ouya-sda123-boot.zip
I tried this on my new 64GB and it wouldn't boot, but it's a brand new card so it might be the problem. I'll try a known good card later.
kairnage said:
Here is a mirror for you.
http://www.mediafire.com/download/hban76kzeys6ybd/ouya-sda123-boot.zip
I tried this on my new 64GB and it wouldn't boot, but it's a brand new card so it might be the problem. I'll try a known good card later.
Click to expand...
Click to collapse
Sorry you had trouble - a couple other things to be wary of:
Have only the one thumb drive connected, at least until it boots - the boot.img looks for the data partition as sda1, the system partition as sda2, and the cache partition as sda3. If some other storage device is connected, it might be labeled sda by the OUYA depending on the order it finds them in and then the thumb drive might become sdb or so on.
It worked for me through a hub, but it may also be worth trying it directly if you have trouble.
Guessing you did this, but give it some time, booting from thumb drive especially on the first boot is slower. It's not too long, but it was long enough that for a moment I thought I'd failed again. Of course the thumb drive I was using wasn't built for speed, so YMMV.
If you're still having trouble - even on my failed attempts at boot images for this (it took me 5 tries at getting the ramdisk to get it bootable), I was still able to connect to it using adb. I ran adb on Linux, running adb as root to bypass all that udev configuration stuff - it's just easier, and if the OUYA/device failed to mount the system partition, it might not match the configuration you expect anyway. If you're able to run "adb shell" but it gives you a permission error about not being able to access the shell binary on the system partition, that probably means you didn't setup the system partition correctly - it's crucial that it get set up with the right permissions, which is why I provided the image so it could just be written directly with dd.
I'm going to try to put together a recovery image that can partition the thumb drive for you and flash an arbitrary update.zip to the thumb drive, but not sure when I will be able to have that ready. The initial version will not be able to patch the boot.img though, so that will still need to be modified like the one I have provided, but maybe in a future version I can get it to also modify the boot.img for loading the information from the thumb drive.
Let me know if you continue to have issues or if you do get it working! And if anyone else has got it working, please post so I can at least know someone got it working. It worked for me, and I don't see why it shouldn't work for anyone else, but if there is something conflicting I want to figure it out.
EDIT: I just remembered when I was looking into the swapping boot/recovery stuff, I noticed in the recovery source that there already are some tools that might be able to help partition the thumb drive and fix permissions included in the recovery source:
https://github.com/CyanogenMod/android_bootable_recovery/tree/jellybean/utilities
Tonight I'll see if I can put together some new instructions using those tools that might be easier and less error-prone. I'm hoping those tools can be applied to the thumb drive anyway, I'll try and figure that out tonight.
~Troop
Trooper_Max said:
Sorry you had trouble - a couple other things to be wary of:
Have only the one thumb drive connected, at least until it boots - the boot.img looks for the data partition as sda1, the system partition as sda2, and the cache partition as sda3. If some other storage device is connected, it might be labeled sda by the OUYA depending on the order it finds them in and then the thumb drive might become sdb or so on.
It worked for me through a hub, but it may also be worth trying it directly if you have trouble.
Guessing you did this, but give it some time, booting from thumb drive especially on the first boot is slower. It's not too long, but it was long enough that for a moment I thought I'd failed again. Of course the thumb drive I was using wasn't built for speed, so YMMV.
If you're still having trouble - even on my failed attempts at boot images for this (it took me 5 tries at getting the ramdisk to get it bootable), I was still able to connect to it using adb. I ran adb on Linux, running adb as root to bypass all that udev configuration stuff - it's just easier, and if the OUYA/device failed to mount the system partition, it might not match the configuration you expect anyway. If you're able to run "adb shell" but it gives you a permission error about not being able to access the shell binary on the system partition, that probably means you didn't setup the system partition correctly - it's crucial that it get set up with the right permissions, which is why I provided the image so it could just be written directly with dd.
I'm going to try to put together a recovery image that can partition the thumb drive for you and flash an arbitrary update.zip to the thumb drive, but not sure when I will be able to have that ready. The initial version will not be able to patch the boot.img though, so that will still need to be modified like the one I have provided, but maybe in a future version I can get it to also modify the boot.img for loading the information from the thumb drive.
Let me know if you continue to have issues or if you do get it working! And if anyone else has got it working, please post so I can at least know someone got it working. It worked for me, and I don't see why it shouldn't work for anyone else, but if there is something conflicting I want to figure it out.
EDIT: I just remembered when I was looking into the swapping boot/recovery stuff, I noticed in the recovery source that there already are some tools that might be able to help partition the thumb drive and fix permissions included in the recovery source:
https://github.com/CyanogenMod/android_bootable_recovery/tree/jellybean/utilities
Tonight I'll see if I can put together some new instructions using those tools that might be easier and less error-prone. I'm hoping those tools can be applied to the thumb drive anyway, I'll try and figure that out tonight.
~Troop
Click to expand...
Click to collapse
I could get into ADB as well but was only seeing the stock system. Actually now I think it was the adapter I was using. At first I assumed it was the microsd because it would hang on boot. But after trying a known good card it did the same. I remembered having issues with an adapter that has the indicator light to show it was connected on a tablet before. I tried a generic Kingston adapter and it sees the cards now. Of course I had already wiped them back to stock partitions. LOL Oh well, start again tomorrow.
For anyone who doen't have a Linux box handy or an extra thumb drive to boot a live USB, here are some quick instructions on how to partition the thumb drive from the OUYA itself. You can connect the thumb drive either before or after booting, it doesn't really matter. I'm using all fastboot/adb commands here so they are easy to recognize. If you're knowledgeable about how to use adb/fastboot, feel free to execute the commands however you're comfortable.
Type all the adb/fastboot commands as specified - if it doesn't begin with adb or fastboot, it isn't a command. All /dev paths you type into the commands should be related to /dev/block/sda (your thumb drive), not any internal partitions or you could mess up your OUYA!!!
Boot up the OUYA normally, then reboot into the bootloader:
adb reboot bootloader
Boot CWM using fastboot:
fastboot boot OuyaCWMrecovery6.0.3.2.img
Print partition information for the thumb drive:
adb shell parted /dev/block/sda print
Model: USB Flash Memory (scsi)
Disk /dev/block/sda: 8128MB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Number Start End Size Type File system Flags
1 32.3kB 6000MB 6000MB primary ext4
2 6000MB 7000MB 1000MB primary ext4
3 7000MB 8000MB 1000MB primary ext4
Click to expand...
Click to collapse
(note in the example output above I already have it partitioned)
Here I was using an 8 GB thumb drive, you can see the size where it says 8128MB.
Delete Partitions from thumb drive: (be sure you don't have anything important on your thumb drive)
adb shell parted /dev/block/sda rm 1
adb shell parted /dev/block/sda rm 2
adb shell parted /dev/block/sda rm 3
(note that you only need to delete partitions that showed up in the table when you printed it, here I deleted 1,2,3, since I had 3 partitions above and the parted commands will say you may need to update your fstab, but that is not relevant here so don't worry about it)
Create 3 primary partitions (adjust the sizes as necessary):
adb shell parted /dev/block/sda mkpart primary 1 6000
adb shell parted /dev/block/sda mkpart primary 6000 7000
adb shell parted /dev/block/sda mkpart primary 7000 8128
Check your paritions again:
adb shell parted /dev/block/sda print
Format your partitions as ext4:
adb shell mke2fs -t ext4 /dev/block/sda1
adb shell mke2fs -t ext4 /dev/block/sda2
adb shell mke2fs -t ext4 /dev/block/sda3
Push the cm10 system image onto the device: (this will take a little while)
adb push ouya-cm10-system.img /tmp/ouya-cm10-system.img
Check that it copied correctly:
adb shell ls -l /tmp
-rw-rw-rw- 1 root root 260308992 Jul 27 2013 ouya-cm10-system.img
-rw-rw-rw- 1 root root 3587 Jan 1 02:12 recovery.log
Click to expand...
Click to collapse
(Note the size... I'm pretty sure copying it /tmp put it into the ramdisk, but our OUYA has enough ram for this)
Write the cm10 system image onto /dev/block/sda2: (this will take a little while)
adb shell dd if=/tmp/ouya-cm10-system.img of=/dev/block/sda2 bs=4M
Check/fix the filesystem:
adb shell e2fsck -fp /dev/block/sda2
Resize the filesystem to fill the partition:
adb shell resize2fs /dev/block/sda2
(Unfortunately, it looks like resize2fs is not included in CWM, so this step would need to be done on the computer)
At this point, the thumb drive should be good, so you just gotta reboot back to the bootloader and boot the boot image:
Reboot to the bootloader again:
adb reboot bootloader
Boot the modified boot image I provided:
fastboot boot ouya-sda123-boot.img
Not sure if that's necessarily easier or harder to follow than the other instructions... I'd probably rather do the partitioning on a desktop with GUIs, but for some who don't have a Linux box handy it might be easier to use the OUYA as a little Linux box
I'm sure you noticed this method still used dd to write the system image, just run from the OUYA instead. I looked into just extracting the system files from the CM10 zip onto the OUYA, but fixing the permissions was not as easy as I thought it would be - the /system partition permissions are actually fixed by the updater script as referred to here:
http://fokke.org/site/content/howto-create-android-updatezip-package
It's actually a scripting language and the script also creates symlinks and does other stuff.
You'd have to extract/modify the updater script to not use the internal partitions and then execute it (I think using the edify program)... still less straightforward than using dd if you have the image handy like I gave you... But I may look into this as another option. The knowledge will certainly be relevant when I look to build a custom CWM recovery to allow you to flash zips to the thumb drive instead of the internal partitions.
~Troop
Thanks to csonger for feedback, which may be handy if you have system partition issues:
csonger said:
I tried both methods (under linux and with ouya as well), but neither of them worked for me. My 8 gig pen couldn't boot.
The first thing is that the system image is 256MB and the tutorial says the System partition needs 512MB or 1GB.
After the boot failures I checked the pendrive. The second (system) partition was 256MB. It is ok when we use dd, but it was full. 100% usage.
First I run the e2fsck command to check for filesystem problems and it found some inode issues on it.
Following the fs fix the resize2fs command did the job and I had got a 1 gig System partition.
After that my pendrive could boot the cyanogenmod image without problems.
May be someone else has the same problem as me. Please share with them.
Thanks for your work:
csonger
Click to expand...
Click to collapse
I didn't have those issues myself for some reason, but I'll investigate and see what instructions I need to update!
Thanks again for the feedback!
~Troop
Finally got my Linux partition back up and running so I can give this a go again.
It's been trying to boot CM for about 15 minutes now, ADB does see it, but I think it might no like the partitions I did.I did 2GB for system and 1GB for cache. I will repartition and try again.
kairnage said:
It's been trying to boot CM for about 15 minutes now, ADB does see it, but I think it might no like the partitions I did.I did 2GB for system and 1GB for cache. I will repartition and try again.
Click to expand...
Click to collapse
That should be fine, I would think... If you continue to have trouble, try running e2fsck and resize2fs on the system partition, as csonger suggested. (In case you didn't already)
I was confused when I came up with the instructions that it worked without me doing resize2fs since I figured that would be needed to expand the filesystem to fill the partition, but somehow it worked anyway and even looked like it was expanded, but I'd probably just forgot to refresh my view after dd.
Note that you can run the commands with just the partition as an argument: (I added -fp to e2fsck to automatically find/fix problems)
e2fsck -fp /dev/sdX2
resize2fs /dev/sdX2
(Where X is the letter for your thumb drive)
Without a size specified, resize2fs resizes the filesystem to fill the partition.
Hope that helps!
EDIT: I went back and fixed the instructions with this addition now that I've tested it a little more. Unfortunately, I noticed that CWM does not include resize2fs and I didn't spot any other programs handy within CWM to do the trick, so you can't quite do this part on the OUYA. I will look into alternate instructions or building my own CWM that includes resize2fs or just plain automates this more.
~Troop
Ever since I bought my Xperia TL (LT30at), I've been upset at the ridiculous allocation of onboard memory: only 1.97GB were available for apps. The rest was allocated to an emulated external SD card, which meant that any app installed on it would lose widgets and suffer from other weird glitches on reboot. I've searched all over the place and couldn't find a solution until @teidus shared his method (per my research, it has been used on older devices from other manufacturers as well). I tried it and adjusted the steps to ensure 100% compatibility with my phone (and added explanations so that other devices could be tested).
See proof in the attached screenshots.
WARNING: You will lose a lot of data if you don't follow this guide!
WARNING: These operations require a rooted device. If you are running unrooted Android 4.3 or newer, you cannot proceed.
NOTE: All of these changes will be reversed if you flash an FTF that includes partition-image.sin.
DISCLAIMER: I'm not responsible for bricked devices and data loss from power surges, hardware damage, or changes to the linked software. Always check the native manuals for all the tools listed here!
Required Downloads:
1. Flashtool, which includes ADB (got questions about it? - ask them here) - requires installation
2. @[NUT]'s dual XZDualRecovery. I'm linking to v2.7.94 BETA installer. If you wish to check for a more recent version, go to his download page and do a page search (CTRL+F) for XT-lockeddualrecovery[version-goes-here].installer.zip. (got questions about it? - ask them here) - unzip to a location you'll remember (say, C:\xperia-resize\recovery\)
3. FTF with your phone's 4.1.2 firmware - save it to the default Flashtool firmware directory (C:\flashtool\firmwares\)
4. File system utilities (attached) - unzip to a location on your phone's physical SD card you'll remember (say, fs-utils\)
The Guide (skip all the way to the bottom for the short guide if you know how to backup and flash FTFs):
1. Back up all data stored on internal storage:
Option 1: Connect your phone to the computer and save it to a computer drive.
Option 2: Open a file manager on the phone and copy it to a physical SD card (/sdcard1).2. Back up your current installation with either Online Nandroid or your current recovery and remove the SD card for safety.
3. Flash your phone with basic 4.1.2 install.
3.1 Turn off the phone.
3.2 Launch Flashtool.
3.3 Click the thunderbolt icon on the left and select Flashmode.
3.4 Select the appropriate 4.1.2 FTF from the list (it'll be there if you saved it to c:\Flashtool\firmwares\).
3.5 Tick "No final verification" checkbox on the right, click Flash, and when prompted, hold the VOLUME DOWN button, and connect the phone to the computer. The operation will take about 10 minutes. DISREGARD THE INSTRUCTIONS IN THE ANIMATION. Xperia T/TL doesn't have a hardware Back button.
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
3.6 When flashing is complete, boot the phone.
4. Connect your phone to the computer and let the computer recognize it.
5. Enable USB debugging.
5.1 Go to (System) Settings -> Developer options and flip the switch in the top-right corner of the screen.
5.2 Look a bit down (toward the middle of the screen) and check USB debugging.6. Install the dual recovery.
6.1 Go to the location on your computer where you extracted dual recovery (e.g., C:\xperia-resize\recovery\).
6.2 Double-click install.bat.
6.3 Press 3 on your keyboard for the option 3. Installation on unrooted ROM.
6.4 Follow the instructions on your computer's screen.
6.5 When you will try to reboot the phone back into Android, the recovery will prompt you to root the OS. Do that.
7. Boot into PhilZ Touch recovery by opening App Drawer, starting NDR Utils, and selecting the appropriate option.
8. When the phone boots into recovery, connect it to the computer.
9. Start ADB.
If you don't know where ADB is, press WIN + R on your computer keyboard, type cmd in the pop-up window, hit ENTER, then copy & paste this command (paste by right-clicking the command prompt window): cd C:\Flashtool\x10flasher_lib. Then type adb shell and hit ENTER again.10. Execute the following commands one-by-one (you can copy & paste, hit enter, and wait for the output):
Code:
umount /storage
parted /dev/block/mmcblk0
print
These commands unmount all drives, launch parted utility for mmcblk0 (onboard flash storage), and show the partition table.11. Record the beginning of the partition Userdata and the end of the partitionSDCard. These are the limits of your new partitions The numbers are in megabytes (MB) unless specified.
For Xperia T/TL, Userdata begins at 1621 and SDCard ends at 15.6GB. Xperia V (and any other device with less than 16GB of storage) will definitely have a different value.
12. Remove the last 2 partitions by executing the following commands:
Code:
rm 14
rm 15
13. Calculate the new end point for Userdata, which is also the new start point for SDCard:
1621 (or another number you recorded in step 11) + additional space = new end. I added 4GB (4 ×1024MB = 4096MB) so my new end point was 7865. I advise to add at least 2GB (2048MB) to feel safe about having enough storage for apps. DON'T TRY TO REMOVE SDCARD ALTOGETHER BY ALLOCATING ALL OF THE AVAILABLE SPACE TO USERDATA! Emulated storage is coded into OS environment and requires additional modifications to work properly.14. Create and name new partitions by executing the following commands (don't copy the text in parentheses):
The system might throw several errors in the process. When prompted, just type ignore and hit ENTER.
Code:
mkpartfs logical ext2 1621 7865 (change the numbers as appropriate for your device)
name 14 Userdata
mkpartfs logical ext2 7865 15.6GB (change the numbers as appropriate for your device and notice the GB to make sure the partition extends to the end of the onboard storage)
name 15 SDCard
quit
These commands create the partitions with ext2 file system and given beginning and end parameters, rename them as required, and close parted.15. "Damage" the internal, emulated SD card so that it could be formatted properly later.
Tap Mounts and storage -> Format /storage/sdcard0 -> default and confirm the operation.16. Insert the physical SD card (sdcard1) that you removed in step 2 back into the device.
Make sure it's mounted by tapping Mounts and storage and checking its status (it should say unmount /storage/sdcard1/). If not, mount it by tapping on the appropriate option.17. Execute the following commands one-by-one in the ADB command prompt:
Code:
cd /storage/sdcard1/fs-utils (the directory name will be different if you didn't follow my conventions)
tune2fs -j /dev/block/mmcblk0p14
tune2fs -O extents,uninit_bg,dir_index /dev/block/mmcblk0p14
e2fsck -fpDC0 /dev/block/mmcblk0p14
These commands convert Userdata partition into the appropriate ext4 filesystem, which is not supported natively in the recovery environment.18. Reboot the device into full Android.
18.1 Remove the physical SD card for safety.
18.2 Tap Back until you make it to the top level of recovery. Then tap the top option Reboot the device.19. When Android loads fully, you'll notice a notification about storage file system. Tap it and follow the prompts to format it.
20. Insert the physical SD card and reboot into the recovery you used to create the backup in step 2 to restore it.
In PhilZ, that's Backup and Restore -> Restore from /storage/sdcard1.21. Reboot back into full Android and copy the files saved from internal storage back into it. Then reboot again so that apps recognize the data.
DONE!
Short guide/list of ADB commands and other events (*enclosed in asterisks*) if you know the safety drill:
Code:
*flash 4.1.2 & install dual recovery*
*boot into PhilZ Touch*
adb shell
umount /storage
parted /dev/block/mmcblk0
print
*record start point of Userdata and end point of SDCard*
rm 15
rm 14
mkpartfs logical ext2 1621 7865
name 14 Userdata
mkpartfs logical ext2 7865 15.6GB
name 15 SDCard
*Mounts and storage -> Format /storage/sdcard0 -> default -> confirm*
*insert physical SD card*
*Mounts and storage -> Mount /storage/sdcard1*
cd /storage/sdcard1/fs-utils
tune2fs -j /dev/block/mmcblk0p14
tune2fs -O extents,uninit_bg,dir_index /dev/block/mmcblk0p14
e2fsck -fpDC0 /dev/block/mmcblk0p14
*remove physical SD card*
*reboot*
*format emulated SD card*
*reboot recovery*
*restore backup*
*reboot*
*restore sdcard0/*
*reboot*
nice, i was looking for a way to increase the /data after switching to Omni and ART
Will try ASAP..
PS: i think this thread should be added in ALL IN ONE THREAD Section:good::victory::victory::victory:
raghulrr said:
PS: i think this thread should be added in ALL IN ONE THREAD Section
Click to expand...
Click to collapse
That's a request to @gregbradley.
#6 would be the most welcomed way to resize partitions, but unfortunately it's not so easy as it was e.g. with ZTE Blade. But altogether it's good to know that you can resize them now! Thanks!
nice tutorial brother... i'll be trying tommorow
Sent from S5J using Tapatalk
Thanks
Thank you I was definetly looking for it
Thanks
@TO: I couldn't test ist but I think made a little mistake:
I think you want want to execute the programms wich are in the Directory in wich you made the cd. So you have to add ./ before each command.
I don't have such a device, to test it by myself: You said you have to make a new virtual SD. Some ROMes have a vold.fstab in /etc. Mayby it't enough to change this file to be able to remove the SD Partition completly.
Partitionator said:
@TO: I couldn't test ist but I think made a little mistake:
I think you want want to execute the programms wich are in the Directory in wich you made the cd. So you have to add ./ before each command.
I don't have such a device, to test it by myself: You said you have to make a new virtual SD. Some ROMes have a vold.fstab in /etc. Mayby it't enough to change this file to be able to remove the SD Partition completly.
Click to expand...
Click to collapse
I was writing the guide as I was executing commands so everything is exactly as I entered it. I obviously removed all steps that turned out to redundant and wrong.
Will that work with the Xperia Arc S?!
What is the difference between your method and the [MOD]{CM10.1}{CM10.2}{CM11}{AOSP}Working Int2Ext for Xperia 2011 [ART-compatible]?
metaljoker said:
Will that work with the Xperia Arc S?!
What is the difference between your method and the [MOD]{CM10.1}{CM10.2}{CM11}{AOSP}Working Int2Ext for Xperia 2011 [ART-compatible]?
Click to expand...
Click to collapse
These are some great questions! This approach is very different from all Int2Ext scripts:
HUGE NOTE: This works only for devices where the onboard NAND capacity is at least 8GB. That means that 2011 devices with 1-4GB of onboard memory are better off with mounting the physical SD card as /data.
This is a semi-permanent modification to the partition table (it'll work fine if you do factory reset and flash a new ROM .zip but the allocation will be reset every time you flash FTF, possibly even if you exclude partition-image because each .sin contains its offset and size bits).
This approach is compatible with locked bootloaders because support of init.d comes from the kernel.
This approach allows you to allocate as much or as little of onboard NAND storage to each of the 2 partitions (/data and /storage/sdcard0) used for storing apps and their data instead of just allocating the entire physical SD card.
This approach allows you to live a life independent of your physical SD card (i.e., your apps still work and your phone boots up just fine even if the physical SD card dies).
bsined said:
These are some great questions! This approach is very different from all Int2Ext scripts:
HUGE NOTE: This works only for devices where the onboard NAND capacity is at least 8GB. That means that 2011 devices with 1-4GB of onboard memory are better off with mounting the physical SD card as /data.
This is a semi-permanent modification to the partition table (it'll work fine if you do factory reset and flash a new ROM .zip but the allocation will be reset every time you flash FTF, possibly even if you exclude partition-image because each .sin contains its offset and size bits).
This approach is compatible with locked bootloaders because support of init.d comes from the kernel.
This approach allows you to allocate as much or as little of onboard NAND storage to each of the 2 partitions (/data and /storage/sdcard0) used for storing apps and their data instead of just allocating the entire physical SD card.
This approach allows you to live a life independent of your physical SD card (i.e., your apps still work and your phone boots up just fine even if the physical SD card dies).
Click to expand...
Click to collapse
That's a great info! There are steps in the thread that aren't clear enough not only for me but also for others, that's for sure.
For those who have partitioned their SDcard, unlocked BL & rooted phones, should have their own line of steps, however I feel like I might mistake something in case I use it.
P.S.:
01. You didn't answer my question whether this is compatible with my phone, did you?!
02. There should be a video demonstration for this method for it is new in this website, don't you think?!
03. unzip to a location on your phone's physical SD card you'll remember (say, fs-utils\) what do you mean by remember "say, fs-utils\" ??
@up
It depend on your partition scheme, which ones are on the end of memory (physical).
Wysłane z mojego GT-I8160 przy użyciu Tapatalka
batlin said:
@up
It depend on your partition scheme, which ones are on the end of memory (physical).
Wysłane z mojego GT-I8160 przy użyciu Tapatalka
Click to expand...
Click to collapse
I have a 1GB partitioned SDcard for the apps! Does this info help?!
How the heck could I install the TeamWin Recovery Project (TWRP) ??!!
metaljoker said:
That's a great info! There are steps in the thread that aren't clear enough not only for me but also for others, that's for sure.
For those who have partitioned their SDcard, unlocked BL & rooted phones, should have their own line of steps, however I feel like I might mistake something in case I use it.
P.S.:
01. You didn't answer my question whether this is compatible with my phone, did you?!
02. There should be a video demonstration for this method for it is new in this website, don't you think?!
03. unzip to a location on your phone's physical SD card you'll remember (say, fs-utils\) what do you mean by remember "say, fs-utils\" ??
Click to expand...
Click to collapse
SD partitioning and status of the bootloader should not have any effect on the process. However, root is required in order to install recovery.
1. Theoretically, this mod is compatible with any device that has the latest version of CWM or TWRP. I've tested it with Xperia TL and Nook Tablet 16GB. However, if your device's total available user storage (/data + /storage/sdcard0) is 4GB or less, you will likely require a physical SD card for storing your apps, user data, and whatever is written to sdcard0 by default.
2. Although I realize that some people learn best by watching instead of reading, this procedure is very quick when your device's recovery supports all the required commands. I repartitioned Nook Tablet in barely 10 minutes. If I were to make a screen cast of it, 90% of the video would be just moving the mouse around or waiting for the phone to boot. The longest operation is to create the journal, which takes about 10 seconds by the phone.
3. "Say" = "for example". Create a folder on the physical SD card and call it "fs-utils".
bsined said:
SD partitioning and status of the bootloader should not have any effect on the process. However, root is required in order to install recovery.
1. Theoretically, this mod is compatible with any device that has the latest version of CWM or TWRP. I've tested it with Xperia TL and Nook Tablet 16GB. However, if your device's total available user storage (/data + /storage/sdcard0) is 4GB or less, you will likely require a physical SD card for storing your apps, user data, and whatever is written to sdcard0 by default.
2. Although I realize that some people learn best by watching instead of reading, this procedure is very quick when your device's recovery supports all the required commands. I repartitioned Nook Tablet in barely 10 minutes. If I were to make a screen cast of it, 90% of the video would be just moving the mouse around or waiting for the phone to boot. The longest operation is to create the journal, which takes about 10 seconds by the phone.
3. "Say" = "for example". Create a folder on the physical SD card and call it "fs-utils".
Click to expand...
Click to collapse
WOW! I am very impressed for answering each question in an informative way! W E L L D O N E! ! :good:
I'll take my chances since I've read your lines. Wish me luck!
I'll get back to you once I encounter the unexpected, or maybe not. We'll see! Here goes nothing...
metaljoker said:
WOW! I am very impressed for answering each question in an informative way! W E L L D O N E! ! :good:
I'll take my chances since I've read your lines. Wish me luck!
I'll get back to you once I encounter the unexpected, or maybe not. We'll see! Here goes nothing...
Click to expand...
Click to collapse
I realized that I've forgotten to include roll-back instructions if anything goes wrong. Here's the short version:
1. You better have made Nandroid backup of your system and have copied the content of your /storage/sdcard0
2. Find a stock, rootable FTF on the forums, download it, and flash it using Flashtool
3. Install recovery on the reflashed device
4. Restore backup from recovery and copy the content back to /storage/sdcard0
ADB Shell
Sorry if this silly questions
I want use your mod on my XV, but I've some problem (on Windows 8.1 Pro 64bit)
1. When I reboot from PhilZ Touch recovery for a first time, i don't get root question
2. Cause i don't get about root question, so i try root from PhilZ Touch recovery menu
3. When I try run adb shell on TWRP recovery, there no device notif (I already install Windows Update : http://support.microsoft.com/kb/2917929 and use Flashtool Driver 1.5)
So, am i mis something? Where i got wrong? Thanks before
thunder888 said:
Sorry if this silly questions
I want use your mod on my XV, but I've some problem (on Windows 8.1 Pro 64bit)
1. When I reboot from PhilZ Touch recovery for a first time, i don't get root question
2. Cause i don't get about root question, so i try root from PhilZ Touch recovery menu
3. When I try run adb shell on TWRP recovery, there no device notif (I already install Windows Update : http://support.microsoft.com/kb/2917929 and use Flashtool Driver 1.5)
So, am i mis something? Where i got wrong? Thanks before
Click to expand...
Click to collapse
I guess is the problem is with Win8 and ADB drivers. However, just to be sure, try attaching the phone to the computer to test ADB when it's booted to PhilZ and full Android. If the phone is not recognized, search the forum for a fix (there should be one). Unfortunately, my only Win8 machine is Surface RT, which doesn't accept ADB drivers, so I can't test anything myself.
bsined said:
I guess is the problem is with Win8 and ADB drivers. However, just to be sure, try attaching the phone to the computer to test ADB when it's booted to PhilZ and full Android. If the phone is not recognized, search the forum for a fix (there should be one). Unfortunately, my only Win8 machine is Surface RT, which doesn't accept ADB drivers, so I can't test anything myself.
Click to expand...
Click to collapse
Yes, i agree. But if I attach phone at full Android and run ADB, it can be recognized. I'll reinstall my windows and will give a report later. Thanks :good:
My dearest friend...
My phone was revered back to original. In order to have it rooted it took me 17 hours
1] Regarding the "Flash your phone with basic" my phone's already rooted which means that I do not need to flash an FTF file. In case I need to, which version?! Would that harm anyhow?
2] I'm stopped at 6.5. Because I do not have the TWRP installed on my phone.
3] My Recovery is CWM-based Recovery v6.0.4.7, if no problem then how can I Start ADB?!!!
Obviously, I'm not even half way. I need your help my friend!
* Xperia Arc S
**DISCLAIMER**
**I WILL NOT BE RESPONSIBLE IF YOUR PHONE DIES.**
**PROCEED AT YOUR OWN RISK**
**IF YOU MESS UP, USE SP FLASH TOOLS TO FLASH ROM IN THE FORMAT ALL+ DOWNLOAD MODE**
A lot of space in the internal storage in sprout4 devices is wasted. This will give you a bit more space in userdata. I put this guide in Android One General, but this will apply to most devices. At least to those devices that use GPT partitions like the Sprouts do.
Requirements:
An Android device (duh)
TWRP Materialized by musfiqus, from here: http://forum.xda-developers.com/cro...p-materialised-twrp-recovery-android-t3088715 It includes the Parted binary. Some other builds of TWRP don’t have that. You need it.
Something resembling intelligence
Patience
A PC (Windows , Linux, Mac, whatever) is optional. The instructions here use a PC, but if you don’t have one, you can type commands in TWRP terminal).
ADB drivers, if you’re using a PC.
A hardware calculator. Using an on-screen calculator so many times is going to get you to go Hulk.
Here’s how to do it.
Reboot to TWRP recovery, and open a terminal / Command Prompt. Take a backup of IMEI.
Type:
adb shell
You’ll get a screen like this:
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
Type :
parted /dev/block/mmcblk0
By default, Parted uses MB as the storage unit. To prevent possible unused space after repartitioning, we’ll use sectors as a unit instead.
Type
unit s
This’ll change to sectors.
Type
print
It’ll give a warning, just type
i
Then:
print free
At the top, you’ll see that the sector size is written. Write this number down somewhere. For my Android One 4GB, the sector size is 512 bytes.
Now, you need to understand what the list means. Each horizontal row shows the details of a partition.
The 1st column shows the partition number.
The 2nd column shows the start offset of that partition. That means that the partition starts at the location mentioned.
The 3rd column shows where the partition ends. Notice that each partition starts exactly 1 sector after the previous one ends.
The 4th column is obviously the size of the partition.
The 5th column is the file system used by the partition. If nothing is written in this column, that means that it’s a binary partition.
The 6th column is the partition name.
You’ll see that the sizes in that list are weird. They’re not in any standard unit you might know. That’s because we used sectors instead of megabytes. The ‘s’ after each number indicates that it’s in sectors. (You can use the default MB unit (1MB=1000 KB. 1KB=1000bytes), or the MiB unit (1MiB=1024 KiB), but that just might leave 1 or 2 MB of space unused. So, I’m using sectors).
Remember that 1 sector = 512 bytes for my phone.
There’s some free space at the top and bottom of the list. Leave that free space there. Do not make partitions using those.
To convert sectors to MiB or KiB:
1s = 512bytes (Use the sector size you wrote down previously for this step, it might not be 512 bytes for you)
1024 bytes = 1 KiB
1024 KiB = 1 MiB
1024 MiB = 1GiB
So, 4833280s = (4833280 x 512) B = 2474639360 B
= (2474639360 / 1024) KiB = 2416640 KiB
= (2416640 /1024) MiB = 2360 MiB
= (2360 / 1024) GiB = 2.30 GiB
We’ll use another terminal window with sizes in MiB now. So open another Parted prompt in a new terminal / command prompt window, but instead of
unit s,
this time, write
unit MiB
Type “print”, “i”, and “print free” again.
Look at my 11th partition. Its size is 8MiB. I know that this logo partition doesn’t need more than 2 MiB. So, I’ll make it smaller.
When you make partitions smaller, all the data inside will be lost. So, we need to back up the partitions first.
Open a 3rd terminal window. Type
adb shell dd if=/dev/block/mmcblk0p11 of=/microSD/p11
The “dd” command copies bytes from the “if=” location, to the “of=” location. The internal storage is /dev/block/mmcblk0. The “p11” after that refers to the partition we are backing up. Notice that in the Parted list, “logo” has a partition number of “11”. So the general command to back up partitions is
adb shell dd if=/dev/block/mmcblk0p<partition_number> of=/microSD/p<partition_number>
From recovery, unmount all partitions except microSD and oem. Then back up partitions 11, and 13 from PC. We will copy the files from OEM instead of using dd. So type
adb shell mkdir /microSD/oem
adb shell cp –a /oem/ /microSD/oem
Go to /microSD/oem/oem/app with TWRP’s file manager, and delete everything there.
Open the 1st terminal with sizes in sectors.
Type
rm 11
This will delete the oem partition. Type
print free
Partition 11 has been replaced with free space. Let’s make the Logo partition 2MiB. 2MiB = 4096s. The start sector is 113920, so the end sector needs to be (113920 + 4096 - 1) = 118015.
The command to create partitions is:
mkpart <name> <start_sector> <end_sector>
name <partition_number> < name>
So, type:
mkpart logo 113920s 118015s
name 11 logo
Some free space has appeared below logo. If you type “print free” on the 2nd terminal, with units set to MiB,
You’ll see that logo is now only 2 MiB, and an extra 6 MiB of free space is available. Also, oem takes 64 MiB. Open the 3rd terminal, from which you took backups, and type
adb shell df /oem
Divide the number under “used” with 1024. That’ll tell you how much space is being used. My oem partition is 64 MiB, but it’s only using about 6MiB. Typical. Not all space in the partition is available for file storage, so I’ll make the oem partition 11 MiB.
Unmount oem from recovery. Open the 1st terminal window, with sizes in sectors.
Type
rm 12
print free
11MiB = 22528s. new End size will be (118016 + 22528 - 1)s = 140543s
Type
mkpart oem 118016s 140543s
name 12 oem
Look at the 1st partition listing in this post. Oem had a ext4 file system. Open the 3rd terminal window. Type:
adb shell make_ext4fs /dev/block/mmcblk0p12
This will make an ext4 file system. Type “print free” in the MiB terminal. Things look good. Copy back the contents of oem. Mount oem from recovery, and type:
adb shell cp -a /microSD/oem/oem/* /oem/
If you get out of space errors, delete oem, and make it a bit larger. Now type:
adb shell chown root:root /oem/oem.prop
adb shell chmod 644 /oem/oem.prop
This will fix permissions of oem.prop.
I won’t change sizes of partition 13. So I’ll make it occupy 33536s from location 140544s. End sector will be (134400 + 33536 – 1)s = 174079s. So do it like the last two times. If you want to change the size of system, read my next post below first. I suggest that you keep the size of system intact. So, delete system, make it again just after expdb, and make an ext4 file system in it, like you did for oem.
Cache is about 128MiB big, but you just need it to be 5MiB. Any more is a waste. Delete cache, make it again right below system, with a size of 5MiB, and make an ext4 file system in it. This is what my partition table looks like after doing that.
So, after all that insanity, I just gained a tiny 182MiB of space. I’m not sure about the gen partition, and metadata is apparently used for encrypting data, so I’ll leave those two untouched. In my screenshots, userdata doesn’t have any file system, but that’s because I just unlocked my bootloader. Userdata is usually ext4. So, I’ll delete userdata,
and make userdata again. No need to calculate the end size again, as userdata will occupy all the free space, start sector will be 2232320s and end sector will be 7438335s. Then put an ext4 file system on userdata,
and done!
Now restore the partition 11 and 13 backup. If partition 11 gives an error, just ignore it.
dd if=/microSD/p11 of=/dev/block/mmcblk0p11
dd if=/microSD/p13 of=/dev/block/mmcblk0p13
Now reboot recovery, flash a new rom, restore IMEI, and reboot. Read the next post to get more space.
Reserved
Nice thread dude, though awesome tutorial
error
daemon started successfully P*
error: device not found
error: device not found
you think one is gonna take such a big risk of repartitioning for little more space?
yash sharma100 said:
you think one is gonna take such a big risk of repartitioning for little more space?
Click to expand...
Click to collapse
you think people will not get irritated by quoting such big post?
prazool007 said:
daemon started successfully P*
error: device not found
error: device not found
Click to expand...
Click to collapse
Install adb drivers for your phone. You need both adb itself and the drivers. Get it here http://forum.xda-developers.com/showthread.php?t=2588979
nice tutoriol but have you tried it
Yes, many times.
Great tutorial.
Just a little question. I'm sprout8 users, but i use sprout 4 rom. And in sprout4, the default data storage is on external memory (SDCard). The internal memory (userdata) is hidden (on rooted device, i can acces it in data/media/0/)
I think is a waste of space just to let 5.75 GB space only to be hidden.
the question: Can i convert the user data partition to make the system partition a little bigger?
Shybet said:
Great tutorial.
Just a little question. I'm sprout8 users, but i use sprout 4 rom. And in sprout4, the default data storage is on external memory (SDCard). The internal memory (userdata) is hidden (on rooted device, i can acces it in data/media/0/)
I think is a waste of space just to let 5.75 GB space only to be hidden.
the question: Can i convert the user data partition to make the system partition a little bigger?
Click to expand...
Click to collapse
Yes, definitely. Just delete userdata, then make it again with a smaller size, delete system, and put the free space in system. Then make the ext4 volumes on them.
will it be possible to give a flashable zip that will do the same as its a very risky process and my device cant be recovgnized in sp tools
1) I had intended to make a flashable zip. But I was too lazy to make one. I'll make it soon.
2) A flashable zip might actually be more risky than this manual method.
3) Unlock your phone's bootloader, and install MTK Vcom drivers, then SP Flash will recognize it.
How to backup IMEI?
Hi. Sorry if this is a noob question. In the article, the first and last steps are to backup/recover the IMEI. Is this done by backing up the "nvram" partition?
ex (backup):
adb shell dd if=/dev/block/mmcblk0p2 of=/microSD/nvram.img
adb pull /microSD/nvram.img
ex (restore):
adb push nvram.img /microSD
adb shell dd if=/microSD/nvram.img of=/dev/block/mmcblk0p2
or is there more to it than that?
Great tutorial!
But how do I restore everything back to normal?
Should I undo the steps or just restoring a TWRP Backup will do?
After I boot in TWRP it says error: "no devices/emulators found". Can you please help me ? I love my cheap phone , I just want to make the system partition a little larger so that I can add some extra apps
actyon20 said:
After I boot in TWRP it says error: "no devices/emulators found". Can you please help me ? I love my cheap phone , I just want to make the system partition a little larger so that I can add some extra apps
Click to expand...
Click to collapse
It says that on PC? That probably means that you didn't install the ADB drivers. Google for "Minimal ADB XDA", and download it from there. Then it should work.
If not, run all the commands in this post from TWRP - > Advanced - > Terminal (no need for PC).
By the way, a larger data partition = more user apps (from Play Store, etc). Larger system = more system apps (not usually needed).
CSAkshay said:
Great tutorial!
But how do I restore everything back to normal?
Should I undo the steps or just restoring a TWRP Backup will do?
Click to expand...
Click to collapse
To undo, you can just flash with SP Flash using the format + download mode.
However, many devices have been getting bricked by SP Flash lately. So I suggest repeating the steps in this post, but use the original offsets and sizes. That will also undo it.
wb407 said:
Hi. Sorry if this is a noob question. In the article, the first and last steps are to backup/recover the IMEI. Is this done by backing up the "nvram" partition?
ex (backup):
adb shell dd if=/dev/block/mmcblk0p2 of=/microSD/nvram.img
adb pull /microSD/nvram.img
ex (restore):
adb push nvram.img /microSD
adb shell dd if=/microSD/nvram.img of=/dev/block/mmcblk0p2
or is there more to it than that?
Click to expand...
Click to collapse
Actually, I'm not exactly sure.
It didn't work for me, so I had to use the IMEI fix zip from these forums to rewrite the IMEI.
out386 said:
It says that on PC? That probably means that you didn't install the ADB drivers. Google for "Minimal ADB XDA", and download it from there. Then it should work.
If not, run all the commands in this post from TWRP - > Advanced - > Terminal (no need for PC).
By the way, a larger data partition = more user apps (from Play Store, etc). Larger system = more system apps (not usually needed).
Click to expand...
Click to collapse
First of all , thanks a lot for the reply.
Second , yes , I tried that from pc with windows 10 64-bit.
My steps:
1. From my phone I enabled debugging.
2. I downloaded "android-sdk_r24.4.1-windows.zip" from //developer.android.com/studio/index. After I launch SDK Manager I installed:
*Android SDK Tools
*Android SDK Platform-Tools
*Google USB Drive
At Environment Variables at Path I added ;C:\android-sdk\tools;C:\android-sdk\platform-tools
When I type "adb" in cmd a long list appears.
3. I downloaded some "ADB Drivers_Updated" and I go in that folder , Open new cmd+shfit and I type "adb shell" and the next text appears ( without lines )
[email protected]_P4Life:/ $-----------------
BEFORE this you said that I must root in TWRP so I google it and I did the following steps :
1. Went in C:\android-sdk\platform-tools
2. Open new cmd+shift and I type "adb reboot recovery" the phone reboots and the next image appears:
//i.imgur.com/JLWdPXU.jpg
After this I'm lost because when I type abd shell in cmd it says "/no emulators found..."
I want to make a bigger partition for "system content apps" because even if I have ~5GB internal memory remaining I saw that there is a 1GB partition that when it's full (programs registry or I don't know what ) you can't install program anymore even if you have an empty SD card attached and the Default Write option in system is set to SD card.
Can you please help me ? This is my first time doing this and my phone is Allview P4 Life . I really like that cheap phone.
as usual, if anything goes wrong, no responsibility etc
The official update tutorial for rooted users doesn't keep app data, only internal storage! This tutorial keeps EVERYTHING
This method allows updating from 4.5.15 (unlocked, encrypted, rooted) to 5.0.2 without any data (sys settings + app data + user data) loss
A FAQ section is present at the end of this post and will be regularly updated.
List of files to download while doing the following steps:
OOS 5.0.2 ROM
Codeworkx TWRP recovery
Latest Magisk
Terms and software used in this guide:
Fastboot / Bootloader = bootloader of the phone, it's a very low level mode of the phone that allows booting into recovery. Can be accessed by using the advanced reboot menu (enable in dev options) or by "adb reboot fastboot".
Recovery = a small operating system on the phone that allows you to do various operations even when the main OS (Android) is broken. This includes flashing ROMs, modifying stuff on the storage, etc. It's the Android swiss army knife. If you can get a phone to boot TWRP, then you can do almost anything.
Magisk = rooting software that uses a systemless method to keep SafetyNet working. Systemless = instead of modifying the system, every change is put in a separate image that is mounted "over" the system. When the system tries to access a file modified by Magisk, instead of reading it from the partition, it reads it from Magisk. It's recommended to use Magisk instead of SuperSU as of 2018.
ADB = tool that allows controlling the phone from your PC through USB. You can use it when you're in Android if USB debugging is enabled in the settings, or when you're in TWRP. Here, we mostly use it for transferring files directly (without MTP) and running commands (using "adb shell")
Note: For this guide you will be required to download and install Magisk. If you don't want your phone to be rooted, then at the end of this guide reboot into TWRP, wipe both caches and re-flash the OS. This will uninstall Magisk and any other root patch. Beware: it will reflash stock recovery, so if you ever want to re-root, you'll need to reboot to fastboot and flash TWRP manually.
Convention for commands that you will have to run:
a command line starting with "C:" means that it should be run on your PC
a command line starting with "~ #" means that it should be run on your phone (through adb shell) while in TWRP
a command line starting with "OnePlus5:/ $" means that it should be run on your phone (through adb shell) while in OxygenOS
Although the commands start with "C:", this is just for readability purposes. You should run everything from inside an empty directory with enough disk space and writing access.
Your phone will have to be plugged in to your PC from the beginning to the end. Also, make sure it has at least 80% battery before beginning, just in case.
I know, the tutorial is huge. This is simply due to the fact that if I just wrote "make a nandroid backup of this and that, flash, and restore the backup while doing this", then some people may encounter problems because not everyone knows how to do a nandroid backup, restore it, etc. Also, there are a lot of things that need to be done precisely that way and not another way, which explains why the tutorial is huge. Also, you may notice that there is a lot of commands to run throughout the tutorial, this is because that way, I'm sure that at the end, you will have done everything like I did it on my phone, so that if you have a problem it's much easier to figure out where it comes from.
Summary of what you need to do (this is only a SUMMARY to give you a preview of what the whole thing looks like, you shouldn't follow it except if you're really an expert since a lot of things need to be done precisely, instead you should follow the easier complete steps below):
Make a Nandroid backup of /data
Backup files on internal storage
Wipe everything (internal storage + /data + system + caches), and then format data (important!)
Push and flash the OOS zip
Wipe caches and reboot (to Oreo!)
When it reboots, make sure everything (features, like Wi-Fi and fingerprint sensor) works. Don't "save anything" though, everything you do will be erased when we'll restore your backup. This is just a "test drive" for Oreo.
Reboot to TWRP, wipe Data and restore the /data backup
Run the three commands to fix Wi-Fi and fingerprints
Reboot (to System) and check everything works (don't do anything, don't change any setting, just make sure it works)
Reboot to TWRP, rename the "Android" folder to "Android_oreo" on sdcard, delete everything else on sdcard and restore your internal files
Rename the freshly restored Android (nougat) folder to "Android_nougat" and rename "Android_oreo" to "Android".
Flash Magisk, wipe dalvik+cache and reboot to System
When in Android, everything should work except some apps won't have their data. This is normal. Open a terminal (either on your phone using Termux or from your PC using adb shell), elevate using su and rename "Android" to "Android_oreo" and "Android_nougat" to "Android" (this is so that it correctly restores permissions)
If everything works fine, delete the "Android_oreo" folder
First, if you have Xposed Framework (systemless or not) installed, uninstall it. Next, if needed, uninstall any Magisk module that is "Nougat-only" to prevent any problems afterwards.
Boot the phone to bootloader/fastboot (either using advanced reboot, or by using volume down button when you start your phone) and boot to the TWRP recovery by doing
Code:
C:\> fastboot boot twrp-3.2.1-0-oreo-8.1-codeworkx-cheeseburger.img
from your PC.
Next, in TWRP, make a backup of /data (using the Backup button). Then, still while in TWRP, run the following commands:
Code:
C:\> adb shell
~ # cd /sdcard
/sdcard # tar cvf twrp.tar TWRP
/sdcard # md5sum twrp.tar
<< md5 checksum of twrp.tar >>
/sdcard # exit
C:\> adb pull -p /sdcard/twrp.tar
When the above command has finished, make sure that the checksum of the received twrp.tar file matches the one previously displayed.
If it doesn't match, delete the file and run adb pull again. Don't continue following this guide until you have received a 1:1 (checksum-wise) backup of /data.
Code:
C:\> adb shell
<< WARNING: dangerous command! double check the following line is correct before pressing enter! >>
~ # rm -rf /sdcard/TWRP
~ # rm /sdcard/twrp.tar
That was for /data. Now, the backup for the internal storage:
Code:
~ # cd /sdcard
/sdcard # du -csh
<< you should see here the total size of sdcard, that'll give you an idea of how long it'll take >>
/sdcard # tar cvf sd.tar element1 element2 element... elementN
<< in the command above, replace "element1..N" by a space-separated list of what you want to have in the backup.
Keep in mind that your list HAS to contain the element "Android" (case is important). It contains the app data.
Let's say for example you want to keep only the photos you have taken (and nothing, nothing else that was on internal storage).
The photos are in the folder DCIM, so the command will look like this:
tar cvf sd.tar Android DCIM
(because you want DCIM, and Android has to be in the list, no matter where)
>>
/sdcard # md5sum sd.tar
<< md5 checksum of sd.tar >>
/sdcard # exit
C:\> adb pull -p /sdcard/sd.tar
When the above command has finished, make sure that the checksum of the received sd.tar file matches the one previously displayed.
If it doesn't match, delete the file and run adb pull again.
Keep in mind that anything you don't put in that list will not be backed up and will be lost!
Now, you have a backup for all the important stuff so we can start doing the real sh*t.
Now, the important step:
Go back to the TWRP home screen, press "Wipe", "Advanced Wipe" and there check "Dalvik / ART Cache", "Cache", "System", "Data" and "Internal Storage". Confirm using the slider at the bottom of the screen. Press the home button, then "Reboot" and "FastBoot". Now, type the same fastboot command as in the previous step to boot the recovery image. You'll enter the recovery as before.
Now, on your PC, in the terminal, type
Code:
adb push -p OnePlus5Oxygen_23_OTA_029_all_1801292040_d71af3d.zip /sideload
(note: here, we are not using "adb sideload", we are really using "adb push"). In TWRP, click Install, in the file manager go to /sideload and select the OOS zip file. Confirm by sliding. If you get an error, go back to home, click Mount and ensure System is not checked. Then try installing again. If it still does not work, reboot to fastboot, type command again, get to the recovery and install again.
When the zip-file is installed, go home, click "Wipe", "Advanced Wipe" and check both caches and confirm. Then, go home, click "Wipe" and then "Format Data". Then, go home, click "Reboot" and then "System". Your phone will now reboot to Oreo. It will take a long time, but do not turn off the phone. Let it run. On my phone, it took on average 2 minutes for that boot.
You'll be greeted by the "first boot" page. It'll ask you if you want to restore a backup or start anew, choose start anew. Connect to your Wi-Fi network and Google account. Follow all the instructions until you get to the home screen. There, make sure everything works (especially Wi-Fi and fingerprint sensor). Don't save your fingerprints yet, they will be erased afterwards. If everything works, you can continue following these instructions. If not, post a comment down there.
Now that you're at the home screen, go in the settings, About Android and click the build number 8 times to enable Developer Options. Go in there and enable advanced reboot. Then, reboot your phone into fastboot/bootloader using the power button. Type the exact same command as before to start TWRP. Once that you are in TWRP, run the following commands:
Code:
C:\> adb push -p twrp.tar /sdcard/
C:\> adb shell
~ # cd /sdcard
/sdcard # tar xvf twrp.tar
/sdcard # cp /data/misc/wifi/WifiConfigStore.xml /sdcard/
In TWRP, click "Wipe", "Advanced Wipe" and check only the "Data" partition. Confirm. Press home, then "Restore" and choose the backup in the list. Confirm to restore. Back to the terminal, we need to run the following commands otherwise Wi-Fi and fingerprints won't work:
Code:
/sdcard # cp WifiConfigStore.xml /data/misc/wifi/
/sdcard # rm /data/misc/wifi/wpa_supplicant.conf
<< WARNING: dangerous command! double check the following line is correct before pressing enter! >>
/sdcard # rm -rf /data/system/users/0/fpdata
/sdcard # rm /data/system/users/0/settings_fingerprint.xml
Note: the command above are ran from your PC in an adb shell while the phone is still in TWRP.
Reboot the phone to system and ensure Wi-Fi and fingerprints are still working. Right now you should already see your old home screen and all your apps, but the internal storage isn't there yet. Reboot in fastboot, run the command to get in TWRP.
Once TWRP has booted, run the following commands:
Code:
C:\> adb push -p sd.tar /sdcard/
C:\> adb shell
~ # cd /sdcard
<< WARNING: dangerous command! double check the following line is correct before pressing enter! >>
/sdcard # rm -rf Alarms Albums DCIM Download Movies Music Notifications Pictures Podcasts Ringtones
/sdcard # ls
<< now, look at the list of files that were printed, and rm anything left that is not called "Android" or "sd.tar"
<< WARNING: dangerous command! double check everything is correct before pressing enter! >>
if when you do rm <the thing> it tells you it's a directory, then do: rm -rf <thething>
if there's a folder called SomeFolder, do "rm -rf SomeFolder"
next, run this:
/sdcard # ls
Android sd.tar <-- expected output
/sdcard # mv Android Android_oreo
/sdcard # tar xvf sd.tar
/sdcard # mv Android Android_nougat
/sdcard # mv Android_oreo Android
If you don't have Magisk somewhere on your sd card, download it and upload it using MTP or adb. Then flash it using the Install button. Clear dalvik/cache and reboot to system.
When the phone has booted (again, it might take time), make sure USB debugging is enabled and run the following commands:
Code:
C:\> adb shell
OnePlus5:/ $ su
<< here, you might see a Magisk screen asking for superuser access. Allow. >>
OnePlus5:/ $ cd /sdcard
OnePlus5:/sdcard $ mv Android Android_oreo && mv Android_nougat Android
Now, try some apps and make sure all the data is there (especially games and Netflix/Hulu/etc). If everything is there, and the phone works properly, go back in the terminal and type:
Code:
OnePlus5:/sdcard $ rm -rf Android_oreo
Optionally, start the TWRP app and flash it, it can always be useful. You can also reboot to fastboot to do that.
Now reboot your phone (normal reboot) one last time.
There, working OOS 5.0.2 / Android 8.0.0 phone with no data loss.
Frequently Asked Questions
How long does the whole thing take?
Highly depends on the amount of data you have on your phone. Since it the USB port only supports USB 2.0, it may take 4 or 5 hours in total.
Will doing this void my warranty?
No.
Will I be able to install future OTA updates using the regular download-reboot-flash-twrp procedure?
Yep. Just use the regular method as you would have on Nougat.
Will I lose my data?
If you follow all the instructions, no. Even if you don't follow them, as soon as you have made a backup of /data and internal storage, then no matter how bad you screw up you could always get a working phone back.
I followed the instructions and now my phone doesn't work
Boot in TWRP, wipe everything, reflash.
questions will be added there in the future
Having WiFi and fingerprint issues
You sure it is a good idea to just delete those files? I would have guessed that I need to replace these (nougat version from backup) with the oreo version to have it working just like before the restore.
Code:
/sdcard # rm /data/misc/wifi/wpa_supplicant.conf
/sdcard # rm -rf /data/system/users/0/fpdata
/sdcard # rm /data/system/users/0/settings_fingerprint.xml
I'm having the issues with wifi and fingerprints. Neither one is working. I'll try to figure out how to fix this.
@zdimension Thanks for this guide, I don't have time to test it yet, but I have a question
pdluke said:
Code:
/sdcard # rm /data/misc/wifi/wpa_supplicant.conf
/sdcard # rm -rf /data/system/users/0/fpdata
/sdcard # rm /data/system/users/0/settings_fingerprint.xml
Click to expand...
Click to collapse
At this point in the procedure, adb shell is still using root (before flashing magisk) ? How is that possible ? Does the adb /sideload preserve root ?
olivier380 said:
@zdimension Thanks for this guide, I don't have time to test it yet, but I have a question
At this point in the procedure, adb shell is still using root (before flashing magisk) ? How is that possible ? Does the adb /sideload preserve root ?
Click to expand...
Click to collapse
These commands should be run while the phone is in TWRP. Also note that adb /sideload is not used here, only adb push.
pdluke said:
You sure it is a good idea to just delete those files? I would have guessed that I need to replace these (nougat version from backup) with the oreo version to have it working just like before the restore.
Code:
/sdcard # rm /data/misc/wifi/wpa_supplicant.conf
/sdcard # rm -rf /data/system/users/0/fpdata
/sdcard # rm /data/system/users/0/settings_fingerprint.xml
I'm having the issues with wifi and fingerprints. Neither one is working. I'll try to figure out how to fix this.
Click to expand...
Click to collapse
If you delete them, they will be generated automatically at the next system boot. But you could also make a backup of those three files before wiping /data, store that somewhere, restore Nougat /data and then restore your backup of those three files. The result would be the same.
Note: actually, not exactly. Erasing the first file won't change anything since it's not used anymore in Oreo, but the two other files contain the fingerprint configuration (list of saved fingerprints). So,
Either you remove the files and you have to save your fingerprints again at next boot
Either you restore them from an Oreo backup and you'll get the fingerprints you had saved during the "first boot" procedure when you rebooted the phone right after flashing the OS
But the result is mostly the same: everything works. Deleting the files ensures you get something clean. If you restore from an Oreo backup I can't guarantee the result (as it may interfere with other files from the Nougat backup).
10 bucks to make a script to do this all for me haha.
@zdimension Thanks for the clarification Another thing you might add to the files to download would be Magisk (optionally). In this kind of guide, I've always find it useful to download everything first.
olivier380 said:
@zdimension Thanks for the clarification Another thing you might add to the files to download would be Magisk (optionally). In this kind of guide, I've always find it useful to download everything first.
Click to expand...
Click to collapse
Oops, forgot to add it
I added the link, and also instructions for how to un-root afterwards for those who would want it.
To improve the guide, here are some ideas :
- You should highlight that rm -rf is a very dangerous command, and that it needs to be checked twice (especially the targeted folder)
- It could be useful to use the du -csh command to check the size of a folder (to estimate the backup time for example).
- As a safety measure, one could md5sum the tar file before and after using adb pull
What do you think ?
olivier380 said:
To improve the guide, here are some ideas :
- You should highlight that rm -rf is a very dangerous command, and that it needs to be checked twice (especially the targeted folder)
- It could be useful to use the du -csh command to check the size of a folder (to estimate the backup time for example).
- As a safety measure, one could md5sum the tar file before and after using adb pull
What do you think ?
Click to expand...
Click to collapse
Thanks for the ideas! I updated the post (and I added a changelog at the bottom for future reference).
Followed guide for successful upgrade from 4.5.15 encrypted, unlocked bootloader w/ Magisk root.
One note, after the first complete wipe and flash of the full ROM, it was getting stuck on first boot and never completed. Discovered that I needed to not just wipe the Data partition but Format it in TWRP, to clear out the old encryption I think. Magisk wouldn't install either until I did this.
@debork thanks for the positive feedback (all the merit goes to @zdimension of course)
@zdimension there are many people in the other thread https://forum.xda-developers.com/oneplus-5/how-to/official-oxygenos-4-5-2-7-1-1-ota-t3627003 that tried (unsucessfully) to upgrade from 4.5.15 to 5.0.1, maybe a link to this topic could be useful for them (if it's not too late).
Regarding the
Go back to the TWRP home screen, press "Wipe", "Advanced Wipe" and there check "Dalvik / ART Cache", "Cache", "System", "Data" and "Internal Storage".
Click to expand...
Click to collapse
I think it should be highlighted in red, since it is the actual "clean flash" (AFAIU, correct me if I'm wrong).
if we only have the BL unlocked non root and stock recovery can we only ota without any loss of data ?
debork said:
Followed guide for successful upgrade from 4.5.15 encrypted, unlocked bootloader w/ Magisk root.
One note, after the first complete wipe and flash of the full ROM, it was getting stuck on first boot and never completed. Discovered that I needed to not just wipe the Data partition but Format it in TWRP, to clear out the old encryption I think. Magisk wouldn't install either until I did this.
Click to expand...
Click to collapse
Thanks for feedback, I will add that to the guide (although it worked with just Wipe for me )
zdimension said:
Thanks for feedback, I will add that to the guide (although it worked with just Wipe for me )
Click to expand...
Click to collapse
Have you rooted the 4.5.15 with Magisk or SuperSU (which is not compatible with Oreo anymore) ?
olivier380 said:
Have you rooted the 4.5.15 with Magisk or SuperSU (which is not compatible with Oreo anymore) ?
Click to expand...
Click to collapse
I stopped using SuperSU when it was sold to that shady company. Also, Magisk is better imo.
quick05 said:
if we only have the BL unlocked non root and stock recovery can we only ota without any loss of data ?
Click to expand...
Click to collapse
Official OnePlus support said that nothing is guaranteed if your bootloader is unlocked. But since you're on stock recovery + non rooted, you could always try. But backup everything first. Some people here on XDA have reported that it doesn't work, though.
Just followed your guide with no problems. Thank you very much!! I can confirm also that you need to format data after the wipe otherwise it gets stuck in a bootloop!
Thanks so much for this. I was able to successfully follow the guide and get upgraded to 5.0.1 without losing any data. In fact, I even messed up one step by failing to include the Android directory in the sdcard.tar backup (perhaps that should be more explicit), but it doesn't seem to have affected everything; all of my apps seem to have retained their data.
A few notes:
1. The file size of twrp.tar was ~14GB but when executing the pull command, it recognized it as only ~1.3 GB. As a result, the pull was not complete until it reached over 1000%. All the more reason to do the md5 check.
2. As others stated, I needed to format the data partition, not just wipe it.
3. I might recommend also including a "summary" version somewhere on what this guide does. Scrolling through the guide the first time, it seemed pretty daunting, but really all that you're doing is: backing up data partition and internal storage; wiping device; flashing Oreo ROM; tweaking a few files; and restoring backed up data and internal storage.
Thank you again so much! Glad to finally be on Oreo.
elight3 said:
Thanks so much for this. I was able to successfully follow the guide and get upgraded to 5.0.1 without losing any data. In fact, I even messed up one step by failing to include the Android directory in the sdcard.tar backup (perhaps that should be more explicit), but it doesn't seem to have affected everything; all of my apps seem to have retained their data.
A few notes:
1. The file size of twrp.tar was ~14GB but when executing the pull command, it recognized it as only ~1.3 GB. As a result, the pull was not complete until it reached over 1000%. All the more reason to do the md5 check.
2. As others stated, I needed to format the data partition, not just wipe it.
3. I might recommend also including a "summary" version somewhere on what this guide does. Scrolling through the guide the first time, it seemed pretty daunting, but really all that you're doing is: backing up data partition and internal storage; wiping device; flashing Oreo ROM; tweaking a few files; and restoring backed up data and internal storage.
Thank you again so much! Glad to finally be on Oreo.
Click to expand...
Click to collapse
Thanks for the feedback! I'll add a summary to the guide.