It would appear that the N7 eMMC controller wear-leveling implementation does not enforce page/block erasure during wear leveling re-mapping.
Further, there is no reason to believe that partition boundaries are relevant to the block remapping.
The upshot of this is that pages (containing arbitrary data) that were once in /data can end up in slack space - for instance in the back end of the boot or recovery partitions. (The valid images written in there are megabytes shorter than the full partition sizes). So, don't give folks Nandroid backups - even if a factory reset has been performed prior to the nandroid creation time or the userdata backup removed. (The boot & recovery partitions are captured as full partition dumps, so you get valid images plus slack space)
I ran a simple test yesterday:
- record the boot partition using "dd"
- boot into bootloader and then "fastboot erase boot"
- boot into custom recovery and capture boot partition (again) using "dd"
- restore boot partition (fastboot or nandroid)
Then I examined the boot partition dumps (e.g. using "hexdump" or "strings"). In my case the "erased" boot partition contained (among other stuff) what looked like pathnames for lots of files stored in /data. Clearly that is the type of stuff that would never be found in the boot partition without block remapping. You can also find the same kind of thing going on in the USP partition - have a look if you are curious.
I suppose this has implications for folks "cleaning up" tablets for resale or return - bits of your data lives on in unexpected places...
cheers
Related
Does anyone notice that? I have a 103mb data.img that I tried to upload and got not a space error. Also tried loading the data.img as system.img with the same error
When trying to load an older 99mb (yes exactly lol) it worked perfectly
did you pull this from your own device?
I've noticing on the rom I'm building (which pushes the limits of /system to load a completeish hero) that sometimes the device will allow you to allocate slightly more or less space. I still don't understand how partitioning works on the device (or why it's in place, I'd like a full single partition instead protected by permissions and avoid having the waste of unused space in boot, or system, or cache... etc...), but again, it seems that your device allows you to cram slightly more if you push it, but once you try to flash it back with fastboot, the specified partition sizes will prevent you from flashing
I just saw an interesting thread in the development section that explains an alternative to the Death SPL. The method there lets you flash ANY rom on any SPL, but I dont really understand how it works.
The thread can be found here: http://forum.xda-developers.com/showthread.php?t=704560
So basically, you shrink the cache partition to allow for more room for the actual ROM(which partition does that go in?)?
To do this do we edit the boot.img in the ROM update.zip? What else do we do?
Could someone explain this in a way a 9th grader could understand?
kingkurry said:
Could someone explain this in a way a 9th grader could understand?
Click to expand...
Click to collapse
Take file by firerat, flash file. omgroflpartitons.
If you don't understand the instructions as they are, wait for it to be perfected before you try anything. This will probably end up being integrated into releases that need it, so you don't need to worry about the specifics at the moment.
Will that patch file work for all ROMs. He said its only been tested with CyanogenMod 5. And I want to understand what im doing, not just do it without thinking about it...
Also, does the recovery patcher decrease the size of the partition that holds the recovery image?
Does it permanently change the size of the recovery partition?
When you flash a ROM, what partition is it being flashed too? Is this the one being increased in size?
What does the boot.img in an update.zip package hold, and is that copied to the boot partition?
Sorry but my curiosity is killing me
OK well to break it down we have 6 partitions on the internal memory:
Misc - Here be dragons
Recovery - Contains recovery system (+seperate recovery kernel) - recovery.img lives here
Boot - Contains kernel & important initialization stuff - boot.img lives here
-------------
System - Contains the whole android system (the "ROM", if you like).. everything else from an update.zip apart from the boot.img
Cache - Used by system and recovery for temporary storage
Userdata - Contains all personal data, downloaded apps, settings etc.
The first three partitions must be left at the default size so don't worry about them.
What this patch does is pass a command to the kernel which remaps the 3 large partitions at boot time. Since we're flashing system images from recovery, we also need to pass the same command to the recovery kernel before attempting to flash the main system, or we'd be writing to one place then telling the kernel to look for it in another.. bad idea.
This method allows any partition setup you like, but the most useful at the moment (and this is the way firerat has set up his scripts to suit cm5) is to make the /system partition just the right size for the "ROM" with a bit of breathing space, make the /cache partition a minimal size for the recovery system to use, then have /userdata fill the remaining space so we can load it up with apps. Since we've reduced cache to a minimal size, it's redirected at boot time to a place on the sdcard instead.. this give us maximum space to divide between /system and /data with no wastage.
Does that help at all..?
Thanks dude. That does help a lot.
Just wondering though, how much breathing space do u need in the system partition?
What does the recovery system use the cache partition for and how do we know what "a minimal size for the recovery system to use" is?
Is it possible to reduce the userdata partition to the minimum possible size a partition can be(if i recall correctly it was 128kb) and use an ext partition on your SD card instead?
If we shrink the Cache partition a lot, does this mean we have to use linux swap to compensate for the lowered amount of cache?
Also do we have to remap the partitions every time we flash a new ROM?
And what are the "dragons"?
kingkurry said:
And what are the "dragons"?
Click to expand...
Click to collapse
He's saying that it's just there. There could be anything from nothing there to a text document containing the ingredients to the cure of AIDs.
Well what about every thing else? Can you guys help me with that? Also what is the total size of all 3 of the big partitions combined?
I need to resize my /system partition on my phone since its out of space. It seems like the only way I'll be able to do it (regardless of what I do), will be to reflash my phone and make sure the partition is large enough this time.
My question is: In the process of flashing, how are partition sizes for data, system, cache, etc. determined? I notice in Odin3 there is a box for "repartition" so I assume that somewhere there must be a way to define the sizes.
Thanks
I'll make this one quick, so I know the userdata.img that comes with stock images is just a blank image that's sole purpose is to format your userdata partition for you, and that you can use fastboot format userdata for the same result, however is it the same deal with the cache.img file? Is the cache file just a blank image used for formatting the cache partition of your device or does it actually contain some data that the device needs to function? Thanks!
I have yet to flash either and have had no problems. I wipe both through twrp before booting up for the first time.
There is nothing of importance in either.
danarama said:
There is nothing of importance in either.
Click to expand...
Click to collapse
So in this sense, formating or erasing the cache is the same effect as flashing the cache.img?
H4X0R46 said:
So in this sense, formating or erasing the cache is the same effect as flashing the cache.img?
Click to expand...
Click to collapse
Yes and no.. Sort of.
The image itself, I don't really know why it exists. When you fastboot flash those images, it does a format first. Actually flashing the image afterwards has no benefit.
They could just put the format commands in the flash-all script. Perhaps they've done it for those who don't use the script to encourage them to format by flashing the image. Only thing I can think of. Or maybe those images are big enough that they wrote 0 byte data over existing data blocks that exist after the format to make them more secure, because really, a format or erase doesn't destroy data. It just makes it inaccessible by the average user. So it could be that they're overwriting a small section of data in the partition as that is the only way to destroy data.
Anyway, it should be rare that you need to format those partitions..
Formatting just erase allocation table, flash an empty.img overwrite and really cleans the data.. I think..
_____________________________________Read more write less and be smart
danarama said:
Yes and no.. Sort of.
The image itself, I don't really know why it exists. When you fastboot flash those images, it does a format first. Actually flashing the image afterwards has no benefit.
They could just put the format commands in the flash-all script. Perhaps they've done it for those who don't use the script to encourage them to format by flashing the image. Only thing I can think of. Or maybe those images are big enough that they wrote 0 byte data over existing data blocks that exist after the format to make them more secure, because really, a format or erase doesn't destroy data. It just makes it inaccessible by the average user. So it could be that they're overwriting a small section of data in the partition as that is the only way to destroy data.
Anyway, it should be rare that you need to format those partitions..
Click to expand...
Click to collapse
Yea it really is kind of confusing that they add these blank images, would it be a safer bet in this case to use the erase command as opposed to format? Or is it a better idea to just flash these blank images? They really made it confusing by adding them. I also have a 64GB nexus 6, so after flashing the userdata.img, it shows I only have 32GB! It requires a factory reset that, as you know, takes AGES on this device! In short, should the format command be used in more rare circumstances as opposed to erase or using the img file?
Use format to fix your issue. The images are known to cAuse this.
Finally I've got everything installed and set up and logged in, etc, after reviving from brick. But then I just realized... This is a 128GB phone and things are saying I'm low on space (kinda)...
Anyone have any idea what can cause this or how to fix? The only thing in my mind that might be relevant is that the Data partition is in F2FS when before my huge issue I had it in EXT4? The OP level 2 support guy used a 7.1.1 factory image to revive my device. Can this matter?
Settings/Storage it's showing SYSTEM is taking up 89GB?!?!?!
Mixplorer says all of Internal Storage is only using 1.62GB or so
Desktop shows the phone as having 24.8GB storage in TOTAL with 6.9GB used
I don't know how to interpret these screenshots at all or what to do about it...
Edit: A full TWRP backup of every partition checked in the options is 7.75GB in total (as displayed in Windows), system image & all!
Edit2: Also in TWRP, there is no option for resizing the Data partition (which is f2fs, every other partition is EXT4) ...
If no one has any ideas... This is the only thing that seems like it makes sense to try doing to switch data from f2fs to ext4 and hopefully not have to clean flash? https://forums.oneplus.net/threads/...-without-losing-internal-storage-data.439999/
But if that doesn't work, and a regular clean flash doesn't work, that's what I'm most concerned about before even messing with it... JUST got it all set up perfectly after reviving from bricked state...
Switch to ext4, as its the most common Linux' file system. f2fs seems to be the culprit. And you'd better done a clean flash.
Erase userdata partition from fastboot and then format it (also via fastboot). Should work.
Don't forget to make a backup of you existing data (including internal).
As warned by many, F2FS is unstable and causes many errors. Try switching to Ext4, via ADB. This ought to give you the corrected partitions.
Confirmed
Just posting to confirm the above messages.
i converted my system to ff2fs and my system was taking up a little over 40Gb.
Reverting back to ext4 sorted it