Database Develop

[Q] How to wipe Data partition after it's been encrypted?

Hey guys,
My work switched to Google Apps for their email so i wanted to test it out on my phone. Doing so required them to encrypt my phone. What this appears to have done is encrypt the data partition. So now when i go into (TWRP) recovery and try to do a backup / wipe / or anything else, it says it is "Unable to mount /sdcard" I'm unable to do a factory reset of any kind so i'm essentially stuck on this rom (ARHD) until i can figure out how to wipe data and get the /sdcard to be not encrypted. Does anyone have any ideas? I'm imagining that with the modified recovery not being able to access the data partition that i'm going to have to do some adb type stuff?
Please help! I want to keep flashing new roms!

Anyone??

I think you can't decrypt your data partition...
Did you try to wipe your data partition with the latest TWRP?

Guich said:
I think you can't decrypt your data partition...
Did you try to wipe your data partition with the latest TWRP?
Click to expand...
Click to collapse
That's the thing, the recovery can't mount /sdcard so wiping isn't an option from recovery

jrun said:
That's the thing, the recovery can't mount /sdcard so wiping isn't an option from recovery
Click to expand...
Click to collapse
Hi! Only TWRP recovery can solve your issue just do this...
Boot to TWRP recovery->Wipe
From there there are 2 options the left is the advance wipe and on the right is Format Data
Just select the format data button and it should reformat the whole data partition.
1st one would prompt so many errors and reformat then reformat it again 1 time and it should fully reformat your phone.
In theory it should reformat data partition even when encrypted but just in case it fails and I won't be able to go back to this thread just boot in bootloader and type
"fastboot erase userdata"

Well none of that works. TWRP doesn't support decryption yet for this device.

sevenalive said:
Well none of that works. TWRP doesn't support decryption yet for this device.
Click to expand...
Click to collapse
What I instructed basically doesn't decrypt the partition. It reformats it.

Riyal said:
What I instructed basically doesn't decrypt the partition. It reformats it.
Click to expand...
Click to collapse
I know what you instructed, it doesn't work. Only way I got it to work. Flash the stock recovery. Boot into system. Take off the encryption, phone reboots and removes the encryption, data is formatted. Then flash TWRP again.

I basically ran into this exact problem. I was hoping that I'd still be able to back up the encrypted device, but forget about it. What a pain in the ass.
Someone should make add something to the guides about how encrypting your device makes nandroid backups impossible.

[Q] Fix missing space (23.03GB vs. 64GB), but keep data

I've run in to the problem some other people were having too. A problem, where after fastboot flashing userdata.img (from stock factory image), I'm left with 23.03GB of storage on my device, even though it's a 64GB model.
I've Googled the problem, and people suggest running "fastboot format data" to fix it. But of course that will wipe all my settings and so forth. My question is if I can back up the data partition using TWRP onto an OTG device. Then run "fastboot format data" and finally restore data again using TWRP.
Would that work? And would it even fix the problem of the missing space.
For reference, the issue has previously been discussed here:
http://forum.xda-developers.com/nexus-6/help/nexus-6-64gb-23gb-free-space-t2953636

stokholm said:
I've run in to the problem some other people were having too. A problem, where after fastboot flashing userdata.img (from stock factory image), I'm left with 23.03GB of storage on my device, even though it's a 64GB model.
I've Googled the problem, and people suggest running "fastboot format data" to fix it. But of course that will wipe all my settings and so forth. My question is if I can back up the data partition using TWRP onto an OTG device. Then run "fastboot format data" and finally restore data again using TWRP.
Would that work? And would it even fix the problem of the missing space.
For reference, the issue has previously been discussed here:
http://forum.xda-developers.com/nexus-6/help/nexus-6-64gb-23gb-free-space-t2953636
Click to expand...
Click to collapse
Just so its clear the fastboot format does the trick. (I also did fastboot and vol scrolled to recovery and did a full wipe before the format)(that was prob overkill)
fastboot format userdata
fastboot format cache
fastboot reboot
I've never done that with twrp. I usually just dump my whole SD card to a hard drive then start fresh. Its a reasonable pain to ensure there are no problems.
Also wouldn't you have been totally wiped anyway if you were going back to stock? Have you been using it with 23gb for a while?

No, haven't been using it for long like that. Only a few days actually.
It's not that I have a lot of files, but I do have a lot of settings and app settings. I know I could probably use Titanium Backup to back that stuff up, but I don't trust that method do get everything and not mess something up.
One more question though. Why format cache too? I saw that suggested in the thread I referenced. But isn't it enough to format data? I guess it makes no difference really, but I'm trying to learn in the process too.

stokholm said:
I've run in to the problem some other people were having too. A problem, where after fastboot flashing userdata.img (from stock factory image), I'm left with 23.03GB of storage on my device, even though it's a 64GB model.
I've Googled the problem, and people suggest running "fastboot format data" to fix it. But of course that will wipe all my settings and so forth. My question is if I can back up the data partition using TWRP onto an OTG device. Then run "fastboot format data" and finally restore data again using TWRP.
Would that work? And would it even fix the problem of the missing space.
Click to expand...
Click to collapse
Yes and no.
Let me explain;
The first problem, which may or may not actually *be* a problem, is whether or not recovery will PERMIT a backup to an OTG. Assuming that it does, it unfortunately will only backup everything on the data partition BESIDES the "media" directory (where the "internal SD card" can be found).
To work around this, perform your backup TO the internal storage, then reboot back to Android, copy *everything" from the "internal storage" path to your computer (which will include the "backup" directory, whatever it happens to be called with the recovery you prefer). Then perform the fastboot format on the data partition, boot into Android skipping all the signin junk, copy everything BACK to the internal storage, reboot into recovery again, and restore it.
ALTERNATIVELY, and probably much easier (definitely much faster, since it should complete within a few seconds)...
I *believe* that most recoveries should include the resize2fs command (though I've never had an actual need for this, so haven't actually tested it), so via ADB into your recovery.....
Code:
umount /data
resize2fs /dev/block/platform/msm_sdcc.1/by-name/userdata
should do the trick.
NOTE HOWEVER, it is generally recommended to backup any partition where you are resizing a filesystem PRIOR to resizing it.
HAVING SAID THAT, I've run resize2fs hundreds of times on hundreds of systems, and never had an issue with a grow operation.
Running resize2fs without a "size" parameter will grow the filesystem to the size of the partition. The partition table still holds the correct size, which is why "fastboot format" fixes the issue.

That was a great explanation, @doitright. Thank you for that. I will try resize2fs at some point.

stokholm said:
No, haven't been using it for long like that. Only a few days actually.
It's not that I have a lot of files, but I do have a lot of settings and app settings. I know I could probably use Titanium Backup to back that stuff up, but I don't trust that method do get everything and not mess something up.
One more question though. Why format cache too? I saw that suggested in the thread I referenced. But isn't it enough to format data? I guess it makes no difference really, but I'm trying to learn in the process too.
Click to expand...
Click to collapse
Resizef2s sounds relatively painless, but does again, as doitright says, require caution by backing up before.
About the cache thing. I think to do a full format, data and cache are on different blocks, so that might be why. But then again the SD card is mounted in /data/media so maybe only format data is needed and the cache is to help with something else or just to be cautious, I don't know.
Now I wonder (and this can be searched here in xda) if TWRP back up data is /data/data or /data minus /data/media? Because fastboot format data I think is all of data i.e. /data. This is worth knowing for the future. Especially since we are flashinging things to our phones and whatever else.

MunkinDrunky said:
Now I wonder (and this can be searched here in xda) if TWRP back up data is /data/data or /data minus /data/media? Because fastboot format data I think is all of data i.e. /data. This is worth knowing for the future. Especially since we are flashinging things to our phones and whatever else.
Click to expand...
Click to collapse
ALL recovery backups are /data/* EXCEPT /data/media, being "backed up" to /data/media/something.
It simply cannot be implemented any other way for devices without separate storage devices, otherwise the backup would back up previous backups, which would be just plain wasteful.
The /cache partition is practically irrelevant. There is quite literally NO REASON to ever worry about it. No reason to wipe it, no reason for format it unless it somehow becomes corrupt.

It doesn't seem like resize2fs is included in TWRP.

stokholm said:
It doesn't seem like resize2fs is included in TWRP.
Click to expand...
Click to collapse
The resolution really is as simple as you originally thought. Its a common issue I've seen a hundred times on the nexus 5.
Take a full TWRP backup and then copy your entire sdcard to PC.
Format data
Boot into android and do initial set up.
Copy sdcard backup back to device whilst booted into android
Restore TWRP backup

You can use the file manager in TWRP to copy your entire sdcard to USB-OTG and copy it back. Personally, I find this easier than copying to PC.

adrman said:
You can use the file manager in TWRP to copy your entire sdcard to USB-OTG and copy it back. Personally, I find this easier than copying to PC.
Click to expand...
Click to collapse
The reason I said to copy it to PC via android was that using MTP or adb in recovery will break the sdcard permissions, when it is copied back. If you're confident OTG in TWRP works differently to adb and MTP in TWRP, cool. I just haven't tested it myself so won't suggest it.

rootSU said:
The reason I said to copy it to PC via android was that using MTP or adb in recovery will break the sdcard permissions, when it is copied back. If you're confident OTG in TWRP works differently to adb and MTP in TWRP, cool. I just haven't tested it myself so won't suggest it.
Click to expand...
Click to collapse
I used OTG via TWRP's file manager, when I wiped to decrypt and everything came back properly. I would assume that would be the case here as well.

adrman said:
I used OTG via TWRP's file manager, when I wiped to decrypt and everything came back properly. I would assume that would be the case here as well.
Click to expand...
Click to collapse
Excellent

rootSU said:
Excellent
Click to expand...
Click to collapse
Channeling Mr. Burns? [emoji1]

adrman said:
Channeling Mr. Burns? [emoji1]
Click to expand...
Click to collapse
Always, aren't you?

From within TWRP there is an easy way to do this even after you have your phone all setup. This issue always happens on my Nexus 5 and Nexus 6.
I did this without doing a backup and after my phone has been used on marshmallow for a week or so.
Steps from within TWRP:
1. Wipe > Advanced Wipe
2. Select the Data partition.
3. Select Repair or Change File System
(Notice the Partition Size)
4. Select Resize
(Wait and shortly after see the partition size be up to full size.)

This doesn't appear to work on 6.0.1 custom Rom with systemless SU 2.61... gave me a bad partition error.
Guess I will have to wipe.
maamdroid said:
From within TWRP there is an easy way to do this even after you have your phone all setup. This issue always happens on my Nexus 5 and Nexus 6.
I did this without doing a backup and after my phone has been used on marshmallow for a week or so.
Steps from within TWRP:
1. Wipe > Advanced Wipe
2. Select the Data partition.
3. Select Repair or Change File System
(Notice the Partition Size)
4. Select Resize
(Wait and shortly after see the partition size be up to full size.)
Click to expand...
Click to collapse

I have come across this thread. Here is what worked for me. (in case somebody needs it in the future)
1. Boot into twrp, type
Code:
adb shell
2. list the mounted partitions
Code:
df
You should see
Code:
df
Filesystem 1K-blocks Used Available Use% Mounted on
tmpfs 1507020 24 1506996 0% /dev
tmpfs 1507020 24 1506996 0% /tmp
/dev/block/mmcblk0p38
253920 264 248416 0% /cache
/dev/block/mmcblk0p42
24143612 23590364 536864 98% /sdcard
/dev/block/mmcblk0p42
24143612 23590364 536864 98% /data
/dev/block/mmcblk0p41
2015408 1965276 33748 98% /system
3. unmount /data and /sdcard
Code:
umount /dev/block/mmcblk0p42
umount /dev/block/mmcblk0p41
4. run
Code:
e2fsck -f /dev/block/platform/msm_sdcc.1/by-name/userdata
resize2fs /dev/block/platform/msm_sdcc.1/by-name/userdata
now you should see:
Code:
df
Filesystem 1K-blocks Used Available Use% Mounted on
tmpfs 1507020 24 1506996 0% /dev
tmpfs 1507020 24 1506996 0% /tmp
/dev/block/mmcblk0p38
253920 264 248416 0% /cache
/dev/block/mmcblk0p42
57306748 23598452 33691912 41% /data
/dev/block/mmcblk0p42
57306748 23598452 33691912 41% /sdcard
5. reboot, you are done

Please help me. Tried to change /data file system and wiped all my files...

I'm panicking very much right now.
I did a big, big stupid and accidentally wiped my entire /data partition, including /sdcard (!!) in TWRP 3.0.4.1 while attempting to change the filesystem of my /data partition from F2FS to EXT4. I did a full backup of my phone, including /data partition (but excluding /data/media/0/ which is the location for /sdcard of course) on my phone prior to attempting to change the filesystem.
I didn't think that this would also wipe /sdcard, and ALL MY FILES INCLUDING MY BACKUP...
I'm frantically searching through Google as we speak for guides to clone these formatted sectors over USB (I'd imagine using ADB shell) to a PC and use Windows file or partition recovery tools to try and save every, if not most files.
Please lend me your knowledge and links to fixing this issue, Android community. <3
In older Android versions the /sdcard partition was it's own partition, which is why I didn't think of copying all my files over to a PC before making changes to /data.

well, when you wipe data from any partition in twrp it sort of resets all space to be overwritten. The data that is marked for overwrite doesn't remain when you full wipe. Since you changed the file system, the data was removed and overwritten by the wipe with blank space.... i hope you had at least some of it backed up on your pc. I never use those apps personally, i always hard backup all my things with a drag and drop onto my pc. I highly recommend it, saves you a disaster. Im very sorry.... i hope you didnt lose anything really important.

OcazPrime said:
well, when you wipe data from any partition in twrp it sort of resets all space to be overwritten. The data that is marked for overwrite doesn't remain when you full wipe. Since you changed the file system, the data was removed and overwritten by the wipe with blank space.... i hope you had at least some of it backed up on your pc. I never use those apps personally, i always hard backup all my things with a drag and drop onto my pc. I highly recommend it, saves you a disaster. Im very sorry.... i hope you didnt lose anything really important.
Click to expand...
Click to collapse
Heh, that's exactly what happened. I'm gonna attempt this guide: https://forum.xda-developers.com/ga...de-internal-memory-data-recovery-yes-t1994705
Basically try to use "dd" to make a RAW copy of "dev/block/dm-0" which according to the mount command I ran in TWRP's terminal shell, is what block /data is used on OnePlus 3T (OOS 4.1.3).
Wish me luck.

[QUESTION] Missing GBs?? Maybe Partitions? Questions Post-Brick!

Finally I've got everything installed and set up and logged in, etc, after reviving from brick. But then I just realized... This is a 128GB phone and things are saying I'm low on space (kinda)...
Anyone have any idea what can cause this or how to fix? The only thing in my mind that might be relevant is that the Data partition is in F2FS when before my huge issue I had it in EXT4? The OP level 2 support guy used a 7.1.1 factory image to revive my device. Can this matter?
Settings/Storage it's showing SYSTEM is taking up 89GB?!?!?!
Mixplorer says all of Internal Storage is only using 1.62GB or so
Desktop shows the phone as having 24.8GB storage in TOTAL with 6.9GB used
I don't know how to interpret these screenshots at all or what to do about it...
Edit: A full TWRP backup of every partition checked in the options is 7.75GB in total (as displayed in Windows), system image & all!
Edit2: Also in TWRP, there is no option for resizing the Data partition (which is f2fs, every other partition is EXT4) ...

If no one has any ideas... This is the only thing that seems like it makes sense to try doing to switch data from f2fs to ext4 and hopefully not have to clean flash? https://forums.oneplus.net/threads/...-without-losing-internal-storage-data.439999/
But if that doesn't work, and a regular clean flash doesn't work, that's what I'm most concerned about before even messing with it... JUST got it all set up perfectly after reviving from bricked state...

Switch to ext4, as its the most common Linux' file system. f2fs seems to be the culprit. And you'd better done a clean flash.
Erase userdata partition from fastboot and then format it (also via fastboot). Should work.
Don't forget to make a backup of you existing data (including internal).

As warned by many, F2FS is unstable and causes many errors. Try switching to Ext4, via ADB. This ought to give you the corrected partitions.

Confirmed
Just posting to confirm the above messages.
i converted my system to ff2fs and my system was taking up a little over 40Gb.
Reverting back to ext4 sorted it

unable to mount internal storage due to encryption

Long story short, I've been trying to get into my internal storage via TWRP but its saying that there are 0MB in the internal storage and that there is nothing in the sdcard folder
I've tried formatting it, hasn't worked
I've tried changing the format to FAT and then back to EXT4
then after that didn't work i tried the same but with EXT2 , it showed the amount of storage but still didn't let me see what was inside of the sdcard folder, whenever i'd go in there its just completely empty
Can I please have some help with this, I've been stuck on this for a couple hours now

Ouch! Please don't say that you just formatted userdata (/data).
You've just killed all your data on your phone.
No, I'm not that experienced with TWRP and I don't know which versions under which circumstances it can mount userdata.
I use custom recoveries and just presume that I can't mount userdata in them.
In the normal system (which mounts and decrypts) I use normal tools to sync or backup.

Renate said:
Ouch! Please don't say that you just formatted userdata (/data).
You've just killed all your data on your phone.
No, I'm not that experienced with TWRP and I don't know which versions under which circumstances it can mount userdata.
I use custom recoveries and just presume that I can't mount userdata in them.
In the normal system (which mounts and decrypts) I use normal tools to sync or backup.
Click to expand...
Click to collapse
I didn't format data it self i meant that i factory reset it, sorry i should of been a lot more clear on that

Oh, ok.
Well, in any case, if it's encrypted you need to mount it not with a simple "mount" command but something fancier using dm.
They don't try to make it easy to do.
Unless you're destroyed your system and you're trying to recover your data, whatever you're trying to do is best done in the normal system.

never mind i found a fix for it
well not really a fix just a way around it
thanks for the help ^^

SoftieIsVibing said:
never mind i found a fix for it
well not really a fix just a way around it
thanks for the help ^^
Click to expand...
Click to collapse
I had the same issue so I flashed a raw firmware after taking full system backup. Now TWRP is working fine.