Database Develop

[Q] DCIM thumbnails taking over storage - need a fix!

Ok so I have an SGS for over a year now, with an external sd of 16GB and cyanogenmod 10 installed (this one). It's fantastic. But, recently I started having some storage problems. I couldn't update anything or download any new app or even take a picture. This seemed odd as I don't have that many applications installed and I keep all my big files on the external sd. So I started clearing caches and moving applications to the sd. Those solutions didn't last long.
Worse, ther weird part was yet to come. When I checked my settings >> storage, I found that there should be more than a 1.5GB of free space. But, If I checked my settings >> Apps >> On SD Card, I could see that there was only around 5MB of space left. Still, I found some old backup that I didn't need anymore, so I deleted that. hooray, there were now 500MB free to use. That is, until the next day when again, there was zero space to be used. I was shocked because during that time I didn't install anything new, I only took a small video of less than a minute, which by itself didn't weight more than 60MB of space.
Something was definetly up. Using File Manager I tried checking where all the data is, and I found that, to my amazement, in the DCIM folder, there was a folder named ".thumbnails" which took up 2GB of space. That is a lot. seaching around the Forums here, it seems I'm not the first one to notice this problem.
And the solution always seems to be "just delete the thumbnails". To me that seems a bit crass and not really a solution. I'm not tech enough to know what are the implications of simply deleting those files, and I'm not sure how long it's going to last anyway, before I need to do it again. So I'd like to know if there's some kind of fix I can install or some configuration I can change that would make sure it doesn't happen again. Or, if anything, some explanation in layman's terms why it's okay to simply delete those files. I'd like to mention that my phone is rooted, and I have CWM, Rom Manager and Terminal Emulator so I can use those if necessary.

AW: [Q] DCIM thumbnails taking over storage - need a fix!
That really is weird. Did you try to fix permissions in cwm recovery? I'm not sure it would even check that directory but it surely doesn't hurt.
Even if it works it will not cure your immediate problem. You will need to delete the thumbnail folder or the files in it. And here is why you don't have to worry.
Thumbnails are small files that store a preview of your images. They get created by your image viewer and are updated whenever it detects a change in the image file. When your viewer doesn't find a thumbnail it needs to read the whole image file, create a preview and show this. The created preview is then saved as thumbnail so it can be loaded next time the program needs to show the image.
In short: if you delete the thumbnails then they will be recreated next time your image viewer shows the images. So there is nothing to lose. You can safely delete the thumbnails.
JUST BE SURE THEY REALLY ARE THUMBNAILS AND NOT THE IMAGES THEMSELVES!

[Q] How is the Z3C without root? Can backups and moving apps to SD be done?

Hi everyone. I'm heavily considering getting a Xperia Z3 Compact, but I have a few hangups that I would like to get some feedback. I've been rooting my devices since I've been using Android. When my Galaxy S4 Active was updated to Android 4.3 and I lost root for 3 months, I hated it. But it's an ongoing battle that I'm tired of fighting and I've found that there are only really a couple apps that I really need root access for. Plus, if I really need to, I can unlock the bootloader and root the Z3C, but I don't want to lose the DRM keys and degrade my device for only a couple programs if I don't have to. So I'm wondering just how necessary they are and whether I can work around this.
The first application that I use all the time is Titanium Backup. I've been using it since my very first Android device and it has been a valuable tool. I've used it to backup applications that I didn't have room for and restore them later, to remove or freeze bloatware, and to upgrade system apps with new versions to save room. These things were a necessity with my old Xperia Play. However, the main function that I really need is the backup of app data. Anything can restore the installed apps, but very little will backup the actual app data.
Or at least so I thought. I've been using Titanium Backup for so long that I didn't really consider switching to any other applications. Plus the GS4A was my first phone on Android 4.x, so I didn't know about the included ADB backup feature built into it. From what I understand, Helium can do an ADB backup without root access, correct? Has anyone used this with the Z3C? I've read that some Sony phones don't support it. Although I've also read that all Motorola phones don't support it, and the GSM version of the Droid Turbo (aka "Moto X Play" last I read) is my other contender against the Z3C.
The next app that I use right now is FolderMount and this one is a bit trickier. Back on the Xperia Play, space was incredibly limited. I used Link2SD to save more space than moving apps to the MicroSD card through the Application Management. I could probably have used it on the GS4A, but I found FolderMount to be much easier to work with. It was also necessary because all moving apps to the SD card did was move them to an emulated SD card on the main device storage, which defeated the entire purpose.
I still don't understand why this is so stupidly designed now and why Google keeps trying to kill off the SD card. Or for that matter why device manufacturers still include only 16GB with SD card expansion (and no way of moving apps properly) or only 32GB without. Neither is enough room for games that can consume a gigabyte or two for massive .obb files. And in all honesty, those .obb files are all that I want to move, which is why FolderMount works so well for that.
So that brings me to the question that I think I already know the answer to: Has anyone found a way to move the .obb files to the external SD card without root access? Either that, or to change it so that the shared storage is the SD card and all the .obb files get downloaded to it by default? I suspect that the answer is still "not without root" here. It just seems asinine that Sony not include this feature if they are going to include an inadequate amount of storage space.
Aside from those issues, I could probably run without root access for most other things. Although root does make it much handier. I just hate needing it to fix or workaround poor design decisions on the part of manufacturers or Google themselves.

Info
MechaBouncer said:
Hi everyone. ...
Can backups and moving apps to SD be done?
Click to expand...
Click to collapse
without Root >> NO
:good:

So that goes for using Helium as well?

http://forum.xda-developers.com/search.php?searchid=312768077

Helium does work with Z3C without root. Switched from my old HTC One to Z3C with Helium. Restored most of the apps without a problem. Not all apps do work though, but this seems to be a problem with adb backup itself, not Helium itself.

MechaBouncer said:
Hi everyone. I'm heavily considering getting a Xperia Z3 Compact, but I have a few hangups that I would like to get some feedback. I've been rooting my devices since I've been using Android. When my Galaxy S4 Active was updated to Android 4.3 and I lost root for 3 months, I hated it. But it's an ongoing battle that I'm tired of fighting and I've found that there are only really a couple apps that I really need root access for. Plus, if I really need to, I can unlock the bootloader and root the Z3C, but I don't want to lose the DRM keys and degrade my device for only a couple programs if I don't have to. So I'm wondering just how necessary they are and whether I can work around this.
The first application that I use all the time is Titanium Backup. I've been using it since my very first Android device and it has been a valuable tool. I've used it to backup applications that I didn't have room for and restore them later, to remove or freeze bloatware, and to upgrade system apps with new versions to save room. These things were a necessity with my old Xperia Play. However, the main function that I really need is the backup of app data. Anything can restore the installed apps, but very little will backup the actual app data.
Or at least so I thought. I've been using Titanium Backup for so long that I didn't really consider switching to any other applications. Plus the GS4A was my first phone on Android 4.x, so I didn't know about the included ADB backup feature built into it. From what I understand, Helium can do an ADB backup without root access, correct? Has anyone used this with the Z3C? I've read that some Sony phones don't support it. Although I've also read that all Motorola phones don't support it, and the GSM version of the Droid Turbo (aka "Moto X Play" last I read) is my other contender against the Z3C.
The next app that I use right now is FolderMount and this one is a bit trickier. Back on the Xperia Play, space was incredibly limited. I used Link2SD to save more space than moving apps to the MicroSD card through the Application Management. I could probably have used it on the GS4A, but I found FolderMount to be much easier to work with. It was also necessary because all moving apps to the SD card did was move them to an emulated SD card on the main device storage, which defeated the entire purpose.
I still don't understand why this is so stupidly designed now and why Google keeps trying to kill off the SD card. Or for that matter why device manufacturers still include only 16GB with SD card expansion (and no way of moving apps properly) or only 32GB without. Neither is enough room for games that can consume a gigabyte or two for massive .obb files. And in all honesty, those .obb files are all that I want to move, which is why FolderMount works so well for that.
So that brings me to the question that I think I already know the answer to: Has anyone found a way to move the .obb files to the external SD card without root access? Either that, or to change it so that the shared storage is the SD card and all the .obb files get downloaded to it by default? I suspect that the answer is still "not without root" here. It just seems asinine that Sony not include this feature if they are going to include an inadequate amount of storage space.
Aside from those issues, I could probably run without root access for most other things. Although root does make it much handier. I just hate needing it to fix or workaround poor design decisions on the part of manufacturers or Google themselves.
Click to expand...
Click to collapse
Titanium backup doesn't work because it is built in a way that requires root to work. On the other hand Helium uses ADB therefore it works without root but requires the device to be connected to a computer for the backup. Tested and it works flawlessly. I heard that the Sony compagnon isn't bad either but it doesn't work with encryption.
Also you can do manually full ADB backups including the APK without any software or root.
You cannot move apps to the sd card but I would say that you don't need too either since the apps can store their data on it.

Thanks -Vulture- and difto. That confirms what I was wondering about Helium. I knew that Titanium Backup wouldn't work without root, so it's good to know that Helium still will for most apps.
And I figured root would be necessary to move apps or their files to the SD card. Unfortunately, a lot of games I have still won't write to it on their own, which is why I've had to resort to FolderMount. It's quite frustrating. I feel like any application that needs to download external files should be able to move those to the SD card.

Where did all my storage go

I have used a reported 6gig out of 11 on my /sdcard0, and yet when I run sd analyst in es explorer it doesn't add up as you can see in the attachment.
Sd analyst seems to be correct because I don't have any huge games installed. So where did all my storage go?

Its all the apps Samsung puts on there that you cant delete.
If somebody starts a class action lawsuit on this device like they did with the iPad I am all over it. Its not even remotely fair that a 16gig device has 8.9gig free because of 3+ gigs of bloatware you cant delete.
I have an SD card, but the lack of app space is concerning me.

Except I'm not running touch wiz. I'm running a CM12 ROM. Any way if I'm reading it right, this is just counting /storage/emulated/legacy the user data partition. Very weird.

barth2 said:
Except I'm not running touch wiz. I'm running a CM12 ROM. Any way if I'm reading it right, this is just counting /storage/emulated/legacy the user data partition. Very weird.
Click to expand...
Click to collapse
That does not make sense? I hope one of the Devs can come up with the answer to this mystery!

Correct me if I'm wrong, isn't it that if you root your device and get a rom let's say CM, as far as I know CM doesn't eat up a lot of space meaning depending on the contents of that rom, that 3gb bloat ware should be gone right?? Also this is what frustrates me with Samsung, we just can't have the option to write over to SD cards which I know can be a liability but look at the what we need to deal with. There's no 32gb version in my country so I'm always cramped up for space and I hate it, having to remove apps and games just to get by.
Sent from my SM-T805

Well I wiped my internal memory (drastic measure) and started over and now it looks correct. Not sure what was behind it.

More information would be great here. I know there's an incredible amount of Google bloat on these devices, but perhaps there is some information missing from the ES File Explorer results. I'd recommend downloading a dedicated app cache cleaner and a dedicated storage analyst app to get an accurate idea of all the things that are taking up storage. Some of the biggest offenders are browsers, but the one I've seen eat up a whole gigabyte on unknowing users' phones is usually the sneaky Google+.
Storage analyzer
https://play.google.com/store/apps/details?id=com.levelokment.storageanalyser
1Tap Cleaner
https://play.google.com/store/apps/details?id=com.a0soft.gphone.acc.free
steelbrachen said:
with Samsung, we just can't have the option to write over to SD cards which I know can be a liability but look at the what we need to deal with.
Click to expand...
Click to collapse
Well, with Google Android Kitkat, it was decided that apps should only be allowed to write to their own sandboxed folders on the SD card anyway. Samsung has always been pretty good about at least allowing file management via a first party app, and even managing which apps can use the SD card are listed in the application manager, so I fail to see how Samsung is at fault here. (Meanwhile, in addition to demanding restrictions across the board to SD cards in Kitkat and then Lollipop, Google's more focused on forcing a social network on their users than writing a file manager for all the devices that DO have SD cards... or providing any decent AOSP apps in general.)
Sent from my Galaxy S5

Without going into too much detail the amount of available space is affected by the partition layout of the device. Esp with cm or custom rom that only takes up a couple hundred mb; the system partition ends up with a lot of unused space. If you were willing the modify your default partition table you could reclaim some usable space. Though I would place this in the advanced user grouping and would not recommend attempting it.

Files disappeared,

Yes, I know how to use google, yes, I tried many things now and I don't know where else to look for help so I'm asking here, a place with many smart heads full of ideas.
I was doing pretty normal thing on my phone this morning, just trying to copy all the files from internal memory onto the PC. Of course I couldn't do that thanks to "Unspecified error" during copying some of the folders, including Downloads and DCIM. I managed to copy only some of the files from internal storage. I thought a reboot might help. And so the nightmare began. All the other files I wanted to copy disappeared from gallery and Total Commander. They must be on the phone though, as there is something on the phone that uses 2gb of memory. I did not delete them, the system just fails to see them. I already tried deleting all the ".nomedia" files, clearing cache and data from Media Storage app, disabling it, rebooting and enabling again but it was of no use. I'm helpless and I don't know what to do, why did android even do this? I lost a lot of photos and images and I'm really upset about that. Any ideas what to do? The phone is not rooted, never played with software in it, it's updated to 5.1

Download any File Manager from Play Store, check if the files are on the phone memory. If not, well, they are somehow lost. Maybe you cut-paste them onto your PC, and the files were deleted, but you got an 'unspecified error' on moving.

Thanks a lot for a response. I use Total Commander on every android phone I had, I can see that my files look like they disappeared completely, android just generated empty generic folders ("Pictures etc) after I cut off the previous folders along with their content so I saved a little bit that I could. On the other hand something is occupying 2gb of data but the phone refuses to show what, so there is a chance my files are there. After copying the whole content of the phone to the folder on desktop it just shows it weight few hundreds KB. I'll be trying to use some recovery software for android. The question is how did this happen at all? It had no right to happen, what if this happened to someone who has completely no idea about that stuff? Who's to blame and who's responsible for helping such a person? Motorola? Google? Anyway, this is the last phone with no micro-sd card slot I used. A memory card is the best backup

You COULD recover your files even if they were deleted if Android still had a mass storage mode. I don't think any of the data recovery softwares play nice with the MTP transfer that android offers. Of course, the SD card is infinitely better, but a rarity these days. I am sorry for your lost data. You seem like you did everything within your reach to get it back.
EDIT: Just to be sure, did you cut-paste the folders on your PC from phone, or just copy-paste. It might be possible for files to be deleted in cut-paste, if some error occurs in copy.

Thanks for contributing. It seems Google knows better then any of us what we need and there's no more Mass Storage option which gives free access to memory. Unfortunately I don't have that option. First I ensured which files are not affected by error by trying to copy them folder after folder. Then there were left files I couldn't copy, it was DCIM with most recent photos (unfortunately) and download where I kept lots of images with cool infographics found on the internet. I thought a reboot would help to get access to them and you know the rest. Files were there no longer. I'm seriously considering switching to Windows Phone, it seem it's too much to ask for a phone that just "werks" nowadays...I have a painful lesson to use memory card as well as cloud synchronizing but didn't give up yet, still trying anything I can I also wrote to Motorola with help request but I don't think they'll help me any better than people here, although it's more their responsibility to fix that stuff, the phone is barely 6 months old.

A lot of photos disappeared, nothing is working to recover it

Hi, yesterday while looking at my gallery I noticed that a lot of photos are missing, for example, my Italy trip photos and videos are gone, not all of them though, there are some left, but videos are all gone. I didn't delete or move them, didn't touch them at all, it just disappeared. I've tried looking at my Onedrive, Google drive backups, but they are all showing current photos, that I can already see. I've also tried cleaning gallery and media storage data in settings or deleting .nomedia files (which cant be actually deleted), but no results. Booting on safe mode, using different recovery software also shows same photos that I already see on my phone. Tried to get help from Samsung Support, sadly they are as competent as I am, all the suggestions were already known. They did suggest to factory reset my phone while backuping everything using smart switch since there might be some malfunction and doing so all the supposedly hidden photos gonna be released. Not really trusting it though, because in backup I can clearly see that the photos that are being backed up are the photos I can already see, and no hidden files are being backed up. I've also restarted my phone like 6-7 times in different circumstances and noticed that everytime my phone boots it loads up 196 pictures in my camera roll, and then after some time the other part of it gets load up. So my question here today is: Is there anything I haven't tried and is it possible to recover the hidden/disappeared photos?
P.s. the photos that I'm concerned about is in my internal storage not in SD card.

After you lose data it's many times to late to do anything. Why aren't you using the SD card as a data drive to at least backup your critical data?
That in turn should be backed up redundantly to at least 2 hdds that are physically and electronically isolated from each other and the PC. Consider this a wake up call...
This isn't looking good. Backed up what's left, now. Try in safe mode.
It may be malware, in any case it sounds like it's factory reset time if you can't ID the root cause.
If you added any downloaded images* to the folders losing data it may be a malware jpeg, if so this jpeg must be deleted or it will continue to destroy and mutilated files in the folder it occupies. I've seen ones that do this in Android. Think. What was done, installed or downloaded?
*All downloaded jpegs should be kept in the download folder until vetted. This will help limit the damage hopefully to just the loose files in there. At least open the jpeg there and look for abnormal behaviors in that folder because you add that jpeg to your database! Antivirus may not detect these little buggers... WYSIWYG

blackhawk said:
After you lose data it's many times to late to do anything. Why aren't you using the SD card as a data drive to at least backup your critical data?
That in turn should be backed up redundantly to at least 2 hdds that are physically and electronically isolated from each other and the PC. Consider this a wake up call...
This isn't looking good. Backed up what's left, now. Try in safe mode.
It may be malware, in any case it sounds like it's factory reset time if you can't ID the root cause.
If you added any downloaded images* to the folders losing data it may be a malware jpeg, if so this jpeg must be deleted or it will continue to destroy and mutilated files in the folder it occupies. I've seen ones that do this in Android. Think. What was done, installed or downloaded?
*All downloaded jpegs should be kept in the download folder until vetted. This will help limit the damage hopefully to just the loose files in there. At least open the jpeg there and look for abnormal behaviors in that folder because you add that jpeg to your database! Antivirus may not detect these little buggers... WYSIWYG
Click to expand...
Click to collapse
Okay first of all, One drive should be responsible for backing up my data, thats why its there, no? I do understand that data back up need to be taken more responsibly, but again why would I keep everything backed up from my phone since everything in there is just basic daily use things, nothing that much of importance, except of course memory photos in my camera roll (which again SHOULD be backed up in my one drive). Second, if I were that careless as an IT student to download malware into my device, why would I move it to my camera roll folder ? (thats the only folder that has photos and videos missing). Third, I did factory reset my phone with a back up from Smart Switch as suggested from so called Technical Support specialist from Samsung, sadly the results are same (All photos I had before factory reset are there, no actual recovery has been made).
Sorry if I might sound rude, but I'm really frustrated. Samsung really let me down big time for the first time in 12 years of using their products.

I feel your pain as I lost an irreplaceable database myself at one point that was 30 years old. It sucks.
The only backup I trust and use is copy/paste/verify for size and if readable.
SmartSwitch can fail you miserably, never use it for critical data... same with Google junk.
In unencrpted hdds I trust.
Maybe your Google account was hacked... again always suspect malware, viruses, rootkits if unexplained behaviors are observed.
If you had a previous SmartSwitch backup saved that probably would have worked. At the time what the Samsung rep told you in lieu of the former was your best shot probably. The data was already lost unless its backed somewhere you overlooked or forgot.
It's also possible you could have a hardware failure in progress, keep a close eye on the device.
The reason I back up they way I do is from hard lessons learned going back to XP Pro and other OS's. Also never clone (copy/paste only) or encrypt data drives. Always use a good SD card that's V30 rated like a Sandisk Extreme. All critical data goes here, and everything you need to do a full reload including installable app copies. App that allow settings to be saved like Poweramp, contacts all go on the data drive as do vid and music libraries. ColorNote will save it's backup directly to the SD card, it's a very useful that can used for bookmarks. Don't forget all passwords as well.
Keep the DCIM and download folders on internal memory. Only these and installed apps go on internal memory. Periodically backup the DCIM files to the SD card but DO NOT use the phrase dcim in the backup copy folder name... and then redundantly back up the SD card periodically. Best to time stagger these backup between 2 or more hdds. Develop a plan, put some thought into it. You have all the hardware you need for a robust dual drive device if you use it!
I have 2 N10+'s and this is how I run them. Zero issues.

@blackhawk. Agree... Never trust Google Photos. Somehow I also lost a lot of my photos. Mostly the ones that I shared (as file). Was that because I deleted them directly using Root Explorer? No. Because I downloaded them to Restored folder and deleted through Explorer? No. Because I deleted them from Sent folder of WhatsApp? No... I tried each one of these and couldn't consistently delete the items from Google photos... But how? Why? Just don't trust the application.

nugroho2 said:
@blackhawk. Agree... Never trust Google Photos. Somehow I also lost a lot of my photos. Mostly the ones that I shared (as file). Was that because I deleted them directly using Root Explorer? No. Because I downloaded them to Restored folder and deleted through Explorer? No. Because I deleted them from Sent folder of WhatsApp? No... I tried each one of these and couldn't consistently delete the items from Google photos... But how? Why? Just don't trust the application.
Click to expand...
Click to collapse
Always redundantly backup critical data and verify it's size is correct, it's readable and complete. Copy/paste only, never clone media files ie music.
A drop, malware, hardware failure, a near lightning strike, a boot loop, a user mistake etc can destroy data instantly. Expect it will happen because eventually it probably will. No such thing as overkill with data backup. Multiple copies in different locations that are time staggered and not connected to a PC preferably on hdd. An earth grounded safe or metal box is ideal to store them in at 50-70F.

Great advice! I see these posts about lost data, and people don't realize, you have to backup this data somewhere. The FIRST step is something like Google Photos or Amazon even. But these are not enough (it's a start, at least).
General recommendation is a 3-2-1 backup, as detailed here: https://www.backblaze.com/blog/the-3-2-1-backup-strategy/
Bottom line, if it's something you don't want to lose, then have a REAL backup beyond just GPhotos or similar.
Especially in today's world of cellphones and flash-drives. Both are easily lost or damaged. At least in the old days you can sometimes recover data from spinny drives (I did a LOT of this work for people, using specialized software)... but flash drives don't work the same, and they likely fail to read completely, leaving ALL the data gone in a flash (pun intended).
Backup, BACKUP, BACKUP!!!
EDIT: Also note, if you use something like Gphotos, you can very-easily download ALL the photos from there. This makes it rather easy to backup those files by simply copying that ZIPped file as necessary. You'll have to do this manually every so often, but it's quite easy to do.

This same thing just happened to me. All of my folders including my camera pics were deleted, except my downloaded folders. All the photos are missing from my backups, all trash bins are empty, and Samsung Customer Service suggested that I do everything I had already done. I thought I'd be fancy and get the ZFold 4, which doesn't have an option to use an SD Card. I even downloaded a recovery program to my laptop and it couldn't find any photos. I'm currently waiting on an adaptor, so I can download photos from my last SD card in my previous phone. I've never been more angrier with a phone than I am right now.