Database Develop

can someone tell me if my thinking is correct

ok so i didnt really know what swap was or compcache with backing was so i spent the day lookin it up and i just want to know if what im getting out of my info is correct.
swap, basically instead of your phone just seeing ram it also sees your swap partition as part of the ram. so it stores app data and etc. to the swap partition thus making your phone run faster and not taking up all your internal ram and make it run slower. and it is managed by linux. by changing your swappiness setting either higher or lower will change how often swap is used.
compcache, basically creates a sort of swap partition inside the ram itself. it compresses what it swaps into this partition to create more space in the ram. im guessing people like this with backing swap because the compressed info can be brought up quicker because its compressed in the phones ram rather than like swap were its on the sd.
backing swap, when compcache is full it uses the swap partition on my sd. but rather than being like linux swap were its controlled by linux, backing swap is managed by compcahe wich works together better than compcache and linux swap.
im a complete noob here lol so if i got something wrong you gotta explain it to me in laymans terms thanx for the help

Yes, that's right. Did you copy and paste those definitions lol?

lol no i did a lot of google searching its all other peoples info but i put it into words that i can understand.lol

Amack that almost deserve a sticky

this post help me understand the way the partition works alittle better now, thanks!

Question about swap and internal G1 memory

Im new to all this swapping stuff etc. so sorry if this is a stupid question. When i type the free command I see that my phone memory is also almsot used up with like 1000-3000k being frees and my swap memory only being half way used (i set it to 64 mb) is there a way i can make the swap be the primary source of memory?
Hope that made sense, Thanks in advance!

No, you can't set virtual memory to have the priority. If you have a user.conf and userinit.sh you can change the swappiness value, which I'm assuming would mean more virtual memory is utilized.

Compcache + Swap - Sorry guys....

Sorry for posting another question relating to this.. I'm having a heck of a time trying to wrap me head around...
ROM: Cyanogen 4.2.10.1
I have a 4GB class 6 SD card in my phone partitioned (rounded):
3300MB fat32
500MB ext3
200MB swap
I'm have the userinit.sh in /system/sd as
Code:
#!/system/bin/bash
#
partition=3 #partition number of swap partition. Change if swap is not partition 3.
mem_limit_kb=14688 #default 14688 (15%). Range 0 - [size of swap partition], larger number = less 'normal' RAM, more 'swap' RAM
swappy=30 #swappiness. larger number = more likely to swap, smaller number = less likely to swap
modprobe ramzswap;
rzscontrol /dev/block/ramzswap0 --backing_swap=/dev/block/mmcblk0p$partition --memlimit_kb=$mem_limit --init;
swapon /dev/block/ramzswap0;
sysctl -w vm.swappiness=$swappy;
exit 0;
as per the Cyanogen wiki...
If I run "free" from the terminal I get
Code:
free
total used free shared buffers
Mem: 97924 96448 1476 0 2940
Swap: 24472 12940 11532
Total: 122396 109388 13008
What I'm having trouble understanding is "mem_limit_kb=14688". What memory is it referring to? The built in RAM?
If so, does that mean to put active programs into the remaning 80MB of RAM and store the "images/pages" of inactive applications into the 14688KB of reserved RAM for Compcache? After the 14688KB of Compcache is filled, should it not then dump everything to the Swap?
The reason I'm confused is that I have 200MB of swap partition, and yet I still load up the browser from memory and it reloads pages... Or is it reloading the page, just this time from a local cache (which would be silly)?
Finally, what exactly does "swappiness. larger number = more likely to swap, smaller number = less likely to swap" actually mean? I don't really understand that at all...

Its not a good idea having both enabled in the user.conf..One will be set as back swap while the other is be set to 1.

To be honest I have no idea.. heh. I just copied it from the Wiki.. To me it made a bit of sense running both... Compache to be first since it should be a bit faster than swap... Atleast that's how I saw it...

stock sd card, hidden stuff?

i might be way off on this but im just being cautious. I decided i was gonna replace the 4gig class 2 sd that comes w/ the nexus with the 4gig class 6 i had in my g1. i decided to use my g1 to format the nexus sd card but noticed 1 thing i guess i just dont understand. When i dropped into parted with the nexus sd card i typed "print" just to see if it was only 1 partition but what i seen was this
Number___Start_______End_________Size______Type_____File System
___1____4194kB_____3965MB_____3961MB_____Primary______fat32
can someone explain to me what would be from 0-4194kB and also how the end size is bigger than the complete size. If that is how its supp to be and im just misunderstanding then please update me on how it actually works or if ive noticed something that needed to be pointed out.
Thanks, veritasaequita

3961MB + 4194kb = 3965MB, the "End" Size is just an Index. Partition starts at 4M, is 3961 MB in size, so it ends at 3965M
About the missing 4M at the start, I'll take a quick look at it.
EDIT:
Looks like all zero to me, if you substract the mbr from the first 4MB. Perhaps somebody at google did some clever aligning? but then 4mb would be huuuuuge

thats kinda what i was thinking, actually my first thought was if it was a 4mb sorta kinda "swap type partition" for performance....prolly not though. im a perfectionist so im not gonna swap cards until i can fig out what or where those 4mb are and have them on my new card....i dunno my ears are smoking now...lol
veritas

well you can always just dd the old card to the new one and then enlarge the fat partition (or in your case: just omit the last step )
but why make such a fuss about 4mb of zeros? i've seen lots of strangely partitioned usb-sticks/flash disks --and-- a lot of people use other cards in ther n1's (because of the crappy size the n1 ships with). if you must you can always make a dd to a new card or store the dump somewhere for restore later (or just put the old card somewhere safe?).
but still, that seems paranoid and not perfectionistic... don't worry so much
ps: yeah, swap with 4mb wouldn't help much and on the other hand they would just mark it as "swap" in the partition table if it were the case.

im following, thanks for the insight bro
veritas

Just a guess, but most modern hard drives have a hardware cache at the start of the disk. i.e 8MB, 16MB, 32MB etc.
Maybe your SD card uses 4MB.
Also, the manufacturers use the non-technical "Billion Bytes" instead of "Giga Bytes" in the fine print. 4000000000 B (Billion Bytes) = 3814 MB

nope, cache would be invisible by definition.
also the 4mb are there just not allocated to any partition. this has nothing to do with how you calculate the size of the disk.

Just out of curiosity could there ever be a possible way someone like perhaps google could store something hidden to everyone else in that 4mb for maybe like a factory backup of the file system or a backup bootloader? Just exercising my brain, thats all...lol

It's nothing basically, all the hard drives in your PC will have a few meg space you can't partition.

veritasaequita said:
Just out of curiosity could there ever be a possible way someone like perhaps google could store something hidden to everyone else in that 4mb for maybe like a factory backup of the file system or a backup bootloader? Just exercising my brain, thats all...lol
Click to expand...
Click to collapse
well of course you can store something there and if the IPL of the phone can access the sd card it certainly could execute that or flash something. there is a broad range of possibilities what you could and couldn't to with the 4 meg (a factory backup would be a little bigger ) and a backup bootloader wouldn't make sense, because you can feed something to the current loader directly. if you do anything worse to the phone like erasing the IPL, well thats what JTAG is for, just program the chips directly.
so of course this could be an evil tracking cookie and we all could get even more paranoid, but still its all zero's and not really "hidden" in any way, just not allocated and directly visible.
what i want to say: every facebook or other kind of web 2.0 shouldn't be worried about 4 unexplainable megs on some sd card

Any possibility to correct the forced 32K cluster size on 32GB SD cards?

Learned last night from Patriot Memory what I already knew without realizing it...with 32GB (only) SD cards on what appears to be any version of Android, the cluster size -must- be 32KB (I had tried both 64KB for performance and 4KB to reduce slack wastage) or you *will* encounter corruption (possible root cause for some of our SD-card-booting brethren?).
Google has no fix according to Patriot and this is a widespread problem (across phones and tablets), though I can't for the life of me figure out why.
Is this a kernel-level issue with the filesystem? Any idea of whether this is something that might get fixed for CM7 to make it even more compelling than it already is?
Rodney

Rodney, I think the answer is no, unless we were to go to NTFS which is not robustly supported. here is a reference for FAT32 cluster sizing and formatting. Informative but some what dated.
http://www.infocellar.com/hardware/harddrive-cluster-sizes.htm
anyway, for 32gb disk , 32k clusters are mandated.
FAT32 offers efficient 4KB clusters for partitions up to 8GB. The cluster size increases with the size of the partition, up to 32 KB for partitions over 32GB

DigitalMD said:
anyway, for 32gb disk , 32k clusters are mandated.
Click to expand...
Click to collapse
Microsoft seems to think differently, as they offer a number of different cluster sizes for that partition using FAT32, all of which work (on their OS). If you're saying it's the Linux FAT32 VFS that's busted, that's fine...but it's still busted.
Rodney

It's not busted, its just the way Fat32 works for compatability. That's why Linux has ext3 and Windows has NTFS. Fat32 is old, but universal.

It is busted, and there is no such "compatibility" rule. The web page you referenced is mistaken.
The _only_ valid rule, in Microsoft's own code, is as follows (assuming 512-byte sectors):
FAT32: 65,526 < Number of clusters < 268,435,446
So for a 32GB device, let's look at the number of clusters we create with various cluster sizes...
64KB: 500,000
32KB: 1,000,000
16KB: 2,000,000
8KB: 4,000,000
4KB: 8,000,000
2KB: 16,000,000
1KB: 32,000,000
512: 64,000,000
(no cluster size smaller than this because a cluster can't be smaller than a logical sector)
All of the above cluster sizes *are* supported by Microsoft, and *should* be supported by any "compatible" implementation of FAT32...thus, there is an _in_compatiblity with Linux's VFS implementation of FAT32 that does not adhere to the standard, and thus causes data corruption.
Don't believe me? Format a 32GB SDcard with 16KB clusters on a Windows box. Copy 100 100MB files to it, mounted directly to Windows, and check each for corruption. Repeat the test after inserting the SDcard into an Android device and USB-mounting it to the same PC. You _will_ see corruption, that _is_ specific to the Android device. I imagine it would be seen under other Linux distributions as well, but I've seen no anecdotal evidence and can't be bothered to test since the evidence is pretty clear.
Obviously, if 32KB were the only permitted/possible cluster size, Patriot wouldn't have had to warn all its customers about the risk of data corruption when _not_ using that cluster size.
All I wanted to know is whether anyone had even looked at it, cared to do so, or were even aware (within the dev community), so that a bit of attention might *possibly* be paid to it to save the long-term pain of users having (silently) corrupted data. It's the worst kind of data loss, and if it's an easy fix, worth doing...don't you think?
Rodney
P.S.: I'm not going to get into any further semantics wars. I am continuing to tire of the ego-positing on the forum. There's nothing wrong with being wrong (I know - I do it a lot!), but continuing to defend your position beyond that is both hopeless and counterproductive.
P.P.S.: It's worth noting this defect does _not_ exist for 16GB and smaller FAT32 partitions in Android. For those playing along at home, many of the files Android creates itself - and many I store on SDcards - are smaller than 32KB, so the whole point of the exercise in adjusting the cluster size downward is to avoid wasting space.

whatever man... just chill

DigitalMD said:
whatever man... just chill
Click to expand...
Click to collapse
Misinformation bugs me (no pun intended), because it leads to defects just like this. There are a great many resources on the Internet that are Just Plain Wrong(TM), Wikipedia notwithstanding (which I personally think gets a bad rap, but that's a whole different subject). If it weren't for the fact that this bug trashes user data without notice, I wouldn't care so much, but I pity the fools that come after me that format their SDcards with smaller cluster sizes thinking they are being clever about space usage only to find a large number of their files are toast. It's not fun.
Rodney

What are you going to do with the sd card?!
If you put 10000 files on it you will only gain 7,8 MB with a 16kb cluster size.

TDO said:
What are you going to do with the sd card?!
If you put 10000 files on it you will only gain 7,8 MB with a 16kb cluster size.
Click to expand...
Click to collapse
Actually, 10,000 1KB files with a 32KB cluster will result in 310MB of wastage - moving to a 16KB cluster size reduces the slack to 150MB. In practice, I have a -lot- more than 10,000 1KB files, largely emulator files...in fact, it's a bit worse, because each of those 1KB ROMs is zipped down considerably, so the real wastage is ~315MB for those files alone. Yes, this does represent only 1% of the total storage, but that's also a single directory.
Besides, I'm anal.
Rodney

http://support.microsoft.com/kb/314463
Maximum cluster size for FAT32 is 32K, but otherwise, sounds like something is busted.
Hard to believe since the FAT32 code should be coming from Linux mainstream which is pretty old at this point.