Database Develop

EXT Filesystems

Okay so when we first had apps to SD, we used an extended partition called EXT2. And then cyanogen and all the other devs decided that ext2 wasn't as good as ext3. Now I realize that there's an ext4. So my question is, is there an infinite list of ext filesystems and is it a coincidence that ext2<ext3<ext4?

The question relies on a bit of history.
ext - (extended file system) which was a file system used in minix and *nix.
ext2 (second extended file system) was a replacement for ext. Primarily this allowed for larger files and longer file names.
ext3 was the replacement for ext2... intuitively enough. the most important thing it added was journaling. There were quite a few other changes but the journal is the reason ext3 is favored for the G1. The short version is that an journaled file system is less likely to become corrupt if not powered off correctly.
ext4 extends ext3. From a G1 perspective the main things it provides is some performance improvements as well as journal checksumming. There are quite a few other changes but most will not be pertinent to an embedded environment. Check wikipedia if you want a complete list.
Ext2 to ext3 is completely backwards compatible - if you make a partition ext3 you can still mount it as ext2, you just lose the journal capability while doing so.
Ext3 to ext4 is not backwards compatible if extents are used. Truthfully, unless you map extents to a smaller multiple of block size of your flash card using extents on an SDHC would probably not be advisable. People enable it with the default 128MB extent size and 4KB block size none the less. It would be a long discussion as to why this is non optimal considering a flash block size of 128KB (page size 64KB I believe) but it is not worth having.... most people enable it and so ext4 in most cases will not be backwards compatible.
EDIT: As of this moment there is not an EXT5 on the horizon. Most next gen file system work on Linux is concentrating on BTRFS and a few others whose names escape me.
EDIT2: added block size to EXT4 extents to be clearer.

JanetPanic said:
The question relies on a bit of history.
ext - (extended file system) which was a file system used in minix and *nix.
ext2 (second extended file system) was a replacement for ext. Primarily this allowed for larger files and longer file names.
ext3 was the replacement for ext2... intuitively enough. the most important thing it added was journaling. There were quite a few other changes but the journal is the reason ext3 is favored for the G1. The short version is that an journaled file system is less likely to become corrupt if not powered off correctly.
ext4 extends ext3. From a G1 perspective the main things it provides is some performance improvements as well as journal checksumming. There are quite a few other changes but most will not be pertinent to an embedded environment. Check wikipedia if you want a complete list.
Ext2 to ext3 is completely backwards compatible - if you make a partition ext3 you can still mount it as ext2, you just lose the journal capability while doing so.
Ext3 to ext4 is not backwards compatible if extents are used. Truthfully, unless you map extents to a smaller multiple of block size of your flash card using extents on an SDHC would probably not be advisable. People enable it with the default 128MB extent size and 4KB page size none the less. It would be a long discussion as to why this is non optimal considering a flash block size of 128KB (page size 64KB I believe) but it is not worth having.... most people enable it and so ext4 in most cases will not be backwards compatible.
EDIT: As of this moment there is not an EXT5 on the horizon. Most next gen file system work on Linux is concentrating on BTRFS and a few others whose names escape me.
EDIT2: added page size to EXT4 extents to be clearer.
Click to expand...
Click to collapse
oo Awesome, THanks for the response. Cleared things up. Another thing...why didn't developers start with EXT4 then?

Ext4 support wasn't built into Android - Cyanogen (I think, don't mean to step on anyone's toes) added it in. And it is still a pretty new file system type, so after patching in support, Cyanogen still had to go back and bring in some newer patches to try and fix bugs.. It's very cutting edge still. Given time, it'll be the "standard" and we'll be poking at some other new file system and wondering why everyone doesn't use it.

The original kernel for android would not have supported ext4, since it was not available at the time of the G1 release.
I am going from memory but I think the cupcake release used a 2.6.27 kernel. This would have been before ext4 support was released for general consumption, though it might have been available under experimental.
Cyanogen and most of the custom ROMS out there are using a 2.6.29 kernel now, which has a general release of ext4. There are some known problems with the stock ext4 release in 29 but Cyanogen backported the fixes from 2.6.30. Since I believe most of the ROMS are "Cyanogized" you should be fine now with any of the current ROMS on a 29 kernel that supports ext4, which is not necessarily all of them. I pretty much stick to Cyanogen's ROMs so I can not say what the others do and do not have.

Saiboogu said:
Ext4 support wasn't built into Android - Cyanogen (I think, don't mean to step on anyone's toes) added it in. And it is still a pretty new file system type, so after patching in support, Cyanogen still had to go back and bring in some newer patches to try and fix bugs.. It's very cutting edge still. Given time, it'll be the "standard" and we'll be poking at some other new file system and wondering why everyone doesn't use it.
Click to expand...
Click to collapse
JanetPanic said:
The original kernel for android would not have supported ext4, since it was not available at the time of the G1 release.
I am going from memory but I think the cupcake release used a 2.6.27 kernel. This would have been before ext4 support was released for general consumption, though it might have been available under experimental.
Cyanogen and most of the custom ROMS out there are using a 2.6.29 kernel now, which has a general release of ext4. There are some known problems with the stock ext4 release in 29 but Cyanogen backported the fixes from 2.6.30. Since I believe most of the ROMS are "Cyanogized" you should be fine now with any of the current ROMS on a 29 kernel that supports ext4, which is not necessarily all of them. I pretty much stick to Cyanogen's ROMs so I can not say what the others do and do not have.
Click to expand...
Click to collapse
Ah Thank-you both of you. Cleared things up alot

This explains it nicely! Ext4 for cyan. Is it still a good idea to run the linux swap partition as well with ext4 and Cyan roms?

yes
you still want to run a linux-swap partion, swap and ext are 2 different things. Swap is where your system extends it's operating memory not storage as to say even a full computer linux distro uses a swap partion<I know my ububtu does> so yes if you want the benefits of swap than you better have a partion for it

Awesome..
Thanks to alritewhadeva for rising this question and thanks much to Saiboogu and JanetPanic for letting us win users what exactly these are for. Couldn't find better explanation even after googling about them.
gridlock32404 said:
you still want to run a linux-swap partion, swap and ext are 2 different things. Swap is where your system extends it's operating memory not storage as to say even a full computer linux distro uses a swap partion<I know my ububtu does> so yes if you want the benefits of swap than you better have a partion for it
Click to expand...
Click to collapse
Regarding this linux-swap.... I do have 32MB partition for that one created before installing the ROM. Now, are cyanogen's roms take advantage of this partition automatically or do I need to "enable" or change setting for that to happen?

If you read the first post on cyan's rom, I believe it does but as compcache, which I believe and I might not be right but I think it is just a better way of swap, I think more efficent

I repartitioned today to ext4. I used cyan's userint.sh and Mike Taylors user.conf. I'm not convinced it is working right at it seems more sluggish at times.

I just use a 96mb linux swap myself that I turn on with swapper and it is quite snappy, there was a thread were people were saying that comp wasn't as good as swap so I decided doing all that was just a hassle so I stuck with tried and true swap

This is the MT user.conf and the one I'm running.
# User.conf by miketaylor00
# General parameters
general{
apps2sd=0 # this is useless here, require a modified a2sd script
media2sd=1 # moves the medias to sd if /system/sd/media exists
}
#compcache related parameters
compcache{
compcache_en=1 # enable(1) or disable(0) compcache
cc_disksize=32 # Ram swap disksize - any number between 1 to 95 should work
cc_memlimit=18 # Limite the memory usage when backing swap is used
cc_backingswap_en=1 # enable or disable backing swap
cc_backingswap=/dev/block/mmcblk0p3 # pointing to the backingswap partition device
cc_swappiness=28 # default 60
}
#Linux swap parameters
#
# linux swap can only be enabled if cc_backingswap_en is set to "0"
#
linux_swap{
linux_swap_en=0 # enable(1) or disable(0) linux swap
linux_swap_partition=/dev/block/mmcblk0p3 # swap partition device
swappiness=30 # default 60
}
#virtual memory
sys_vm{
sys_vm_en=1 #enable(1) or disable(0) virtual memory configurations
page_cluster=3 # default 0
laptop_mode=0 # default 0
dirty_expire_centisecs=3000 # default 3000
dirty_writeback_centisecs=500 # default 500
dirty_background_ratio=5 # default 5
dirty_ratio=10 # default 10
}
#cpu clock
proc_cpu{
proc_cpu_en=1 #enable(1) or disable(0) user cpu configurations
# freqency options
# 19200
# 122880
# 128000
# 245760
# 384000
# 528000
scaling_min_freq=192000 # default 245760
scaling_max_freq=528000 # default 528000
sampling_rate=2000000 #default 200000 depending on kernel version
powersave_bias=0 # default 0, CM3.9.6 default uses 200
up_threshold=45 # default 40, percent cpu usage before going up a speed step
}

gridlock32404 said:
If you read the first post on cyan's rom, I believe it does but as compcache, which I believe and I might not be right but I think it is just a better way of swap, I think more efficent
Click to expand...
Click to collapse
Compcache is different to swap.
Swap is a virtual extension of the RAM, which is why it gives us a performance increase. Although adding x amount of swap space isn't the same as adding x amount of RAM.
Compcache compresses what's in RAM, which in theory increases the amount of RAM
Compressing pages and keeping them in RAM virtually increases its capacity. This allows more applications to fit in given amount of memory.
Click to expand...
Click to collapse

I flashed to Cyan's lastest this morning. I assume that user.conf and userint.sh was replaced with Cyans own?
I'm super fuzzy on all this, as I will be running Cyan's roms from now on, I just need a system/setup/ext that's optimized for his roms. I didn't have much luck with MT's for whatever reason. It ran, just lagged.

[SOLVED] Best swap size

Seen lots of comments saying best linux-swap=32MB, 64MB or even 128MB! Has anyone actually tested these conclusively? Would love to hear your results..

I've tried everything from 16 to 128mb for research purposes. I found that 32mb gets full sometimes and 64mb+ lags when it gets over a certain size. I use 48mb now, just to be different.

32MB, there's no point making it bigger.

Thanks guys...

Edit your thread name and add [SOLVED] prefix if you can.

Mine is set to 28MB and swappiness at 20.
I think its perfect, but no point in going over 32MB.

thanx .. no idea how to edit thread-name?

I have my linux swap set at 32MB with a swappiness of 60, with it at 60 I find it hard to fill it up.

I know this thread is old, but I don't see a reason why I should make a new thread if this one is right on topic for what I wanna talk about..
I first made an over sized swap partition of 256MB ( inexperienced -- I just got into smartphones and modding them a week ago ) and it didn't help performance much. I made a 32MB partition a couple days ago ( swappiness is at 90 ) and it really increased performance of my set.

swap
If you have a RAM equal or lower than 256
SWAP = 2 * RAM
Example
SWAP = 2 * 256Mb
SWAP = 512 Mb
If your RAM is greater just use a RAM of 512Mb

How do you set swappiness?
I used amon ra to setup swap and ext but no swapiness in there.
If its done in console that's fine, I'm comfortable there.
SPL: 0013d
Radio: 2708
Recovery: Amon Ra V1.7
Rom: CM-RC1-DS-2708port_S

@Phil;
echo $swappiness_value > /proc/sys/vm/swappiness
==
Powered by Android

Thank you very much.
SPL: 0013d
Radio: 2708
Recovery: Amon Ra V1.7
Rom: CM-RC1-DS-2708port_S

Compcache + backing swap in newer CM ROMs

Hello!
I have a G1 using CM 4.1.999, and my microSD is partitioned with an ext4 for apps2sd and a 64MB linux-swap partition.
I am using the user-configurable userinit.sh with user.conf.
I am trying to understand the cc_disksize and cc_memlimit settings.
My understanding is that it used to be that no matter how I set these I will get a compcache of 15% of total RAM with a backing swap using the entire linux_swap partition.
But I am a bit confused as this rom has compcache 0.6, and I am not sure if that limitation still applies after .5x.
If that is the case, I want to confirm... if I set cc_disksize to 0 and set cc_memlimit to 32, I will get a RAM compcache of 32MB, and it will use the entire linux_swap partition as backing swap.
Is this correct?
Thanks for any help you can give me in understanding this!

Sorry for the bump, but I'm hoping a little more clarification might be a little more answerable...

Well i cant give you a definitive answer, but i can point you in the right direction.
If you don't get much help here, you can try PM'ing miketaylor00
he spent considerable amount of time talking about the subject and his user.conf is implemented in JACxROM
you can send him a link to this q&a and maybe he'll help you out. im sure a lot of people would like more clarification
i also believe the defaulting to 15% of ram was fixed in 0.6 compcache
i havent messed with these settings much. i messed around with them when they first came out but stopped after 0.6 and just started using whatever was included default in the rom. swappiness was pretty much the only settings i changed

bradycl_84043 said:
Hello!
I have a G1 using CM 4.1.999, and my microSD is partitioned with an ext4 for apps2sd and a 64MB linux-swap partition.
I am using the user-configurable userinit.sh with user.conf.
I am trying to understand the cc_disksize and cc_memlimit settings.
My understanding is that it used to be that no matter how I set these I will get a compcache of 15% of total RAM with a backing swap using the entire linux_swap partition.
But I am a bit confused as this rom has compcache 0.6, and I am not sure if that limitation still applies after .5x.
If that is the case, I want to confirm... if I set cc_disksize to 0 and set cc_memlimit to 32, I will get a RAM compcache of 32MB, and it will use the entire linux_swap partition as backing swap.
Is this correct?
Thanks for any help you can give me in understanding this!
Click to expand...
Click to collapse
Yes, your assumptions are correct. If you have backing swap enabled it will use the entire swap partition. You can check the stats with this command:
Code:
su
rzscontrol /dev/block/ramzswap0 -s
The disksize value when you have backing swap enabled should always equal the size of the swap partition. I recommend a 32MB swap partition if you are using a ROM that doesn't have BFS and a 64MB swap if it does have BFS.
The memlimit is the amount of uncompressed data that can be used by compcache. It usually compresses down to about 20-25% of the uncompressed size so with a 32MB setting it will only use about 5-8MB of RAM. I've been using a 32-48MB memlimit lately and it works great. If you are on a BFS ROM I would go with 48MB. I believe that CM is using BFS I just wanted to give the info for both in case someone else reads this down the road. There is a lot of info about the stats on this page.
http://code.google.com/p/compcache/wiki/StatsExplained
If you have any more questions shoot me another PM. I hope this helped.

What's a good partition for a CM rom?

Fairly new to CM, currently running the latest version (v4.2.12.2). I kept my partition from a previous build (Swap = 128 MB and Ext3 = 640 MB).
So what's a recommended partition for a CM build to run smoothly?

BoomBoomPOW said:
Fairly new to CM, currently running the latest version (v4.2.12.2). I kept my partition from a previous build (Swap = 128 MB and Ext3 = 640 MB).
So what's a recommended partition for a CM build to run smoothly?
Click to expand...
Click to collapse
CM doesn't use swap so it's ok to get rid of that unless you frequently try roms that need swap. And I would just use 512 for ext3, either size is an ungodly amount of apps

For donuts i like EXT=128MB & of course the rest if for Fat32.
I normally dont install 40>X apps.

[Q] largest rom?

what is the largest rom that i can flash to my g1 with danger spl?

Any rom taht uses the (death,danger,haykuro) SPL can be flashed devs try to make it fit with out danger, but sometimes its jus needed for all the extras

yerand said:
what is the largest rom that i can flash to my g1 with danger spl?
Click to expand...
Click to collapse
DeathSPL makes no difference regarding the maximum size of ROM you can flash since we can now load custom partition tables with ANY SPL. The maximum size that you can store INTERNALLY is about 208 MB (would require userdata and cache on the sdcard though.
Technically, anything less than or equal to 208 can be stored internally, though theoretically, you could store part of a ROM on the SDCARD, which is up to 32 GB, so 32GB+208MB - a little bit of space for userdata+cache. If you want everything internal, then it is up to you to figure out how you want your storage allocated.