Database Develop

[HACK] mount sdcard as ext4

These patches add ext4 support for your sdcard (in the Nexus S - the internal card.) There are a number of reasons you might want to do this - performance (ext4 is faster, plus mounting ext4 is basically instantaneous, which is very nice on reboots...no more checking sdcard), it's more efficient, you get a bunch of file system security features (if you care...)
You should not apply them if you don't really know what you are doing. The patches are resonably benign. Converting your sdcard partition to ext4 is NOT NOT NOT benign and you can really hose yourself doing it.
I'm not going into great depth with these instructions. If you don't understand them, play around with building cyanogen, installing it, etc until they are crystal clear.
To swap your sdcard to ext4:
* Apply the patches and rebuild cyanogen. Rebuild it. They will change your recovery image and vold.
* Install vold and flash (and/or boot) the recovery image.
* Boot into recovery, mount your sdcard and back it up to your computer (i.e., not nandroid, copy the files.) Nandroid would be a good idea too.
* Format your media partition to ext4 (the one that is vfat, by-name is media, mine is partition 3) mkfs.ext4 can be found by googling, or you can use make_ext4fs from /system/bin. I used mkfs.ext4.
* You should be able to manually mount that.
* Push your files back.
* sync and reboot.
Et Voila, you should have an ext4 sdcard partition.
THIS IS VERY DANGEROUS.
YOU CAN REALLY HOSE YOURSELF DOING IT.
Here are the patches:
http://review.cyanogenmod.com/#change,1705
This one adds ext4 support to vold, thus allowing gingerbread to mount ext4 partitions for the sdcard (which on the Nexus S, is the 'media' partition.)
and:
http://review.cyanogenmod.com/#change,1716
This is a change to recovery, making it seamlessly mount ext4 partitions for /sdcard.
Code:
# mount
/dev/block/platform/s3c-sdhci.0/by-name/system on /system type ext4 (ro,noatime,barrier=1,data=ordered,noauto_da_alloc)
/dev/block/platform/s3c-sdhci.0/by-name/userdata on /data type ext4 (rw,nosuid,nodev,noatime,barrier=0,data=ordered)
[b]/dev/block/vold/179:3 on /mnt/sdcard type ext4 (rw,nosuid,nodev,noexec,noatime,barrier=1,data=ordered,noauto_da_alloc)[/b]
/dev/block/vold/179:3 on /mnt/secure/asec type ext4 (rw,nosuid,nodev,noexec,noatime,barrier=1,data=ordered,noauto_da_alloc)
tmpfs on /mnt/sdcard/.android_secure type tmpfs (ro,relatime,size=0k,mode=000)

....and if you turn on USB Storage, the sdcard shows up as an ext4 disk...
As expected, but this is going to be one for the Linux geeks.
Wow, it is VERY nice to have that sdcard mount instantly on boot. VERY nice.

oh the read/write speeds

If anyone is interested in playing with this, I submitted the change to cyanogen:
http://review.cyanogenmod.com/#change,1705
The tricky bit is getting the filesystem created. You can use:
/system/bin/make_ext4fs.
That leaves me with a recovery which will not currently mount sdcard as ext4 unless I fiddle with fstab and mount it myself. On to look at that.

...and, here's a patch to the recovery which will let it mount /sdcard as either ext4 or vfat, making this change reasonably seamless.
http://review.cyanogenmod.com/#change,1716

i would love to see a Quadrant or a benchmark after this

this is and awesome hack to use with App 2 SD
it'll be lag less
the only annoyance, is that if you want to MOUNT USB on a Windows PC, then you'll need a software to read EXT4
for anyone interested this will be handy
http://www.ext2fsd.com/?page_id=16
i'll definitely be doing this MOD, as i'll be running anything i can from the Internal SD

So this will only work on CM7?

Please some of the genius here may post a step-by-step tutorial or howto for this one? It will be very apreciated.

No, it will work with any build, but you need to rebuild vold in order to get it to mount. The easiest way to get them outside of the CM tree would be to build cm and then copy the vold out and use that. It should work with any ROM.
The latest versions - which are in CM's gerrit - include modifications to the sdcard utility to use fuse to mount the filesystem using FAT semantics (i.e. - bypass security). Frankly, this is a waste. It slows everything down, noticeably, and app problems are few and easily fixed if you know how to fire off a chmod.
I haven't provided step by step instructions since it can seriously hose your phone if you don't know what you are doing.

DebauchedSloth said:
No, it will work with any build, but you need to rebuild vold in order to get it to mount. The easiest way to get them outside of the CM tree would be to build cm and then copy the vold out and use that. It should work with any ROM.
The latest versions - which are in CM's gerrit - include modifications to the sdcard utility to use fuse to mount the filesystem using FAT semantics (i.e. - bypass security). Frankly, this is a waste. It slows everything down, noticeably, and app problems are few and easily fixed if you know how to fire off a chmod.
I haven't provided step by step instructions since it can seriously hose your phone if you don't know what you are doing.
Click to expand...
Click to collapse
Wait so it's a waste? It isn't worth using ?

Anderdroid said:
Wait so it's a waste? It isn't worth using ?
Click to expand...
Click to collapse
If done right this would increase the read/write speeds. For example, your boot up time would be much faster and ext4 is more effecient. So it's not a waste but it has to be done correctly otherwise is could do more damage than good

Using the version I posted to CM gerrit, which bounces the file system through fuse and mimics FAT security, is still better than FAT (IMO), but it's not nearly as fast as just mounting the filesystem as Ext4 - though it is more compatible.
I've been running mine mounted as straight ext4 for a couple of weeks. Probably the single best mod I've made to any of my phones. There are occasional app bits that I need to fix manually (such Dropbox resetting file ownership), but it's worth it for the quicker boot up and faster overall operation.

Here you find the ROM CyanogenMod 7 compiled for Nexus One with ext4 sdcard mount support:
- https://github.com/diegostamigni/nexus/tree/master/one

Is the only reason to pipe it through fuse to get it to mount on m$? If so I'm not going to bother - I only use linux almost exclusively these days

diego.stamigni said:
Here you find the ROM CyanogenMod 7 compiled for Nexus One with ext4 sdcard mount support:
- https://github.com/diegostamigni/nexus/tree/master/one
Click to expand...
Click to collapse
No version for Nexus S?

Looking forward to this develpoing, as the ext4 conversion (done via modaco rom beta2 in recoverymode) made a HUGE difference to the galaxy tab i have (halved the loading times of everything). ^^
Unfortunately im far to noob to help out though ...will be keeping a keen eye on it all and learning

Forgive my ignorance, but why don't phones come like this?

NicholasQ said:
Forgive my ignorance, but why don't phones come like this?
Click to expand...
Click to collapse
window$ more then likely.
window$ can't even see Linux file systems without special drivers installed.
if i mess this up would it brick my phone? i'm really wanting to try this but i don't want to turn my Nexus S into a paperweight

so if we did this, would the windows computer be able to read the mounted sd card? is there any other things that would be needed after this mod? or just format to ext4 and be done with it?

Alternate Webtop partition

I was hoping to be able to figure all of this out on my own (so that I could impress everybody with my brilliance, of course! ), but I've hit a brick wall, so I figured I would see whether anybody had any suggestions.
The short version: I'd like to be able to create an alternate and larger /osh partition that we can run Ubuntu off of so that we can install software more freely as well as not worry about any future software updates that might conflict with the partition contents.
My thought was that it should be possible to have an alternate partition to /osh that's used to run from (whether it's mounted on /osh or not is largely irrelevant).
However, the problem I'm hitting up against is a matter of timing. The best entry point I've found to modify behaviour is /system/bin/mountosh when it's called by /init.rc, especially since it's responsible for most of the webtop setup. The catch is that by the time mountosh is called, the only partitions mounted are through p16, meaning that internal storage (p18) isn't available and the SD card (mmcblk1) isn't available either, meaning that there's no way to use either (a loopback file on the former, a partitioned SD card for the latter). I suppose it would theoretically be possible to create a loopback file on p16 (data), but that's only 2 GB and would conflict with other uses.
Does anybody know how to force internal storage and/or the SD card to be mounted earlier? Or is this going to require hacking a kernel (and bootloader and...) that we don't have the source code to yet? If I can get past this piece, then I'll be able to help other users get a webtop that would be in much better shape (especially since this would let us start using apt without having to worry so much about space issues). Thanks!

I am not sure off hand at the moment, but you might have a look at /etc/fstab and see if you can have the sdcard mounted earlier there. While having it mounted earlier may have some unexpected side effects; I presume the driver for the sd card reader is compiled into the kernel, so I don't see a reason it can't be mounted earlier.
But, not having an atrix anymore I can't test this outright.

Fenny said:
I am not sure off hand at the moment, but you might have a look at /etc/fstab and see if you can have the sdcard mounted earlier there. While having it mounted earlier may have some unexpected side effects; I presume the driver for the sd card reader is compiled into the kernel, so I don't see a reason it can't be mounted earlier.
But, not having an atrix anymore I can't test this outright.
Click to expand...
Click to collapse
/etc/fstab is pretty meaningless, unfortunately. Most of the mounting is performed within /init.rc, and I see no entries for the external storage there. As it stands, the entry for the internal storage is more in the format of "on X, do Y" rather than the more traditional procedural format, so I'm not sure how much control I can have over that.
Also, second thoughts tell me that putting a filesystem file in /mnt/sdcard would be stupid, because it would cause all kinds of issues when attempting to USB mount the partitions from a computer.

I think the best way to attack this would be to pivot to another filesystem structure on the external sdcard. I'll take a look at init.rc later today and see what can be done.

agentdr8 said:
I think the best way to attack this would be to pivot to another filesystem structure on the external sdcard. I'll take a look at init.rc later today and see what can be done.
Click to expand...
Click to collapse
So you'd think. And that's what I did. I have a nice 2 GB ext3 partition on my SD card with the contents from the /osh partition (p13) rsync'ed over. But, only the mmcblk0 partitions are available when mountosh runs (up through p16), with no mmcblk1 partitions. Naturally, /dev/block/vold is empty as well.
I might need to poke some contacts to see if it's possible to detect mmcblk1 earlier, but in the meantime, I might just create a filesystem file in /data (p16). Hell, /data is almost 2 GB large, and I'm only using a little over 500 MB in it, so there's plenty of free space there. Why did they make it so large? Do people really download that much crap to their phone?

Grr...
So, even creating a file in /data and mounting that via loopback doesn't work, even if it's mounted on top of the existing /osh partition. I'm guessing that they have some level of verification.... So yeah. Something else is missing here.

I had two sucessful ways of overcoming this. The first way was to move /usr onto /data, and to symlink it back. But, I wanted a way of not modifying /osh, so I ended up creating a init.d script to bind a copy of /osh over /osh. I don't have access to a dock or HDMI monitor at the moment, so I can't confirm if the webtop launches with either of these methods, but full functionality worked in the command line.

shawnbuck said:
I had two sucessful ways of overcoming this. The first way was to move /usr onto /data, and to symlink it back. But, I wanted a way of not modifying /osh, so I ended up creating a init.d script to bind a copy of /osh over /osh. I don't have access to a dock or HDMI monitor at the moment, so I can't confirm if the webtop launches with either of these methods, but full functionality worked in the command line.
Click to expand...
Click to collapse
Yeah, that's the thing. Everything worked on the command line, but X itself refused to start, even after I completely disabled TOMOYO (even when running /etc/init.d/startX.sh by hand).
I might decide to poke at it another time, but I'll at least wait for the SBF situation to sort itself out first.

Need help rebuilding emmc

Ok fellas, my nook color has been out of commission for over 2 months. I can boot from sd but not from internal emmc.
Sam has been helpin me but it has been difficult due to him being in a different part of the world. From what i've been told my emmc is corrupt and my partitions need to be rebuilt or formatted.
I have tried the different fixes here and still i am stuck on the N screen.
I am trying to return my NC to stock form.
I will let Sam chime in also. I know there are a lot of talents on this forum and pretty sure this can be fixed.
Thanks everyone..
Sent from my SGH-T959 using XDA App

Hi,
As Wendell said, I've tried helping him with his corrupt emmc.
The problem is:
The emmc semms to be corrupt, or at least some sectors of it. Wendell cannot return it, because it boots to CWR. That's the strange part, though the disk seems totally corrupt, it still seems to be able to boot. I guess some sectors and still working. But I don't know of any way to find out which..
This is what I know so far:
The internal emmc generally returns I/O error when trying to use fdisk or parted.
But, somehow, depending on the bs and count, I can dd /dev/zero to it.
Even after having dd zeros to it, it still seems to boot to internal CWR. So, the dd isn't working. Still don't know why, or how to remedy to it.
If someone could give us a clue or two here, it would be very much appreciated. I must admit I'm really no HDD expert..
Thanks!!
Sam

http://nookdevs.com/Flash_back_to_clean_stock_ROM
Have you guys tried this? Or has an internal EMMC rom been flashed?

Any idea how the partition became corrupt? Did you dual-boot?

Partition became corrupt after flashing a couple of the very first OC kernels. Not sure if thats what caused it but wouldn't boot past N screen. Since then i have been trying to fix it by doing the several fixes to no avail and possibly damaging the emmc even more.
Sent from my SGH-T959 using XDA App

fprice02 said:
http://nookdevs.com/Flash_back_to_clean_stock_ROM
Have you guys tried this? Or has an internal EMMC rom been flashed?
Click to expand...
Click to collapse
Please correct if I'm wrong but i think i need the stock recovery for this method. Unfortunately, the stock was replaced with CWR and now i cannot even get rid of CWR off the internal. If I could do it again, I would not place CWR on internal but rather run it off sd. I installed CWR via rom manager because the sd method was not yet available.

Try option number 3 in the thread I made in the general forum. the clockworkmod recover + stuck at " " screen. Flashing cwm via rom manager can cause corrupted /boot partition
Sent from my Vortex using Tapatalk

Woot , thanks for your response.
I have tried those methods to no avail. I can currently boot into CWR via 'home+N' but here is the crazy part, I cannot use the zip that sam made to remove CWR from emmc or even try to update to 3.0.1.0 via the CWR zip. My CWR on the emmc will not budge at all.
Sam, has even tried to erase my emmc totally from being able to boot to CWR just so I can get it looked at by B&N but CWR will not budge.

Excuse my ignorance, but will linux run off the sd card on the nook?
What about using something like GParted? Perhaps the USB bootable image on the sd card might work?

I know you are all trying to help. But please keep in mind that there aren't any obvious answers here. The emmc is corrupt. You can't flash anything to it.. and as I already stated, parted won't work either, it returns I/o error.
The only strange thing is that the nook still boots.. I can't understand that as the emmc lacks any partition table.
We need someone with hdd repairing experience. None of the answers found on existant threads will help Wendell..
Thank you.
Sam
Sent from my Desire HD using XDA App

samuelhalff said:
The only strange thing is that the nook still boots.. I can't understand that as the emmc lacks any partition table.
Click to expand...
Click to collapse
The boot may be from real hardware/non-eemc rom.
I know you said it's a emmc error - but have you tried to flash back to stock using CWM off of the SD with thecubed's MonsterRoot pack?

Yes
Sent from my SGH-T959 using XDA App

I'm sure you have tried this... but just in case...
Can you fdisk the partitions... then dd image img files to it?
I don't know if you will have any way of saving mmcblk0p2 and 3 to save your device specific information.

I believe sam has already tried
Sent from my SGH-T959 using XDA App

Here is what i would try.
First thing, get a cwm sd card setup that is bootable and has a decent amount of space so you can backup your emmc. The cwm sd card will allow us to adb in, but not worry about having any emmc partitions mounted (booting to a full android image would want at least the mmcblk0p2 partition mounted).
Make sure you have adb setup and working, so you can adb in once you've booted into cwm from the sd card. Make sure you know how to use adb pull and adb push to get files onto and off of the device easily.
Boot from the card, adb shell into the device. make sure when you are doing dd commands you are pulling from and writing to image files in a directory on the sd card, not on the emmc. you probably shouldn't even have any emmc partitions mounted at all.
Note: dd and bzip2 are both on the cwm 3.0.10 ramdisk, dunno about the earlier versions
I'd make a backup image of your emmc just in case things go from bad to worse:
Code:
dd if=/dev/mmcblk0 | bzip2 -9 > backup.img.bz2
This will take a while.
Now you'll need a MBR from someone with a stock partition table (i'm dual booting so mine probably won't work, otherwise i'd make one for you). Any volunteers?
Code:
dd if=/dev/mmcblk0 of=mbr.img bs=512 count=1
Once you have this file, just dump the image onto your emmc.
Code:
dd if=mbr.img of=/dev/mmcblk0
once you have a working partition table, you should be on your way to using the standard recovery techniques peppered throughout the forums.
btw, this won't fix any corruption in the extended partition table, that would take a little more effort (i'd have to read up on where/how that info is saved). Also obviously if it's a hardware problem rather than just corrupted data you have more problems than this will fix.
p.s. cwm loading witch a corrupt partiton table isn't that weird imho. the bootloader has barely over 400 bytes to load up the boot file and hand things off, it's probably not taking a close look at the partition map when it does this.

I know am probably being a stupid noob...have you tried the 8 failed boots?
Sent from my NookColor using Tapatalk

wendellc said:
Woot , thanks for your response.
I have tried those methods to no avail. I can currently boot into CWR via 'home+N' but here is the crazy part, I cannot use the zip that sam made to remove CWR from emmc or even try to update to 3.0.1.0 via the CWR zip. My CWR on the emmc will not budge at all.
Sam, has even tried to erase my emmc totally from being able to boot to CWR just so I can get it looked at by B&N but CWR will not budge.
Click to expand...
Click to collapse
Again..probably being a noob...are the file permissions set as ro? Mayhaps you need to change permissions?
Sent from my NookColor using Tapatalk

Lol.
Thanks for all your input.
DD and fdisk return i/o error. DD only works if I use i.e 1M as BS.
It seems there isn't any partition table on the emmc.
I guess some sectors are dead. The idea would be to identify which ones and erase them.
Although simply being able to remove the bootloader would allow Wendell to bring it back to the shop..
Maybe someone has a clue as to how to delete completely the first X sectors of emmc..?
Thanks.
Sent from my Desire HD using XDA App

samuelhalff said:
Lol.
Thanks for all your input.
DD and fdisk return i/o error. DD only works if I use i.e 1M as BS.
It seems there isn't any partition table on the emmc.
I guess some sectors are dead. The idea would be to identify which ones and erase them.
Although simply being able to remove the bootloader would allow Wendell to bring it back to the shop..
Maybe someone has a clue as to how to delete completely the first X sectors of emmc..?
Thanks.
Sent from my Desire HD using XDA App
Click to expand...
Click to collapse
Did you read my post? It is exactly telling you how to replace the first sector of the emmc. you can source it from /dev/zero with a bs=512 and a count=1 if you really wanna wipe the bootloader, but better would be just getting it working yeah? all our nooks have the same partition map (unless we are dual booting) and that map is stored on the first sector. My instruction tell how to pull the first sector with dd, and how to push it back on. notice i'm accessing the mmc directly? not the partitions? so unless his hardware is bad (or the extended partition is corrupt too) then my instructions will work. He just needs a copy of the first sector of a working nook. I'd put mine up but i'm not sure it would work, i have two extra partitions from stock. It probably wouldn't matter, but i'm not 100% sure on that, so i'd rather it be someone else that makes the image from a nook with a stock partition map (the contents of the partitions can be whatever, as long as they are all in the same place as stock).

Hi,
Thanks. Yes, I read your post. dd, for some reason, only works if I set bs to at least 1M. I don't have the NC, but maybe Wendell could confirm that.
Wendell: go to "adb shell" and type [dd if=/dev/zero of="/dev/block/mmcblk0" bs=512 count=1] without the [].
Please confirm that this returns an I/O error.
But for some reason, using 1M or greater as bs works, but the bootloader still isn't erased.. it still boots to CWR..
I'm pretty sure all the I/O error i've had mean that there's a hardware failure.. But that's my opinion, nothing more..
Thanks.

Can we repartition to make swap?

I've been thinking about this for a few days now. Then, today, xda posted an article on repartitioning the Xperia to gain some advantage. So it seems like a good time to pose this question for us HD+ owners.
This a "feasibility study" question. I'm not a developer. I can hardly hack my way though Mint, let alone the HD+. I'm asking if the following possible?
(If yes, maybe a nice developer will do it for us? )
Can we shrink and repartition the /factory partition in the HD+ (as they shrink partitions in the Xperia) and use the leftover space as /swap?
Background
I've been reading leapinlar's excellent treatises on the partition structure in the HD+ (see Item 16) and how the /factory partition, meant for automatic disaster recovery, doesn't do us much good if you've installed a custom ROM and a custom recovery. (See all of Item A12.)
Among other things, the /factory partition contains an image of the stock B&N ROM. Which will cause bootloops if triggered when you're running CM.
But, if I understand it correctly, the /factory partition isn't a total waste. It duplicates some important data from the /rom partition (devconf), which could come in handy IF you put the stock recovery back on in an attempt to fix your HD+. (Then it could rebuild the /rom partition if it had become corrupted.)
So, as it stands now, /factory is 448 MB that's mostly useless to us. Can't we put it to good use? Like /swap? Or direct zRAM to it?
Leapinlar has already created some repartitioning tools, as has the Xperia dev now. So I'm hoping a lot of the hard work has been done already.
Proposal
(I apologize in advance is my terminology isn't exactly correct.)
Is it possible to:
1) remove the factory.zip from the /factory partition and then shrink the partition to accommodate only the /rom backup files. (I'm thinking that all factory zips are the same size, so once the new smaller partition is calculated, it ought to be the same for everyone's HD+);
2) create a new partition from the left over space, calling it (new) partition #11;
3) format partition #11 as swap
4) easily tell CM 10 or CM 11 to use the /swap? I'm hoping this can be done with just a init.d script. (Maybe an fstab tweak too?)
5) Or perhaps dedicate #11 for zRAM, as a kernel developer has done with useless space in the Galaxy Tab 2. In essence, we'd have almost 1.5GB of RAM.
Feasible? Yes/No? I realize that dinking with partitions can damage my HD+. It's a risk I'm willing to take.

I'm not going to get involved with this, I'm just going to add a warning. The partition structure for the HD/HD+ is different than the Nook Color and most other devices. The NC and other devices like SDs use DOS partitioning. The HD/HD+ is like the Nook Tablet that uses GPT partitioning. The two partitioning schemes are not compatible. So don't try to use my tools in the NC forum. If you want to know more about GPT partitioning on the Nook Tablet and get some tools, search the Nook Tablet forum.
Sent from my BN NookHD+ using XDA Premium HD app

leapinlar said:
The partition structure for the HD/HD+ is different than the Nook Color and most other devices. The NC and other devices like SDs use DOS partitioning. The HD/HD+ is like the Nook Tablet that uses GPT partitioning. The two partitioning schemes are not compatible.
Click to expand...
Click to collapse
Good to know! Now we have more data than we had before.

Looks like a lot of the work has been done already
So, I found this thread in the xda forums all about nook tablet partitions. They even talk about resizing! (Or at least, about making a different size /userdata partition.)
There is also this here about repartitioning Nook partitions.
Looks like parted would be easier for what I'm looking to do, shrink the /factory partition and create a new partition.
I read here that I can use a live Linux CD to mount the factory.img. From there, I presume I can delete the factory.zip and create a new image?
I'm not a linux geek, but I presume I need to be on a live Mint CD to use parted in ADB? (Update: Apparently I am to use the Android SDK for ADB in Windows. Downloaded already. Am climbing the steep learning curve.)

First half of Step 1, Done
Well, it took longer than it should have, but I made a new, smaller factory7.partition.img file. It's 1 MB.
I started by booting a Mint 16 Live CD.
Then I mounted the OEM factory partition image I made beforehand using the dd command, per leapinlar's instructions. (See his Item 16.)
Code:
sudo mount -o loop partition7.factory.img {some.folder}
That image contained 4 zips: factory,zip; fsck.zip; rombackup.zip; romrestore.zip.
Dummy me, I tried deleting the factory.zip inside the mounted image. It occurred to me later that that would be like trying to delete a file from a mounted CDROM's iso image. Can't be done.
So I copied the 4 zips to a new folder. Then I could deleted factory zip, which was 433.3 MB.
That left less than 1MB for the other 3 zips.
If there's an elegant way of modifying an .img and resaving it, I couldn't find it. (I spent a lot of time with mkfs.yaffs2. But this .img was not a yaffs.(Didn't pass a "sanity check" when I tried to mount the new .img I had made with mkfs.yaffs2.)
So I brute forced an imaging solution by copying the 3 copied zips to a 1MB ext4 partiton on a USB stick. (I made the partition with gparted and used chmod 777 to clear permissions in the partition.)
Then I made an image of the new 1MB partition using the dd command. In my case
Code:
sudo dd if=/dev/sdd2 of=new.partition7.factory.img
(I probably don't have the leading /'s right here. Am truly hacking my way through all this. I'm sure most of you could do this whole project in a few minutes.)
So now I have a shiny new 1MB custom factory image just wating to be used.
Update: I think I'll redo this, and leave 4 zips, in case the recovery mechanism is looking for a "factory.zip" I'll made a dummy file and archive it, calling it "factory.zip."
The next task is to get ADB working, delete the original large partition7 in the HD+, create a new 1MB partition7 in its place, and then restore my new image to partition 7 using the dd command in reverse.
After that, then to create partition11 out of the unallocated 432MB and format that as swap.
Update: It occurs to me that I will have to backup partitions 8, 9, and 10 with dd. Then I'll have to destroy them, recreate them in new locations, and then restore them. Fortunately, an xda developer has already written the instructions on how to do all this.
I see that CM 10 supports swapon, so I'm hoping it will be downhill after the partition work. I'm hoping CM will automagically find the new swap partition, once I enable swap.
If you know different, I'd appreciate hearing from you. Will using swap in this memory burn the memory out quickly?

Better yet, swap file instead of swap partition
So, I was thinking more about this idea and how I'd have to destroy and recreate p8, p9 and p10 to make a new swap partition at p11. Aside from that being a lot of work and more prone to errors with all those operations, a new p11 would never get TRIM'd by the OS.
So now the plan is to reduce /factory (p7) to 1MB and increase /system (p8) by 443 MB. Then I'll put a swap file in /system.
Only have to work on two partitions that way and /system gets TRIM'd. (Or at least Lagfix will trim it.)
Whatdaya think?
I suppose it's easy enough to put a swap file in /data to get a feel if there's any performance gain from swap in eMMC. Someone wanna show me the code to do this? (I've read about how to create a swap file. Not sure how to mount it at emmcblk10.) Never mind, Found it on xda! Am currently running 500MB swap on /data to see what it's like.

PMikeP said:
So, I was thinking more about this idea and how I'd have to destroy and recreate p8, p9 and p10 to make a new swap partition at p11. Aside from that being a lot of work and more prone to errors with all those operations, a new p11 would never get TRIM'd by the OS.
So now the plan is to reduce /factory (p7) to 1MB and increase /system (p8) by 443 MB. Then I'll put a swap file in /system.
Only have to work on two partitions that way and /system gets TRIM'd. (Or at least Lagfix will trim it.)
Whatdaya think?
I suppose it's easy enough to put a swap file in /data to get a feel if there's any performance gain from swap in eMMC. Someone wanna show me the code to do this? (I've read about how to create a swap file. Not sure how to mount it at emmcblk10.) Never mind, Found it on xda! Am currently running 500MB swap on /data to see what it's like.
Click to expand...
Click to collapse
I guess this is a dumb question, but if you can put a swap file in /system and /data, why can't you delete the factory.zip from /factory and put the swap there? Why create a new partition?
Sent from my BN NookHD+ using XDA Premium HD app

leapinlar said:
I guess this is a dumb question, but if you can put a swap file in /system and /data, why can't you delete the factory.zip from /factory and put the swap there? Why create a new partition?
Click to expand...
Click to collapse
First, I'm abandoning my bright idea. Turns out that swap is slow (even on eMMC) and possibly destructive to the eMMC with all the writes. (Hmmm . . . I wonder how many writes /cache gets compared to a swap file? (IOW, would a swap file be any more destructive than cache is?))
There were 1 second pauses every now and then, presumably while swapping when I had a swap file mounted in /data. Even tho I could get more free memory with a swap file than with zRAM, zRAM is much faster. And since zRAM is supported by the OS, it's the path of least resistance for me.
But to answer your question, it's not a dumb question. I'm a dumb user.
First, I don't know how to mount the /swap on the /factory partition. Can it be done? I don't see /factory in root explorer when running. Whereas I can see /system and /data. (So, to use Windows talk, it seems like /factory is "hidden" when running the OS?) By the same token, I don't know if I can delete the factory.zip from /factory in situ. If it can be done, it probably requires ADB and fastboot (?). But I haven't learned about those yet. So I'd have to use my new image trick, from my post above, to "delete" factory.zip from /factory.
Second, even if I could mount a swap in /factory, I don't think I'd want to. (Although I agree it's be a lot easier (and less dangerous) than deleting and recreating partitions.) I don't think /factory gets TRIM' by the OS. And I presume I would want it TRIM'd every now and then or else the swap would get slower over time than it already is.
I notice that Lagfix offers to trim /system and /data (and /cache). But not /factory. As above, if /factory is not visible during operation of the OS, that would be one reason why Lagfix doesn't trim it.
Or it could simply be that Lagfix doesn't see the need to trim /factory. Since /factory is supposed to be static, there wouldn't be any reason for Lagfix (or CM) to trim it.
Whether /factory can be trimmed with the fstrim command while running, I don't know.
Anyway, my idea for swap wasn't such a good idea. So I've abandoned the idea.
It still bugs me, as an engineer who compulsively tries to optimize everything, to carry that wasted 443 MB around in /factory.
I don't think we need an extra 443 MB in /system or /data. (Or /cache.) But if we ever do, we know where to find it.

/factory is not automatically mounted in stock or CM. So to get access to it you would need to mount it with a script command. And I would assume you need script commands to set up /swap so that should not be a problem. But /factory is formatted fat32 and /data and /system are ext4 so that might interfere too.
And manual trim commands can be run to trim the mounted /factory. You don't need the app lagfix to do it.
But I agree, I'm not really sure you gain anything with swap on this device.
Edit: as PMikeP pointed out to me in a PM, I was mistaken. /factory is ext4, not fat 32. Thanks for the correction. That should make it easier.
Sent from my BN NookHD+ using XDA Premium HD app

[Q/Dev suggestion] Space for another big ROM slot in internal storage?

Taken from CM12 thread as this started to become off-topic. The issue I'm trying to address is how to stuff a huge Lollipop ROM directly into internal storage, without creating a virtual slot, to increase system performance and have overall cleaner solution. Original inspiration: @Mentor.37's custom Safestrap for unused partitions, which has way too small /data for me unfortunately. (explanation)
sd_shadow said:
Septfox said:
This in mind, is there any way to repartition the internal (stock) storage to decrease the size of /cache/ and create a larger /data/ partition, or is it not possible without modifying the bootloader?
Click to expand...
Click to collapse
Code for mounting the unused preinstall and webtop partitions to SD storage http://forum.xda-developers.com/showthread.php?p=59253593
Click to expand...
Click to collapse
This allows you to mount the partitions either as a new storage or to a specific directory only, in other words it does not merge the storages, just adds a mount point next to (or possibly if modified a bit on the top of) other storage. While this may be useful for taking some inherent load off /storage/sdcard0/ (i.e. mounting /dev/block/webtop to /storage/sdcard0/Downloads) for people not swapping their sdcards, it doesn't help in Septfox's intention of enlarging /data/.
I face the same problem as Septfox: I'd love to use the storage intended for running system for it, not virtually mounted storages created in the storage intended for storing media and support data. However, the outline of the storage is intended for way older and less robust system so even the /system/ is not quite enough (667 MB, which tightly fits CM12 with a small GApps package) and /data/ is also not enough for heavier use (3.22 GB). Therefore I would like to merge it with currently unused partitions: maybe join preinstall to system and webtop to data, making both big enough.
One alternative would be to mount the 1.4 GB webtop as /data/app, which currently makes about 2/5 of occupied space of my /data. Is this possible? At which point of system startup is the script in /system/etc/init.d executed (is it done by Safestrap or the ROM itself?), and at which point might the system first need to access /data/app that contains the APKs of user-installed apps? All the really needed stuff (compiled executables) is in the /data/dalvik-cache, right? Here I'm on a really thin ice, don't know much about Android's architecture, so sorry if this is a major bullsh*t - just throwing my idea in Technically what I'm talking about is such modification of the script:
Code:
#!/system/bin/sh
mount -o rw,remount /data # Not too sure about this
# - depending on whether /data is already rw or not.
# For that I would need to know when is the script
# executed. I'm almost sure this is not necessary
# though. In original script this was done to allow
# writing into /storage/.
mkdir /data/app
#mount -o ro,remount / - unnecessary, see above
#mount -t ext3 /dev/block/preinstall /storage/preinstall
# I don't see any use for small preinstall partition.
mount -t ext4 /dev/block/webtop /data/app
chmod 777 /data/app # not sure about this either,
# probably should be 771
Clean version:
Code:
#!/system/bin/sh
mount -o rw,remount /data # Unnecessary?
mkdir /data/app
mount -t ext4 /dev/block/webtop /data/app
chmod 777 /data/app # Possibly 771 instead.
Is that possible to run, or will it bootloop, what do you think?
Even better IMO would be to delete preinstall and webtop and shrink the Stock data down to maybe 200 MB so the original Stock system is still present, preventing the phone from bricking and accommodating Safestrap. The remaining space could be divided between Safe system (say 800 MB) and Safe data (over 4 GB). Are we able to do this somehow, maybe by customizing Safestrap a bit more? Or are partitions in /dev/block locked by bootloader? Also are all these and Internal storage located on the same physical chip, or are there two separate memories in the D4? Attached proposal of repartitioned layout Sizes taken from here and here.
Developers and experienced users, I would love to hear your opinion, mainly on whether repartitioning internal storage or at least mounting webtop to /data/app could work on D4. Thanks!
Addition to the original post:
I have a spare D4 with shattered screen and not working SIM slot that I bought for spare parts. Apart from GSM (or telephony altogether? I didn't try, as only option would be emergency call which I don't want to abuse, and we don't have CDMA networks here) it works fine though. I can try meddling with formatting/partitioning - it won't be too big deal if it gets bricked beyond possibility of SBF restore.

Replies so far, taken from the original CM12 thread:
sd_shadow said:
I don't think repartitioning is possible without high risk of hard bricking the device, and yes the locked bootloader does limit what can be done.
This is quite off topic, and you should start a new thread if you are going to continue.
Click to expand...
Click to collapse
Good call, started new thread
lucize said:
tried to change the type of preinstall partition and the device would boot into fastboot: (invalid cg hab (cg: ebr, status: 0x0056)
so a recovery is needed, I'll try a resize if I can compile the tools in safestrap. but I think it would not work
later edit: used fdisk to resize in safestrap and it broke again so it can't be done
Click to expand...
Click to collapse
Thanks for trying! What do you mean by changing type?
As I added here in the first post, I have a spare shattered but mostly working D4 I wouldn't be too sad to see bricked - I can try out more dangerous stuff if you point me in the right direction. I'd like to help testing different approaches to repartitioning if you have some potentially harmful ideas that you wouldn't try on your own phone.
Curious question aside: where is bootloader (and fastboot) stored? Is it sitting on some other small partition? Can we mount it to be read, or even to write there? What/where is that protection that keeps us from unlocking the bootloader?
Also, concerning mounting the free partitions to other system partitions: do you think it's possible to mount them as a folder in /data (whichever we use)? Does it matter what filesystem the partitions use? What part of boot-up runs the /system/etc/init.d/ scripts? Do you think attached proposals 2 or 3 are feasible? It would still have the stock system untouched for an emergency use and Safestrap storage, but its data would be shared with the safe system, with webtop mounted as /data/app or /data/data (1.4 GB should suffice I hope) which would leave us with nice 3.2 GB for the rest. Of course, preinstall's 600 MB for /system is hardly enough for CM12 with Pico version of PA GApps so the rest of used Google apps would inflate /data a bit - but still this is probably the best option we have now.

Actually, I find the stock /system/ to be adequate. Yea, it's a tight fit, but CM12+PA Micro Gapps slots in with 30-someodd megabytes to spare, and there shouldn't be any real need for additional space on top of that. Though, I suppose there could be trouble if CM starts including larger apps.
Init.d is done by the ROM itself, and has to be enabled at build time; CM12 actually has it shut off for whatever reason (Slimkat did as well, I would imagine CM11 also did). There's an app called "Universal Init.d" in the Store, but by the time it can get around to executing the scripts, it's obviously way too late to be screwing with vital partitions : \
Worth pointing out that Safestrap appears to have proper ADB access, you might be able to do something with partitions thataway. I don't know enough about partitions and mounting in Linux to take a jab at it. Woop, looks like that's already a no-go. Maybe the bootloader does a check or three to make sure the partitions are all in order, and throws a critical error if not.
If absolutely all else fails, there's still symlinking large apps into the newly-accessible partitions that can be done, either manually or with Link2SD/similar apps. It's a bit of a hassle, but an option nonetheless.

Well, stock /system is fine but I want to keep it untouched - mainly because I don't want to install CM12 as an update over stock JB, and also to keep myself from SBFing (at least in the long run) because of trivial issues. When this option is off the plate, the remaining partitions don't seem to offer enough space to run CM11 in a non-virtual slot (as Mentor.37 offers with his modified Safestrap with a "Safe" slot with 600M system and 1.4G for data).
Symlinking is an ugly solution - used it for a while, never liked it, mainly because symlinks broke once SD got unmounted or mounted as Mass Storage, with problems remounting afterwards.
Since we can modify the system freely, I'm sure that if necessary, we could make a neater solution than using an app to create symlinks after boot. Question how early in the boot process can we add some scripts (by flashing some zipped patch over the ROM) goes to more knowledgeable devs though...

first time I just used t option in fdisk to change partition type to whatever without changing size, the second time I resized some of them and every time after 1st reboot the M logo would appear for 1 second and from now on it would go straight into fastboot without M logo, so it seems that something in bootloader is verifying the layout or something and if is not good it would stop.
I don't think that it's possible to brick it for good, use rsd to recover
Regards

Thanks to @lucize for his trying, sadly it seems we can't repartition at all
@Mentor.37, is there a source to your customized SS 3.75 available? I'd like to try and mess around with it to put the Proposal 3 from my second post's attachment to work.
Could mounting and symlinking be done from Safestrap, or does the ROM do it itself? I have ideas of various cross-linking of folders on the partitions to kind of emulate repartitioning - but I'd have to try if it works and it would need to be done either before the system boot or shortly in the process. See attachment - that's a first draft I presume for this that the user keeps stock system only for keeping the phone bootable no matter what happens in other than stock slots, so stock data would be utilized only for root and safestrap, leaving most of its 3.2 GB free.

LuH said:
Thanks to @lucize for his trying, sadly it seems we can't repartition at all
@Mentor.37, is there a source to your customized SS 3.75 available? I'd like to try and mess around with it to put the Proposal 3 from my second post's attachment to work.
Could mounting and symlinking be done from Safestrap, or does the ROM do it itself? I have ideas of various cross-linking of folders on the partitions to kind of emulate repartitioning - but I'd have to try if it works and it would need to be done either before the system boot or shortly in the process. See attachment - that's a first draft I presume for this that the user keeps stock system only for keeping the phone bootable no matter what happens in other than stock slots, so stock data would be utilized only for root and safestrap, leaving most of its 3.2 GB free.
Click to expand...
Click to collapse
I believe that it's impossible because symlinking "works" after kernel is loaded and safetrap take place (is loaded) before a kernel is loaded....

If I'm not mistaken, symbolic linking is a filesystem thing, something like a shortcut only more complex (after all, it's completely transparent to anything accessing it). ADB probably has commands to make links, terminal emulators in Android definitely can. I think the problem you face is making sure the mountpoint, path, etc are exactly the same both in SS and Android.
Probably better to just do both the mounting and symlinking under Android, so you can be 100% sure that everything matches. We already know /cache/ is unused most of the time under LP, and can (probably) even be unmounted while booted if needbe; why not experiment with it, rather than going straight for the other more important partitions, until you're sure what you're thinking will work?
Edit: Wikipedia has a big ol' writeup on symlinks under various systems, looks like a fun read.

rblanca said:
I believe that it's impossible because symlinking "works" after kernel is loaded and safetrap take place (is loaded) before a kernel is loaded....
Click to expand...
Click to collapse
I guess then this could be implemented in some early booting stage of the system by some custom .zip being patched over, kind of the way Mentor.37's ramdisks are. Unfortunately I don't have nearly enough knowledge to do it.
What I hope could be done more easily is altering SS to maybe use not only shared cache but shared data as well - then webtop could be used for safe system, leaving more then enough space for it, and we still would have fairly usable 3.2 GB data for it. Maybe we could even assign preinstall as stock data? @Mentor.37, I'd really love to hear your opinion on this, or maybe even get your alternated safestrap's source so I could try it myself

Sorry guys, I'm dropping this. I tried CM12 in stock slot and it doesn't help the system as much as I hoped for, so there's no need for this from my side.
I also mistook the process of installing ROM in Stock. I thought I have to upgrade the stock system to the new ROM in order to keep Safestrap in it, but it turns out it's independent and when "wiping" stock /system it leaves the Safestrap there, so I can easily do a clean install of a new ROM in the stock slot

LuH said:
I tried CM12 in stock slot and it doesn't help the system as much as I hoped for, so there's no need for this from my side.
Click to expand...
Click to collapse
About that, could you compare the difference regarding performance between CM12 on the safe slot and CM12 on the stock slot?
I currently got it running on safe slot but I'm thinking about trying to install it to stock slot, now that I sold my Lapdock and don't need Weptop mode anymore.
But I only would do it if it increases the performance of the D4.

Shani Ace said:
About that, could you compare the difference regarding performance between CM12 on the safe slot and CM12 on the stock slot?
I currently got it running on safe slot but I'm thinking about trying to install it to stock slot, now that I sold my Lapdock and don't need Weptop mode anymore.
But I only would do it if it increases the performance of the D4.
Click to expand...
Click to collapse
I don't really remember unfortunately, been running it from stock slot for a while now. Before it definitely was way more laggy than now, but that's also when CM12 for D4 was in a VERY early phase.
It definitely increases the performance, I'm just not sure how much. I don't see any reason why leave original system in stock slot though, so no reason to run CM from the safe slot. I don't like the idea of mounting the ext fs with system running from it from fat storage, it's bound to generate some unnecessary overhead.
In case of any major screw-up, sbf is your friend and AFAIK can't get messed up itself It's a good idea to have the factory cable available though, just in case it for whatever reason dies on you with low battery.

Ok, still good to know, thanks! What's sbf?
Well when I got my D4 a few months ago, I thought installing on the stock slot would be dangerous (in terms of bricking) and since I had the Lapdock, I wanted to keep the stock ROM.
But since then I've read that many users have CM12 running on stock slot, so now I want to do that, too. It's just so annoying when the whole phone freezes for half a minute or so and I think that's mostly due to the limited memory (although I have 170-300 MB free most of the time).
Do I have to consider anything special or different while installing than on an install on safe slot?

Shani Ace said:
Ok, still good to know, thanks! What's sbf?
Click to expand...
Click to collapse
see
Q12: What is a SBF?

Shani Ace said:
It's just so annoying when the whole phone freezes for half a minute or so and I think that's mostly due to the limited memory (although I have 170-300 MB free most of the time).
Click to expand...
Click to collapse
Same here, I suspect it's also connected with throttled data transfer requests of multiple apps at once, but the RAM is IMO the biggest factor. In Linux RAM is almost always full and the "free" space is occupied by cache (filesystem cache or whatever else app's cache) ready to be deleted - maybe the phone is stuttering when freeing the "free" space for other use.
Weird is that my brother with almost vanilla L 5.1 on Nexus 4 has system consuming about 100M less RAM than my CM12 - device-specific drivers maybe?
Shani Ace said:
Do I have to consider anything special or different while installing than on an install on safe slot?
Click to expand...
Click to collapse
If you mean on stock slot, just remember that you need to have some system installed there even in order to boot into safestrap - delete old, install new and ONLY then reboot, otherwise you're facing sbf Learned the hard way It doesn't brick your phone (meaning you can unbrick it), but it's lengthy and annoying...

@sd_shadow: Thanks, now I know. Feels kinda stupid having asked one of the FAQ's - can't remember when that happened before.
LuH said:
Same here, I suspect it's also connected with throttled data transfer requests of multiple apps at once, but the RAM is IMO the biggest factor. In Linux RAM is almost always full and the "free" space is occupied by cache (filesystem cache or whatever else app's cache) ready to be deleted - maybe the phone is stuttering when freeing the "free" space for other use.
Weird is that my brother with almost vanilla L 5.1 on Nexus 4 has system consuming about 100M less RAM than my CM12 - device-specific drivers maybe?
Click to expand...
Click to collapse
Ah that's interesting, didn't know that about Linux before. Well I always imagined that when there's little memory left, the drive somehow gets slowed down because there would'nt be enough space for some temp files or executions. But I don't really have a clue.
On the other hand, different devices showing different and inconclusive results like that is something I already experienced many years ago. ^^
LuH said:
If you mean on stock slot, just remember that you need to have some system installed there even in order to boot into safestrap - delete old, install new and ONLY then reboot, otherwise you're facing sbf Learned the hard way It doesn't brick your phone (meaning you can unbrick it), but it's lengthy and annoying...
Click to expand...
Click to collapse
Yeah I know, I had read about it before. But you can't really miss all those hints everywhere, so I think the community takes good care of spreading that time-saving information.
Actually thanks for encouraging me, because last night I flashed CM12.1 and everything onto the stock slot! Today I installed and configured most of the things I needed (including int/ext storage swap) and I have to say, it really runs better! It might not be as smooth as with a more recent smartphone, but it's definitely snappier than CM12 on the safe slot was (now it's gone and will rest in peace^^). It may sometimes take a few seconds, but most of the time it's very fluid, a great, noticeable improvement.
Additionally, now that I don't have to household with the memory anymore, I installed all the apps that I had left out on the safe slot install and even installed a couple of huge games from the Play Store (NFS:MW, NBA Jam, Batman Dark Knight, Injustice) onto my microSD card and it still runs almost without any hick-ups! I haven't tested the games yet, though.
EDIT: It really seems to have something to do with the amount of memory left on /system. After all that installing I was down to ~ 200MB (which would've been normal on the safe slot) and the phone started lagging a little bit more. But after moving some games and apps to the SD I have more than 500MB 600MB - okay after deleting the cache it's 1,26GB - free on /system and the phone runs smoothly again.

Shani Ace said:
EDIT: It really seems to have something to do with the amount of memory left on /system. After all that installing I was down to ~ 200MB (which would've been normal on the safe slot) and the phone started lagging a little bit more. But after moving some games and apps to the SD I have more than 500MB 600MB - okay after deleting the cache it's 1,26GB - free on /system and the phone runs smoothly again.
Click to expand...
Click to collapse
I guess you're talking about /data, not /system (different partitions, /system is usually read-only and system is installed there with stuff you flash from safestrap, everything else goes to /data or /sdcard). Yup, when /data is becoming full, you're gonna experience some strange behavior.
/data is the first "Internal storage" in Storage settings, /system is not shown there, /sdcard is the second "Internal storage" and /sdcard-ext, or "SD card", is the actual microSD in default CM12 setup. I guess you have the last two switched though.