YAFFS instead of Ext2, Ext3, Ext4 - Captivate Android Development
What are the chances of being able to ditch all the concerns that go along with Ext2, Ext3, and Ext4 hacks to improve read-write on the Captivate by going with YAFFS or YAFFS2?
http://en.wikipedia.org/wiki/YAFFS
YAFFS is an optimized filesystem for use with NAND flash memory, would it provide the stability and speed we're looking for?
I know this idea has been discussed lightly before in the various "lag fix" threads, but I was wondering if users with knowledge could enlighten the rest of us on the YAFFS filesystem.
I may be way wrong here, but judging by a few commands I've seen regarding busybox install issues, is it possible the stock kernel supports this already?? In which case could we modify the current lag fixes to use it instead?
Since this is related and I already posted this in Nuka's thread:
http://forum.xda-developers.com/showthread.php?t=750663
Those wacky gents over in the I9000 section have managed to get YAFFS2 compatibility into the kernel. This is something we should probably look into once we've figured out the wake-lag issue (maybe sooner )
In other news, assuming we can get YAFFS2 plugged into our kernel, how feasible (albiet probably complicated) would it be to actually REPLACE RFS and then load the OS on top of YAFFS2? There's just something about cascading file systems written on top of each other that sounds like trouble to me.
Zilch25 said:
Since this is related and I already posted this in Nuka's thread:
http://forum.xda-developers.com/showthread.php?t=750663
Those wacky gents over in the I9000 section have managed to get YAFFS2 compatibility into the kernel. This is something we should probably look into once we've figured out the wake-lag issue (maybe sooner )
Click to expand...
Click to collapse
Nice find!
I agree, the wake-up-lag issue is bothersome. Looking forward to mimocan possibly helping us out with his solution that he offered the Samset people.
kennethpenn said:
What are the chances of being able to ditch all the concerns that go along with Ext2, Ext3, and Ext4 hacks to improve read-write on the Captivate by going with YAFFS or YAFFS2?
http://en.wikipedia.org/wiki/YAFFS
YAFFS is an optimized filesystem for use with NAND flash memory, would it provide the stability and speed we're looking for?
I know this idea has been discussed lightly before in the various "lag fix" threads, but I was wondering if users with knowledge could enlighten the rest of us on the YAFFS filesystem.
Click to expand...
Click to collapse
Using a FS designed for Flash media will help with some of the fixes but not all. For example the mimocan fix it should help, as it will reduce overhead and increase the longevity of the Flash. Also since the mimocan fix is on the external card, it is properly un-mounted at shut down so this helps stop corruption.
Conversly it won't really do anything for the ext2 fix because that is a virtual disk sitting on top of another FS, which is not un-mounted at shutdown.
Potentially a custom ROM could replace the FS on the internal flash, but Samsung must have did what they did for a reason. It is hard to tell if they did what they did because they were lazy / didn't know any better, or they knew something we don't.
Bjd223 said:
Potentially a custom ROM could replace the FS on the internal flash, but Samsung must have did what they did for a reason. It is hard to tell if they did what they did because they were lazy / didn't know any better, or they knew something we don't.
Click to expand...
Click to collapse
Samsung use rfs because for some misguided and twisted reason some manager there thinks it is good and proprietary so they have a leg up on the competition. They go on about how quick rfs mounts as a benefit (yaffs and other flash files systems are slow to mount).
RFS isn't exactly terrible in the right places - the /dbdata symlink hack works because RFS-on-NAND is faster than RFS-on-SD, even though raw reads on the internal SD are faster than on the OneNAND partitions. I suspect it simply interacts poorly with devices that already do their own remapping and wear-leveling, or perhaps that the performance hit is a result of using much larger blocks on /data. I wonder what would happen if one just formatted /data with smaller blocks than 16KiB?
kennethpenn said:
YAFFS is an optimized filesystem for use with NAND flash memory
Click to expand...
Click to collapse
That already explains it all. I do not think it can be used on SD. This is copy-paste from UBIFS, but I believe it is the same for YAFFS:
This is why UBIFS does not work on MMC cards and the like - they look like block devices to the outside world because they implement FTL (Flash Translation Layer) support in hardware, which simply speaking emulates a block device on top of the built-in raw flash.
Click to expand...
Click to collapse
For the ext2 hack with loop device (file formated as ext2) I have no idea if it is any better.
Why do I feel like i'm the only one without the wake up lag issue? .
tbae2 said:
Why do I feel like i'm the only one without the wake up lag issue? .
Click to expand...
Click to collapse
I bet u r on stock kernel
Sent from my SAMSUNG-SGH-I897 using XDA App
Doesn't the Captivate already use yaffs2? Try running "mount" in a terminal emulator or in adb shell.
junknuts said:
Doesn't the Captivate already use yaffs2? Try running "mount" in a terminal emulator or in adb shell.
Click to expand...
Click to collapse
no, it uses rfs for everything...
yaffs2 is not really any better than rfs, both file systems, however, are not intended for use on SD memory. In other words you would be exerting great effort to get yaffs2 as your file system for the NAND part of your phone, and get little to no improvement. As for the mimocam fixes and the like that use your SD card or Internal Memory (Internal SD Card essentially), ext2, 3, or 4 would still be better options than yaffs2 or rfs as they are intended for true NAND Flash memory not block based SD memory.
tbae2 said:
Why do I feel like i'm the only one without the wake up lag issue? .
Click to expand...
Click to collapse
Ok, silly question...what is "wake up lag"? Either thats too technical of a term, or too vague, I dont really get it.
derek4484 said:
Ok, silly question...what is "wake up lag"? Either thats too technical of a term, or too vague, I dont really get it.
Click to expand...
Click to collapse
When you don't use your phone for a while, the phone goes into a "sleep" mode where the screen is totally black and does not respond to any touch. To "wake up" the phone, you press the power button, after which you can see the phone locked screen, and the screen responds to touch again. On a normal Captivate with the precompiled kernel installed by Samsung, or on a Captivate where you have flashed the same precompiled kerenel via Odin, the delay between pressing the power button, and seeing the phone locked sceen is usually less than a second or so. On phones where a kernel compiled from the Samsung released source code (either with or without any modifications) is flashed on, there is a much longer delay between the power button press, and the phone waking up to show the phone locked screen. This is what we mean by wake up lag. We still don't know what causes the rebuilt kernels to lag on wake up.
*Moved question to General*
Flash memory: NAND vs SD?
Hello,
Mr Taco above made a distinction that I can't figure out. He suggests that YAFFS is for "NAND" memory and not "block based SD." Here is my confusion.
From what I have read, it appears as though typical flash multimedia drives, such as MMC and other SD cards, are all created from NAND chips (as opposed to the less-memory-dense NOR chips). So I do not understand the distinction between "NAND" and "SD."
Also, he mentions "block based" devices. But as far as I can tell, "block level" has to do with the *interface* to storage and not the nature of the storage itself. For example, one could map a bunch of random access memory and present it as a block device (say, so it could be treated as a disk with a filesystem). Similarly, mmap and other techniques use disk storage with a memory-like interface (as opposed to a block interface).
So here are some questions I have that might help me figure this out:
1. Presumably the Captivate has some true RAM, that is, non-NAND chips that give truly random access memory that is not persistent upon shutdown.
2. That RAM has a normal (non-MTD) interface (512MB?).
3. Besides this RAM, the Captivate has an internal NAND-based flash drive (2gb).
4. It also has another internal NAND-based flash drive (16GB).
5. It has an external slot for a micro sd card, a NAND-based flash device accepting up to 32GB.
If these are all true, I have questions.
A. Is mimocam's (sp?) solution to use actual RAM, synced to flash, to load certain partitions?
B. Or is his solution implemented on the internal 2GB flash?
C. Certainly, other solutions have used external and internal flash (the ext2 solution, for example).
D. So, isn't it still true that any solution that would rely on the 2GB, internal 16GB or external NAND-flash would profit from using YAFFS?
Thank you in advance,
-0
zeroaltitude said:
Mr Taco above made a distinction that I can't figure out. He suggests that YAFFS is for "NAND" memory and not "block based SD." Here is my confusion.
From what I have read, it appears as though typical flash multimedia drives, such as MMC and other SD cards, are all created from NAND chips (as opposed to the less-memory-dense NOR chips). So I do not understand the distinction between "NAND" and "SD."
Click to expand...
Click to collapse
I'll repeat once again, it is from UBIFS page, but afaik it is the same for yaffs and others:
One thing people have to understand when dealing with UBIFS is that UBIFS is very different to any traditional file system - it does not work on top of block devices (like hard drives, MMC/SD cards, USB flash drives, SSDs, etc). UBIFS was designed to work on top of raw flash, which has nothing to do with block devices. This is why UBIFS does not work on MMC cards and the like - they look like block devices to the outside world because they implement FTL (Flash Translation Layer) support in hardware, which simply speaking emulates a block device on top of the built-in raw flash.
Click to expand...
Click to collapse
"One thing people have to understand when dealing with UBIFS is that UBIFS is very different to any traditional file system - it does not work on top of block devices (like hard drives, MMC/SD cards, USB flash drives, SSDs, etc). UBIFS was designed to work on top of raw flash, which has nothing to do with block devices. This is why UBIFS does not work on MMC cards and the like - they look like block devices to the outside world because they implement FTL (Flash Translation Layer) support in hardware, which simply speaking emulates a block device on top of the built-in raw flash."
Although I do appreciate the attempt to answer my questions, this reply is very puzzling.
My question was why did Mr Taco make a *distinction* between SD and NAND as far as flash memory goes. The reply, quoted above, *lumps SD and NAND devices together*. Furthermore, the primary topic of the quotation is yet another filesystem, UBIFS, that is not even under discussion (at least, not in my post it wasn't).
So, I'm happy to read up on UBIFS, however, I don't see how that can possibly answer the specific questions I have. I hope someone with some familiarity with device drivers or filesystem internals will drop by this thread. Otherwise, it's back to the research .
-0
NAND flash is typically configured so that for each block of stored data, there is a sub-block for supplementary data. This is used for wear-levelling, etc. "Raw" NAND devices such as kernel mtd devices expose this as well as the data portion of each block, and filesystems like yaffs2 and ubifs incorporate wear-levelling into the filesystem design, and rely on having direct access to raw flash.
SD cards contain a controller that does its own wear-levelling, and expose only the data segment of each block. Yaffs2 and ubifs will simply not work on such a device, nor are they really appropriate choices for it. A block-oriented filesystem is needed here, and because we can't guarantee graceful shutdowns, it needs to have some sort of data integrity scheme. Ext3/4 are journalled, Samsung's RFS is FAT with copy-on-write added, and the other possibilities here are nilfs2 and btrfs - but there are no backports of reasonable recent btrfs versions for the kernel Samsung has provided to us.
Unhelpful said:
nilfs2 and btrfs - but there are no backports of reasonable recent btrfs versions for the kernel Samsung has provided to us.
Click to expand...
Click to collapse
Thanks. I'm a bit boring to explain the basic things ppl can google first.
About fs. Do we need metadata or block level journaling?
We can consider ReiserFS or JFS? btrfs sounds nice tho.
Related
[concept] Android with NTFS (or any FS that supports files +4GB)
Hello Sometimes i want to move file that's bigger then 4G to a sd-cart. Unfortunately FAT32 don't support this So I asked google and google didn't know eather, but he gave me interesting project. Android-x86 Android-x86 supports NTFS as file system. So i was wandering, if in kernel i turn module ntfs on, will i be possible to convert fat32 partition to ntfs? will it work then? [edit] I know I can change fat32 to ext3/4 but will it be mounted to /sd-cart/ so i'll be able to write/read it from the phone?[/edit]
really interesting any news about this??? anyway some notice about other platform??? how is it possible that still today don't exist an operative system for smartphone with filesystem support more than old fat32 4gb data??? this has no sense for me
confiq said: [edit] I know I can change fat32 to ext3/4 but will it be mounted to /sd-cart/ so i'll be able to write/read it from the phone?[/edit] Click to expand... Click to collapse I realise I'm replying to an old post but as there's been recent activity on this thread... I'd just use an ext2 partition. If you're running A2SD you probably have one already. I run MCR 3.2 on my Hero & this partition is mounted (as a result of A2SD) on /system/sd & is obviously read/write. Max filesize under ext2/3 is 16GB for a 1KB blocksize.
anything happen with this? with the wealth of tablets coming out with the ability to mount host powered usb harddrives it would be great to be able to read/write (or even just read) ntfs natively within Android. formatting to Ext3 is the only option at the moment and is a bit of a ballache if i want to use the drive for anything else or take it to a friends etc.
thefunkygibbon said: anything happen with this? with the wealth of tablets coming out with the ability to mount host powered usb harddrives it would be great to be able to read/write (or even just read) ntfs natively within Android. formatting to Ext3 is the only option at the moment and is a bit of a ballache if i want to use the drive for anything else or take it to a friends etc. Click to expand... Click to collapse Havent heard anything yet. But you can use the EXT2/3 FS driver for windows to make your life a little easier. I use it when I pull hdds out of my NAS. http://www.fs-driver.org/
Swap Partition When Using Apps2sd
Hey guys, I am planning to format my sdcard to fat32/ext_ partitions for use of apps2sd. I have a few questions I would like answered first though. -Is a swap partition required? I have little space on my sd and would rather not use one. If it is required, what size/file system would you suggest? I take it there is no invisible swap partition by default? -Ext 2, 3 or 4 for my apps2sd partition? - I am planning to make my apps2sd partition only about 200mb, will that work well? Thanks.
- swap: depending on rom, but in general: no - i would recommend ext2, you can use 3 or 4 as well. keep in mind that ext4 doesn't work on any 2.1 rom until we have the kernel sources (after official release). - it will work, the size only determines how many apps you can install there. average size of an app is ~1mb, though can go from a few kb to several mb (i.e. copilot.apk has 14mb).
Thanks a lot Any particular advantage of ext2 over ext3?
tbman1996 said: Thanks a lot Any particular advantage of ext2 over ext3? Click to expand... Click to collapse Ext3 and ext4 wear out your sdcard quicker, because of journaling features. Since, compared to a system or cache partition, the sdcard is not written to as much, and system crashes where the mobile is shut down during a write operation are rare for Android, journaling doesn't offer any significant advantage. The unnecessary journaling however, accounts for extra writes to the flash memory chips, which in turn equals extra wear. http://en.wikipedia.org/wiki/Ext2 said: ext2 is still the filesystem of choice for flash-based storage media (such as SD cards, SSDs, and USB flash drives) since its lack of a journal minimizes the number of writes and flash devices have only a limited number of write cycles. Click to expand... Click to collapse
SDCARD Mounting Paradigm - Cyanogenmod7
There are two sdcard mounting paradigms. Currently Cyanogenmod for the tablet is set to use the one that matches phones. 1) Internal SDCARD mounted to /SDCARD, External to /SDCARD2(or equivalent) Pros -Internal sdcard is easily accessible Cons -External sdcard isn't mounted to computer over usb 2) Internal SDCARD mounted to /EMMC, External to /SDCARD Pros -External SDCARD is default and mounted to computer over usb -Works well for phones with small internal storage sizes Cons -Your internal storage is hard to access, not mounted over usb I took this GIT commit and rolled the changes back for CM7. I then recompiled CM7 to use the option 1 mounting paradigm. Let me know which you prefer. I personally like option 1 because it doesn't rely on having an external sdcard. Additionally the internal sdcard is not easily removable like the external, so being able to access it over usb is important. If there are reasons for options 2, let me know. Download: CM7RC0 with Option 1 Mounting Paradigm(Mounting over USB works): http://www.multiupload.com/NJFGUW04A2 Gapps: http://android.d3xt3r01.tk/cyanogen/gapps/gapps-gb-20110120-signed.zip
Very cool, I was planning to do that myself this weekend. I played with CM7 yesterday for a few minutes but with no SDCard in the "external" slot there were some limits to have useful the OS was.... I'll probably still pull the source down this weekend, but I can flash your ROM tonight and start having a more detailed look at CM7.
Hey Rothnic, nice call, I agree that the CM7 default mounting scheme is both somewhat confusing and somewhat less useful with the G Tablet. Quick question - you packaged up those performance packs with updated drivers for TNT etc. I know this may be a dumb question, but are these the same drivers currently being used in CM7? Or are they Froyo specific and incompatible with a Gingerbread ROM build because of library dependencies or structural changes? I'm just trying to figure out if there's some way to speed up 2d graphics stuff in CM7 without waiting for more drivers from NVidia.
rcgabriel said: Hey Rothnic, nice call, I agree that the CM7 default mounting scheme is both somewhat confusing and somewhat less useful with the G Tablet. Quick question - you packaged up those performance packs with updated drivers for TNT etc. I know this may be a dumb question, but are these the same drivers currently being used in CM7? Or are they Froyo specific and incompatible with a Gingerbread ROM build because of library dependencies or structural changes? I'm just trying to figure out if there's some way to speed up 2d graphics stuff in CM7 without waiting for more drivers from NVidia. Click to expand... Click to collapse CM7 uses the Nvidia libraries from TnT 3588. I haven't gotten to merging in the newer "performance pack" ones that TnT doesn't include. The libraries are Froyo specific, and were for many devices. I believe the CM team has built wrappers so that it can use the froyo specific libraries. I'm not sure on the acceleration issue.
Ok I got 2 questions: Do we have any clue how does Incredible (for example) works? It has 8gb internal plus External, they both mount on usb. Doesnt the fact that we can mount the internal storage on the computer, etc etc, can be a reason for the corruption of partitions? (maybe unplugged before unmounting by mistake or stuff like that..). That may have happen to almost everyone, everything starts crashing, FC, etc, all because of the repartitioning. Maybe paradigm 2 its better and correct. Another quick thing, is it possible to mount internal on /scard and external on /sdcard/sdcard2? That would make both accesible on usb mount no? Thanks
great work rothnic I tried to play with cm7 last night but was getting frustrated by this issue. Thanks for doing this!!
+1 for /SDCARD here... thanks, rothnic
Thanks for the efforts. Oh, option 1until I can get my hands on a 32GB micro SD, but probably still favor /sdcard and /sdcard2 anyhow. Also, can't wait for hardware acceleration with this Rom, best one so far.
I like option 1 over option 2 for the gTab. What would really be cool is an option 3. Internal + External = virtual SDCard.
+1 for Option 1.
If it is going to be a 'true' Cyanogen port, it should probably use the default Cyanogen behavior, which is Option 2.
This is meant as a discussion for the merits of each. For my use case option 1 makes sense. Id like to hear why option 2 makes sense to others. How do you access your 14gb of internal storage?
slaughts said: If it is going to be a 'true' Cyanogen port, it should probably use the default Cyanogen behavior, which is Option 2. Click to expand... Click to collapse This is handled in the device specific config files so in no way makes it different. Each device has its own specific mounting scheme.
I prefer option 1, because it does not depend on having the external inserted. Sent from my VEGAn-TAB-v1.0.0B5.1 using Tapatalk
danielsjam said: I prefer option 1, because it does not depend on having the external inserted. Sent from my VEGAn-TAB-v1.0.0B5.1 using Tapatalk Click to expand... Click to collapse This has to be the most compelling reason on why it should be mounted as sdcard and sdcard2.
thebadfrog said: This has to be the most compelling reason on why it should be mounted as sdcard and sdcard2. Click to expand... Click to collapse I agree. I prefer internal being SDCARD.
A vote for Option 1. As a temporary solution, I go into the Terminal Emulator and type Code: su mount /emmc/ /sdcard/ and it works gksmith said: I like option 1 over option 2 for the gTab. What would really be cool is an option 3. Internal + External = virtual SDCard. Click to expand... Click to collapse You mean.............just like Windows Phone 7?
It depends on the read/write speed of the sdcard. But since the internal sdcard is shared with other system partitions it makes sense for the external sdcard to be mounted at /sdcard, especially if you have media on it. Processes that read/write to /sdcard (like Gallery) can't mean anything good for performance. What we should have is both the sd and emmc to have /data, /system, and /sdcard mdraid partitions created and mounted/initialized for a nice perf increase. I wonder if its possible with android. It certainly is supported in the kernel with the right config. I should check to see if there are arm binaries for mdadm. Then i could create a test partition to measure performance. Oh and CIFS is what i use to xfer files. There are plenty of ways to access partitions without mounting them as a drive. I think that usually leads to trouble. There is even file expert which lets you create smb shares... there prob solved, map network drive.
Blades said: It depends on the read/write speed of the sdcard. But since the internal sdcard is shared with other system partitions it makes sense for the external sdcard to be mounted at /sdcard, especially if you have media on it. Processes that read/write to /sdcard (like Gallery) can't mean anything good for performance. What we should have is both the sd and emmc to have /data, /system, and /sdcard mdraid partitions created and mounted/initialized for a nice perf increase. I wonder if its possible with android. It certainly is supported in the kernel with the right config. I should check to see if there are arm binaries for mdadm. Then i could create a test partition to measure performance. Click to expand... Click to collapse How would you handle people who do not have an external sdcard?
One more for option 1.
Any Chance Cyan can implement this into his rom?
http://code.google.com/p/synergy-kingdom/wiki/WhatIsGodMode Looks like it yields really good results
You call request features over on the issue tracker. Not sure if it would make it into 7.1 though. Sent from my T-mobile G2 using Tapatalk
Sorry but where can I find the issue tracker lol I looked around but didnt see anything that jumped out at me
LOL. No, this wouldn't help. First off, this is strictly for devices with an MTD storage chip. We have eMMC. eMMC incorporates wear levelling automatically, and takes the extX partition straight up. If you look at all the I/O benchmarks of HTC VISION and compare them with GN1, you'll notice that the Vision blows the GN1 out of the water. That is because the function of the yaffs partition in your link is built in to the eMMC chip already!!! Nothing to gain by adding a crazy layer of uselessness!!! What this sillyness does is it keeps the base MTD partitions formatted as yaffs2 partitions (because you can't partition an MTD device using a conventional filesystem). It then creates a single file in each of those filesystems and formats that FILE as an extX filesystem, then loopback mounts those to the usual places. What we do with eMMC is we just format the eMMC partitions to extX straight away and mount as normal.
[Q] Create FAT32 Partition for Mass Storage Mode?
I just got a Nexus 4 and have found to my despair that it doesn't have Mass Storage Mode due to the sdcard becoming integrated with normal data. I really want Mass Storage mode, as I use my phone as a memory stick for various educational purposes, including running various executables directly from the Mass Storage. Is it possible to create a 1GB, or so, FAT32 partition from the normal ext4 space and allow that to be mounted as Mass Storage? If someone could point me in the right kind of direction for that, I would be much obliged.
This would require much more work than you would think. You can't just go around changing partition sizes and layout of the nand without the bootloader throwing a Wobbler. I mean it probably could be done with some extensive work but it means you probably wouldn't be able to use fastboot anymore because the offsets would be wrong Sent from my Nexus 4 using xda app-developers app
I see, well that's disappointing. I really used mass storage a lot and media transfer doesn't cut it for my needs. Damn.
Yes, this would be possible. However, I haven't done it personally and can see it being a little bit involved. From doing a search, it doesn't look like anyone has approached this either. Creating the image is the easy part. But to replace the stock fuse setup / supplement it gets tricky. You'd need to extract the ramdisk from your boot and make some changes there at the least. Further, you'd probably have to decompile your framework to modify res/xml/storage_list.xml. Overall, I think it's doable but would require some research comparing with sources for other devices (with SD cards). I guess it depends on your level of aptitude / desire. Might be better off attaching a usb stick to your keychain. To get the ball rolling, here's a way you could create a vfat image: Code: dd if=/dev/zero of=/data/sdcard.img bs=1M count=1024 mkfs.vfat /data/sdcard.img That would write 1MB of 0s 1024 times (1GB of 0s) and then format it as FAT. Perhaps you could research having that "partition" be an add-on, supporting UMS, while still maintaining the default fuse setup.