Related
I found this post in some other section!!
i am just giving the same information here for my fellow g3 mates!!
Full credit go to neoKushan
here the original thread!! http://forum.xda-developers.com/showthread.php?t=742351
Nows here what we are here for!!! Clear your doubts guys!!!
Ok, so here's the deal, in a very longwinded way that should hopefully explain everything and answer ALL questions.
You have an SD card in your phone and, a bit like normal PC Hard Drives, you can "partition" them (split them into two or more sections of different filesystems). Normally, your SD card is just one big FAT32 partition, which is fine for storing your pics, messages, emails, etc.
Now, other then your Phone's SD card, your phone will have its own internal flash memory (or "NAND") storage. Tradditionally with Android, you could only install applications to this NAND storage, you cannot install them onto your SD card. So if you have an empty 32GB SD card, but only 5Mb of internal phone storage, you still wont be able to install many apps, if any at all.
This was done to protect the apps from things like piracy - it's not easy to access the location where apps are installed on your phone's internal storage (normally impossible without root), so you can't for example buy an app, copy it, refund it, then install it again.
Still, this is no good for those of us who like to install lots and lots of apps, legitimately, as we run out of internal storage very quickly.
So Google came up with a way to install apps to the SD card. A folder is created called something like .android_secure and this stores (I believe) encrypted versions of applications, but there's a few catches:
1) Apps aren't automatically stored here, you have to manually "move" them
2) Not all apps are capable of being moved, in fact most apps aren't, the developer needs to update their app and allow it. Some apps aren't and wont be updated and some developers may not want to allow it for whatever reason.
3) Not all app data is moved, most of it is but some data is left on your phone so many people still run out of internal storage quickly.
4) You can force ALL apps to be moved to this area by default, but it breaks incompatible ones - such as Widgets, which are unable to load due to the SD card not being "prepared".
So that's Froyo's version. Before Froyo existed, some very clever people came up with a thing called "Apps2SD". Remember I said that your SD card normally is one big FAT32 partition? Well, Apps2SD works by having your SD card patitioned into TWO filesystems. A normal FAT32 partition for your usual stuff and a secondary "EXT" partition. EXT is just a filesystem, like FAT32 or NTFS, but it's the filesystem used by Android internally. The SD card is normally FAT32 because it's a "universal" filesystem, that just about any machine will be able to read, whereas EXT filesystems are generally Linux only, but I digress.
EXT has several different versions. The most common one you'll see is ext3. The main difference between ext2 and ext3 is "journaling", which is just a fancy way of saying that should an operation (such as copying, writing or reading) be interrupted unexpectedly (say, by you turning your phone off), then no data should be lost or corrupted. You know how when you turn your phone on, it says "preparing SD card"? It takes a few minutes, but what it's actually doing is checking that the FAT32 partition hasn't been damaged, because FAT does NOT have journaling. If you used a computer back in the Windows 98 days, you may remember that lovely blue "Scandisk" screen that had to run every time you didn't shut your computer down correctly - that's the same thing. But then Windows 2000/XP came along with NTFS, which also has journaling, meaning you had less chance of loosing data. But I digress once more.
So you have your SD card partitioned into EXT and FAT32. Generally it doesn't matter if it's ext3 or ext4, but you don't get any real advantage with ext4 over ext3 in this instance. Apps2SD then runs a special script on your phone which "symbolically links" the folder from your phone's internal storage where your apps are normally stored, to the ext partition on your SD card. A symbolic link is a bit like a shortcut for folders, except it's transparent to the OS: In other words, Android doesn't know that when it's installing it's apps to the internal phone storage, it's actually being stored on the SD card. This effectively boosts your internal phone memory from the previous 5mb that you had in my example above, up to whatever size you made the ext partition on your SD card (often 512Mb or 1Gb, but it depends on how many apps you install).
Plus, because it's "journaled", it doesn't need to be "prepared", meaning it's ready to go as soon as the phone starts - so your widgets and apps work immediately (unlike "forced" Froyo Apps2SD, where widgets disappear).
The catch with Apps2SD is that whatever space the ext partition takes up is taken away from the SD card. So if you have a 4Gb card (with something like 3.5Gb of actual storage) and you make a 512Mb ext partition, your SD card will "shrink" to 3Gb. The space isn't actually lost, it's just being used by the ext partition. If you reformat your card, you'll get it back.
Finally, there's a difference between "Apps2SD" and "Apps2SD+". Remember I said that your apps are stored on a special folder inside your Phone's NAND storage? Well, that was a bit of a lie. It's actually stored in TWO places. There's a second area which is called the Davlik Cache. You don't really need to worry about what this is for (Hint: IT's to do with the Java runetime your phone uses to run apps), all you need to know is that apps use it to store data, which also eats up internal phone memory. Apps2SD+ moves davlik cache to the ext partition on your SD card as well, freeing up even more space. Some people believe that this may come at the cost of performance, as the internal NAND memory should be faster than your SD card (Which is why you also get people arguing over which "class" SD card is better for Apps2SD - the logic being that a faster SD card means less impact from this move), but the truth of the matter is that your applications will be running from your Phone's RAM anyway, so performance isn't really impacted at all. Since most apps are only a few hundred Kb's in size, or a couple of MB at the most, it's a non-issue.
Finally, any recent version of Apps2SD/Apps2SD+ should work with an SD card that is or isn't formatted with an ext partition. It'll check for this partition when your phone first boots and if it's not there, just use internal phone storage.
Having an ext partition WITHOUT Apps2SD+ shouldn't cause any issues, either, so you can format your SD card whenever you're ready.
So in summary:
Apps2SD "fakes" your phone's internal memory and puts it all on a hidden section of your SD card.
Apps2SD+ pushes even more content to the SD card, freeing up even more space on the phone itself.
"Froyo" Apps2SD has various limitations that "old" apps2SD does not, but is much easier to handle as it doesn't involve any kind of "partitioning".
I don't really claim to be an expert but I wrote this and nobody really chimed in to say anything was wrong, so hopefully it applies here.
I'll try to keep an eye on this thread in case anyone has questions.
I don't really claim to be an expert but I wrote this and nobody really chimed in to say anything was wrong, so hopefully it applies here.
I'll try to keep an eye on this thread in case anyone has questions.
Click to expand...
Click to collapse
Thanks a lot buddy.. People may know all that but you have written in such a simple manner that even a small kid will understand..
Acid lestitious 2.0. Fugumod 2.2 build 1.9
thanx soo muxh for this info...it's getting really confusing out there with everyone claiming this type is better than that blah blah blah etcetc....this is very helpful
thanks again
JohnstonF said:
thanx soo muxh for this info...it's getting really confusing out there with everyone claiming this type is better than that blah blah blah etcetc....this is very helpful
thanks again
Click to expand...
Click to collapse
You're welcome
Acid lestitious 2.0. Fugumod 2.2
Hello all,
I have CM9 nightly installed on a 32GB microSD and I have just run into a weird problem. I still have 27GB left open BUT when I try to download magazines and install apps, it's saying I'm out of room or running out of room. Anyone have any idea why that might be?
Is it possible it's installing things to the EMMC? If so, how do I change that?
Sorry if this a n00b question. Thanks in advance for your help.
mikelav456 said:
Hello all,
I have CM9 nightly installed on a 32GB microSD and I have just run into a weird problem. I still have 27GB left open BUT when I try to download magazines and install apps, it's saying I'm out of room or running out of room. Anyone have any idea why that might be?
Is it possible it's installing things to the EMMC? If so, how do I change that?
Sorry if this a n00b question. Thanks in advance for your help.
Click to expand...
Click to collapse
Android has 3 major areas where stuff is stored. The system partition holds all the OS stuff plus has an area for system apps. The data partition is were downloaded apps get stored normally. The media or sdcard area is normally used for music, videos, pictures and also where some apps store any working data.
The system and data areas are of relatively fixed sizes even though they are all on the actual SD card. For your 32GB SD card it is probably "460M system, 975M data, rest is FAT for sdcard" from veryGreen post.
So your error message is being triggered probably because the 975M data partition is full. Typically this enough to hold about 150 - 200 apps depending on their size, but some games can take quite a lot.
You can check memory usage by going into settings and looking under apps.
What can you do about it? On an SD card install the simplest way is to use ANdroids capability to move apps from the standard data partition to the sdcard partition. Not all apps can be moved but many can and this will then free up space in your data partition.
Get the Apps2Sd app to help you manage this process.
I haven't run from a sdcard in quitevawhile, buy it sounds like the card didn't get repartitioned after making it into a bootable. After you burn an image to a card, you need to use an application like Easus, a disk partitioning tool (free for home use for windows, I believe) t repartition the card and make use if the remaining space.
mateorod said:
I haven't run from a sdcard in quitevawhile, buy it sounds like the card didn't get repartitioned after making it into a bootable. After you burn an image to a card, you need to use an application like Easus, a disk partitioning tool (free for home use for windows, I believe) t repartition the card and make use if the remaining space.
Click to expand...
Click to collapse
I think the partitioning must have worked otherwise it wouldn't function at all.
You could change data partition size to give more space to the data partition if you are careful. I've done that on emmc prior to install. I've never tried it on an SD card after install and it's possible it might muck something else up. However, the basic point is that apps and related content go into data by default. The big part of the SD card left over from the initial install is intended for media.
Compare it to a phone. The data partition in the internal phone memory is fixed and can run out of space if lots of apps are installed. If you plug an SD expansion card into a phone it allows you to store lots music, video, etc but unless you move apps to the SD card then your original data space is unchanged. That's why app users can complain if a memory hungry app can't be moved to SD.
Maybe you can teach me something here. I am not sure I understand.
When I formatted an 8 gig card to run cm7.1, I was able to use it to boot but had the rest of memory unavailable for use. It wasn't until I redid the process and then reformatted the partitions to have the remaining space available. Only then was I able to use the remaining 6 gigabyte or whatever. I had the four partitions in both instances, and the card worked, it just wasn't available because all the space was allocated to the wrong partition. Which I rectified with Easus, I'm pretty sure.
What I guess I am saying, isbthat there are some tools that could be used, although I maybe wrong. It just seems counter-intuitive that the OP should have to use App2SD and otherworkarounds when theybhave 32 gigs to play with. I think Easus lets you define those partitions anyway you like, with the 29 gigs or so that is left once you allocate the system stuff.
Like I sad, I maybe just don't understand this very well, it is something I did when I was first learning about rooting, before I figured there was no real reason not to go internal.
mateorod said:
Maybe you can teach me something here. I am not sure I understand.
When I formatted an 8 gig card to run cm7.1, I was able to use it to boot but had the rest of memory unavailable for use. It wasn't until I redid the process and then reformatted the partitions to have the remaining space available. Only then was I able to use the remaining 6 gigabyte or whatever. I had the four partitions in both instances, and the card worked, it just wasn't available because all the space was allocated to the wrong partition. Which I rectified with Easus, I'm pretty sure.
What I guess I am saying, isbthat there are some tools that could be used, although I maybe wrong. It just seems counter-intuitive that the OP should have to use App2SD and otherworkarounds when theybhave 32 gigs to play with. I think Easus lets you define those partitions anyway you like, with the 29 gigs or so that is left once you allocate the system stuff.
Like I sad, I maybe just don't understand this very well, it is something I did when I was first learning about rooting, before I figured there was no real reason not to go internal.
Click to expand...
Click to collapse
The various partitions have different purposes. It's not that they are unavailable for use. You want each area to have sufficient room for what you want but not necessarily too big because that then is wasted and is subtracting from the other areas.
First there is the boot partition containing the boot images. This can be small.
Next you have the system partition (~500MB) which can be fairly small because the OS does not take a lot of room and the system apps are relatively fixed in size and you don't need to add extra to that.
Next you have the data partition where downloaded apps and some of their associated data lives. You want this to be fairly big to accommodate a decent number of apps but it doesn't need to be huge. Apps vary in size from 100s of Kbytes up to say 20MB or more. So a 1GB data partition can typically hold about 200 apps.
On an SD card that then leaves the rest of space for a sdcard partition where media can be stored like video, pictures and music plus some apps will also require some working space on there.
So on an SD card install the main balance is between data and sdcard. If you were to make the data partition larger to accommodate more downloaded apps then you reduce the amount of space for music, video etc. But you do want enough space to hold a decent number of apps. The standard verygreen SD card installer sets the balance at ~1GB data and the rest sdcard for media. Now if you never wanted to put much media files on the SD card and you want to be able to download thousands of apps then that would be an argument for setting the balance the other way.
Now if you install to the internal memory the same scenario applies but you have an additional partition confusingly called emmc. Your boot, system and data areas are on the internal memory. The left over internal area is the emmc partition and the sd card is now normally set up to be a single sdcard partition.
Both the emmc and the sd card are typically used to hold media files.
The size of the data and emmc partition in the internal memory can be varied before you do the install by some partitoning zip tools and there is a thread dedicated to that.
By default as supplied new Nooks have a 5GB internal data partition and a 1GB emmc. Many people think that is not a great choice as it is really difficult to run out of app space with 5GB and it means there is only 1GB internal space for media and the unused data space is wasted. By repartitoning to say 2GB data and 4GB emmc then you get plenty of space for apps and release space for more media.
You are right that using something like Apps2Sd would seem unnecessary when you have lots of free space. It is effectively a work-around to let you use some of the sdcard as extra data area if you run out of the data area that has been allocated. Actually on an SD card install there is not much of a downside in that moving an app from data to sdcard as it is still all on the same SD card. For an internal memory install it is nice to have a big enough data partiton to make moving apps to the SD card unnecessary.
Hi. Did my usual +3 hours googling but came empty handed.
Could size-agnostic CM7 SD installer script be modified so that /Data partition is set to 2Gb instead of 1Gb? I know that I can always do the "move to SD" routine but Android keeps bugging me that I`m out of space even when I have 170Mb free on /data. And also moving apps back and forth from internal to SD somehow wrecked my system process. As a result I now have constant rebooting of the launcher as my only Android experience. Uninstalling recent apps did not work. Neither did freeing up space on SD and denying root to all apps that previously asked for it.
zuzka said:
Hi. Did my usual +3 hours googling but came empty handed.
Could size-agnostic CM7 SD installer script be modified so that /Data partition is set to 2Gb instead of 1Gb? I know that I can always do the "move to SD" routine but Android keeps bugging me that I`m out of space even when I have 170Mb free on /data. And also moving apps back and forth from internal to SD somehow wrecked my system process. As a result I now have constant rebooting of the launcher as my only Android experience. Uninstalling recent apps did not work. Neither did freeing up space on SD and denying root to all apps that previously asked for it.
Click to expand...
Click to collapse
If you are willing to start from scratch you can burn the image again to the SD, but don't put any zips in it. Boot and let the script create the new partitions. Then put the card back in the PC and use Mini-Tool Partition Manager to shrink partition 4 (sdcard) and grow partition 3 (data) That program will handle the ext partition (3) as long as there is no data written to it yet. You can get that program free on the internet. Do a Titianium backup of your apps and data and restore later.
Edit: Now after thinking about it, you may be able to doctor your existing card. Partition 4 is fat so you can shrink it with data in it. Take the space off the front. Then I think the program will let you grow partition 3 into the empty space even with data already in it. It is just trying to move it with data does the program balk. Windows does not know how to read ext. Of course if you are running linux, just use gparted to do everything.
Yes, you could do that or you could use a program like easeus partition manager and move the partitions around after the first boot of the card.
Edit (while typing): It looks like you might be talking about an internal installation in which case you don't need to use size agnostic SD installer. Just repartition the internal the way you want it (there are threads for this) then flash your CM7.
Edit 2: beat out by leapinlar while editing 1
JP
Thanks for the idea of repartitioning prior to installing CM7. I`ll also try to repartition SD the way it is now- with data. I was concerned that /data partition is unmountable under Ubuntu when /boot and /system can be read without problems.
Hi again. I`ve managed to srink /sdcard partition under Windows and sort of grow /data partition under Ubuntu LiveCD (gparted froze right after doing its magic).
Boot loop on my Nook stopped as free space on /data increased from 175Mb to 1.2Gb. So I guess I wont be doing a fresh install at the moment.
Question is, exactly how much free space does CM7 need on /data partition? In a real world?
(at ~200Mb it starts bugging me about free space but overall Android stays stable)
zuzka said:
Question is, exactly how much free space does CM7 need on /data partition? In a real world?
(at ~200Mb it starts bugging me about free space but overall Android stays stable)
Click to expand...
Click to collapse
Original Nook Color had 1GB for data. B&N upped it to 5GB on the new Nook Colors. I personally would make it 2GB unless you are going to be adding some really big apps.
Sent from my Nook Color running ICS and Tapatalk
Hello.
I'm using the latest aurora and I'd like to discuss an idea I had.
Since the U8800 has plenty internal memory and it's quite fast comparing to an average sd card, it'd be great if we could use the internal memory for the tasks that need "speed" instead of "space".
Running apps from the sd card is the perfect example!
So the idea would be, while keeping the external sdcard mounted on /mnt/sdcard, having /mnt/sdcard/Android/data, for instance, point to the «internal sdcard path»/Android/data.
What do you think would be the best solution:
- hacking the mount points?
- using links?
- other solution?
Any developer or filesystem savvy user can point the advantages or dangers of using this solution?
- e.g.: the system or some apps may check if the external sd-card is mounted before trying to read/write to /mnt/sdcard/Android/data... that would mean the system/those apss may think that path isn't reachable with the ext-sdcard unmounted, altough they can be read.
Suggestions? Rants? Ideas?
Regards!
I have wondered if the /data/app or maybe the whole /data partition could be mounted in the internal storage partition. That would give 2gb space for apps and associated files instead of 3-400 ish and would mean no need to move apps to sd card which would mean faster loads at startup due to internal memory being faster.
VuDuCuRSe said:
Hello.
I'm using the latest aurora and I'd like to discuss an idea I had.
Since the U8800 has plenty internal memory and it's quite fast comparing to an average sd card, it'd be great if we could use the internal memory for the tasks that need "speed" instead of "space".
Running apps from the sd card is the perfect example!
So the idea would be, while keeping the external sdcard mounted on /mnt/sdcard, having /mnt/sdcard/Android/data, for instance, point to the «internal sdcard path»/Android/data.
What do you think would be the best solution:
- hacking the mount points?
- using links?
- other solution?
Any developer or filesystem savvy user can point the advantages or dangers of using this solution?
- e.g.: the system or some apps may check if the external sd-card is mounted before trying to read/write to /mnt/sdcard/Android/data... that would mean the system/those apss may think that path isn't reachable with the ext-sdcard unmounted, altough they can be read.
Suggestions? Rants? Ideas?
Regards!
Click to expand...
Click to collapse
Well this is known for a while now . Quoted from my thread here:
vold.fstab - CHANGE THE DEFAULT SDCARD: -thanks to julle131
There is this file named vold.fstab at your /system/etc/ folder, which controls which of your sdcards (Internal/External) will be mounted when you connect your phone to the PC. The last 2 lines are the only that matter. These are the two possibilities:
EXTERNAL SDCARD MOUNTED (DEFAULT):
dev_mount sdcard /mnt/sdcard auto /devices/platform/msm_sdcc.4/mmc_host
dev_mount emmc /mnt/sdcard1 14 /devices/platform/msm_sdcc.2/mmc_host
INTERNAL SDCARD MOUNTED:
dev_mount sdcard /mnt/sdcard 14 /devices/platform/msm_sdcc.2/mmc_host
dev_mount emmc /mnt/sdcard1 auto /devices/platform/msm_sdcc.4/mmc_host
change them as you wish with a root explorer or before flashing.
Click to expand...
Click to collapse
Make this change and every app will use the internal sdcard. Gallery works ok with images to external sdcard, it just might take a while for the first refresh. But I don't know if you'll notice any difference in terms of speed....
Cheers
spirosbond said:
Well this is known for a while now . Quoted from my thread here:
Make this change and every app will use the internal sdcard. Gallery works ok with images to external sdcard, it just might take a while for the first refresh. But I don't know if you'll notice any difference in terms of speed....
Cheers
Click to expand...
Click to collapse
That's not what he talked, I think. He said that the apparel would be installed on the external sd card, but the data would be moved to the internal one. And we have speed, but the main card would be the external one.
I think its a great idea if I understood it right!
Sent from my U8800 using xda premium
mrasquinho said:
That's not what he talked, I think. He said that the apparel would be installed on the external sd card, but the data would be moved to the internal one. And we have speed, but the main card would be the external one.
I think its a great idea if I understood it right!
Sent from my U8800 using xda premium
Click to expand...
Click to collapse
Oh I see! Sorry about that. As far as I know apps are not choosing were to put their data. They just say to the OS "I want these things to be at the sdcard" and the OS puts them at the correct place. So you can't "hack" the app to "see" at somewhere else. On the other hand the OS initialises on boot which card is the primary and which is the secondary. So my knowledge stops there and I can't imagine a way to make this separation between app data and the rest...
I hope for someone else's help!
PaulMilbank said:
I have wondered if the /data/app or maybe the whole /data partition could be mounted in the internal storage partition. That would give 2gb space for apps and associated files instead of 3-400 ish and would mean no need to move apps to sd card which would mean faster loads at startup due to internal memory being faster.
Click to expand...
Click to collapse
Maybe this cannot be done, because I think that the mapping of the partitions and mounts comes from the bootloader, but there is already the solution of resizing the data partition using the space of internal storage.
PaulMilbank said:
I have wondered if the /data/app or maybe the whole /data partition could be mounted in the internal storage partition. That would give 2gb space for apps and associated files instead of 3-400 ish and would mean no need to move apps to sd card which would mean faster loads at startup due to internal memory being faster.
Click to expand...
Click to collapse
That's a great idea!
I almost don't use the so called "internal sd card", therefor I have 2GB of fast empty space being wasted.
Why not simply mounting the internal sd-card partition in the data folder and the data partition on the internal sd card folder?
I only use the internal sd card to hold ringtones and other small stuff that I need to be in accessible 100% of the time, so a few hundred MB are enough!
Me gusta!
Every ROM flashes it's own fstab, right?
So all we'd need to do would be, editing the fstab before flashing it.
-----------------
Regarding the my first idea... I don't think one can simply mount a path on another path. We'd need a "virtual" device pointing to /mnt/sdcard-internal/data/ then mount that virtual device on /mnt/sdcard-external/data/.
My knowledge of unix mount/filesystems is not much more than what's explained here: https://help.ubuntu.com/community/Mount
OK, just found this:
http://askubuntu.com/questions/11079/mount-a-directory-to-look-like-a-drive
there is a FUSE filesystem named bindfs that probably does exactly what you want. For example the following:
bindfs -n /media/USB-HDD-01/ISO/ /home/johnc/ISO-images
Click to expand...
Click to collapse
would result in ISO-images to show up as a mounted filesystem in the Places sidepanel in Nautilus. The -n is required because otherwise bindfs tries to use the allow_other FUSE option, which by default is not allowed for regular users (if you use bindfs in /etc/fstab this is not a problem).
The first directory you give is the existing one, the second directory you give is an empty directory under which the contents of the existing one will appear.
Click to expand...
Click to collapse
So, bindfs depends on FUSE... and there's a port of fuse to android: https://github.com/seth-hg/fuse-android but looking at the README I get the feeling that it's not really working. The last step would be having a kernel with FUSE support... and regarding that, I have no idea, how easy it is to get a kernel like that for our U8800 :\
Finding such a solution to expand the DATA partition indirectly would be great!
I used to apply genokolar's custom partition method as "1.2G DATA + 1.46G INTERNAL SD" before, but i found out that it has some issues with Official 2.3 Roms. One of them -the most important IMHO- is the BOOTLOOP problem occurring in some specific situations!!!
See the details in my post here:
http://forum.xda-developers.com/showpost.php?p=25853141&postcount=134
(Seems that I don't encounter the bootloops on Custom Roms, but i'm not %100 sure and never will be )
So i unfortunately gave up using it, and revert to the stock partition .
I will be grateful to the one who will find a better and reliable solution. Hope to see it soon...
I still have to gather more information on the official update "modus operandi".
My goal is to simply reformat both partitions and then swap their mount points.
If the official update formats the partitions before applying the flashing official ROM files, then partitioning changes shouldn't cause any failure on the official update.
But like I said, I'm still yet to confirm that info.
Gen's idea was great (I learned a lot just by looking at his scripts) but resizing partitions has its dangers.
Since "no one" really uses the internal memory, why not simply exchange its room with /data, right?
This would be awesome! I would love to have 2gb for apps, instead of cluttering my phone.. plus the 2gb of internal storage are just a waste for me.. I never use them.. so this would be great!
Note from the Author -
I am moving on to the N5 now and ditching my S3. I will continue to maintain this thread, however - please do PM me if you think that something needs to be changed or updated in this thread as I doubt I will be answering questions within the thread as much. Please don't PM support questions to me. Only PM updates that need to be made in the thread.
It's been a blast!
Regards
Dan
S3 Storage (Data Loss Recovery / Prevention / Info)
This thread is intended to give you an overview of some of the Storage of the S3 from a Data Loss and recovery perspective. It is not intended to cover USB sticks or mods to Swap / Mount other storage. It is solely to cover day-to-day data concerns and give a background to how these things work
Please note, if you have recently swapped between Android 4.1 and 4.2 and cannot find your sdcard data, you need to read [Info] Flashed 4.2? Can't find your /sdcard data?
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
Before we get started...
Here are a couple of threads you should get familiar with before posting on XDA.
Forum Rules - use Search before posting
Post Questions or Support queries in Q&A, NOT General
Backround of Android storage (Pre-S3)
Firstly, I think we need to understand how Android worked historically as this will help us to understand how the S3 works now.
A legacy android device, lets pretend the HTC desire on Android 2.2 as this was a standard configuration at the time. It had 2 major partitions (and several minor ones which are irrelevant to this topic). It has the /system partition and the /data partition. These were partitions of the internal NAND flash memory.
/system is where the Android operating system lives. The user could not delete or change anything in here (unless you were rooted). All the software that came with the phone was installed in the /system partition
/data is where all the userdata goes. Whenever you installed an app from the market, it installed to /data/app and it put all it's important data into /data/data. Also any system settings you changed (Wallpaper, ringtones etc) all were stored in /data/data. When you did a factory reset, it wiped /data and everything in it.
Of course, having these 2 partitions was not enough for everyday use. There was no where to store your music, photos, documents etc. /data is just for app data and settings. So this is where /sdcard comes in
/sdcard is the Android mount point for the External SD card in this legacy android device. This means that when you inserted a Micro SD card, Android used /sdcard as it's internal reference to where the card's storage is. The /sdcard was a necessity before you could take photos. Over time, bigger apps started to put other data here. For example, a GPS / Map application would store its apk (application package) in /data/app and store your personal configuration settings in /data/data but it may download and store offline maps somewhere on the /sdcard. In older devices, the internal Memory (NAND Flash) was usually too small to allow much data on it. Many users would have to root to get more storage space or keep uninstalling apps to keep the "low on space" warnings at bay
How the S3 is different
Well, the S3 is substantially different. There are of course SOME similarities. For example, the S3 still has internal NAND Flash Memory. This is often referred to as the eMMC (Embedded MultiMedia Card) - which still contains the /system and /data partitions, used in exactly the same way.
The main difference is /sdcard. The S3 is designed specifically so using an external micro sd card is NOT a necessity. It has a larger NAND Flash internal memory (eMMC) so it can also have an "internal SD card". This is where people start to get confused. The entire internal memory is an eMMC which is essentially an internal SD card, however a partition of that internal memory is /sdcard.
OK, I know - let me explain. /sdcard is a mount point that Android uses to know where to store /sdcard data. But on the S3, instead of storing it on a required external sd card, it points to an internal memory partition. Now here is the clever bit. The /sdcard actually points to /data/media (or /data/media/0 in Android 4.2 onwards). So you continue to have your /data partition, but within /data you have:
/data/app
/data/data
/data/media
The clever thing is that the file system that android uses for /system and /data is a Linux file system called Extended FS. In our case, we use the Ext 4 file system. This is important to understand because these file systems do not work with Windows so an external SD card would usually be Fat 32 file system, or exFAT so we could plug it into our windows computers and read the contents. What Samsung have had to do is use the FUSE file system to allow /sdcard (or /data/media) to exist as a FAT file system within the EXT 4 file system. Clever stuff. But it has it's pro's and cons...
You lost me at file system
All electronic systems that have an operating system and store data use a file system. Think of it in it's simplest form. Imagine a school text book. It has lots of chapters about different things. It has a "Table of Contents" in the first few pages, telling you where each chapter in the book is so if you want to know what page chapter 13 is on, you look in the contents and find the page and go straight there - The alternative is looking through each page individually to find the chapter. Not a quick process.
Well data storage works the same. When you put a file on a hard drive, sdcard, USB stick (or whatever) it is written to a specific location. When it is written to this location, the location is added to the file system. So when you put word.doc onto the drive, The file system is informed of the (very complicated) location of the file. When you tell Windows, Android (or whatever) that you want to open word.doc, the operating system consults the File system and goes to retrieve the data from its true and real (yet very complicated) location on the drive.
There are many file system types still in use today. Usually they are operating system specific. For example, Ext 4 is a Linux file system (and Android by Proxy as it is Linux kernel based) and Windows cannot read Ext file systems. Similarly, exFAT is a Microsoft file system (also used for sdcards on the S3) and cannot be used (easily) on Linux machines. Since most everyday users of the S3 are Windows users, you can see now hopefully why it was important for Sasmung to use "magic" and implement FUSE to allow an exFAT file system to be used for /sdcard, within the Ext 4 partition of /data
Did I REALLY need to know ALL that?
You know what? Probably not but it may go some way to help understand the limitations we will cover later on.
A bit more info for the S3
Obviously, the internal memory supplied with the S3 may not be enough for all users so they added the ability to add expanded memory in the form of the Micro SD card. Android uses the Mount point of /extSdCard now, instead of /sdcard like it used for legacy devices - because /sdcard is already in use elsewhere.
One thing many of you have probably noticed is that with the S3, there is no option to mount the /sdcard or /extSdCard as USB Mass storage on your computer. You must use MTP or PTP.
PTP - Photo Transfer Protocol. When you connect your S3 to your computer using PTP, Your computer sees it as a camera. It will show photos on your "camera" and will set about implementing the default camera options (such as suggesting you import your photos) etc. It won't show documents or other media necessarily.
MTP - Media Transfer Protocol. When you connect your S3 to your computer, it will be seen as a media player. This should allow you full access to all the files on there, including word documents and the such like.
One of the reasons for this is that because /sdcard points to something using the FUSE file system and is not a true partition, it would be difficult to allow it to be used as USB Mass Storage. It may or may not be possible but the biggest advantage of using MTP / PTP is that the computer and the S3 can both access the internal memory at the same time. With USB Mass Storage (UMS) this is quite awkward and can result in errors.
Deleting data
This is partially why we needed to understand a little about file systems. So I could explain to you how data is handled when it is deleted.
As I explained earlier, when you write a file to memory, a corresponding entry is written to the file system to advise the OS where the data is. Sure, you may think you are writing the file to /sdcard/documents/work directory on the internal memory, but in reality these directories or folders do not actually exist at a memory hardware level. The data is written to a block and the file system is informed where that block is, how big the file is, what directory it should appear in to the OS etc. When a file is written the memory, the OS see's the available space go down and the used space go up. All this information comes from the file system.
When you delete the file, the actual data is NOT deleted. It remains where it is on the memory. The block is not overwritten. When the OS is told by the user to delete the file, the File system entry is deleted. This changes the free/used space as the file system is no longer accounting for the data, however the truth is the data still exists. When the next request to write a file to the memory comes from the OS, the file system will think the block where the old data was is empty and will overwrite it.
It is this difference between the file system and reality that allows data to be recovered by external software. if you do not write any data to the memory, external software can scan the memory for data whilst bypassing the file system all together. Ff course the window is small. You only have a very limited time to recover data before the file system allows the data to be overwritten with a new entry.
This is not just true of a deleted file. Even formatting the memory (which is actually just re-creating a new, blank file system) leaves all the data in tact behind-the-scenes and can all still be recovered until you start writing data to the memory. Cool huh?
Wow, all this time I've been stressing, is it really that simple?
Awwww snap! You got me. No it is not that simple. All this PC software, example: Piriform recuva only works on a computer drive. In windows, imagine this is anything with a Drive Letter. C: drive etc.
The only way to get a drive letter on your sdcard is to use USB Mass Storage mode, which as previously discussed - is not possible on your S3 (unless you are rooted, you can mount USB mass storage in custom recovery or use a UMS app from Play). The alternative is to use a card reader on your PC and put the sdcard in it.
There are also apps like Undelete for root users - which again, you guessed it - requires root. So if you're not rooted, it's simplest to use a card reader which can be bought for peanuts.
It's worth mentioning, NONE OF THE ABOVE will work with /sdcard on internal memory. It is not possible to get your data back once deleted from internal memory. Once gone, it's gone forever. You can only restore from /extSdCard (removable, external SDcard)
Phone won't boot, can I get my data back from internal memory?
Let's start by saying, it depends why your phone won't boot. If it's an SDS (Sudden Death Syndrome) type issue, where your internal eMMC (NAND FLash memory) has failed, then no. However, if you believe this is not the case then you can get your /sdcard data using adb BUT you need a custom recovery to be flashed via Odin before you do this. Read [REF] Understanding the basics before rooting your S3
However, if out of curiosity - you do still want to get your data off, using adb , read below:
Pre requisite is having adb "installed" on your windows PC. Download THIS file and follow the instructions in the readme.
You need to observe the following. For android 4.1.x and earlier, /data/media for android 4.2.x and newer, /data/media/0 - I will assume 4.2.2 for this guide,.
1) Boot into recovery, connect usb and go to "mounts and storage". Toggle the "mount data" options to mount these partitions. Tip, when mounted, the option then becomes "unmount data"
2) Open "cmd" in Windows and type the below code, which will copy all your data to a folder called sdcard on your windows desktop
Code:
adb pull /data/media/0/ c:\users\rootsu\desktop\sdcard
Also note, this assumes you have windows vista or newer. Also, it assumes your windows username is rootsu.
That's it, simple.
Display and Digitiser won't work, can I get my data back from internal memory?
You can use adb and a custom recovery to pull data from your /sdcard or even app data from /data/data
Pre requisite is having adb "installed" on your windows PC. Download THIS file and follow the instructions in the readme.
You need to observe the following. For android 4.1.x and earlier, /data/media for android 4.2.x and newer, /data/media/0 - I will assume 4.2.2 for this guide,.
1) Boot into recovery, connect usb and go to "mounts and storage". Toggle the "mount data" options to mount these partitions. Tip, when mounted, the option then becomes "unmount data"
2) Open "cmd" in Windows and type the below code, which will copy all your data to a folder called sdcard on your windows desktop
Code:
adb pull /data/media/0/ c:\users\rootsu\desktop\sdcard
Other things you may want to pull.....
Code:
adb pull /data/data/com.android.providers.telephony/databases/mmssms.db c:\users\rootsu\desktop\sdcard
Code:
adb pull /data/data/com.android.providers.contacts/databases/contacts2.db c:\users\rootsu\desktop\sdcard
Also note, this assumes you have windows vista or newer. Also, it assumes your windows username is rootsu.
That's it, simple.
Data corruption
When data becomes corrupt, there's really not a lot you can do. The file system knows where the data is already. If it's corrupt, you're stuck. Most common causes of corruption are:
1) Dirty unmount of /sdcard. SD card pulled out whilst it is being written to / phone shuts off whilst being written to. SOMETIMES - Plugging the card into a card reader in windows, Windows will ask to fix it and MAY fix it.
2) Fake SD card. This is really the MOST common. Get a card reader ans use:
h2testw.exe for windows to test your card in a card reader. Set it to read the full size of the card, which will take hours but well worth it.
If you get a result like this:
Code:
Warning: Only 63995 of 63996 MByte tested.
The media is likely to be defective.
3.8 GByte OK (8072512 sectors)
58.6 GByte DATA LOST (122989248 sectors)
Details:2 MByte overwritten (4096 sectors)
...Then you have a fake card, that is really 4 GB. I'll explain this.
Commonly, fake cards are reprogrammed to "think" they are high capacity cards, such as 32 GB or 64 GB to defraud buyers out of money. This is common on eBay (Never buy cards from eBay).
When these cards are formatted, the file system also thinks it is this fake capacity. Normally, when a card is full, the file system will report to the OS there is no more space and this prevents additional writes to the card. However, in the case where the card is 4GB and the File system thinks it is 64 GB, the tricked file system doesn't know the card is full. The file system keeps allowing data to be written to the card, over writing the existing data but without replacing the file system entries. The file system thinks data that has been overwritten hasn't been overwritten so when you try to open one of these files, it is essentially "corrupt" or non-existent.
Preventing data loss
Time to wise up guys. It is possible to recover data off your removable media, but internal memory - very unlikely. No apps on your PC or Android will help with deleted data. So you need to backup.
Dropbox - Use dropbox to automatically upload your photos to online storage.
Foldersync - Use FolderSync to upload important sdcard files to your dropbox account, or better yet - got a computer thats always on at home? Set foldersync to schedule a sync over wifi whilst you're asleep.
Other info
Interesting tidbits
Quite an exhaustive reference guide you got here rootSU thanks this will sure come handy for all of us :good:
Cheers
Thank you very much for taking the time to write this. It's a non academic approach to a sum of keywords and all of them are explained in such a manner that it would be almost impossible to misunderstand
Nice!
Nice post!
There are a few other interesting tidbits of info that might be worth mentioning:
- eMMC has an internal micro-controller that runs very specific firmware (and SDS was mainly caused by a bug in that firmware)
- eMMC (just like SSD) has specific writing/erasing limits and commands to deal with that - as a very general idea it can write about 4k at a time but can only erase in much larger blocks - like 64k (at least, but a 16GB model could have a much bigger block); normally on the same erase-size block there is very special list maintained, and based on that list wear leveling is implemented;
- all flash-based memory AGES - there is only a limited amount of erase/writes cycles possible before a point where the info is no longer reliably-stored; in some models that value can be incredibly small! to avoid writing more to some regions than other a mechanism call wear leveling is implemented; that one can have a big impact on both speed and reliability (but really don't expect it to create miracles)
- since it is very important for the speed and reliability of the flash memory to return unused blocks to this internal lists there are special TRIM commands that informs the firmware that the block can be garbage-collected; with an OS that supports TRIM, when a file is erased the blocks are also TRIMmed; this is one extra level that makes recovery basically impossible under normal circumstances
- this does not really mean that things are completely impossible to recover, just that you might need to spend so much on it that recovery would be impractical for any item worth less than 100000 US$ to 1 million US$
EDIT
- also just as with SSD it is not a bad idea to keep a good percentage of the flash memory free - IMHO at least 4GB for 16GB models, 6-8GB for 32GB models - that will improve performance since fragmentation (CLARIFICATION - free-space fragmentation) will grow much slower
- unfortunately there is no program for eMMC similar to smartctl (or any other SMART-data reading program) on normal SATA/IDE/SCSI disks - there seem to be some proprietary commands that are somehow similar but those are generally undocumented.
xclub_101 said:
Nice post!
Click to expand...
Click to collapse
I've put a link to your post in post 1. Where as it's not strictly relevant to my point, it is interesting stuff....
Fragmentation isn't an issue on ssd's. Its an issue on hdd because the head must physically move to another area of the Platter to get the data. That's the slow down. Defrag of a hdd moves all the used blocks (data) together so the actuator doesn't need to move much.
Performance degrades over time on ssds because every write, if data already exists must be erased too. But this hasn't really been an issue so much since TRIM became widely available.
-----------------------
Sent via tapatalk.
I do NOT reply to support queries over PM. Please keep support queries to the Q&A section, so that others may benefit
rootSU said:
...
Fragmentation isn't an issue on ssd's.
...
Click to expand...
Click to collapse
Thank you.
Also sorry for the misunderstanding with my contraction - what I wanted to say was free space fragmentation - that one does matter a lot on solid-state memory because of the garbage collection and some controllers have been famous for having a huge drop in performance with little free space - I will try to also correct that post.
xclub_101 said:
Sorry, I used a misleading contraction - what I wanted to say was free space fragmentation - that one does matter a lot on solid-state memory because of the garbage collection and some controllers have been famous for having a huge drop in performance with little free space - I will try to also correct that post.
Click to expand...
Click to collapse
Yep, it's true about Garbage collection, but TRIM *should* handle this nicely as should "over provisioning" although probably, some cheap SSD's may not over provision.
EDIT> Actually (sorry everyone for off topic) if you're interested in SSD's, these articles are a "fun" read... (I put fun in speech marks as it depends how geeky you are )
http://en.wikipedia.org/wiki/Trim_(computing)
http://www.pcworld.com/article/2038...-ssds-what-makes-these-speedy-drives-hum.html
Update to post 1:
Note from the Author -
I am moving on to the N5 now and ditching my S3. I will continue to maintain this thread, however - please do PM me if you think that something needs to be changed or updated in this thread as I doubt I will be answering questions within the thread as much. Please don't PM support questions to me. Only PM updates that need to be made in the thread.
It's been a blast!
Regards
Dan
Awesome bits of info. This is the game changer. I learned a whole lot just by reading here in XDA. I've only been using Android for a few weeks but thanks to XDA, I've already rooted, installed a bunch of apps and kept my OCD in check.
my device memory has corrupted and when i start recovery mode i get "E: faild to mount /cash (invalid argument) "