[ale] musings on the insides of an ssd - part 2

[Ron Frazier (ALE)]

Hi all,

This is part 2 of a 2 part message.

You've got the key bits of data right there. In this context, a warranty without an endurance rating means nothing. An endurance rating without a data rate means nothing. Unfortunately, many articles and ads don't give you this data.

Doing a bit of quick math. 5 yr = 1825 days. 1825 days * 20 GB = 36.5 TB of total data that you can write to the drive before it is no longer guaranteed to be fully reliable. 36.5 TB / 180 GB of capacity = 202. Thus, you can completely fill, or refill, the drive with data 202 times.

You may say to yourself, I'll never write 36 TB of data to this drive. Well, you may not, but your pc might. There are long checklists that you can and should look for any give type of pc to optimize it for an ssd. It's different for linux, windows, mac. I don't have them memorized, but I'll just say here that your pc can foil your best laid plans behind your back.

Here are just five examples from the linux world.

Note, reading from the memory cells does not hurt the memory cells. Only writing and erasing them.

In many cases, linux will alter the meta data of a file even if you just read the file. It stores the "date accessed". This causes some data to be written to the disk every time you touch a file, whether you alter it at all. When you consider years of usage, and tens of thousands of files, this adds up, and uses up the write endurance of your ssd. Most of the ssd optimization checklists recommend turning this off.

Another place the os can sabotage your ssd is your swap file or partition. By definition, the swap file or partition will start being exercised when the pc runs lower on ram than it wants to be for whatever it's doing. From experience, I can tell you that opening 70 or so tabs in Firefox, email, Libre Office, and a few other random things will start hitting the swap file unless you have lots of ram. If you've ever watched the drive light of a pc with swap being used, you may see it flickering continuously. Hitting swap can add many GB of data writes to the drive in a very short time. In the case above, you've only got 36,500 GB of writes to play with for the entire life of the drive. Most checklists recommend putting your swap on a spinning drive.

You get email, right? You must, because you're reading this message. I'll bet you have dozens or hundreds of emails coming in all the time. In my case, I have thousands, from mailing lists and such. Every email adds to the database if you store them locally, which may be mysql or something. Every email uses up some of the drive's write life. How much, depends on your usage. I have email archives going back a decade, which encompass 10 GB or so. The average tech user, maybe 1 GB. The average grandma, maybe 50 MB. It depends. Here's another thing. As you delete messages, usually they're flagged as being deleted, but actually still exist. Later the database may be compacted, which involves essentially rewriting the remaining messages back to the hard drive and deleting the old file. So, if you have an 8 GB file with 2 GB of deleted messages, the system could decided to rewrite the remaining 6 GB. That comes off your usage tally of the drive's life.

What about podcasts, vidcasts, and media. My tablet downloads 1 GB of podcasts / week, about 20 shows of around 1 hour of audio. I listen to them, and most eventually get erased. Not that I plan to change, but, that does contribute to using up my flash memory. If that were video on a desktop pc, it could occupy 20 X the space or more.

What if you edit certain types of documents or media files? What if your system always makes a backup copy of every file, or different versions? What if it edits a temp file and then saves the file? Etc., etc.

All these scenarios can balloon the amount of data being written to the drive to much more than what you'd think. On a hdd, you don't worry. The drive is either working or it's not. When it starts to throw errors, you deal with it. But, you might want to think about what you're doing to your SSD. Everything you write to it pushes it, in a definable and predictable way, toward its death bed.

We've discussed the concept of data scrubbing here before. This is a good practice with either a hdd or ssd. It is a good idea to use a diagnostic utility to periodically read every sector on the drive. This lets the controller know if any sectors are weak and need error correction or to activate over provisioning. If you have evidence that the drive is throwing errors, and a read scan doesn't fix them, it can be a good idea to do a full drive WRITE / READ scan. Yes, depending on the diagnostic, you might use up 1 or 2 of the 202 full writes this example drive has of it's life. However, some errors can only be detected by the controller by trying to write to the cells then read them back. If necessary, they are mapped out and spares brought on line. Thus, you could extend the life of the drive past the point where it might otherwise have to be replaced by letting the controller know, in detail, which cells it can depend on.

I've mentioned over provisioning a couple of times. This is simply providing more memory cells than what are needed to provide the rated capacity of the drive. In the hdd world, we'd think of them as spare sectors. Over provisioning allows the controller to deallocate cells that are getting flaky and use the spares instead. This extra space is also used for drive optimization and management. This extends the useful life of the drive. In general, I think it's a good idea to pay for some over provisioning.

The Maximum PC article made a very interesting statement. It said that if you see a drive rated at 256 GB, there is no over provisioning. But, if it says 240 GB, there is 16 GB of over provisioning.

Wow, is it really that simple? I never thought about that. Can you really just subtract the drive capacity from the nearest binary multiple to find the over provision amount? If so, it sounds like that Intel 180 GB drive I mentioned has 76 GB of spare space. I don't know for sure if that's true, but it sounds good.

What concerns me is that, 5 years down the road or less for heavy users, and later for casual users, a bunch of people that didn't properly consider drive endurance, or who bought cheaper products with MLC or TLC flash, may find that their data is starting to mysteriously vanish.

I wonder if there are any utilities that can monitor the total data written to your existing hdd over time, so you could know, for example, if the Intel drive's 20 GB / day limit is a problem for you.

The controller is a critical part of the SSD. However, looking for a specific name may not be as important as before. There are sometimes performance differences between compressible data (like text) and non compressible data (like iso's) that you may wish to consider.

The drive should support TRIM, as should your OS, which allows the os in the background to erase blocks of data that are no longer needed, so the cells will be ready for writing when needed. Some drives have automatic garbage collection which does some of this work without requiring the os to trigger it.

Finally, if you're buying a motherboard, you might want to consider whether it supports ssd caching. This allows you to cache your most frequently used programs and data automatically to a small ssd, but have most of your big data and programs on a spinning hdd where space is cheaper. This is what a hybrid hdd does, but ssd caching technology allows you to create the same effect by bringing your own hdd and ssd and combining them.

I mentioned a while back in another thread that some drives support a data field in the smart data that shows the ssd drive's estimated remaining life based on the usage to the current time. The controller tallies up the total data written and makes that data available. You should look for a drive with this feature and monitor its progress toward its eventual doom.

Also, I remember mentioning previously, but it bears repeating, that some ssd warranties are based on the endurance rating and not the clock. So, if you write 60 GB / day to the drive I mentioned above, which is rated for 20 GB / day, your warranty may expire in 1.67 years and not 5 years.

Finally, here's a piece of anecdotal advice, which I cannot prove the worth of. But, run the drive. Yes, just run the drive. I don't trust an ssd lying on the shelf for a year to hold my data nearly as much as I would an hdd. Could be wrong. But, by running the drive, you allow the controller to do its maintenance, analysis, garbage collection, whatever it does. It cannot do that when it's powered off.

I've had the memory in my GPS go wonky a couple of times. I've had to wipe and restore the unit's software and reset the settings. I'm pretty sure those failures occurred after prolonged times of non use. Since I've been running the unit every day, I believe I've had fewer problems.

Hope this is helpful.

Sincerely,

Ron



--

Sent from my Android Acer A500 tablet with bluetooth keyboard and K-9 Mail.
Please excuse my potential brevity if I'm typing on the touch screen.

(PS - If you email me and don't get a quick response, you might want to
call on the phone.  I get about 300 emails per day from alternate energy
mailing lists and such.  I don't always see new email messages very quickly.)

Ron Frazier
770-205-9422 (O)   Leave a message.
linuxdude AT techstarship.com
Litecoin: LZzAJu9rZEWzALxDhAHnWLRvybVAVgwTh3
Bitcoin: 15s3aLVsxm8EuQvT8gUDw3RWqvuY9hPGUU