[ale] If you own (almost) ANY HDD, repartition with 4k boundaries

Ron Frazier (ALE) atllinuxenthinfo at techstarship.com
Thu Jan 3 21:52:27 EST 2013


You're welcome.  Glad to help.  Maybe it will help someone else avoid 
some potholes.  I'm sure I'll need the help on other issues later.

Sincerely,

Ron

On 1/3/2013 9:25 PM, Jim Kinney wrote:
> Nice write-up. Thanks!
>
> On Thu, Jan 3, 2013 at 9:00 PM, Ron Frazier (ALE) 
> <atllinuxenthinfo at techstarship.com 
> <mailto:atllinuxenthinfo at techstarship.com>> wrote:
>
>     Hi all,
>
>     I recently posted a message about misaligned partitions, to which
>     a couple of people replied with info about 4k sectors for HDD's. 
>     I decided to post this message with a different subject because
>     this can have a much broader impact than many people probably
>     know.  Some of you may already know this.  Others may not.  I
>     haven't purchased any new drives lately, so I'm just learning
>     about it.
>
>     To reiterate what happened, I replaced a 500 GB hybrid SSD HDD
>     with a new 750 GB one of newer vintage.  I cloned the hard drive
>     using a 3 year old version of Acronis True Image, which had always
>     worked before.  When I brought up the new drive in Ubuntu 11.10
>     Disk Utility, for each partition I clicked on, it said the
>     partition boundary was misaligned, and performance would suffer. 
>     Based on the responses to the other threads, I started doing research.
>
>     I'm not the best at explaining this, so bear with me.  See also
>     the links and quotes below.
>
>     Essentially, all HDD's made after January 2011 are now employing
>     4096 byte or 4 KB sectors instead of the traditional 512 byte
>     sectors.  The firmware in the drive reports sectors as though they
>     were 512 so legacy software will work.  Also, traditionally, the
>     first partition of an MBR formatted HDD started on sector 63,
>     after the MBR.  So, if the data from a 512 byte sector drive is
>     cloned onto a 4 KB sector drive, the partition boundaries will be
>     offset and the data will not align properly with the new physical
>     media.  You might think, so what, the firmware is translating, no
>     problem.  Wrong!  As it turns out, misaligned partitions can lead
>     to write performance which is 10 times slower.
>
>     There are the equivalent of 8 - 512 byte sectors in 1 - 4 KB sector.
>
>     Let's say you want to read 512 bytes, what your software thinks is
>     1 sector.  It sends the request to the HDD.  The firmware finds
>     the actual 4 KB sector with what you want in it.  It then extracts
>     the 512 bytes you want and sends that to you.  That works pretty
>     well.  Writing is a whole other matter.
>
>     Here's a quote from the Seagate article below:
>
>     "A write process can be more complicated, particularly when data
>     the host computer attempts to write is a subset of a physical 4K
>     sector. In these cases, the hard drive must first read the entire
>     4K sector containing the targeted location of the host write
>     request, merge the existing data with the new data and then
>     rewrite the entire 4K sector. (On the next revolution.)"
>
>     So, if you're writing 512 bytes, the firmware finds and reads the
>     4 KB sector that contains the area you want to write, replaces the
>     512 bytes you're writing while retaining the rest, and writes the
>     sector back to the disk.  It has to wait for the platter to spin
>     around again before it can rewrite the sector.  At 7200 RPM, this
>     takes an additional 8.3 mS.  Lets say you're writing 1024 bytes
>     and that bridges two 4 KB sectors.  The system has to read both
>     sectors, merge 512 bytes into one, then 512 bytes into the other,
>     then write them back on the next revolution.  You get the idea.
>
>     The IBM article below indicates that this can lead to a 10 TIMES
>     speed degradation if doing small writes.
>
>     You fix at least part of the problem by realigning both the start
>     and end locations of each partition on the disk to 4 KB
>     boundaries.  The standard industry practice seems to be to align
>     them on on 1 MiB boundaries, which is a multiple of 4 KB.
>
>     So, you want your first partition to start at 1 MiB, which is
>     sector number 2048 rather than 63.  I would leave 1 MiB between
>     partitions, and make the next partition start and end on 1 MiB
>     boundaries.
>
>     I should note that you don't want to do this with Windows XP from
>     what I've read.  It expects the 1st partition to be at 63, period.
>
>     In my case, I cloned a disk with 512 byte sectors to a disk with 4
>     K sectors, but I kept the partition arrangement exactly the same,
>     which had always worked before.  So, everything on the clone ended
>     up being misaligned.
>
>     Here's why I worded the subject the way I did.
>
>     If you have an SSD, you MUST align the partitions properly for
>     optimum performance.  They have 4K sectors.
>
>     If you have a HDD with 4K sectors, you MUST align it properly for
>     optimum performance.
>
>     If you have a hybrid SSD HDD with 4K sectors, you MUST align it
>     properly for optimum performance.
>
>     If you have a hybrid SSD HDD with 512 byte sectors on the
>     platters, you MOST LIKELY need to align it properly for optimum
>     performance.  Very hard to find any data on this.
>
>     AND, if you have a lowly old HDD with 512 byte sectors, and you
>     EVER MIGHT clone it to any of the others, you probably need to
>     align it properly for optimum performance of the drive that you
>     clone it to.
>
>     As was pointed out in the other thread, some cloning software
>     understands the 4K sectors.  The older stuff, including my 3 year
>     old Acronis True Image, does not.  However, I'm pretty sure that
>     if I align the partitions of an old HDD to 1 MiB boundaries, then
>     do an identical clone, that should be preserved.  If I let it
>     expand the partitions on the new HDD, I don't know.  Even with new
>     software that understands, you might have to turn on an option
>     such as align to 4K, align to 1 MiB, align to target, etc.
>
>     I looked at the manual for Terabyte Unlimited's cloning software
>     and I may replace Acronis with that.  The product looks pretty
>     impressive, and the manual mentions 4K sectors.  I'm not afraid to
>     pay for something this complex and critical if I have to.  I've
>     heard good things about Paragon software but don't know anything
>     about it.
>
>     You can check to see if you have an alignment conflict with Disk
>     Utility (in Ubuntu).  I don't know what program that actually
>     runs.  It will only show a message if the drive is partitioned the
>     old way and it has 4K sectors, apparently.  If the drive is
>     partitioned the old way and you have 512 byte sectors, it won't
>     complain.  The version of Disk Utility in Ubuntu 11.10 does not
>     partition the drive properly by default.
>
>     Because of the potential problems of having ANY drives around with
>     512 byte sector alignment (also called cylinder alignment, I
>     think), I've decided to repartition all my boot drives and am
>     considering doing the archival drives.  I'm putting everything on
>     1 MiB boundaries whether the actual drive has 512 byte or 4K sectors.
>
>     The only way I could get into trouble with that is if I partition
>     a drive with old software or if I clone a drive with old software
>     and resize the partitions.
>
>     You can use the latest version of Gparted from gparted.org
>     <http://gparted.org> to repartition the drive non destructively
>     with data on it.  You should backup the data first.  Don't use the
>     one in Ubuntu, for example.  It's old and buggy.  Download the ISO
>     for the Gparted live CD and use that so you have the newest version.
>
>     This lifehacker article tells how to do it.
>
>     http://lifehacker.com/5837769/make-sure-your-partitions-are-correctly-aligned-for-optimal-solid-state-drive-performance
>
>     It's a little dated.  The UI for Gparted has changed slightly. 
>     There is a drop down box for alignment.  Set this to MiB.  There
>     is no longer an align to cylinders check box.
>
>     So, here's what I did to all my boot drives.
>
>     For the first partition, I simultaneously reduce the size and move
>     the boundaries to allow 2 MiB before and after.
>
>     Then, I move the partition to the left by 1 MiB as the article
>     indicates.
>
>     I then move each other partition that has at least a 2 MiB
>     boundary to the left to within 1 MiB.  If any partition doesn't
>     have at least 2 MiB to it's left, I first move it to the right up
>     to 2 MiB, then back to 1 MiB.  This is due to an anomaly in the
>     way Gparted calculates space, as mentioned in the article.
>
>     When, I'm done, there is 1 MiB before the first partition,
>     although you cannot see it on the Gparted drive map.  I have 1 MiB
>     unused spaces between partitions.  And, every partition begins and
>     ends on a 1 MiB boundary.
>
>     This will optimize performance if the drive has 4K sectors, or if
>     it is ever cloned to a drive that does.
>
>     Doing this, particularly moving the boot partition away from
>     sector 63 will probably break your boot loader.  The article talks
>     about how you can fix a Windows installation with a Windows
>     install disk.  If you don't have one handy, you may not want to
>     try this.
>
>     See the Gparted faq (below) for info on fixing a Linux boot loader.
>
>     Here's how to check if you've finished the job properly using Gparted.
>
>     Click on the first partition, right click, and select information.
>
>     The first sector should be at 2048.
>     Look at the total sectors.  In one case, for me, it's 307,183,616.
>     Get out an engineering calculator that can handle big numbers. 
>     Enter this in and divide by 2048.
>     In my case, the answer is 149,992.  If this is a whole number,
>     with no decimal, you're good.
>
>     The unused space between each partition should have 2048 total
>     sectors, or a multiple of that.
>
>     For each other partition, check that the <first sector> / 2048 is
>     a whole number
>     and that the <total sectors> / 2048 is a whole number.
>
>     If all the numbers check out, you have optimized your partition
>     structure for drives with 4K sectors.
>
>     Below are a number of links and some quotes that I found helpful.
>
>     I hope this info is helpful to you.
>
>     Sincerely,
>
>     Ron
>
>     ---------------------------------------
>
>     http://en.wikipedia.org/wiki/Advanced_Format
>     http://www.zdnet.com/blog/storage/are-you-ready-for-4k-sector-drives/731
>
>     http://wdc.custhelp.com/app/answers/detail/a_id/5655
>     https://ata.wiki.kernel.org/index.php/ATA_4_KiB_sector_issues (geeky)
>     http://gparted.org/faq.php
>     http://gparted.org/display-doc.php?name=help-manual
>     http://gparted.org/display-doc.php?name=help-manual#gparted-fix-grub-boot-problem
>
>
>     ---------------------------------
>
>     http://www.seagate.com/tech-insights/advanced-format-4k-sector-hard-drives-master-ti/
>     (good)
>
>     "As all hard drive manufacturers have agreed to transition to the
>     Advanced Format sector design by January 2011, the industry must
>     adapt to and embrace this change to minimize potential negative
>     side effects. "
>
>     "A write process can be more complicated, particularly when data
>     the host computer attempts to write is a subset of a physical 4K
>     sector. In these cases, the hard drive must first read the entire
>     4K sector containing the targeted location of the host write
>     request, merge the existing data with the new data and then
>     rewrite the entire 4K sector. (On the next revolution.) "
>
>     "Fdisk: GNU Fdisk is a command line utility that partitions hard
>     drives. Versions 1.2.3 and above support Advanced Format drives. "
>
>     "Parted: GNU Parted is a graphical utility for partitioning hard
>     drives. Versions 2.1 and above support Advanced Format drives. "
>
>     ------------------------------
>
>     http://www.terabyteunlimited.com/index.htm - cloning software
>     looks cool
>     http://goinglinux.com/ - Just did a podcast, # 195, on SSD's.
>
>     from prior thread - http://jrfom.com/2010/05/03/4kb-sectors/
>     from prior thread - http://lwn.net/Articles/377895/
>
>     -----------------------
>
>     from prior thread -
>     http://www.ibm.com/developerworks/linux/library/l-4kb-sector-disks/index.html?ca=dgr-lnxw074KB-Disksdth-LX
>     (good) (dated)
>
>     "Most modern file systems use data structures that are 4096 bytes
>     or larger in size. "
>
>     "Writes of misaligned data structures, on the other hand, require
>     the disk's firmware to first read two sectors, modify portions of
>     both sectors, and then write two sectors.
>     The biggest write performance effects occurred with small file
>     creation (extracting the kernel tarball). Effects on tarball
>     extraction ranged from 1.04 (for ext4fs) to 25.53 (for ReiserFS),
>     with a mean of 10.9. The second-best performer in this test was
>     XFS, with a figure of 1.82. Because these figures are ratios of
>     unaligned to aligned performance, a value of 10.9 means that a
>     tarball extraction that takes 10 seconds on a properly aligned
>     partition takes 109 seconds on an improperly aligned partition—a
>     huge difference! The XFS figure of 1.82 means that this 10-second
>     operation takes 18.2 seconds on an improperly aligned partition. "
>
>     -- 
>
>     (To whom it may concern.  My email address has changed.  Replying to former
>     messages prior to 03/31/12 with my personal address will go to the wrong
>     address.  Please send all personal correspondence to the new address.)
>
>     (PS - If you email me and don't get a quick response, you might want to
>     call on the phone.  I get about 300 emails per day from alternate energy
>     mailing lists and such.  I don't always see new email messages very quickly.)
>
>     Ron Frazier
>     770-205-9422  <tel:770-205-9422>  (O)   Leave a message.
>     linuxdude ATtechstarship.com  <http://techstarship.com>
>          
>
>
>     _______________________________________________
>     Ale mailing list
>     Ale at ale.org <mailto:Ale at ale.org>
>     http://mail.ale.org/mailman/listinfo/ale
>     See JOBS, ANNOUNCE and SCHOOLS lists at
>     http://mail.ale.org/mailman/listinfo
>
>
>
>
> -- 
> -- 
> James P. Kinney III
> ////
> ////Every time you stop a school, you will have to build a jail. What 
> you gain at one end you lose at the other. It's like feeding a dog on 
> his own tail. It won't fatten the dog.
> - Speech 11/23/1900 Mark Twain
> ////
> http://electjimkinney.org
> http://heretothereideas.blogspot.com/
> ////
>
>
> _______________________________________________
> Ale mailing list
> Ale at ale.org
> http://mail.ale.org/mailman/listinfo/ale
> See JOBS, ANNOUNCE and SCHOOLS lists at
> http://mail.ale.org/mailman/listinfo
>    

-- 

(To whom it may concern.  My email address has changed.  Replying to former
messages prior to 03/31/12 with my personal address will go to the wrong
address.  Please send all personal correspondence to the new address.)

(PS - If you email me and don't get a quick response, you might want to
call on the phone.  I get about 300 emails per day from alternate energy
mailing lists and such.  I don't always see new email messages very quickly.)

Ron Frazier
770-205-9422 (O)   Leave a message.
linuxdude AT techstarship.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ale.org/pipermail/ale/attachments/20130103/a6c49a1c/attachment-0001.html>


More information about the Ale mailing list