[ale] If you own (almost) ANY HDD, repartition with 4k boundaries
Ron Frazier (ALE)
atllinuxenthinfo at techstarship.com
Thu Jan 3 21:52:27 EST 2013
You're welcome. Glad to help. Maybe it will help someone else avoid
some potholes. I'm sure I'll need the help on other issues later.
Sincerely,
Ron
On 1/3/2013 9:25 PM, Jim Kinney wrote:
> Nice write-up. Thanks!
>
> On Thu, Jan 3, 2013 at 9:00 PM, Ron Frazier (ALE)
> <atllinuxenthinfo at techstarship.com
> <mailto:atllinuxenthinfo at techstarship.com>> wrote:
>
> Hi all,
>
> I recently posted a message about misaligned partitions, to which
> a couple of people replied with info about 4k sectors for HDD's.
> I decided to post this message with a different subject because
> this can have a much broader impact than many people probably
> know. Some of you may already know this. Others may not. I
> haven't purchased any new drives lately, so I'm just learning
> about it.
>
> To reiterate what happened, I replaced a 500 GB hybrid SSD HDD
> with a new 750 GB one of newer vintage. I cloned the hard drive
> using a 3 year old version of Acronis True Image, which had always
> worked before. When I brought up the new drive in Ubuntu 11.10
> Disk Utility, for each partition I clicked on, it said the
> partition boundary was misaligned, and performance would suffer.
> Based on the responses to the other threads, I started doing research.
>
> I'm not the best at explaining this, so bear with me. See also
> the links and quotes below.
>
> Essentially, all HDD's made after January 2011 are now employing
> 4096 byte or 4 KB sectors instead of the traditional 512 byte
> sectors. The firmware in the drive reports sectors as though they
> were 512 so legacy software will work. Also, traditionally, the
> first partition of an MBR formatted HDD started on sector 63,
> after the MBR. So, if the data from a 512 byte sector drive is
> cloned onto a 4 KB sector drive, the partition boundaries will be
> offset and the data will not align properly with the new physical
> media. You might think, so what, the firmware is translating, no
> problem. Wrong! As it turns out, misaligned partitions can lead
> to write performance which is 10 times slower.
>
> There are the equivalent of 8 - 512 byte sectors in 1 - 4 KB sector.
>
> Let's say you want to read 512 bytes, what your software thinks is
> 1 sector. It sends the request to the HDD. The firmware finds
> the actual 4 KB sector with what you want in it. It then extracts
> the 512 bytes you want and sends that to you. That works pretty
> well. Writing is a whole other matter.
>
> Here's a quote from the Seagate article below:
>
> "A write process can be more complicated, particularly when data
> the host computer attempts to write is a subset of a physical 4K
> sector. In these cases, the hard drive must first read the entire
> 4K sector containing the targeted location of the host write
> request, merge the existing data with the new data and then
> rewrite the entire 4K sector. (On the next revolution.)"
>
> So, if you're writing 512 bytes, the firmware finds and reads the
> 4 KB sector that contains the area you want to write, replaces the
> 512 bytes you're writing while retaining the rest, and writes the
> sector back to the disk. It has to wait for the platter to spin
> around again before it can rewrite the sector. At 7200 RPM, this
> takes an additional 8.3 mS. Lets say you're writing 1024 bytes
> and that bridges two 4 KB sectors. The system has to read both
> sectors, merge 512 bytes into one, then 512 bytes into the other,
> then write them back on the next revolution. You get the idea.
>
> The IBM article below indicates that this can lead to a 10 TIMES
> speed degradation if doing small writes.
>
> You fix at least part of the problem by realigning both the start
> and end locations of each partition on the disk to 4 KB
> boundaries. The standard industry practice seems to be to align
> them on on 1 MiB boundaries, which is a multiple of 4 KB.
>
> So, you want your first partition to start at 1 MiB, which is
> sector number 2048 rather than 63. I would leave 1 MiB between
> partitions, and make the next partition start and end on 1 MiB
> boundaries.
>
> I should note that you don't want to do this with Windows XP from
> what I've read. It expects the 1st partition to be at 63, period.
>
> In my case, I cloned a disk with 512 byte sectors to a disk with 4
> K sectors, but I kept the partition arrangement exactly the same,
> which had always worked before. So, everything on the clone ended
> up being misaligned.
>
> Here's why I worded the subject the way I did.
>
> If you have an SSD, you MUST align the partitions properly for
> optimum performance. They have 4K sectors.
>
> If you have a HDD with 4K sectors, you MUST align it properly for
> optimum performance.
>
> If you have a hybrid SSD HDD with 4K sectors, you MUST align it
> properly for optimum performance.
>
> If you have a hybrid SSD HDD with 512 byte sectors on the
> platters, you MOST LIKELY need to align it properly for optimum
> performance. Very hard to find any data on this.
>
> AND, if you have a lowly old HDD with 512 byte sectors, and you
> EVER MIGHT clone it to any of the others, you probably need to
> align it properly for optimum performance of the drive that you
> clone it to.
>
> As was pointed out in the other thread, some cloning software
> understands the 4K sectors. The older stuff, including my 3 year
> old Acronis True Image, does not. However, I'm pretty sure that
> if I align the partitions of an old HDD to 1 MiB boundaries, then
> do an identical clone, that should be preserved. If I let it
> expand the partitions on the new HDD, I don't know. Even with new
> software that understands, you might have to turn on an option
> such as align to 4K, align to 1 MiB, align to target, etc.
>
> I looked at the manual for Terabyte Unlimited's cloning software
> and I may replace Acronis with that. The product looks pretty
> impressive, and the manual mentions 4K sectors. I'm not afraid to
> pay for something this complex and critical if I have to. I've
> heard good things about Paragon software but don't know anything
> about it.
>
> You can check to see if you have an alignment conflict with Disk
> Utility (in Ubuntu). I don't know what program that actually
> runs. It will only show a message if the drive is partitioned the
> old way and it has 4K sectors, apparently. If the drive is
> partitioned the old way and you have 512 byte sectors, it won't
> complain. The version of Disk Utility in Ubuntu 11.10 does not
> partition the drive properly by default.
>
> Because of the potential problems of having ANY drives around with
> 512 byte sector alignment (also called cylinder alignment, I
> think), I've decided to repartition all my boot drives and am
> considering doing the archival drives. I'm putting everything on
> 1 MiB boundaries whether the actual drive has 512 byte or 4K sectors.
>
> The only way I could get into trouble with that is if I partition
> a drive with old software or if I clone a drive with old software
> and resize the partitions.
>
> You can use the latest version of Gparted from gparted.org
> <http://gparted.org> to repartition the drive non destructively
> with data on it. You should backup the data first. Don't use the
> one in Ubuntu, for example. It's old and buggy. Download the ISO
> for the Gparted live CD and use that so you have the newest version.
>
> This lifehacker article tells how to do it.
>
> http://lifehacker.com/5837769/make-sure-your-partitions-are-correctly-aligned-for-optimal-solid-state-drive-performance
>
> It's a little dated. The UI for Gparted has changed slightly.
> There is a drop down box for alignment. Set this to MiB. There
> is no longer an align to cylinders check box.
>
> So, here's what I did to all my boot drives.
>
> For the first partition, I simultaneously reduce the size and move
> the boundaries to allow 2 MiB before and after.
>
> Then, I move the partition to the left by 1 MiB as the article
> indicates.
>
> I then move each other partition that has at least a 2 MiB
> boundary to the left to within 1 MiB. If any partition doesn't
> have at least 2 MiB to it's left, I first move it to the right up
> to 2 MiB, then back to 1 MiB. This is due to an anomaly in the
> way Gparted calculates space, as mentioned in the article.
>
> When, I'm done, there is 1 MiB before the first partition,
> although you cannot see it on the Gparted drive map. I have 1 MiB
> unused spaces between partitions. And, every partition begins and
> ends on a 1 MiB boundary.
>
> This will optimize performance if the drive has 4K sectors, or if
> it is ever cloned to a drive that does.
>
> Doing this, particularly moving the boot partition away from
> sector 63 will probably break your boot loader. The article talks
> about how you can fix a Windows installation with a Windows
> install disk. If you don't have one handy, you may not want to
> try this.
>
> See the Gparted faq (below) for info on fixing a Linux boot loader.
>
> Here's how to check if you've finished the job properly using Gparted.
>
> Click on the first partition, right click, and select information.
>
> The first sector should be at 2048.
> Look at the total sectors. In one case, for me, it's 307,183,616.
> Get out an engineering calculator that can handle big numbers.
> Enter this in and divide by 2048.
> In my case, the answer is 149,992. If this is a whole number,
> with no decimal, you're good.
>
> The unused space between each partition should have 2048 total
> sectors, or a multiple of that.
>
> For each other partition, check that the <first sector> / 2048 is
> a whole number
> and that the <total sectors> / 2048 is a whole number.
>
> If all the numbers check out, you have optimized your partition
> structure for drives with 4K sectors.
>
> Below are a number of links and some quotes that I found helpful.
>
> I hope this info is helpful to you.
>
> Sincerely,
>
> Ron
>
> ---------------------------------------
>
> http://en.wikipedia.org/wiki/Advanced_Format
> http://www.zdnet.com/blog/storage/are-you-ready-for-4k-sector-drives/731
>
> http://wdc.custhelp.com/app/answers/detail/a_id/5655
> https://ata.wiki.kernel.org/index.php/ATA_4_KiB_sector_issues (geeky)
> http://gparted.org/faq.php
> http://gparted.org/display-doc.php?name=help-manual
> http://gparted.org/display-doc.php?name=help-manual#gparted-fix-grub-boot-problem
>
>
> ---------------------------------
>
> http://www.seagate.com/tech-insights/advanced-format-4k-sector-hard-drives-master-ti/
> (good)
>
> "As all hard drive manufacturers have agreed to transition to the
> Advanced Format sector design by January 2011, the industry must
> adapt to and embrace this change to minimize potential negative
> side effects. "
>
> "A write process can be more complicated, particularly when data
> the host computer attempts to write is a subset of a physical 4K
> sector. In these cases, the hard drive must first read the entire
> 4K sector containing the targeted location of the host write
> request, merge the existing data with the new data and then
> rewrite the entire 4K sector. (On the next revolution.) "
>
> "Fdisk: GNU Fdisk is a command line utility that partitions hard
> drives. Versions 1.2.3 and above support Advanced Format drives. "
>
> "Parted: GNU Parted is a graphical utility for partitioning hard
> drives. Versions 2.1 and above support Advanced Format drives. "
>
> ------------------------------
>
> http://www.terabyteunlimited.com/index.htm - cloning software
> looks cool
> http://goinglinux.com/ - Just did a podcast, # 195, on SSD's.
>
> from prior thread - http://jrfom.com/2010/05/03/4kb-sectors/
> from prior thread - http://lwn.net/Articles/377895/
>
> -----------------------
>
> from prior thread -
> http://www.ibm.com/developerworks/linux/library/l-4kb-sector-disks/index.html?ca=dgr-lnxw074KB-Disksdth-LX
> (good) (dated)
>
> "Most modern file systems use data structures that are 4096 bytes
> or larger in size. "
>
> "Writes of misaligned data structures, on the other hand, require
> the disk's firmware to first read two sectors, modify portions of
> both sectors, and then write two sectors.
> The biggest write performance effects occurred with small file
> creation (extracting the kernel tarball). Effects on tarball
> extraction ranged from 1.04 (for ext4fs) to 25.53 (for ReiserFS),
> with a mean of 10.9. The second-best performer in this test was
> XFS, with a figure of 1.82. Because these figures are ratios of
> unaligned to aligned performance, a value of 10.9 means that a
> tarball extraction that takes 10 seconds on a properly aligned
> partition takes 109 seconds on an improperly aligned partition—a
> huge difference! The XFS figure of 1.82 means that this 10-second
> operation takes 18.2 seconds on an improperly aligned partition. "
>
> --
>
> (To whom it may concern. My email address has changed. Replying to former
> messages prior to 03/31/12 with my personal address will go to the wrong
> address. Please send all personal correspondence to the new address.)
>
> (PS - If you email me and don't get a quick response, you might want to
> call on the phone. I get about 300 emails per day from alternate energy
> mailing lists and such. I don't always see new email messages very quickly.)
>
> Ron Frazier
> 770-205-9422 <tel:770-205-9422> (O) Leave a message.
> linuxdude ATtechstarship.com <http://techstarship.com>
>
>
>
> _______________________________________________
> Ale mailing list
> Ale at ale.org <mailto:Ale at ale.org>
> http://mail.ale.org/mailman/listinfo/ale
> See JOBS, ANNOUNCE and SCHOOLS lists at
> http://mail.ale.org/mailman/listinfo
>
>
>
>
> --
> --
> James P. Kinney III
> ////
> ////Every time you stop a school, you will have to build a jail. What
> you gain at one end you lose at the other. It's like feeding a dog on
> his own tail. It won't fatten the dog.
> - Speech 11/23/1900 Mark Twain
> ////
> http://electjimkinney.org
> http://heretothereideas.blogspot.com/
> ////
>
>
> _______________________________________________
> Ale mailing list
> Ale at ale.org
> http://mail.ale.org/mailman/listinfo/ale
> See JOBS, ANNOUNCE and SCHOOLS lists at
> http://mail.ale.org/mailman/listinfo
>
--
(To whom it may concern. My email address has changed. Replying to former
messages prior to 03/31/12 with my personal address will go to the wrong
address. Please send all personal correspondence to the new address.)
(PS - If you email me and don't get a quick response, you might want to
call on the phone. I get about 300 emails per day from alternate energy
mailing lists and such. I don't always see new email messages very quickly.)
Ron Frazier
770-205-9422 (O) Leave a message.
linuxdude AT techstarship.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ale.org/pipermail/ale/attachments/20130103/a6c49a1c/attachment-0001.html>
More information about the Ale
mailing list