[ale] OT converting old videos, cataloging dvds, storing forever

[Matt Hessel]

When my 8 month old tried to buff the hardwood floor with some choice discs
from our dvd collection, I decided to find a better way to handle video.

I ripped each one and converted to h.264, then stored on my computer
upstairs, then set up streaming to the ps3 downstairs for the kids to watch.

I had to experiment a bit to get the results I wanted, but now I get a
normal 2 hour dvd to a bit over 1 gig with 5.1 stereo output.

VHS conversion will leave a good bit of noise and artifacting, so it
doesn't compress down as much, but you should be able to put your library
in a much smaller amount of space then the MPEG 2 it sounds like you are
using.

These days I normally see 2 TB drives for about $90-120 depending on how
long you can wait for it to cough out the data.

I would set up two of those in a mirror and play with conversion tweaks to
see how small you can get them without degrading playback.
 On Oct 16, 2012 12:33 PM, "Ron Frazier (ALE)" <
atllinuxenthinfo at techstarship.com> wrote:

> HI all,
>
> After a few hours of Googling and reading forums and such, I've got some
> new data that I wanted to share.  My little video conversion project
> seems to be hitting a snag.  The 800 hours of videos I have are in the
> category of important to me, but not critical.  If I lose them, I won't
> die tomorrow.  So, the snag I'm hitting is the cost in terms of storage
> cost and time to convert them.  I don't have a huge budget, certainly
> not all at once.  I wanted to share my results, and see if you guys
> think I'm missing something.
>
> In terms of time, I think it will take me 10 - 15 minutes of tinkering
> with equipment, software, discs, cases, labels, etc. per one hour show
> to convert.  More if I do edits and cut commercials and sync the sound,
> etc.  Perhaps less if I store only to HDD and don't edit.  That adds up
> to 200 hours of my time, a non trivial amount.  Obviously the video tape
> must play, but I don't necessarily have to watch it.
>
> Here is what I believe to be the state of the art in storage options.
>
> I'm not considering other types of tape at this point.  I'm also not
> considering SSD, based on data retention issues I mentioned in another
> post.
> Option 1 is using HDD's to store the videos.
> Option 2 is some other type of disc, bluray, dvd, or mdisc.
> Option 3 is online storage.
> Based on my reading, I have concerns about the long term reliability of
> bluray, so I'm probably dismissing it.
>
> Let's discuss discs first.  Assume a single layer single sided DVD can
> hold ~1 hour of good quality video at a data rate of about 8 Mbps.  For
> a HDD, assume 4.7 GB / hr of video.  Note, in some of the forums where
> people burn LOTS of DVD's and spend hours checking their error rates
> etc, it was mentioned that 16x write speed DVD's are not reliable.  So,
> if available, I'm mentioning only 8x DVD's here.  There are also many
> types of dyes, and very confusing data, which I'm not going to try to
> sort out here as I don't understand it all.  There are may types of
> printable surfaces on the top of the disc, for automated printing of
> labels.  Lightscribe and inkjet surfaces are examples.  These usually
> increase the cost.  I don't need them.  For my purposes, writing on the
> disc with a DVD safe sharpie pen is enough.  Based on my reading, double
> layer discs suffer more compatibility problems so I'm not considering
> them.  Double sided discs are much more likely to get scratched and
> cannot easily be labeled so I'm not considering them.  It is actually
> very hard to figure out all the DVD media options, but I've selected a
> few to look at.  Prices are Amazon, if they sell the product in question.
>
> Gotta account for the case for the DVD, about $ 0.25 each.
>
> The DVD-R format seems to be the most universally compatible.  I'm not
> trying to list all discs here, just get representative prices.  Of
> conventionally constructed discs, Verbatim seems to have a good
> reputation, but there are varying reviews.  Longevity estimates for
> conventional disc media range from 5 years anecdotal to 50 years from
> the factories.  I will say that I have some 5 year old Maxell DVD-R's
> that are still readable, but I will probably copy them to archival discs.
>
> VERBATIM 94852 DVD-r media 8x 4.7gb 50-pk spindle (shiny silver) - $ 19
> / 50 = $ 0.38 / disc.  Cost with case (cwc) = $ 0.63 / disc
> Verbatim Shiny Silver DataLife Plus 8x DVD-R Spindle, 50 Discs - also $
> 19 / 50
>
> It seems that JVC Taiyo Yuden is the top of the heap in terms of quality
> for conventional media.  Taiyo Yuden bought part of JVC as I understand
> it and they're now shipping with JVC part numbers and packaging.  Many
> people still know the Taiyo Yuden name and so that's usually listed in
> the product listings.  Apparently, some TY discs themselves are
> unlabeled, so if you want TY, make sure you get TY.
>
> 100 Taiyo Yuden/JVC 8X DVD-R 4.7GB Silver Thermal Lacquer - $ 29 / 100 =
> $ 0.29 / disc.  cwc = $ 0.54 / disc  Better product, cheaper price.  Cool.
>
> Still, I don't consider these discs long term storage.  The next step up
> is archival grade discs, which are much more expensive.  Archival grade
> discs usually have a solid gold layer to minimize corrosion and maximize
> reflectivity.  Some have gold only.  Some have gold and silver, which is
> cheaper - less gold, more silver.  They also usually have an extra hard
> coating to prevent scratches.  Conventional DVD's are insanely easy to
> scratch.
>
> Verbatim 95355 UltraLife 4.7 GB 8x Gold Archival Grade DVD-R, 50-Disc
> Spindle - $ 93 / 50 = $ 1.86 / disc.  cwc = $ 2.11 / disc
>
> My research tells me that MAM-A / Mitsui is state of the art in archival
> grade discs.  They have lots of data on their website.  Their gold /
> silver DVD discs are projected to last 83 years.  Their gold DVD discs
> are projected to last 100 years.  And their gold CD discs are projected
> to last 300 years.  Also, note the same risk as above in buying
> unlabeled discs.
>
> http://www.mam-a.com/
>
> MAM-A Mitsui Gold Archive 8x DVD-R with no logo in bulk - 50 Count = $
> 117 / 50 = $ 2.34 / disc.  cwc = $ 2.59 / disc  (100 year discs)
> MAM-A 8x SILVER PLUS GOLD DVD-R Branded Gold Surface 50pk Beehive with
> Hard Coat - $ 93 / 50 = 1.86 / disc.  cwc = $ 2.11 / disc  (83 year
> discs)  (factory price)
> MAM-A 84019 Silver plus Gold DVD-R 16x 4.7GB MAM-A Logo 50pk Spindle -
> 84019 - $ 80 / 50 = $ 1.60 / disc.  cwc = $ 1.85 / disc  (83 year
> discs)  (dealer price)
>
> This last one is probably the best compromise between price and longevity.
>
> Finally, if you want longevity for 1000 years, there's m-disc, as
> mentioned previously.  This is overkill for what I'm doing, but here are
> the prices.
>
> M-DISC DVD+R 4X Cake Box 50 Pack - $ 136 / 50 = $ 2.72 / disc.  cwc = $
> 2.97 / disc.  That's only $ 0.38 / disc more than the 100 year discs.
> Not to bad if you need that kind of lifespan.
>
> So, the best I can do for an archival grade disc is $ 1.85 / disc.
>
> Now, what about the HDD option.  Here's a 3 TB drive I found on Amazon.
>
> Western Digital Caviar Green 3 TB SATA III 64 MB Cache Bare/OEM Desktop
> Hard Drive - WD30EZRX - $ 140
>
> Let's say I want two for redundancy, since this is NOT archival media.
> I would definitely spinrite this thing a couple of times / year to
> refresh the magnetic fields.  That process would probably take 5 days.
> 2 HDD - $ 280
>
> How about a nice dual drive case with fan.
>
> Vantec Dual 3.5-Inch SATA to USB 3.0 and eSATA with JBOD/RAID 0/1
> External Hard Drive Enclosure (NST-400MX-S3R) - $ 83
>
> So the total cost is - $ 363
>
> And we can store 3000 GB / 4.7 GB per hour = 638 hours of video
> approximately equivalent to 638 DVD's.
>
> So, cost per equivalent DVD works out to - $ 363 / 638 = $ 0.57 / "disc"
> or hour.  This is slightly more than the cost of cheap DVD's when cost
> of cases is accounted for but substantially less than the cost of
> archival DVD's.
>
> So, here's how I see my choices.
>
> - 83 year archival DVD's
>
>     * relatively low up front cost for 50 pack
>     * substantially larger long term cost
>     * some additional cost for storage rack, etc.  (and where do I put it)
>     * discs are easy to access and play in dvd player
>
> - two big hard drives
>
>     * relatively high up front cost
>     * substantially lower long term cost
>     * minimal space required
>     * movies are harder to access since I have to hook up computer to tv
> and start player, etc.
>
> There is the possibility of online storage.  Storing 5 GB on Amazon S3
> will cost me $ 0.75 / month for 1 hour of video.  It will cost me about
> $ 0.75 each time I retrieve it.  Given the quantity of data, not very
> practical.
>
> None of these choices are really great for me.  What I may do is to
> select the best 50 - 100 hours of source video and put them onto 83 year
> archival discs.  Even that would cost me $ 185.  I'm not sure it's worth
> it.  If I'm comparing the HDD to the 83 year archival discs, the hard
> drive becomes more economical than the discs once I store 196 hours of
> video.  At this point, it seems there are no good (within my budget)
> options to store 800 hours of video.  One other option is to convert
> everything to cheap DVD's, then refresh it and recopy it every 5 - 10
> years.  That doesn't sound too attractive either.
>
> PS, from a hardware perspective, the Hauppauge USB-Live2 analog video
> digitizer looks cool.  Haven't tried it and don't know about it's
> drivers or software.  I can get mdisc / bluray drives in the $ 50 - 100
> range.
>
> Let me know what you think, and whether I've left out anything major.
> This took two hours to type and I didn't proofread it, so sorry for any
> errors.
>
> Sincerely,
>
> Ron
>
>
> On 10/15/2012 4:45 PM, JD wrote:
> >> I also wanted to share a great way to track and catalog DVD's that
> >> you've purchased.  It's an android app called MyMovies (free - limited
> >> to 50 titles) and MyMovies Pro ($6 - unlimited).  It's ultra cool.  Just
> >>
> > Hopefully, someone else has found a fantastic Linux video editor and
> will be
> > able to let us all know about it.
> >
> >
> >
> > Previously I used a F/LOSS web server + back-end DB to catalog DVDs and
> > recordings.  Over time, that became too much effort and I didn't like
> that the
> > program forced me to a specific DB and the export was only to HTML
> files.  Tried
> > a Java desktop client too, but it was toooo damn slow.  It didn't play
> nice on
> > my network.
> >
> > Ended up using the time-proven 'ls -Rl' technique with numbered DVDs.
>  With a
> > large number of data DVDs holding TV recordings in xvid/avi or
> h.264/mkv, a
> > quick 'egrep -i' searches by title for the specific numbered disk
> containing the
> > recorded show/movie.  As a CLI person, the most-used desktops all have a
> script
> > that will ssh over to the main box with all the dvd-0xyz.txt files and
> perform
> > the query.  For more information, XBMC has the catalog with actors,
> directors,
> > year, genre, title,  etc ....  grep is just so useful and text is so
> easy to
> > search.
> >
> > I love that the grep clearly returns the numbered data DVD I need to
> load.
> >
> > As to converting VHS to MPEG or XVID or h.264, almost any $20 USB TV
> adapter can
> > record SD content to MPEG2.  Just pick from a MythTV compatible device
> to be
> > happiest long term.  If you want to record OTA HD content, I'd suggest
> more
> > effort for the selected hardware.  I love the HD-Homerun network tuners.
> Those
> > work well with OTA except 1 channel that doesn't come in at my house. It
> is
> > hardly a major issue, since 69 other channels do come in OTA from
> Marietta.
> >
> > For finding commercials and editing them out, I searched for Linux-based
> > solutions for years and finally gave up. Comskip does run nice under
> WINE, I've
> > never found a Linux-based editor that will accept the comskip created
> cut files
> > (there's like 6 different formats that comskip will create) as the
> starting
> > point for where to cut commercials.  Comskip is about 90% accurate.
>  Comskip can
> > output cut marker files that XBMC and mplayer will honor without
> altering the
> > source video file. That can be handy when you just want to skip
> commercials, but
> > not waste time actually modifying the file contents.
> >
> > Instead of fighting it, I bought - yes, purchased a Windows tool called
> > VideoRedo TV Suite.  It appears to use comskip code internally to create
> cut
> > markers.  There's a point where $50 is worth it to me.  That $50 program
> has
> > been heavily automated here to generate the cut marker files overnight.
> It takes
> > about 45 seconds of human effort to confirm the cut locations and
> request the
> > modified file be saved.  If you are willing to live with 90% accurate
> cuts, it
> > could all be automated.  Networks have changed up the markers commonly
> searched
> > by VRDP and comskip to trick them into keeping commercials and removing
> some of
> > the desired content.  Older stations with less money like RTV are almost
> 100%
> > accurate with cut locations.
> >
> > At this point, my systems are almost automatic with just a few manual
> > "confirmation" places.  WTV -->  MPEG2 -->  h.264/mkv with CC1 and CC3
> closed
> > captions included.  If there are multiple audio channels in the stream,
> that can
> > require special processing, but that usually happens with a few PBS
> programs
> > with audio tracks for the deaf or background-only audio.
> >
> > I've blogged about almost all of these solutions over the years. The
> most recent
> > articles are about building your own DB4 antenna for $20 or so that
> actually
> > beat some Antennas Direct models.
> >
> >
> > And just to be clear, I also wimped out and use Windows7 to record TV.
> It is
> > just so easy to setup with free schedule data. That OS runs inside a
> virtual
> > machine under KVM and has been doing it nicely for over a year.  THAT
> was not an
> > easy thing to accomplish to get satisfactory results. Microsoft did
> (does?) some
> > things trying to protect Joe-end-user from doing foolish things, like
> blocking
> > recording directly to NAS storage.
> >
> > I suspect there are hundreds of years of knowledge lurking related to
> this problem.
> > _______________________________________________
> > Ale mailing list
> > Ale at ale.org
> > http://mail.ale.org/mailman/listinfo/ale
> > See JOBS, ANNOUNCE and SCHOOLS lists at
> > http://mail.ale.org/mailman/listinfo
> >
> >
>
> --
>
> (To whom it may concern.  My email address has changed.  Replying to former
> messages prior to 03/31/12 with my personal address will go to the wrong
> address.  Please send all personal correspondence to the new address.)
>
> (PS - If you email me and don't get a quick response, you might want to
> call on the phone.  I get about 300 emails per day from alternate energy
> mailing lists and such.  I don't always see new email messages very
> quickly.)
>
> Ron Frazier
> 770-205-9422 (O)   Leave a message.
> linuxdude AT techstarship.com
>
> _______________________________________________
> Ale mailing list
> Ale at ale.org
> http://mail.ale.org/mailman/listinfo/ale
> See JOBS, ANNOUNCE and SCHOOLS lists at
> http://mail.ale.org/mailman/listinfo
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.ale.org/pipermail/ale/attachments/20121016/975822d3/attachment-0001.html