[ale] OT converting old videos, cataloging dvds, storing forever

[Ron Frazier (ALE)]

Hi Matt,

Thanks for the note.  JD also mentioned h.264.  I'm certainly considering that.  I'm certainly considering a hdd solution.  However, I'd prefer something that exists on a dvd and I can just pop it into a stand alone dvd video player.  That's much easier to access in my case.  I don't have any kids around any more.  I also don't have anything that can stream video to the tv.  I can connect the tv to the computer by moving cables around, but it's a pain.  However, I could always change my mind.

Can you encode an h.264 dvd that's playable on a standard player?

Sincerely,

Ron


Matt Hessel <matt.hessel at gmail.com> wrote:

>When my 8 month old tried to buff the hardwood floor with some choice
>discs
>from our dvd collection, I decided to find a better way to handle
>video.
>
>I ripped each one and converted to h.264, then stored on my computer
>upstairs, then set up streaming to the ps3 downstairs for the kids to
>watch.
>
>I had to experiment a bit to get the results I wanted, but now I get a
>normal 2 hour dvd to a bit over 1 gig with 5.1 stereo output.
>
>VHS conversion will leave a good bit of noise and artifacting, so it
>doesn't compress down as much, but you should be able to put your
>library
>in a much smaller amount of space then the MPEG 2 it sounds like you
>are
>using.
>
>These days I normally see 2 TB drives for about $90-120 depending on
>how
>long you can wait for it to cough out the data.
>
>I would set up two of those in a mirror and play with conversion tweaks
>to
>see how small you can get them without degrading playback.
> On Oct 16, 2012 12:33 PM, "Ron Frazier (ALE)" <
>atllinuxenthinfo at techstarship.com> wrote:
>
>> HI all,
>>
>> After a few hours of Googling and reading forums and such, I've got
>some
>> new data that I wanted to share.  My little video conversion project
>> seems to be hitting a snag.  The 800 hours of videos I have are in
>the
>> category of important to me, but not critical.  If I lose them, I
>won't
>> die tomorrow.  So, the snag I'm hitting is the cost in terms of
>storage
>> cost and time to convert them.  I don't have a huge budget, certainly
>> not all at once.  I wanted to share my results, and see if you guys
>> think I'm missing something.
>>
>> In terms of time, I think it will take me 10 - 15 minutes of
>tinkering
>> with equipment, software, discs, cases, labels, etc. per one hour
>show
>> to convert.  More if I do edits and cut commercials and sync the
>sound,
>> etc.  Perhaps less if I store only to HDD and don't edit.  That adds
>up
>> to 200 hours of my time, a non trivial amount.  Obviously the video
>tape
>> must play, but I don't necessarily have to watch it.
>>
>> Here is what I believe to be the state of the art in storage options.
>>
>> I'm not considering other types of tape at this point.  I'm also not
>> considering SSD, based on data retention issues I mentioned in
>another
>> post.
>> Option 1 is using HDD's to store the videos.
>> Option 2 is some other type of disc, bluray, dvd, or mdisc.
>> Option 3 is online storage.
>> Based on my reading, I have concerns about the long term reliability
>of
>> bluray, so I'm probably dismissing it.
>>
>> Let's discuss discs first.  Assume a single layer single sided DVD
>can
>> hold ~1 hour of good quality video at a data rate of about 8 Mbps. 
>For
>> a HDD, assume 4.7 GB / hr of video.  Note, in some of the forums
>where
>> people burn LOTS of DVD's and spend hours checking their error rates
>> etc, it was mentioned that 16x write speed DVD's are not reliable. 
>So,
>> if available, I'm mentioning only 8x DVD's here.  There are also many
>> types of dyes, and very confusing data, which I'm not going to try to
>> sort out here as I don't understand it all.  There are may types of
>> printable surfaces on the top of the disc, for automated printing of
>> labels.  Lightscribe and inkjet surfaces are examples.  These usually
>> increase the cost.  I don't need them.  For my purposes, writing on
>the
>> disc with a DVD safe sharpie pen is enough.  Based on my reading,
>double
>> layer discs suffer more compatibility problems so I'm not considering
>> them.  Double sided discs are much more likely to get scratched and
>> cannot easily be labeled so I'm not considering them.  It is actually
>> very hard to figure out all the DVD media options, but I've selected
>a
>> few to look at.  Prices are Amazon, if they sell the product in
>question.
>>
>> Gotta account for the case for the DVD, about $ 0.25 each.
>>
>> The DVD-R format seems to be the most universally compatible.  I'm
>not
>> trying to list all discs here, just get representative prices.  Of
>> conventionally constructed discs, Verbatim seems to have a good
>> reputation, but there are varying reviews.  Longevity estimates for
>> conventional disc media range from 5 years anecdotal to 50 years from
>> the factories.  I will say that I have some 5 year old Maxell DVD-R's
>> that are still readable, but I will probably copy them to archival
>discs.
>>
>> VERBATIM 94852 DVD-r media 8x 4.7gb 50-pk spindle (shiny silver) - $
>19
>> / 50 = $ 0.38 / disc.  Cost with case (cwc) = $ 0.63 / disc
>> Verbatim Shiny Silver DataLife Plus 8x DVD-R Spindle, 50 Discs - also
>$
>> 19 / 50
>>
>> It seems that JVC Taiyo Yuden is the top of the heap in terms of
>quality
>> for conventional media.  Taiyo Yuden bought part of JVC as I
>understand
>> it and they're now shipping with JVC part numbers and packaging. 
>Many
>> people still know the Taiyo Yuden name and so that's usually listed
>in
>> the product listings.  Apparently, some TY discs themselves are
>> unlabeled, so if you want TY, make sure you get TY.
>>
>> 100 Taiyo Yuden/JVC 8X DVD-R 4.7GB Silver Thermal Lacquer - $ 29 /
>100 =
>> $ 0.29 / disc.  cwc = $ 0.54 / disc  Better product, cheaper price. 
>Cool.
>>
>> Still, I don't consider these discs long term storage.  The next step
>up
>> is archival grade discs, which are much more expensive.  Archival
>grade
>> discs usually have a solid gold layer to minimize corrosion and
>maximize
>> reflectivity.  Some have gold only.  Some have gold and silver, which
>is
>> cheaper - less gold, more silver.  They also usually have an extra
>hard
>> coating to prevent scratches.  Conventional DVD's are insanely easy
>to
>> scratch.
>>
>> Verbatim 95355 UltraLife 4.7 GB 8x Gold Archival Grade DVD-R, 50-Disc
>> Spindle - $ 93 / 50 = $ 1.86 / disc.  cwc = $ 2.11 / disc
>>
>> My research tells me that MAM-A / Mitsui is state of the art in
>archival
>> grade discs.  They have lots of data on their website.  Their gold /
>> silver DVD discs are projected to last 83 years.  Their gold DVD
>discs
>> are projected to last 100 years.  And their gold CD discs are
>projected
>> to last 300 years.  Also, note the same risk as above in buying
>> unlabeled discs.
>>
>> http://www.mam-a.com/
>>
>> MAM-A Mitsui Gold Archive 8x DVD-R with no logo in bulk - 50 Count =
>$
>> 117 / 50 = $ 2.34 / disc.  cwc = $ 2.59 / disc  (100 year discs)
>> MAM-A 8x SILVER PLUS GOLD DVD-R Branded Gold Surface 50pk Beehive
>with
>> Hard Coat - $ 93 / 50 = 1.86 / disc.  cwc = $ 2.11 / disc  (83 year
>> discs)  (factory price)
>> MAM-A 84019 Silver plus Gold DVD-R 16x 4.7GB MAM-A Logo 50pk Spindle
>-
>> 84019 - $ 80 / 50 = $ 1.60 / disc.  cwc = $ 1.85 / disc  (83 year
>> discs)  (dealer price)
>>
>> This last one is probably the best compromise between price and
>longevity.
>>
>> Finally, if you want longevity for 1000 years, there's m-disc, as
>> mentioned previously.  This is overkill for what I'm doing, but here
>are
>> the prices.
>>
>> M-DISC DVD+R 4X Cake Box 50 Pack - $ 136 / 50 = $ 2.72 / disc.  cwc =
>$
>> 2.97 / disc.  That's only $ 0.38 / disc more than the 100 year discs.
>> Not to bad if you need that kind of lifespan.
>>
>> So, the best I can do for an archival grade disc is $ 1.85 / disc.
>>
>> Now, what about the HDD option.  Here's a 3 TB drive I found on
>Amazon.
>>
>> Western Digital Caviar Green 3 TB SATA III 64 MB Cache Bare/OEM
>Desktop
>> Hard Drive - WD30EZRX - $ 140
>>
>> Let's say I want two for redundancy, since this is NOT archival
>media.
>> I would definitely spinrite this thing a couple of times / year to
>> refresh the magnetic fields.  That process would probably take 5
>days.
>> 2 HDD - $ 280
>>
>> How about a nice dual drive case with fan.
>>
>> Vantec Dual 3.5-Inch SATA to USB 3.0 and eSATA with JBOD/RAID 0/1
>> External Hard Drive Enclosure (NST-400MX-S3R) - $ 83
>>
>> So the total cost is - $ 363
>>
>> And we can store 3000 GB / 4.7 GB per hour = 638 hours of video
>> approximately equivalent to 638 DVD's.
>>
>> So, cost per equivalent DVD works out to - $ 363 / 638 = $ 0.57 /
>"disc"
>> or hour.  This is slightly more than the cost of cheap DVD's when
>cost
>> of cases is accounted for but substantially less than the cost of
>> archival DVD's.
>>
>> So, here's how I see my choices.
>>
>> - 83 year archival DVD's
>>
>>     * relatively low up front cost for 50 pack
>>     * substantially larger long term cost
>>     * some additional cost for storage rack, etc.  (and where do I
>put it)
>>     * discs are easy to access and play in dvd player
>>
>> - two big hard drives
>>
>>     * relatively high up front cost
>>     * substantially lower long term cost
>>     * minimal space required
>>     * movies are harder to access since I have to hook up computer to
>tv
>> and start player, etc.
>>
>> There is the possibility of online storage.  Storing 5 GB on Amazon
>S3
>> will cost me $ 0.75 / month for 1 hour of video.  It will cost me
>about
>> $ 0.75 each time I retrieve it.  Given the quantity of data, not very
>> practical.
>>
>> None of these choices are really great for me.  What I may do is to
>> select the best 50 - 100 hours of source video and put them onto 83
>year
>> archival discs.  Even that would cost me $ 185.  I'm not sure it's
>worth
>> it.  If I'm comparing the HDD to the 83 year archival discs, the hard
>> drive becomes more economical than the discs once I store 196 hours
>of
>> video.  At this point, it seems there are no good (within my budget)
>> options to store 800 hours of video.  One other option is to convert
>> everything to cheap DVD's, then refresh it and recopy it every 5 - 10
>> years.  That doesn't sound too attractive either.
>>
>> PS, from a hardware perspective, the Hauppauge USB-Live2 analog video
>> digitizer looks cool.  Haven't tried it and don't know about it's
>> drivers or software.  I can get mdisc / bluray drives in the $ 50 -
>100
>> range.
>>
>> Let me know what you think, and whether I've left out anything major.
>> This took two hours to type and I didn't proofread it, so sorry for
>any
>> errors.
>>
>> Sincerely,
>>
>> Ron
>>
>>
>> On 10/15/2012 4:45 PM, JD wrote:
>> >> I also wanted to share a great way to track and catalog DVD's that
>> >> you've purchased.  It's an android app called MyMovies (free -
>limited
>> >> to 50 titles) and MyMovies Pro ($6 - unlimited).  It's ultra cool.
> Just
>> >>
>> > Hopefully, someone else has found a fantastic Linux video editor
>and
>> will be
>> > able to let us all know about it.
>> >
>> >
>> >
>> > Previously I used a F/LOSS web server + back-end DB to catalog DVDs
>and
>> > recordings.  Over time, that became too much effort and I didn't
>like
>> that the
>> > program forced me to a specific DB and the export was only to HTML
>> files.  Tried
>> > a Java desktop client too, but it was toooo damn slow.  It didn't
>play
>> nice on
>> > my network.
>> >
>> > Ended up using the time-proven 'ls -Rl' technique with numbered
>DVDs.
>>  With a
>> > large number of data DVDs holding TV recordings in xvid/avi or
>> h.264/mkv, a
>> > quick 'egrep -i' searches by title for the specific numbered disk
>> containing the
>> > recorded show/movie.  As a CLI person, the most-used desktops all
>have a
>> script
>> > that will ssh over to the main box with all the dvd-0xyz.txt files
>and
>> perform
>> > the query.  For more information, XBMC has the catalog with actors,
>> directors,
>> > year, genre, title,  etc ....  grep is just so useful and text is
>so
>> easy to
>> > search.
>> >
>> > I love that the grep clearly returns the numbered data DVD I need
>to
>> load.
>> >
>> > As to converting VHS to MPEG or XVID or h.264, almost any $20 USB
>TV
>> adapter can
>> > record SD content to MPEG2.  Just pick from a MythTV compatible
>device
>> to be
>> > happiest long term.  If you want to record OTA HD content, I'd
>suggest
>> more
>> > effort for the selected hardware.  I love the HD-Homerun network
>tuners.
>> Those
>> > work well with OTA except 1 channel that doesn't come in at my
>house. It
>> is
>> > hardly a major issue, since 69 other channels do come in OTA from
>> Marietta.
>> >
>> > For finding commercials and editing them out, I searched for
>Linux-based
>> > solutions for years and finally gave up. Comskip does run nice
>under
>> WINE, I've
>> > never found a Linux-based editor that will accept the comskip
>created
>> cut files
>> > (there's like 6 different formats that comskip will create) as the
>> starting
>> > point for where to cut commercials.  Comskip is about 90% accurate.
>>  Comskip can
>> > output cut marker files that XBMC and mplayer will honor without
>> altering the
>> > source video file. That can be handy when you just want to skip
>> commercials, but
>> > not waste time actually modifying the file contents.
>> >
>> > Instead of fighting it, I bought - yes, purchased a Windows tool
>called
>> > VideoRedo TV Suite.  It appears to use comskip code internally to
>create
>> cut
>> > markers.  There's a point where $50 is worth it to me.  That $50
>program
>> has
>> > been heavily automated here to generate the cut marker files
>overnight.
>> It takes
>> > about 45 seconds of human effort to confirm the cut locations and
>> request the
>> > modified file be saved.  If you are willing to live with 90%
>accurate
>> cuts, it
>> > could all be automated.  Networks have changed up the markers
>commonly
>> searched
>> > by VRDP and comskip to trick them into keeping commercials and
>removing
>> some of
>> > the desired content.  Older stations with less money like RTV are
>almost
>> 100%
>> > accurate with cut locations.
>> >
>> > At this point, my systems are almost automatic with just a few
>manual
>> > "confirmation" places.  WTV -->  MPEG2 -->  h.264/mkv with CC1 and
>CC3
>> closed
>> > captions included.  If there are multiple audio channels in the
>stream,
>> that can
>> > require special processing, but that usually happens with a few PBS
>> programs
>> > with audio tracks for the deaf or background-only audio.
>> >
>> > I've blogged about almost all of these solutions over the years.
>The
>> most recent
>> > articles are about building your own DB4 antenna for $20 or so that
>> actually
>> > beat some Antennas Direct models.
>> >
>> >
>> > And just to be clear, I also wimped out and use Windows7 to record
>TV.
>> It is
>> > just so easy to setup with free schedule data. That OS runs inside
>a
>> virtual
>> > machine under KVM and has been doing it nicely for over a year. 
>THAT
>> was not an
>> > easy thing to accomplish to get satisfactory results. Microsoft did
>> (does?) some
>> > things trying to protect Joe-end-user from doing foolish things,
>like
>> blocking
>> > recording directly to NAS storage.
>> >
>> > I suspect there are hundreds of years of knowledge lurking related
>to
>> this problem.
>> > _______________________________________________
>> > Ale mailing list
>> > Ale at ale.org
>> > http://mail.ale.org/mailman/listinfo/ale
>> > See JOBS, ANNOUNCE and SCHOOLS lists at
>> > http://mail.ale.org/mailman/listinfo
>> >
>> >
>>
>> --
>>
>> (To whom it may concern.  My email address has changed.  Replying to
>former
>> messages prior to 03/31/12 with my personal address will go to the
>wrong
>> address.  Please send all personal correspondence to the new
>address.)
>>
>> (PS - If you email me and don't get a quick response, you might want
>to
>> call on the phone.  I get about 300 emails per day from alternate
>energy
>> mailing lists and such.  I don't always see new email messages very
>> quickly.)
>>
>> Ron Frazier
>> 770-205-9422 (O)   Leave a message.
>> linuxdude AT techstarship.com
>>
>> _______________________________________________
>> Ale mailing list
>> Ale at ale.org
>> http://mail.ale.org/mailman/listinfo/ale
>> See JOBS, ANNOUNCE and SCHOOLS lists at
>> http://mail.ale.org/mailman/listinfo
>>
>
>
>------------------------------------------------------------------------
>
>_______________________________________________
>Ale mailing list
>Ale at ale.org
>http://mail.ale.org/mailman/listinfo/ale
>See JOBS, ANNOUNCE and SCHOOLS lists at
>http://mail.ale.org/mailman/listinfo


--

Sent from my Android Acer A500 tablet with bluetooth keyboard and K-9 Mail.
Please excuse my potential brevity.

(To whom it may concern.  My email address has changed.  Replying to former
messages prior to 03/31/12 with my personal address will go to the wrong
address.  Please send all personal correspondence to the new address.)

(PS - If you email me and don't get a quick response, you might want to
call on the phone.  I get about 300 emails per day from alternate energy
mailing lists and such.  I don't always see new email messages very quickly.)

Ron Frazier
770-205-9422 (O)   Leave a message.
linuxdude AT techstarship.com