[ale] Problem restoring past large file in cpio

[Greg Freemyer]

On 10/18/06, Chuck Huber <chuck at cehuber.org> wrote:
>
> > Message: 7 Date: Wed, 18 Oct 2006 11:31:24 -0400 From: "Danny Cox" <
> DCox at icc.net>
> > Subject: Re: [ale] Problem restoring past large file in cpio
> > Wed, 2006-10-18 at 10:59 -0400, Chuck Huber wrote:
> >>> The problem I'm encountering is that cpio won't restore past a
> >>> large file (a DVD image of about 3.5G).  The listing it created
> >>> during the backup has many files past it that are needed.
> >
> > You might try pax.  It can read cpio archives, and get you
> > through/past the large file.  You might try telling either one to
> > restore everything EXCEPT that large file, and then work on just
> > getting that file back.
>
> Hmmm... I tried using pax and got the following error when trying to
> list the archive:
>     blackbird:/ # pax -f blackbird.20061007.full.cpio
>     pax: Failed stat on blackbird.20061007.full.cpio: Value too large
> for defined data type
>
>     ATTENTION! pax archive volume change required.
>     Ready for archive volume: 1
>     Input archive name or "." to quit pax.
>     Archive name >
>
> This is interesting in that I was unaware of a file size limit on cpio.
> The size of the .cpio is just in excess of 10G.  When I try to get a
> table of contents (cpio -it)  The last file listed is the file
> immediately prior to the large one.  This would indicate that cpio can't
> seek past the 2G point.
>
> When I try to restore a specific file in front of the large one, all is
> well.  When I try to restore a file from beyond the large one, it fails
> with something on like "skipping 1xxxxx bytes of garbage.
>
> With all that already written, I tried:
>
>     cat xxx.cpio | pax
>
> and was able to get a TOC down through and including the large .iso
> file, but nothing past it.
>
> All this would seem to indicate that the cpio format itself has a 2G
> file size limit (32-bit signed integer for file sizes?).  In this case,
> the large .iso starts before the 2G mark, but ends well beyond it.  I
> haven't confirmed this - just a hypothesis.  That being the case would
> render all my backups useless beyond 2G, leaving about 8G inaccessible.
>
> Ideas on how to get past this would be greatly appreciated.  (Our
> primary server is down until I get this fixed.)
>
> Thanks,
>     - Chuck
>
> --
> Chuck Huber
> Duke Pro, Inc.
> Office: 828-684-6022
> Cell: 828-778-3776
> Home: 828-683-7324
> _______________________________________________
> Ale mailing list
> Ale at ale.org
> http://www.ale.org/mailman/listinfo/ale
>

Chuck,

I just did a little googling and it does seem that the CPIO backup format
does not support files over 2GB.  The trouble is likely that there is file
size info in the cpio header prior to each file.  Since your file was over 2
GB, the filesize causes problems.  If you could edit your archive file to
simply delete the iso file and its associated header, then the overall size
of your archive should not be a problem.

Possibly easier than trying to edit a 10 GB file, you could try to:
 split your 10GB file into 1 GB pieces (or whatever is small enough you can
edit it)  (use "split")
 delete the initial segments prior to the end of the big file
 edit the segment that contains the last section of the big file and delete
it
 manually recreate a cpio header at the start of the edited segment
 cat all the remaining/edited segments into cpio and continue the restore

The difficult part is left as an exercise for the reader.
 ie.
Determining where to stop deleting at the end of the big file.  There is a
cpio file header at the start of the next file that you have to leave in
place.

Determining how long a cpio header is (max of 12 bytes based on a quick test
I just did).  (You can get the byte values by doing a od -c | head of any of
your cpio archives.)

I would create some real simple text files and build some cpio archives from
them.  By looking closely at those you should be able to figure out how big
the cpio file header is from that how big the cpio archive header is.

FYI: For some reason I don't think cpio uses a file trailer.  It does have a
archive trailer, but that should not have any impact on you.

HTH
Greg
-- 
Greg Freemyer
The Norcross Group
Forensics for the 21st Century
-------------- next part --------------
An HTML attachment was scrubbed...