jb at riseup.net
Tue Jun 21 06:08:51 PDT 2005
On Tue, Jun 14, 2005 at 11:44:34PM -0400, mtoups at indymedia.org wrote:
> On Tue, 14 Jun 2005 jb at riseup.net wrote:
> >>however there is 82GB of stuff in the hashes/
> >>directory which i think was something jb set up
> >>to save space on duplicated files. since they're
> >>hard links it is kind of time consuming to actually
> >>find what other files are linking to those.
> >this directory can be deleted. if i remember correctly, there
> >were about 30 gigs of duplicates. would be cool to ensure rsync
> >handles hard links coorectly before rsync'ing, because otherwise
> >the size of it all may very well be more than 200 gigs.
> the -H option to rsync preserves hard links, but they
> warn you that it is computationally expensive
so, i'm running this tool on paranode, might take a while given that
there are now about 600,000 files there.. i suppose we could make some
batch for rsync - like use some criteria (size or date) and run a few
hundred rsync instead of a few big ones. as hard linked files have
the same size and dates, rsync could be slow on a smaller set.
i'll let you know when the hard link thing is done, not quite sure when that
will be though :]
More information about the imc-tech