[IMC-Tech] Mirrorservers

[jb]

On Tue, Jun 14, 2005 at 11:44:34PM -0400, mtoups at indymedia.org wrote:
> On Tue, 14 Jun 2005 jb at riseup.net wrote:
> 
> >>however there is 82GB of stuff in the hashes/
> >>directory which i think was something jb set up
> >>to save space on duplicated files.  since they're
> >>hard links it is kind of time consuming to actually
> >>find what other files are linking to those.
> >
> >this directory can be deleted.  if i remember correctly, there
> >were about 30 gigs of duplicates.  would be cool to ensure rsync
> >handles hard links coorectly before rsync'ing, because otherwise
> >the size of it all may very well be more than 200 gigs.
> 
> the -H option to rsync preserves hard links, but they
> warn you that it is computationally expensive
>

heya,

so, i'm running this tool on paranode, might take a while given that
there are now about 600,000 files there..  i suppose we could make some 
batch for rsync - like use some criteria (size or date) and run a few
hundred rsync instead of a few big ones.  as hard linked files have
the same size and dates, rsync could be slow on a smaller set.

i'll let you know when the hard link thing is done, not quite sure when that
will be though :]

bye
jb