[axxs-sysadmin] severe database corruption problems on indymedia.org.nz

[Sebastian]

rehi..

thanks aketus for your efforts, hopefully the cron job works reliably.

i just had the idea that maybe the reason for the corruptions is some
kind of inconsistency between the database tables in memory and on the
hard drive?
wasn't there a server crash a few days ago? maybe the database was
restarted with corrupted tables and thus the impossibility for mysql to
write the fixed tables to disk?

after repairing, i tried to flush the tables myself, but our user lacks
the RELOAD privilege:

mysql> flush tables;
ERROR 1227 (HY000): Access denied; you need the RELOAD privilege for
this operation


so perhaps it is a good idea to if somebody @ axxs could:

- repair all tables
- shutdown the database
- start the database again

in quick sucession? or try the flush as (database) root?


thanks,
 Sebastian


* aketus <aketus at anarchobase.com> [2007-02-07 17:58 +1300]:
> er... I meant 'script to repair all tables' not 'all databases'.. sorry
> 
> Quoting aketus <aketus at anarchobase.com>:
> 
> > I've been helping run the REPAIR TABLE and yes I agree, it's weird
> > that it keeps getting corrupted. Unfortunately I don't know why, I
> > wondered if it was the way dada handles php to mysql activity, but
> > then why would it start playing up now if it didn't in the past.
> >
> > My other theory: I actually just thought of it then: We recently did a
> > MySQL upgrade on axxs. I'm wondering if dada is having trouble
> > handling MySQL 4.1. The more I'm thinking about it, the more I think
> > this may have something to do with it. I don't recall any of these
> > errors occuring prior to MySQL being upgraded.
> >
> > Good idea with the cron. I was going to write a script that repaired
> > all databases when manually run outside of phpmyadmin or terminal, but
> > a cron is probably more reliable for the time being.
> >
> > Quoting Sebastian <aotearoa at kodeaffe.de>:
> >
> >> for 1 week or 2, we have a recurrent problem of database corruption.
> >> we did not change any code that touches database stuff (actually none at
> >> all until today). we see error messages like:
> >>
> >> file not found: articles.MYI
> >>
> >>
> >> or when running mysqlcheck:
> >>
> >> nzdada.articles
> >> warning  : Table is marked as crashed
> >> error    : Key in wrong position at page 339968
> >> error    : Corrupt
> >>
> >>
> >> or just now:
> >>
> >> Unknown ID Error
> >>
> >> Database failed to return a unique record for this query, or you are not
> >> authorized to access that record.
> >>
> >>
> >> a simple SQL command 'repair table <table name>' or 'mysqlcheck
> >> --auto-repair' seems to fix it, but only temporarily. i installed a cron
> >> script today which runs mysqlcheck every 5 minutes.
> >>
> >> i am crawling the internet at the moment for possible answers, but
> >> perhaps any one of you has an idea?

-- 
 _   ascii ribbon campaign     .oOo.   GCSd-s:+aC++ULB+++W++M+PS+++Y+
( )
 X   Fuenf Tonnen Flachs!
/ \  
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://lists.indymedia.org/pipermail/axxs-sysadmin/attachments/20070207/091ec08f/attachment.pgp