[axxs-sysadmin] [SOLVED] Re: severe database corruption problems on indymedia.org.nz

[Sebastian]

kia ore koutou...

it looks like our issue is resolved: i created a new database, loaded
all the data from the old database into it and switched dada to use the
new one.

and the load on the server seems to be low since the last reboot.
choice, axxs!

cheers,
 Sebastian


* Sebastian <aotearoa at kodeaffe.de> [2007-02-07 18:25 +1300]:
> 
> thanks aketus for your efforts, hopefully the cron job works reliably.
> 
> i just had the idea that maybe the reason for the corruptions is some
> kind of inconsistency between the database tables in memory and on the
> hard drive?
> wasn't there a server crash a few days ago? maybe the database was
> restarted with corrupted tables and thus the impossibility for mysql to
> write the fixed tables to disk?
> 
> after repairing, i tried to flush the tables myself, but our user lacks
> the RELOAD privilege:
> 
> mysql> flush tables;
> ERROR 1227 (HY000): Access denied; you need the RELOAD privilege for
> this operation
> 
> 
> so perhaps it is a good idea to if somebody @ axxs could:
> 
> - repair all tables
> - shutdown the database
> - start the database again
> 
> in quick sucession? or try the flush as (database) root?
> 
> 
> * aketus <aketus at anarchobase.com> [2007-02-07 17:58 +1300]:
> > er... I meant 'script to repair all tables' not 'all databases'.. sorry
> > 
> > Quoting aketus <aketus at anarchobase.com>:
> > 
> > > I've been helping run the REPAIR TABLE and yes I agree, it's weird
> > > that it keeps getting corrupted. Unfortunately I don't know why, I
> > > wondered if it was the way dada handles php to mysql activity, but
> > > then why would it start playing up now if it didn't in the past.
> > >
> > > My other theory: I actually just thought of it then: We recently did a
> > > MySQL upgrade on axxs. I'm wondering if dada is having trouble
> > > handling MySQL 4.1. The more I'm thinking about it, the more I think
> > > this may have something to do with it. I don't recall any of these
> > > errors occuring prior to MySQL being upgraded.
> > >
> > > Good idea with the cron. I was going to write a script that repaired
> > > all databases when manually run outside of phpmyadmin or terminal, but
> > > a cron is probably more reliable for the time being.
> > >
> > > Quoting Sebastian <aotearoa at kodeaffe.de>:
> > >
> > >> for 1 week or 2, we have a recurrent problem of database corruption.
> > >> we did not change any code that touches database stuff (actually none at
> > >> all until today). we see error messages like:
> > >>
> > >> file not found: articles.MYI
> > >>
> > >>
> > >> or when running mysqlcheck:
> > >>
> > >> nzdada.articles
> > >> warning  : Table is marked as crashed
> > >> error    : Key in wrong position at page 339968
> > >> error    : Corrupt
> > >>
> > >>
> > >> or just now:
> > >>
> > >> Unknown ID Error
> > >>
> > >> Database failed to return a unique record for this query, or you are not
> > >> authorized to access that record.
> > >>
> > >>
> > >> a simple SQL command 'repair table <table name>' or 'mysqlcheck
> > >> --auto-repair' seems to fix it, but only temporarily. i installed a cron
> > >> script today which runs mysqlcheck every 5 minutes.
> > >>
> > >> i am crawling the internet at the moment for possible answers, but
> > >> perhaps any one of you has an idea?

-- 
 _   ascii ribbon campaign     .oOo.   GCSd-s:+aC++ULB+++W++M+PS+++Y+
( )
 X   Agenda 23
/ \     - Soziale Zwietracht, Schrumpfen und Tradition
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://lists.indymedia.org/pipermail/axxs-sysadmin/attachments/20070212/a77c3bea/attachment.pgp