[Deskaheh-sysadmin] deskaheh status

[finn c.]

Hi all,

Apologies for the lack of communication on Deskaheh. Been totally snowed
under with a number of crises. I've been doing a bit of work on Deskaheh
which seems to have kept it stable for the last few days, and Ski has
been updating some of the software [at least, ski, can you post some
info on what you've been doing?].

Load is down to a respectable 0.2 -> 0.4 most of the time. I'm pretty
sure the spikes http://deskaheh.nysindy.org/mrtg are the result of
software compilation.

_Load monsters_

Hamilton Indymedia was misconfigured:
* File permissions were wrong, and didn't allow www to write to a
  number of the sites directories [needed for rss and mysql backups]
* The upgrade from 0.98.1 --> 0.98.2 didn't alter the prefs mysql table,
  so admins couldn't alter the prefs
* The path for logs was incorrect.

I've fixed these, and noticed another problem - curl refuses to execute
http://hamilton.indymeida.org/cronmaster.php [used as a master cron 
executor]. Wget also fails. I've disabled hamilton's cronjob
until we can sort out what is wrong.

Since I disabled this cronjob, I noticed that mysql is actually stable
for once [we used to suffer from constant runaway mysql processes that
would suck all available cpu power]. And I also noticed that
*occasionally* curl will fetch cronmaster.php, but throw a whole bunch
of errors <!-- quoted sql SELECTS --> , not particularily helpful.

I'll have to individually go through the cronjobs and see what is causing 
the problem, and then reenable hamilton's cronjob. Good to have finally
located the cause of the mysql stress though.

Bots:
We were getting slammed by bots. Notibly IBM's almaden, and also Slurp
[yahoo photobot] and Googlebot. I was kind of curious to see what load
they were causing, so I dropped a robots.txt in nz, hamilton and
bigmuddy imc sites that denied all bots. This halved our load.

I'm going to allow Googlebot back in, but deny it access to all but
features and the newswire on dada sites [i'm not going to touch the mir
sites, as they aren't a problem]. What do people think about letting
Slurp in?

_Other issues_

PHP:
PHP was compiled manually by Ski, and is now in need of an upgrade for
security. I'm not really familiar with FreeBSD, and I'm not sure of the
best way to upgrade it safely. Any help would be appreciated, I don't
really want to break this. :) 

Apache/Mysql tuning:
Is there any reason why mod_perl is active? It saves us 1meg on each
process if it is turned off. Tell me now before I disable it.

Can anyone point me towards the mysql configuration? I've had a good
poke around the system, and I can't find it anywhere. Does it exist?

There's probably a whole lot more tuning that could be done, but I'm not
going to mess with anything that ain't broke. Someone with more
experience may want to take a poke through the configs though.

Dada Mirroring:
We need to get Dada mirroring up asap. Hamilton's is mirrored, but is
'pending verification'. Anyone know how to actually verify? Manually
retriggering the cronjob doesn't seem to help. I'll ask on the dada dev
list for help.

I tried to get Aotearoa mirrored a while back, but it refused to play.
If anyone can give me a hand that would be appreciated.

I'm not sure about Bigmuddy, given that they are not an official IMC.
Has anyone asked for mirroring space for them?

Dada caching:
We should also have a look at installing the caching code written by
John Hatton: http://dega.indypgh.org/~jonhattan/dada_caching.html
Apparently the speedup is quite large.

seeya,
finn

--
pgp encrypted mail welcome - 
keyid: CD564868      keyserver: pgp.mit.edu
E9E6 E3A3 4891 48D8 A95E  3F78 90D7 CA42 CD56 4868
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://lists.indymedia.org/pipermail/deskaheh-sysadmin/attachments/20050222/723b77cb/attachment.pgp