Mar 192010
 

During the rebuild of cncpics.com I took a decent look at the log file.

It looks like there are a large number of robots that are attempting to harvest the images, causing huge amounts of database lookup and then eventually an invalid URL type error.

Because of the way that the gallery software works, anything that looks like a gallery URL is handled by the database.

By ‘fishing’ for images the robots are causing a lot of database reads and ┬álog file writes. Which adds to the load on the server. Which in turn reduces the performance for the legitimate users.

To alleviate the load, we have changed the URL pattern for albums and images. All of the old links will land on the ‘404’ page.

http://cncpics.com/404

While I realize that most of the 404’s are not humans, it amuses me to think that the robots are getting familiar with Homer J.

 Posted by at 10:33 am

 Leave a Reply

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

(required)

(required)