June 2012

Monthly Archives

TCPWebLog – Collecting and Aggregating Web Logs on the Cloud

TCPWebLog is a Free Open Source Software system to collect and aggregate Web-type logs (i.e. Apache, Varnish, PHP, FTP …) from multiple GNU/Linux computers running on a Cloud.

When a user connects to a Website hosted at CatN, the response may come from different Web servers running on different virtual and physical machines. This first introduced the problem of needing to aggregate the logs from multiple cluster nodes to a central log server, and then split them up again based on the virtual host. Here at CatN, for each virtual host we used to write on a central place distinct log files for Apache Access log, Apache Error Log, Varnish, etc. These files are then rotated and processed using other software tools which required a significant administrative overhead.

Read More

How to create TCPWebLog RPM packages for Enterprise Linux

This is a short hands-on tutorial on creating TCPWebLog RPM packages for Enterprise Linux, including, but not limited to, Red Hat Enterprise Linux (RHEL), CentOS and Scientific Linux (SL).

TCPWebLog is a Free Open Source Software system designed to collect and aggregate Web logs (i.e. Apache and Varnish) from multiple GNU/Linux computers running on a Cloud. For more information please read the blog post: TCPWebLog – Collecting and Aggregating Web Logs on the Cloud.

Read More

Making sales with G-Cloud

I’m happy to announce that we have made our first sales to UK central government through the G-Cloud programme. This success has come from over a year of work attempting to break into the central government hosting market.

You can find our seller profile on the Cloud Store or view our G-Cloud Services page on the CatN website.

Regular readers of the blog will know about our experiences with various projects such as the Innovation Launchpad. Take a look at some of the links in the search results below if you wish to familiarise yourself.

Read More

Diary of a Linux n00b

If you’d asked me about Linux 3 months ago I’d have muttered about Ubuntu before quickly changing the subject to whether anyone wanted a cuppa and made a run for the nearest door; I can now proudly say I’ve configured my first LAMP stack and even edited a database with positive results!

One of the things I wanted to get my head around from week 1 was Linux, I had seen it used but that was about as far as it went. The email saying ‘why don’t you have a go at setting up a LAMP stack?’ had me quaking in my boots, but the one thing I did know about Linux was that the community support was great – so off I went to ask ‘where on earth do I begin?’
Read More

DevOps Ninja Assassinates Web Logs!

Hello, I’m Ben and I recently joined the CatN team as a Systems Engineer / DevOps Ninja.

My first proper task was to get statistics working for two sites hosted on an early alpha of vCluster version 2; housepricecrash.co.uk and insanelymac.com. This proved to be quite tricky and required a good understanding of the way things worked.

First off I looked at the PHP script on our syslog1 server that runs daily on the cron and fetches stats from AWStats for the previous day.  This uses a MySQL database to find which sites it will fetch stats for, and screen scrapes an AWStats page for each one to do so. Read More