Zen-Dreams

Servers are the keystones of any organisation working with computers, and to ensure productivity, you must make sure there will be the fewest downtimes possible. To have a correct Level of service, one of the first things to do is to elaborate a monitoring plan.

Why server monitoring has to be performed?

This question might sound ridiculous, but it’s important to know the reasons why you should monitor your servers.

First, if you have plans for the future, then you must monitor your servers, monitoring will allow you to know how the present is and where to go.

• Disk Usage: How is you space used and how is it growing over time?
• Memory usage: How is your system memory used? Do you plan to have more users in the future?
• CPU usage: Will you need to upgrade the number of processors installed on the server if your number of users increases?

Monitoring will also allow you to prevent incidents, or at least identify them before your customers do. If you know how is going your server, you might notice a possible failure and take actions before it (for ex: /var nearly full, one of your process is going crazy, etc…).

How to build a monitoring plan?

First, you have to get your architecture design and identify any SPOF⁽¹⁾ once you have identified them, make sure they will be monitored first and with a good frequency.
Having identified your SPOF, you can now identify what also needs to be monitored and define a frequency for each item.

• CPU
• Memory
• Disk Usage
• System logs
• Specific processes (databases, webservers, application servers…)
• IPCS⁽²⁾
• System Backup logs
• Application logs
• Network usage
• Network connections

When you have defined what you want to monitor, define actions that are to be taken when reaching certain levels (which we will call alarms), for example send an email, raise an application error, etc… One thing that could be done is to define multiple levels such as warning, critical error, severe error with different actions for each state.

Choose your system.

Having defined you plan, you now have to choose a monitoring system, there are many available on the market and each of them as its purpose, pros and cons.
Here is a short list of systems you can have a look at:

• Tivoli
• Sysload
• BigBrother
• Nagios (OpenSource)
• Catci (OpenSource)
• rcp.rstatd
• SNMP
• MS Perfmon

Take actions!

When an alarm is set, take actions as soon as possible, any kind of actions that could set the level to normal state. You must not stay in the raised level, as your service level will decrease exponentially.

After having your state back to normal, analyse what happened and take actions to prevent this in the future. Update your monitoring if necessary, and also make sure to archive all events so that you will be able to prevent recurring problems more easily This is important to analyse problems after they happen and not before taking any corrective action because you cannot know how long the analysis will take. Also make sure to save everything that will help you analyse the problem, to do so you can define actions in your alarm to take a “snapshot” of the system automatically for analysis purpose.

Maintain the system.

Make sure your monitoring is always up-to-date; update it every time you change something on your server. Regularly check that you are not missing events happening on your system.

(1): A Single Point Of Failure is a part of your architecture that will cause the entire system to stop working if it breaks.
(2): Inter process Communication system, might contain Shared memory, semaphores and message queues