Archive

Posts Tagged ‘logging’

Mythic Monday – The Aging Lion and the Fox

December 14th, 2009 Josh No comments

Another one of Aesop's fables that isn't that well known is that of the aging lion and the fox. You can click the link and read it, but for those of you that are linkaphobic, here's a short version:

A lion was getting old and having trouble hunting. He decided, instead, to pretend to be sick and went back to his cave, moaning all the way. Over time, as each of his neighbors stopped by to check on him, he ate them.

Then, one day a fox came by and asked how the lion was doing. The lion moaned and asked the fox to come closer. The fox then observed that the footprints all led into the cave, and none came out.

Clearly, the fox is the fable animal to be. He's smart. He's observant. He's... umm... red and furry? (Are Greek foxes red? . . .  Yes, after googling a bit, it seems that the red fox is global, and the grey fox is only native to the Americas... which has nothing whatsoever to do with this blog entry.)

No, the point of this blog entry is that of evidence. If the lion had been wise, he would have either wiped the tracks after each meal or (more preposterously) fabricated tracks going back out. The fact that he didn't, is what allowed the fox to escape and presumably tell the other animals what the lion had been up to (and Aesop, since he wrote it down). So, not only was the lion caught, but he lost his lovely little racket and probably starved to death shortly thereafter.

Most attackers are aware of this story (sorta), and do take some effort to reduce evidence. A burglar usually wears gloves, a bank robber usually wears a mask, and a hacker usually clears system logs. So, if we want to make it hard for the lion to wipe away the footprints, we have a few options. The first is to replace the dirt outside his den with fast-setting concrete... which would prove somewhat troublesome if you analyze this ridiculous analogy too far. The second is to set up a camera trap and record everyone who enters the cave. (For those purists who would point out that there were no cameras in ancient Greece, let's just say that Hephaestus is there cranking out a vase for each animal. (Happy now, picky people?))

In the modern world, we actually use both of these techniques. Instead of fast-setting concrete, we have a hard drive technology called WORM, or Write Once Read Many. With this drive, you can store the logs in such a way that they cannot be altered. They are, however, quite expensive and can be difficult to set up properly. Instead, we generally prefer to use the camera/vase trap system. For this, we use one of many remote-logging technologies. The simplest is probably the venerable syslog server.

This solution simply involves setting up a dedicated server and installing one of the many syslog systems on it. Then you do a bit of configuration on each of the other servers you have and basically tell them to go log over there. Whenever there is an event, it goes over the network and is stored off the server. That way, if an attacker gets in, even if they wipe their own traces, there is a backup elsewhere that is (in theory) a lot harder to alter.

Of course, you still have to actually be the fox and look at the logs now and then, but at least you'll be safe from a smart lion.

Tags: , , ,

Related posts

Categories: Mythology Tags: , , ,

Small Business Defense – Remote Logging and Analysis

February 26th, 2009 Josh No comments

The first thing to realize when it comes to protecting your logs from attackers is that if the logs aren't there, they can't be attacked.  At a minimum, you should consider setting up a remote logging server.  This does not have to be a brand new top-of-the-line server.  It can be an older server, a workstation or a virtual machine.  The big thing to keep in mind is that it will need a lot of disk space.  Depending on your network, it may also need a very fast network connection.

A nice free option to use is syslog.  It's not as user friendly as some of the commercial systems, but you can't beat the price.  For this tool, you just install one of the syslog-compatible systems on your remote server and configure each of your other systems to log to it.  There are Windows tools and guides so you can capture those logs as well.

Of course, there are some commercial options as well.  These often include enhanced tuning and searching.  Splunk , Snare and LogLogic are known in the industry.

The second thing to consider when looking at logs is that you actually have to look at them.  Remote logging may get the logs away from the attacker, but if it also gets them away from you, they're not terribly effective.  Most of the log management tools fall into three categories:

  • those that find problems and alert you
  • those that let you search the log
  • those that help you visualize the data.

Before looking at any of the many tools out there, ranging from application-specific to purpose-specific to problem analysis, you should first consider what you care the most about.  Logging involves a lot of data, and if you start with alerting before you tune anything, you'll be drowning in it.  Similarly, it doesn't make much sense to put considerable analysis time into an application that isn't business critical.

Instead, it's best to start by getting all of your logs in one place, and focusing on doing that well.  That's a large project in of itself. Once that's done, start looking at the sizes of the log files that you're creating and work on reducing them. Odds are that at least one of your logs was set to maximum verbosity for testing something and never set back. Once you know that all of your logs have the data they need in them and as little garbage as possible, start with the biggest and look for a free tool that helps you pull out the important information. Then, move on to the next. Yes, it will take a lot of time and many tools. It may not look pretty, but it will work.

And, after all, working is what matters the most.

Then, later, once you have a greater level of inspection than you've ever had, you'll know enough to seriously consider the big log management players. There's no point in spending lots of money until you know what you're spending it on.

Tags: ,

Related posts

Small Business Attack – Changing Logs

February 25th, 2009 Josh No comments

In I.T., we love logs.  They're organic, they float, they burn and you can build houses out of them!  Of course, we also like the other kind of logs as well.

The kind of logs I want to talk about are the ones that keep track of what's going on with your systems.  They are intended to make it easier to reconstruct strange behavior and trace issues between systems. System administrators will check the logs to see if there are problems involving CPU, memory or disk usage. Network administrators can use them to trace network congestion and connectivity issues. Developers can use them to find out why certain programs aren't functioning properly. Also, security professionals can use them to help identify attackers and how far they penetrated a system or network.

At least, in theory we can. There's one problem: attackers can write logs too.

A common technique that attackers use is to erase or modify the logs after they successfully compromise a system. They can cover up vulnerabilities, erase their tracks and make things appear to be running OK even when they're not. They can also read the logs and use the information in them to identify other targets.

If you have a system that is backed up on a regular basis, an attacker can find those logs and use them to identify the backup server. Once they know that, they can focus their efforts on getting the data that's over there. They can use logs to identify which users might have elevated permissions on other systems. They can also use them to determine what "normal" activity looks like, so they can hide their activities in places you can't find them.

Like many things, it's a double-edged sword.

You need the logs, because they're useful to you, but they're also useful to the attackers, so what can you do?

Tags: , ,

Related posts

Categories: Business Security Tags: , ,