What are the pros & cons of web server log analysis over web based analytics like google analytics

Question

What are the advantages of web server log file analysis over web services like google analytics?

What are the advantages of google analytics like tools over web server log file analysis?

Answer 1

Pure log file analytics:

• doesn't need JavaScript
• limited to HTTP/S requests
• needs to be on-site
• data is only as recent as last log file update
• takes more work to differentiate bots from humans

Pure javascript analytics:

• can capture mouse movements/clickstream, length of time user spends on page and other user behavior
• more easily distinguish visitors behind proxies or NAT routers
• can provide real-time analytics
• can be more easily integrated with email and social media analytics
• requires JavaScript to work
• can slow down browser slightly
• every page/link needs to be tagged

There are also analytics platforms that combine both log file analysis and javascript tags. As well, Pion provides an analytics platform that combines log file analysis and packet sniffing.

Answer 2

A few thoughts:

Web based tools like Google Analytics require JavaScript to function completely. Some of them even requires JavaScript to be turned onto function at all. So if a user doesn't have JavaScript enabled your site statistics won't be accurate or complete. They also may slow your site down as the user has to wait for that code to fully download and be parsed fort it to work. If the stats provider's server is slow at that moment your page will seem to be slow to the user.

On the plus side, they are easy to set up as they only require a small snippet of code to be placed in your pages and you're up and running. They also tend to stay on top of updated bot and browser lists which they can apply immediately with no work from you required so their reports are always current.

Web based tools like Awstats don't require any client side and thus will offer a complete set of statistics from every user. If you use a web host that provides a basic control panel one of these is usually included so no set up is required.

On the downside if you don't have a server with a pre-installed control panel or want to use a different one then is included you'll have to install one yourself. Installing software on web servers, particularly *nix systems, is not for the lighthearted. As browsers and bots continually change you'll need to make sure you keep this software up to date otherwise you'll have a lot of "unknown" bots and browsers in your stats which isn't particularly helpful.

Answer 3

IMHO comparing Web server logs file analysis to Google Analytics it's like comparing apples to pears. 1 day of server log file can be 1000 text lines to read through or more (depending on the number of visits on your site), this is totally unreadble unless you want to spend the rest of your life analyzing it line by line.

Google Analaytics can be somehow compared to another graphic tools to analyze your webistes' visists like the excellent Awstats which is included in the vast majority of all hosting plans (comes with cPanel). Awstats is based on server log analysis (as you mention in your question), but it shows you result in a graphical form very easy ot read.

Google Analytics vs Awstast (still IMO)

Google Analytics pros

• more detailed,
• you can setup goals and a bunch of other stuff,
• you can easily track Adwords returns,
• you can setup a recurring email with a snapshot of the Analytics to be sent to your customers

Google Analytics cons

• too detailed (for small/mid sites that do not care much about the number of visits per day, but they can easily quantify visists better and easier by month view or even year view)
• very slow to setup an account (slow panel irritating me and wasting my time)
• requires you to place a snippet of Javascript code in ALL your webisites pages (at least in the ones you want to see in Analytics results), this is another waste of time for me and it slows down a bit your page renedering (moreover you migt get somehow irritating JS errors).
• if you are sending them to your customer for free, what happens if one day Big G tells you that from now on you have to pay for them.

Awstast pros

• Gives you out already more than enough results to analyze (and easier to read for your customers)
• Very easy to read by monthly and yearly visits.
• You don't need to setup anything, it comes with vast majority of webhosting services (for sure with all the ones with cPanel).
• It shows you also 404 errors due to broken links to your site (also broken links to your site coming from your site), so you can easily track them and fix your site links immediately.

Awstats cons

• If you want ot exclude a page from analysys you need to play with Awstatst configuration on server (easy once you have done it once, but still harder than just removing a Js snippet for the page/s you want ot exclude)
• If you want to see all the bunch of stuff shown by Google Analytics: goals, ROI, pingback, maps, and even where was my mother when I was visting your site, you simply can't!

Answer 4

It really comes down to flexibility & the amount of work you want to do. If you run a small to medium website and want some generic analytics about your traffic, by all means, throw your data into Google Analytics and never look back. It's an excellent system that reliably gives you timely insight into user behavior on your website.

If your site is complicated & has many unconventional user flows (lots of AJAX?) that you'd like to follow, you might not be able to shoehorn things into GA. Also, if you want to correlate multiple sources of data with your web traffic (eg user information from your database), you're going to need to start thinking about a custom solution as well.

Beyond the capabilities of the tools, you need to look at the tradeoffs between client and server-side logging. Server-side logging always works, regardless of what the client does but, unfortunately, it is only able to gain a limited amount of information from the HTTP request. Client-side logging can be more flexible & gather more information but you have to worry about setting up an extra layer of services to collect them & there's the chance that you might miss some small portion of your traffic.