3

I am looking for a solution to monitor the health (DOWN or UP) of my web pages. Let's say I have three services,

site.com/data
site.com/user
site.com/id

Now, what I can do is that I can create a GET request which will be served every second and check if response status code is 200 OK. If it is anything else than 200 OK, then it will trigger an alert. But, I don't want to hit the server continuously, as it will create unnecessary logs. I am using a mongodb logger. What is the recommended way to achieve this?

Dinesh Ahuja
  • 131
  • 1
  • Monitoring also depends on your deployment model as well. Most deployments in cloud supports health checks and liveness checks. – Here_2_learn Jun 07 '18 at 05:56

5 Answers5

5

I see two way to solve this problem:

  1. Check your logs often and ensure there was an access with code 200 in the last N seconds/minutes and no code 5xx meaning there's a server side error. (the often should be coherent with how long you are ok with no entries)

  2. Keep an active check but:

    • use a HEAD request (so there's less data to return by the server)
    • use a custom user agent for the check
    • configure your log exporter to ignore this user agent.
    • take a larger check interval, 1 second is probably over obsessive

Depending on how often your sites are called you may wish to mix both methods to have a proper check even if there's no traffic from real clients.

Tensibai
  • 11,366
  • 2
  • 35
  • 62
1

One possibility would be to monitor the responses to requests from other site users. For example I'd check if there were 200 OK responses and skip hitting the server is such responses were sent in the most recent X minutes (up to you to determine which time interval is best fit for your application).

Maybe checking on the (server side) errors as well and hitting the server as well if a sudden change in the error occurrence rate is detected - just to be sure.

Dan Cornilescu
  • 6,730
  • 2
  • 19
  • 44
1

While I like the other great answers already here, I'd like to add:

use GET against a different port

You could have your app listen on a different port which isn't logged. haproxy and such are happy to check for health on port A for a service that is live on port B.

don't log canary checks

You could designate certain data items for monitoring and other tests to retrieve and not log requests against those designated items.

set a lower TTL for canary entries

It is a bit scary to cut off logging, so maybe you could delete the entries related to logging more aggressively since it is going into a database? This avoids checking a code path that turns out to not fail when the user experience is failing.

end-to-end checks are pretty important

Having something that is able to

  • see whether it can get to you
  • and whether your thing is doing its thing
  • and whether your thing can talk to everything it needs

often ends up being pretty vital to detect issues that you wouldn't catch by focusing on any one part of this. Paying the price for logging is usually negligible compared to the value and peace of mind you get from end-to-end checks.

chicks
  • 1,848
  • 1
  • 12
  • 29
0

You can use Uptime Robot, it offers a very good free layer (up to 50 sites free).

You can configure your sites (they are called monitors) in many ways. A simple GET (your case), ping and more. In case you want to have an idea, the screen to add a monitor look like this:

enter image description here

You can also configure your email to receive alerts, it has more options but I haven't explored which one of them are on the free tier, but in the other options you can find SMS, slack, hipchat and more.

It has a very good dashboard (I never use it though, I just get the email notifications) where you can see the status of the health checks and the type of error if it's the case.

enter image description here

enter image description here

The only downside on the free tier is the five minutes interval, I would prefer to use a two or three minutes interval but most of the sites don't need one second accuracy on the health check anyway.

Gepser Hoil
  • 1,312
  • 1
  • 15
  • 19
  • 1
    I fail to understand how this would avoid hitting the service with get request and pilling up 'monitoring' logs... – Tensibai Jun 07 '18 at 14:12
  • @Tensibai fair enough, my thinking behind this approach is he doesn't want to hit his service every second, generating a lot of noise. With this approach you are still getting some logs but every 5 minutes, that is 300 times less logs. I could be wrong and this solution may not fit his needs, but anyway I just wanted to give him another option so he can choose better. – Gepser Hoil Jun 07 '18 at 15:16
0

One method is that you do TCP-only monitoring. In this scenario, you establish a three way handshake (or use tcp-half-open monitoring in which no connection is established - you merely look for an [syn, ack] to your [syn], and don't [ack] the [syn, ack] back) and then close the connection assuming it is the web server listening on that port.

A better option however, is to figure out how to exclude your health monitor traffic from logging, though I am unable to give specifics on doing that with rails/mongodb logger.

James Shewey
  • 3,734
  • 1
  • 15
  • 38