0

Developing a process that should read data at consistent intervals. The time period to read data varies depending on the network. I thought this should be straightforward but I can never get consistent timing. Looking for a more consistent and stable system that responds well to network speed variability.

currently I am using a model that follows

|<--read data-->|<--post process-->|<--sleep x seconds to maintain period-->|
|<------------------------------known data rate---------------------------->|

My code does something like

data_rate = 5   # Hz
while 1:
    # read in data
    rd_start = time.time()
    data = getdata()
    rd_stop = time.time()

    # Post processing
    pp_start = time.time()
    rate = 1.0/(rd_start - oldstart) if oldstart else data_rate
    old_start = rd_start
    print rate
    post_process(data)
    pp_stop = time.time()

    sleep_time = 1.0/data_rate - ((rd_stop-rd_start) + (pp_stop-pp_start))
    sleep_time = sleep_time if sleep_time>0 else 0

    time.sleep(sleep_time)

I also have some logic that changes the update rate (data_rate) if the network is having trouble meeting that speed (sleep times are consistently negative) but that is working correctly.

For some reason my data rate is never consistent (And runs at about 4.92 Hz when it stabilizes). Also this method is pretty unstable. What is the better way to do this? Threading.Timers() comes to mind?

Could the consistent offset in frequency be caused by errors with time.sleep()? How accurate is python's time.sleep()?

Community
  • 1
  • 1
Paul Seeb
  • 5,758
  • 3
  • 25
  • 38
  • Have you been able to ascertain where the variation is being introduced? – Marcin May 04 '12 at 17:05
  • The data is read off a pretty heavily accessed server. I think the variation in access time is a result of network traffic. Still I dont understand why I cant hit 5Hz even when I know the time of the read and post processing. – Paul Seeb May 04 '12 at 17:09
  • I imagine there is an issue with the arithmetic in your sleep time calculation - either you are losing a little bit in your calculation, or there is an issue with the interface to sleep rounding the value you are passing it. – Marcin May 04 '12 at 17:16
  • Python is probably one of the worst languages to do this in. You're getting 98% of your interval, the interval being 200ms. Are you running this on linux? I guarantee you won't be able to hit 5Hz exactly in windows. – Falmarri May 04 '12 at 17:22
  • I am on windows. I am actually OK with slight offset from the desired frequency as long as it is consistent. I get even better results when I average ten runs through the loop and then update the time based on the average but I was hoping for a loopwise dynamic solution. Stability is probably my biggest problem when I incorporate the period adjustments to accommodate for when the network is particularly slow. – Paul Seeb May 04 '12 at 17:26

0 Answers0