14

Now we have used TensorFlow to train and export an model. We can implement the inference service with this model just like how tensorflow/serving does.

I have a question about whether the tf.Session object is thread-safe or not. If it's true, we may initialize the object after starting and use the singleton object to process the concurrent requests.

tobe
  • 1,631
  • 1
  • 20
  • 38

1 Answers1

17

The tf.Session object is thread-safe for Session.run() calls from multiple threads.

Before TensorFlow 0.10 graph modification was not thread-safe. This was fixed in the 0.10 release, so you can add nodes to the graph concurrently with Session.run() calls, although this is not advised for performance reasons; instead, it is recommended to call sess.graph.finalize() before using the session from multiple threads, to prevent accidental memory leaks.

Graham
  • 7,035
  • 17
  • 57
  • 82
mrry
  • 123,190
  • 25
  • 396
  • 396
  • 1
    Great. Thanks a lot. Is it necessary to add this in document? Any one uses TensorFlow may implement the inference service for their models and it is important to do it on the right way. – tobe Aug 03 '16 at 04:02
  • Can a minimal example using Keras be provided. – Adrian Hood Sr Apr 12 '19 at 22:27