How to deploy and serve prediction using TensorFlow from API?

python tensorflow tensorflow-serving
rayhan picture rayhan · Jan 27, 2016 · Viewed 7.5k times · Source

From google tutorial we know how to train a model in TensorFlow. But what is the best way to save a trained model, then serve the prediction using a basic minimal python api in production server.

My question is basically for TensorFlow best practices to save the model and serve prediction on live server without compromising speed and memory issue. Since the API server will be running on the background for forever.

A small snippet of python code will be appreciated.

Answer

Jarek Wilkiewicz picture Jarek Wilkiewicz · Feb 16, 2016

TensorFlow Serving is a high performance, open source serving system for machine learning models, designed for production environments and optimized for TensorFlow. The initial release contains C++ server and Python client examples based on gRPC. The basic architecture is shown in the diagram below.

enter image description here

To get started quickly, check out the tutorial.

Related questions

How to export Estimator model with export_savedmodel function

are there any tutorials available about export_savedmodel ? I have gone through this article on tensorflow.org and unittest code on github.com, and still have no idea about how to construct the parameter serving_input_fn of function export_…

Read More
TensorFlow: How to predict from a SavedModel?

I have exported a SavedModel and now I with to load it back in and make a prediction. It was trained with the following features and labels: F1 : FLOAT32 F2 : FLOAT32 F3 : FLOAT32 L1 : FLOAT32 So say I want to …

Read More
Using deep learning models from TensorFlow in other language environments

I have a decent amount of experience with TensorFlow, and I am about to embark on a project which will ultimately culminate in using a TensorFlow trained model in a C# production environment. Essentially, I will have live data which …

Read More