用Python\/Keras\/Flask\/Docker在Kubernetes上部署深度学习模型

步骤一：在Google Cloud上创建用户

我在Google Compute Engine上创建了一个对外提供服务的容器化深度学习模型，当然Google平台并不是必须的，只要能够安装Docker，随便选择平台模式。 

进入Google云平台，点击左侧屏幕选择Compute Engine，启动Google Cloud VM。然后选择“Create Instance”，可以看到已经运行的实例。 

下一步选择计算资源。默认设置就足够，因为只是演示，我选择了4vCPUs和15G内存。 

选择操作系统和磁盘大小。我选择了CentOS 7，100G硬盘。建议磁盘大于10G，因为每个Docker容器有1G大小。

 

最后一步是配置允许HTTP/S工作的防火墙策略。建议选择全部透明，以便减少麻烦。 

选择“Create”，一切进展顺利。 

步骤二：用Keras创建深度学习模型

SSH登录到虚机开始建立模型。最简单方式就是点击虚机下方的SSH图标，会在浏览器中打开一个终端。

 

1、删除预装Docker

sudo yum remove docker docker-client docker-client-latest docker-common docker


    
-latest docker-latest-logrotate docker-logrotate docker-selinux docker-engine-selinux docker-engine

sudo yum install -y yum-utils device-mapper-persistent-data lvm2
sudo yum-config-manager — add-repo https://download.docker.com/linux/centos/docker-ce.repo
sudo yum install docker-ce

    
sudo systemctl start docker
sudo docker run hello-world

Hello from Docker!
This message shows that your installation appears to be working correctly.To generate this message, Docker took the following steps: 1. The Docker client contacted the Docker daemon. 2. The Docker daemon pulled the "hello-world" image from the Docker Hub.    


    
(amd64) 3. The Docker daemon created a new container from that image which runs the    executable that produces the output you are currently reading. 4. The Docker daemon streamed that output to the Docker client, which sent it    to your terminal

global graph
graph = tf.get_default_graph()
...
with


    
 graph.as_default():
    preds = model.predict(image)

mkdir keras-app
cd keras-app

# USAGE
# Start the server:
#   python app.py
# Submit a request via cURL:
#   curl -X POST -F image=@dog.jpg 'http://localhost:5000/predict'



    
# import the necessary packages
from keras.applications import ResNet50
from keras.preprocessing.image import img_to_array
from keras.applications import imagenet_utils
from PIL import Image
import numpy as np
import flask
import io
import tensorflow 


    
as tf
# initialize our Flask application and the Keras model
app = flask.Flask(__name__)
model = None
def load_model():
    # load the pre-trained Keras model (here we are using a model
    # pre-trained on ImageNet and provided by Keras, but you can
    # substitute in your own networks just as easily)
    global model
    model = ResNet50


    
(weights="imagenet")
    global graph
    graph = tf.get_default_graph()
def prepare_image(image, target):
    # if the image mode is not RGB, convert it
    if image.mode != "RGB":
        image = image.convert("RGB")



    
    # resize the input image and preprocess it
    image = image.resize(target)
    image = img_to_array(image)
    image = np.expand_dims(image, axis=0)
    image = imagenet_utils.preprocess_input(image)
    # return the processed image
    return image



    
@app.route("/predict", methods=["POST"])
def predict():
    # initialize the data dictionary that will be returned from the
    # view
    data = {"success": False}
    # ensure an image was properly uploaded to our endpoint
    if flask.request.method == "POST":
        


    
if flask.request.files.get("image"):
            # read the image in PIL format
            image = flask.request.files["image"].read()
            image = Image.open(io.BytesIO(image))
            # preprocess the image and prepare it for classification
            image = prepare_image(image, target=(224,


    
 224))
            # classify the input image and then initialize the list
            # of predictions to return to the client
            with graph.as_default():
                preds = model.predict(image)
                results = imagenet_utils.decode_predictions(preds)
                data["predictions"] = []
                


    
# loop over the results and add them to the list of
                # returned predictions
                for (imagenetID, label, prob) in results[0]:
                    r = {"label": label, "probability": float(prob)}
                    data["predictions"].append(r)
                # indicate that the request was a success
                data


    
["success"] = True
    # return the data dictionary as a JSON response
    return flask.jsonify(data)
# if this is the main thread of execution first load the model and
# then start the server
if __name__ == "__main__":
    print(("* Loading Keras model and Flask starting server..."
        "please wait until server has fully started"))
    load_model()



    
    app.run(host='0.0.0.0')

keras
tensorflow
flask
gevent
pillow
requests

FROM python:3.6
WORKDIR 


    
/app
COPY requirements.txt /app
RUN pip install -r ./requirements.txt
COPY app.py /app
CMD ["python", "app.py"]~

sudo docker build -t keras-app:latest .

sudo docker run -d -p 5000:5000 keras-app

[gustafcavanaugh@instance-3 ~]$ sudo docker ps -a
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS                    PORTS                    NAMES
d82f65802166        keras-app           "python app.py"     About an hour ago   Up About an hour          0.0.0.0:5000->5000/tcp   nervous_northcutt

9、测试模型

现在可以测试此模型。用狗的照片作为输入，可以返回狗的品种。在Adrian的示例中都有该图片，我们也使用它们，并保存自工作目录下，命名为dog.jpg。 

执行命令：

curl -X POST -F image=@dog.jpg 'http://localhost:5000/predict'

{"predictions":[{"label":"beagle","probability":0.987775444984436},{"label":"pot","probability":0.0020967808086425066},{"label":"Cardigan",


    
"probability":0.001351703773252666},{"label":"Walker_hound","probability":0.0012711131712421775},{"label":"Brittany_spaniel","probability":0.0010085132671520114}],"success":true}

Login Succeeded

REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE keras-app           latest              ddb507b8a017        About an hour ago   1.61GB

#Format
sudo docker tag <your image id> <your docker hub id>/<app name>
#My Exact Command - Make Sure To Use Your Inputs
sudo docker tag ddb507b8a017 gcav66/keras-app

#Format



    
sudo docker push <your docker hub name>/<app-name>
#My exact command
sudo docker push gcav66/keras-app

5、创建Kubernetes集群

在Google Cloud Home界面，选择Kubernetes Engine。 

创建新集群： 

选择集群内节点资源，因为要启动三个节点（每个节点4vCPU和15G内存），至少需要12vCPU和45G内存。 

连接集群，Google’s Kubernetes自动会在VM上安装Kubernetes。 

在Kubernetes中运行容器：

kubectl run keras-app --image=gcav66/keras-app --port 5000

gustafcavanaugh@cloudshell:~ (basic-web-app-test)$ kubectl get pods
NAME                         READY     STATUS    RESTARTS   AGE
keras-app-79568b5f57-5qxqk   1/1       


    
Running   0          1m

kubectl expose deployment keras-app --type=LoadBalancer --port 80 --target-port 5000

gustafcavanaugh@cloudshell:~ (basic-web-app-test)$ kubectl get service
NAME         TYPE           CLUSTER-IP     EXTERNAL-IP     PORT


    
(S)        AGE
keras-app    LoadBalancer   10.11.250.71   35.225.226.94   80:30271/TCP   4m
kubernetes   ClusterIP      10.11.240.1              443/TCP        18m

$ curl -X POST -F image=@dog.jpg 'http://35.225.226.94/predict'
{


    
"predictions":[{"label":"beagle","probability":0.987775444984436},{"label":"pot","probability":0.0020967808086425066},{"label":"Cardigan","probability":0.001351703773252666},{"label":"Walker_hound","probability":0.0012711131712421775},{"label":"Brittany_spaniel","probability":0.0010085132671520114}],"success":true}