飞桨模型部署至docker并使用FastAPI调用（三）-API部署

MWHLS • • API/SDK, Python, 虚拟机/docker, 计算机服务, 计算机语言, 飞桨
已阅： 44

飞桨模型部署至docker并使用FastAPI调用

目录
本地 get 调用 fastapi
目录树 - docker
predict.py - 模型推理代码
startup.py - 启动 fastapi 服务
本地 Python post 调用 fastapi
目录树 - 宿主机
post.py - 宿主机中，本地 python 使用 post 访问部署于 docker 中的 fastapi
main.py - fastapi 代码
predict.py - 推理代码
startup.py - 启动 fastapi 服务
参考文献

本地 get 调用 fastapi

  • 中间的调错和测试省略了，只展示最终结果，毕竟环境会骗你，代码不会。

  • 运行 startup.py，并访问 http://127.0.0.1:8000/，终端输出如下：

    INFO:     Will watch for changes in these directories: ['/root/code']
INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
INFO:     Started reloader process [21292] using statreload
/usr/local/lib/python3.8/site-packages/paddle/tensor/creation.py:130: DeprecationWarning: np.object is a deprecated alias for the builtin object. To silence this warning, use object by itself. Doing this will not modify any behavior and is safe. 
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
if data.dtype == np.object:
INFO:     Started server process [21294]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
2022-06-18 07:26:02 [WARNING]   Cannot find raw_params. Default arguments will be used to construct the model.
2022-06-18 07:26:02 [INFO]      Model[BIT] loaded.
------------------ Inference Time Info ----------------------
total_time(ms): 4260.3, img_num: 1, batch_size: 1
average latency time(ms): 4260.30, QPS: 0.234725
preprocess_time_per_im(ms): 121.90, inference_time_per_batch(ms): 4132.60, postprocess_time_per_im(ms): 5.80
INFO:     127.0.0.1:32956 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:32956 - "GET /favicon.ico HTTP/1.1" 404 Not Found

  • 网页展示如下（仅展示部分 base64）：

    {"message":"iVBORw0KGgoAAAANSUhEUgAAAoAAAAH...SK+Z8VWmji1wgxWwAAAABJRU5ErkJggg=="}

目录树 - docker

root
└─ code
   ├─ datasets
   │  └─ infer
   │     ├─ before.png
   │     ├─ label_no_use.png
   │     └─ later.png
   ├─ inference_model
   │  ├─ .success
   │  ├─ model.pdiparams
   │  ├─ model.pdiparams.info
   │  ├─ model.pdmodel
   │  ├─ model.yml
   │  └─ pipeline.yml
   ├─ main.py
   ├─ predict.py
   └─ startup.py
  • main.py - fastapi 代码
# main.py
from fastapi import FastAPI
from predict import predict
import base64

app = FastAPI()
img_before_base64 = base64.b64encode(open("/root/code/datasets/infer/before.png", "rb").read()).decode('utf8')
img_after_base64 = base64.b64encode(open("/root/code/datasets/infer/later.png", "rb").read()).decode('utf8')

@app.get('/')
def index():
    img_variation_base64 = predict(img_before_base64, img_after_base64)
    return {'message': img_variation_base64}

predict.py - 模型推理代码

  • 中英命名有点小冲突，predict是预测，inference是推理，但是参考的几篇里面，都是讲推理，但调用的是 predict，不过按我现在的理解，这俩玩意是一个东西，问题不大。
  • Copilot 帮我写了一些直接转换的函数，然而不知道咋回事结果不是很正确，只好用临时文件解决，感觉降低了不少性能。
from paddlers.deploy import Predictor
from PIL import Image
from matplotlib import pyplot as plt
from io import BytesIO
import base64
import tempfile

def base64_to_img(img_base64):
    # base64 to PIL img
    img_pil = Image.open(BytesIO(base64.b64decode(img_base64)))

    # PIL save as tmp file
    img_tf = tempfile.NamedTemporaryFile()
    img_pil.save(img_tf, format='png')

    return img_tf, img_pil

def predict(img_before_base64, img_after_base64):
    # ref: https://aistudio.baidu.com/aistudio/projectdetail/4184759
    # build predictor
    predictor = Predictor("/root/code/inference_model", use_gpu=False)

    # base64 to tmp file and PIL img
    img_before_tf, img_before_pil = base64_to_img(img_before_base64)
    img_after_tf, img_after_pil = base64_to_img(img_after_base64)

    # predict
    res_pred = predictor.predict((img_before_tf.name, img_after_tf.name))[0]['label_map']

    # result to PIL img
    img_variation_pil = Image.fromarray(res_pred * 255)

    # show with before and after
    plt.figure(constrained_layout=True);  
    plt.subplot(131);  plt.imshow(img_before_pil);  plt.gca().set_axis_off();  plt.title("Before")
    plt.subplot(132);  plt.imshow(img_after_pil);  plt.gca().set_axis_off();  plt.title("After")
    plt.subplot(133);  plt.imshow(img_variation_pil);  plt.gca().set_axis_off();  plt.title("Pred")
    img_variation_tf = tempfile.NamedTemporaryFile()
    plt.savefig(img_variation_tf)

    # plt to base64
    img_variation_base64 = base64.b64encode(open(img_variation_tf.name, "rb").read()).decode('utf8')

    # close tmp file
    img_before_tf.close()
    img_after_tf.close()
    img_variation_tf.close()

    return img_variation_base64

startup.py - 启动 fastapi 服务

import os

os.chdir('/root/code')
path_main = 'main'
command = f'uvicorn {path_main}:app --reload'
os.system(command)

本地 Python post 调用 fastapi

  • fastapi 能所有文件都能热更新，NB。

  • 运行 startup.py 启动 docker 中的 fastapi 服务。

  • 运行 post.py 在宿主机中调用 docker 中的 fastapi 服务。

  • docker 终端输出如下：

    INFO:     Will watch for changes in these directories: ['/root/code']
INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
INFO:     Started reloader process [10056] using statreload
/usr/local/lib/python3.8/site-packages/paddle/tensor/creation.py:130: DeprecationWarning: np.object is a deprecated alias for the builtin object. To silence this warning, use object by itself. Doing this will not modify any behavior and is safe. 
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
if data.dtype == np.object:
INFO:     Started server process [10058]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
2022-06-18 13:19:13 [WARNING]   Cannot find raw_params. Default arguments will be used to construct the model.
2022-06-18 13:19:13 [INFO]      Model[BIT] loaded.
------------------ Inference Time Info ----------------------
total_time(ms): 4365.2, img_num: 1, batch_size: 1
average latency time(ms): 4365.20, QPS: 0.229085
preprocess_time_per_im(ms): 127.90, inference_time_per_batch(ms): 4233.30, postprocess_time_per_im(ms): 4.00
INFO:     127.0.0.1:34572 - "POST /predict HTTP/1.1" 200 OK

  • 宿主机终端输出如下：

    post consume: 5.9285101890563965, all consume: 8.314009189605713
done

  • 代码所在目录生成推理结果图片：

    • pred.png（分辨率为 1024x1024，位深度为 8bit，与训练集中标签一致）

目录树 - 宿主机

test
├─ before.png
├─ label_no_use.png
├─ later.png
├─ post.py
└─ pred.png

post.py - 宿主机中，本地 python 使用 post 访问部署于 docker 中的 fastapi

# post.py
import requests
import base64
import time

def post_predict(img_before_base64, img_after_base64):
    url = 'http://localhost:8000/predict'
    data = {'img_before_base64': img_before_base64, 'img_after_base64': img_after_base64}
    res = requests.post(url, json=data)
    return res.json()

def main():
    # test files
    img_before_base64 = base64.b64encode(open("./before.png", "rb").read()).decode('utf8')
    img_after_base64 = base64.b64encode(open("./later.png", "rb").read()).decode('utf8')

    # post to predict
    time_start = time.time()
    result_post = post_predict(img_before_base64, img_after_base64)
    img_variation_base64 = result_post['img_variation_base64']
    time_consume = result_post['time_consume']

    # output
    with open('./pred.png', 'wb') as f:
        f.write(base64.b64decode(img_variation_base64))
    print(f'post consume: {time_consume}, all consume: {time.time() - time_start}')
    print('done')

if __name__ == '__main__':
    main()

main.py - fastapi 代码

  • 新增 post 方法，删除了一些测试代码
# main.py
from fastapi import FastAPI
from predict import predict
from pydantic import BaseModel
import time

class PredictRequest(BaseModel):
    img_before_base64: str
    img_after_base64: str

app = FastAPI()

@app.get('/')
def index():
    # index
    return 'running'

@app.post('/predict')
def predict_post(request: PredictRequest):
    # predict by post
    time_start = time.time()
    img_variation_base64 = predict(request.img_before_base64, request.img_after_base64)
    time_consume = time.time() - time_start
    return {'img_variation_base64': img_variation_base64, 'time_consume': time_consume}

predict.py - 推理代码

  • 稍微优化了下，但这里肯定还能优化，临时文件肯定是没必要的，不过还是先跑起来再说。
# predict.py
from paddlers.deploy import Predictor
from PIL import Image
from matplotlib import pyplot as plt
from io import BytesIO
import base64
import tempfile

def base64_to_img(img_base64):
    # convert base64 to tmp file and PIL img
    # base64 to PIL img
    img_pil = Image.open(BytesIO(base64.b64decode(img_base64)))

    # PIL save as tmp file
    img_tf = tempfile.NamedTemporaryFile()
    img_pil.save(img_tf, format='png')

    return img_tf, img_pil

def predict(img_before_base64, img_after_base64):
    # predict the variation field from two images, and return the base64 of variation field
    # build predictor
    predictor = Predictor("/root/code/inference_model", use_gpu=False)

    # base64 to tmp file and PIL img
    img_before_tf, _ = base64_to_img(img_before_base64)
    img_after_tf, _ = base64_to_img(img_after_base64)

    # predict
    res_pred = predictor.predict((img_before_tf.name, img_after_tf.name))[0]['label_map'] * 255

    # result to PIL img
    img_variation_pil = Image.fromarray(res_pred).convert('L')

    # save PIL img
    img_variation_tf = tempfile.NamedTemporaryFile()
    img_variation_pil.save(img_variation_tf, format='png')
    img_variation_base64 = base64.b64encode(open(img_variation_tf.name, "rb").read()).decode('utf8')

    # close tmp file
    img_before_tf.close()
    img_after_tf.close()
    img_variation_tf.close()

    return img_variation_base64

startup.py - 启动 fastapi 服务

# startup.py
import os

os.chdir('/root/code')
path_main = 'main'
command = f'uvicorn {path_main}:app --reload'
os.system(command)

参考文献

  1. 将matplotlib绘制的图形直接以base64格式传递到html使用
  2. Renderer problems using Matplotlib from within a script
  3. Python调用get或post请求外部接口
  4. 去除plt.savefig()的白边
  5. 数据可视化--matplotlib图表控制输出分辨率
  6. PIL模块储存单值图为png格式后单值图全黑的情况的解决方案

版权声明：
作者：MWHLS
链接：http://panwj.top/4085.html
来源：无镣之涯
文章版权归作者所有，未经允许请勿转载。

THE END
分享
二维码
打赏
< <上一篇
下一篇>>