Bento ML classifier batch inference with csv file input

[ElisaPiccin]

IrisClassifier_svm.zip

Hi,

I'm trying to build an HTTP post request passing a csv file to my packed model.
I'm looking for the equivalent of this command from bentoml CLI, which works fine:
bentoml run IrisClassifier:latest predict --format csv --input-file iris_5.csv

This is how I produced my iris_5.csv:
iris = datasets.load_iris()
X = iris.data
pd.DataFrame(X).head(5).to_csv('iris_5.csv', header=False, index=False)

With
curl -i --header "Content-Type: text/csv" --request POST --data iris_5.csv localhost:5000/predict
I get an error
HTTP/1.0 400 BAD REQUEST
...
DataframeInput Wrong input format: iris_5.csv.

Here is my main.py to train and pack the model.

from sklearn import svm
from sklearn import datasets
import os
import pickle
from bentoApi import IrisClassifier
from sklearn.preprocessing import StandardScaler

if __name__ == "__main__":
    # from sklearn import datasets
    iris = datasets.load_iris()
    # Load training data
    iris = datasets.load_iris()
    X, y = iris.data, iris.target
    # Model Training
    clf = svm.SVC(gamma='scale')
    clf.fit(X, y)
    # Create a iris classifier service instance
    iris_classifier_service = IrisClassifier()
    # Pack the newly trained model artifact
    iris_classifier_service.pack('model', clf)
    # saving prediction service
    saved_path = iris_classifier_service.save()

This is my IrisClassifier prediction service class:

import pandas as pd

from bentoml import env, artifacts, api, BentoService
from bentoml.adapters import DataframeInput
from bentoml.frameworks.sklearn import SklearnModelArtifact

@env(infer_pip_packages=True)
@artifacts([SklearnModelArtifact('model')])
class IrisClassifier(BentoService):
    """
    A minimum prediction service exposing a Scikit-learn model
    """

    @api(input=DataframeInput(), batch=True)
    def predict(self, df: pd.DataFrame):
        """
        An inference API named `predict` with Dataframe input adapter, which codifies
        how HTTP requests or CSV files are converted to a pandas Dataframe object as the
        inference API function input
        """
        return self.artifacts.model.predict(df)

[bojiang]

curl -i --header "Content-Type: text/csv" --request POST --data iris_5.csv localhost:5000/predict ❌
curl -i --header "Content-Type: text/csv" --request POST --data-binary @iris_5.csv localhost:5000/predict ✅

See the doc: https://docs.bentoml.org/en/latest/api/adapters.html#dataframeinput

curl -i \
  --header "Content-Type: text/csv" \
  --request POST \
  --data-binary @file.csv \
  localhost:5000/predict

By the former command, the data of HTTP request would be a string "iris_5.csv” rather than content of the file.

[ElisaPiccin]

Hi @bojiang thanks for your reply. I tried the second curl and received this output:
DataframeInput Wrong input format: 5.1,3.5,1.4,0.24.9,3.0,1.4,0.24.7,3.2,1.3,0.24.6,3.1,1.5,0.25.0,3.6,1.4,0.2.

This is the content of the csv file:
5.1,3.5,1.4,0.2
4.9,3.0,1.4,0.2
4.7,3.2,1.3,0.2
4.6,3.1,1.5,0.2
5.0,3.6,1.4,0.2

The curl is not catching new lines in the same way of the CLI command:
bentoml run IrisClassifier:latest predict --format csv --input-file iris_5.cs

Thanks for your valuable support,
Elisa

[cryanzpj]

Hi @bojiang , I have spotted the same issue.

when doing curl -i --header "Content-Type: text/csv" --request POST --data @iris_5.csv localhost:5000/predict, it seems the entire csv was interpreted as a giant flat list, instead of a valid JSON object.

However, the same dataset worked fine for bentoml CLI.

Can you please take a look at this issue?

Thanks,
Yuhao

[bojiang]

@ElisaPiccin @cryanzpj. sorry for the late reply.
My bad. It should be

curl -i \
  --header "Content-Type: text/csv" \
  --request POST \
  --data-binary @file.csv \
  localhost:5000/predict

to keep new lines. Break-lines would be stripped with --data.

See https://curl.se/docs/manpage.html#-d

When -d, --data is told to read from a file like that, carriage returns and newlines will be stripped out.

I'm drafting a new PR to correct the Docs.

[cryanzpj]

Thanks Bojiang, it works like a charm!

[ElisaPiccin]

Thanks @bojiang!! 😊