kserve 中的 ConnectionResetError (104, 'Connection reset by peer)?

问题描述 投票:0回答:0

我已将模型部署为 Kserve 中的 docker 镜像。当我尝试从模型 pod 中卷曲请求模型时,我收到类似“ConnectionResetError (104, 'Connection reset by peer)”的错误。这是我的 handler.py 文件

import os
import json
import logging
from PIL import Image
import io
import torch
from transformers import DetrForObjectDetection
from ts.torch_handler.base_handler  import BaseHandler
from detectron2.structures import Boxes, ImageList, Instances, BitMasks, PolygonMasks
from detectron2.modeling import META_ARCH_REGISTRY, build_backbone, detector_postprocess
from detectron2.utils.visualizer import Visualizer, VisImage
from detectron2.layers import batched_nms
import torchvision.transforms as T
import torch.nn.functional
import numpy as np
import base64

这是我的推理服务

apiVersion: serving.kserve.io/v1beta1
kind: InferenceService
metadata:
  name: custom-model
  namespace: user1
spec:
  predictor:
    containers:
      - name: kserve-container
        image: <Dockerhub>/<Docker image >
        env:
          - name: "MODEL_NAME"
            value: "table_det"
          - name: "PROTOCOL"
            value: "v1"
        ports:
          - containerPort: 8085
        readinessProbe:
          failureThreshold: 3
          periodSeconds: 100
          successThreshold: 1
          tcpSocket:
            port: 8085
          timeoutSeconds: 10
          initialDelaySeconds: 120
          periodSeconds: 30
        resources:
          limits:
            cpu: "1"
            memory: 2Gi
          requests:
            cpu: "1"
            memory: 2Gi

i have deployed the Infrernece service file with Docker image.
i got the services like

Inference service {true}
ksvc {true}
Revision {true]
route {true}
virtualservice {true}
All service created by the service is created successfully without any error but i'm not able to establush the connection between the ingress of istio and the model . give me the best solution : thank you

I need solution and how the connection happens between the kserve and model deployed. i want to set perfect connection between the model and kserve. 
kubernetes istio knative knative-serving kubeflow-kserve
© www.soinside.com 2019 - 2024. All rights reserved.