Comments (5)
sakoush
commented on June 9, 2024
1
datatype should be a value from https://kserve.github.io/website/0.8/modelserving/inference_api/#tensor-data-types
from seldon-core.
hpadakanti0618
commented on June 9, 2024
thank you @sakoush for the above details , i tried with BYTES but i got out of range exceptions and i am thinking of issue with shape value in below input payload , any thoughts ?
curl -X 'POST' 'http://<DOMAIN>/v2/pipelines/matchmaker-kafka-poc/infer' -H 'accept: */*' -H 'Content-Type: application/json' -d @example_request_payload.json
here is my payload
{
"inputs": [
{
"name": "sample_request",
"shape": [
-1
],
"datatype": "BYTES",
"data": [
{
"sr_id": 0,
"create_datetime": "2023-09-15 16:01:08",
"task_id": 40163,
"met_id": 26,
"user_session_id": "f90ce453-3413-4a56-a411-f2208326353e",
"matching_session_id": "6c7b8bbf-a609-402b-afc8-6ed7f073fdad",
"service_professionals": [
{
"sp_id": 76022220,
"sp_type": "Leads_Pro",
"utilization": 2.39745,
"linked_sp_id": null
}
]
}
]
}
]
}
Error -
time="2023-12-12T16:28:41Z" level=debug msg="Seldon model header matchmaker-kafka-poc.pipeline and seldon internal model header [matchmaker-kafka-poc.pipeline]" func=inferPipeline source=GatewayHttpServer
2023/12/12 16:28:41 http: panic serving 10.180.13.224:47916: runtime error: makeslice: len out of range
goroutine 12430 [running]:
net/http.(*conn).serve.func1()
/usr/local/go/src/net/http/server.go:1850 +0xbf
panic({0x1ffbb40, 0x25c1f20})
/usr/local/go/src/runtime/panic.go:890 +0x262
go.opentelemetry.io/otel/sdk/trace.(*recordingSpan).End.func1()
/go/pkg/mod/go.opentelemetry.io/otel/[email protected]/trace/span.go:359 +0x2a
go.opentelemetry.io/otel/sdk/trace.(*recordingSpan).End(0xc0002fb680, {0x0, 0x0, 0x14?})
/go/pkg/mod/go.opentelemetry.io/otel/[email protected]/trace/span.go:398 +0x8ee
panic({0x1ffbb40, 0x25c1f20})
/usr/local/go/src/runtime/panic.go:884 +0x212
github.com/seldonio/seldon-core/scheduler/v2/pkg/kafka/pipeline.convertTensors(0xc000343440)
/build/scheduler/pkg/kafka/pipeline/v2.go:413 +0x198
github.com/seldonio/seldon-core/scheduler/v2/pkg/kafka/pipeline.convertToInferenceRequest({0xc000034380, 0x271, 0x380})
/build/scheduler/pkg/kafka/pipeline/v2.go:433 +0xb8
github.com/seldonio/seldon-core/scheduler/v2/pkg/kafka/pipeline.convertRequestToV2({0xc000034380?, 0x25c2960?, 0x200?}, {0x0, 0x0}, {0x0, 0x0})
/build/scheduler/pkg/kafka/pipeline/v2.go:197 +0x32
github.com/seldonio/seldon-core/scheduler/v2/pkg/kafka/pipeline.ConvertRequestToV2Bytes({0xc000034380?, 0xc0006027c0?, 0x4?}, {0x0?, 0x25c03c0?}, {0x0?, 0x10?})
/build/scheduler/pkg/kafka/pipeline/v2.go:189 +0x28
github.com/seldonio/seldon-core/scheduler/v2/pkg/kafka/pipeline.(*GatewayHttpServer).infer(0xc0002de780, {0x25eac60, 0xc000343260}, 0xc0002cdf00, {0xc000477740, 0x14}, 0x0?)
/build/scheduler/pkg/kafka/pipeline/httpserver.go:151 +0xfd
github.com/seldonio/seldon-core/scheduler/v2/pkg/kafka/pipeline.(*GatewayHttpServer).inferPipeline(0xc0002de780, {0x25eac60, 0xc000343260}, 0xc0004532a0?)
/build/scheduler/pkg/kafka/pipeline/httpserver.go:227 +0xf3
net/http.HandlerFunc.ServeHTTP(0x25e9df0?, {0x25eac60?, 0xc000343260?}, 0x2334958?)
/usr/local/go/src/net/http/server.go:2109 +0x2f
go.opentelemetry.io/contrib/instrumentation/github.com/gorilla/mux/otelmux.traceware.ServeHTTP({{0x23229af, 0xf}, {0x25cb800, 0xc0003d2b40}, {0x25ea300, 0xc000507788}, {0x25d50a0, 0xc000436aa0}}, {0x25e9df0, 0xc0001d8460}, ...)
/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/github.com/gorilla/mux/[email protected]/mux.go:145 +0x593
github.com/gorilla/mux.CORSMethodMiddleware.func1.1({0x25e9df0, 0xc0001d8460}, 0xc000571260?)
/go/pkg/mod/github.com/gorilla/[email protected]/middleware.go:51 +0xaa
net/http.HandlerFunc.ServeHTTP(0xc0002cd800?, {0x25e9df0?, 0xc0001d8460?}, 0xc0006b79e0?)
/usr/local/go/src/net/http/server.go:2109 +0x2f
github.com/gorilla/mux.(*Router).ServeHTTP(0xc0002d9500, {0x25e9df0, 0xc0001d8460}, 0xc0002cd700)
/go/pkg/mod/github.com/gorilla/[email protected]/mux.go:210 +0x1cf
net/http.serverHandler.ServeHTTP({0x25dc700?}, {0x25e9df0, 0xc0001d8460}, 0xc0002cd700)
/usr/local/go/src/net/http/server.go:2947 +0x30c
net/http.(*conn).serve(0xc0003dd900, {0x25eb608, 0xc000465ad0})
/usr/local/go/src/net/http/server.go:1991 +0x607
created by net/http.(*Server).Serve
/usr/local/go/src/net/http/server.go:3102 +0x4db
2023/12/12 16:28:43 max retry time elapsed: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp: lookup seldon-collector.seldon-mesh on 172.20.0.10:53: no such host"
from seldon-core.
hpadakanti0618
commented on June 9, 2024
i just change the shape to positive value as 1 and i see below new error , just an Fyi
below is the payload and error
{
"inputs": [
{
"name": "sample_request",
"shape": [
1
],
"datatype": "BYTES",
"data": [
{
"sr_id": 0,
"create_datetime": "2023-09-15 16:01:08",
"task_id": 40163,
"met_id": 26,
"user_session_id": "f90ce453-3413-4a56-a411-f2208326353e",
"matching_session_id": "6c7b8bbf-a609-402b-afc8-6ed7f073fdad",
"service_professionals": [
{
"sp_id": 76022220,
"sp_type": "Leads_Pro",
"utilization": 2.39745,
"linked_sp_id": null
}
]
}
]
}
]
}
2023/12/12 20:07:30 http: panic serving 10.180.13.224:50320: interface conversion: interface {} is map[string]interface {}, not string
goroutine 12531 [running]:
net/http.(*conn).serve.func1()
/usr/local/go/src/net/http/server.go:1850 +0xbf
panic({0x20268a0, 0xc0004d9b90})
/usr/local/go/src/runtime/panic.go:890 +0x262
go.opentelemetry.io/otel/sdk/trace.(*recordingSpan).End.func1()
/go/pkg/mod/go.opentelemetry.io/otel/[email protected]/trace/span.go:359 +0x2a
go.opentelemetry.io/otel/sdk/trace.(*recordingSpan).End(0xc0002fb380, {0x0, 0x0, 0x4be9f4?})
/go/pkg/mod/go.opentelemetry.io/otel/[email protected]/trace/span.go:398 +0x8ee
panic({0x20268a0, 0xc0004d9b90})
/usr/local/go/src/runtime/panic.go:884 +0x212
github.com/seldonio/seldon-core/scheduler/v2/pkg/kafka/pipeline.copyToTensor({0xc0003d3fc0?, 0x1, 0x200?}, 0xc0003894a0, 0xc0000121f8?)
/build/scheduler/pkg/kafka/pipeline/v2.go:377 +0x71f
github.com/seldonio/seldon-core/scheduler/v2/pkg/kafka/pipeline.convertTensors(0xc0003894a0)
/build/scheduler/pkg/kafka/pipeline/v2.go:422 +0x514
github.com/seldonio/seldon-core/scheduler/v2/pkg/kafka/pipeline.convertToInferenceRequest({0xc0000c4000, 0x279, 0x380})
/build/scheduler/pkg/kafka/pipeline/v2.go:433 +0xb8
github.com/seldonio/seldon-core/scheduler/v2/pkg/kafka/pipeline.convertRequestToV2({0xc0000c4000?, 0x25c2960?, 0x200?}, {0x0, 0x0}, {0x0, 0x0})
/build/scheduler/pkg/kafka/pipeline/v2.go:197 +0x32
github.com/seldonio/seldon-core/scheduler/v2/pkg/kafka/pipeline.ConvertRequestToV2Bytes({0xc0000c4000?, 0xc0003d3d00?, 0x4?}, {0x0?, 0x25c03c0?}, {0x0?, 0x10?})
/build/scheduler/pkg/kafka/pipeline/v2.go:189 +0x28
github.com/seldonio/seldon-core/scheduler/v2/pkg/kafka/pipeline.(*GatewayHttpServer).infer(0xc0002de780, {0x25eac60, 0xc0003893e0}, 0xc00069ec00, {0xc0004768c0, 0x14}, 0x0?)
/build/scheduler/pkg/kafka/pipeline/httpserver.go:151 +0xfd
github.com/seldonio/seldon-core/scheduler/v2/pkg/kafka/pipeline.(*GatewayHttpServer).inferPipeline(0xc0002de780, {0x25eac60, 0xc0003893e0}, 0xc00021b170?)
/build/scheduler/pkg/kafka/pipeline/httpserver.go:227 +0xf3
net/http.HandlerFunc.ServeHTTP(0x25e9df0?, {0x25eac60?, 0xc0003893e0?}, 0x2334958?)
/usr/local/go/src/net/http/server.go:2109 +0x2f
go.opentelemetry.io/contrib/instrumentation/github.com/gorilla/mux/otelmux.traceware.ServeHTTP({{0x23229af, 0xf}, {0x25cb800, 0xc0003d2b40}, {0x25ea300, 0xc000507788}, {0x25d50a0, 0xc000436aa0}}, {0x25e9df0, 0xc0004fe000}, ...)
/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/github.com/gorilla/mux/[email protected]/mux.go:145 +0x593
github.com/gorilla/mux.CORSMethodMiddleware.func1.1({0x25e9df0, 0xc0004fe000}, 0xc0004d9920?)
/go/pkg/mod/github.com/gorilla/[email protected]/middleware.go:51 +0xaa
net/http.HandlerFunc.ServeHTTP(0xc00069e500?, {0x25e9df0?, 0xc0004fe000?}, 0x800?)
/usr/local/go/src/net/http/server.go:2109 +0x2f
github.com/gorilla/mux.(*Router).ServeHTTP(0xc0002d9500, {0x25e9df0, 0xc0004fe000}, 0xc00069e400)
/go/pkg/mod/github.com/gorilla/[email protected]/mux.go:210 +0x1cf
net/http.serverHandler.ServeHTTP({0x25dc700?}, {0x25e9df0, 0xc0004fe000}, 0xc00069e400)
/usr/local/go/src/net/http/server.go:2947 +0x30c
net/http.(*conn).serve(0xc0003dd7c0, {0x25eb608, 0xc000465ad0})
/usr/local/go/src/net/http/server.go:1991 +0x607
created by net/http.(*Server).Serve
/usr/local/go/src/net/http/server.go:3102 +0x4db
2023/12/12 20:07:31 max retry time elapsed: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp: lookup seldon-collector.seldon-mesh on 172.20.0.10:53: no such host"
from seldon-core.
hpadakanti0618
commented on June 9, 2024
Hello @sakoush any thoughts ?
from seldon-core.
sakoush
commented on June 9, 2024
BYTES Usually represents strings or binary payload, not a dictionary as in your example. I suggest that you that a look at the content-types of mlserver docs here https://mlserver.readthedocs.io/en/latest/user-guide/content-type.html
from seldon-core.
Related Issues (20)
- Graph Deployments to Reference Existing Model Deployments
- Payload logging is not available with Triton server in Seldon core V1 HOT 1
- Seldon Pipeline Inspect fails with error `Failed to resolve 'seldon-kafka-0.seldon-kafka-brokers.seldon-mesh.svc:9092'` HOT 1
- how to configure a nginx ingress on seldon-mesh service for grpc request
- A/B Model Traffic split is not working as expected after edit of Experiment CR HOT 1
- Add ability to list namespaces in which a given seldon-core operator can control SeldonDeployments
- SeldonV2 - Ability to set component level labels/annotations in SeldonConfig/SeldonRuntime
- Shape and datatype validation on `Pipeline` inference request
- otel-collector-1 container not starting: error decoding 'exporters': unknown type: "jaeger" for id: "jaeger" HOT 2
- Clarification on possibilities to set the model in status ModelAvailable via Python APIs
- Triton inference server metrics is not supported HOT 3
- How to set type of K8s Service created by SeldonDeployment
- Seldon controller manager giving error
- Relax PyYAML constraints in MLflow server HOT 1
- Support multi Hosts in seldon istio vs for Seldon V1
- Operator reconciliation fails when setting fractional milli resource requests
- Router to All with different payloads
- Upgrade V1 Python microservice Cryptography Package
- seldon-container-engine container spec does not allow additional securityContext parameters
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from seldon-core.