Pardon for asking... beginners questions - but I am coming from implementing Pinecone in my application and would like to implement Milvus as well, as an option for the user to choose the vector DB that can be run locally... However, I stumble on some immediate questions:

1. Is the Collection here equivalent to Index on Pinecone? (the Collection in Pinecone is used for snapshots of vectors in the Index). Ans what is Index in Milvus? Where do I find basic definition of the terminology being used?
2. If I intend to run Milvus locally, is my endpoint: "https://localhost:19530" ?
3. I installed WSL2 and Ubuntu, I also install NuGet package in my project in VS2022, however, can you please clarify to connection/link between this and the DB itself - how do I make it run or make sure it is running? I cannot find anything in the documentation pertaining to C# SDK - I know that my WSL2 is running because I see it on the screen...

See https://milvus.io/docs/release_notes.md

Milvus.Client --version 2.2.2-preview.6
调用InsertAsync方法没有问题，调用Search和Query方法都会卡住不返回，我milvus服务端是用docker-compose启动的单节点版本

Milvus 2.3.0 introduced an Upsert operation. From the release notes:

Users now can use the upsert API in Milvus for updating or inserting data. It is important to note that the upsert API combines search, delete, and insert operations, which may result in a degradation of performance. Therefore, it is recommended to use the insert APIs for specific and definitive insertions, while reserving the upsert APIs for more ambiguous scenarios.

Guys its not building for me, I am getting the error that Grpc.FieldData is not found, but you have a FieldData class, can I just replace it with that? Also can I upgrade to .net8? And I don't need .net

Milvus 2.3.0 introduced support for Count. From the release notes:

In previous releases, users would often use the num_entities API to retrieve the total number of entities in a collection. However, it is important to note that the num_entities API only applies to entities within sealed segments. Making frequent calls to the flush API can result in the creation of numerous small segments, which can negatively impact the stability of the system and the performance of data retrieval in Milvus.

In this release, Milvus introduces the count statement as an alternative solution for users to obtain the number of entities in a collection without relying on the flush API.

Apikey support

For a serverless cluster, you have obtained an API key.

Restful api

zilliz doc

See https://milvus.io/docs/gpu_index.md

The current Github Actions Nuget publish workflow looks like it publishes any tag to nuget.org, which seems problematic - we should be able to use tags for other purposes as well without a nuget package immediately getting pushed to nuget.org.

At the very least, we should perform the deployment via a Github environment; this environment would require manual authorization for pushing to nuget.org, and would ideally also contain the nuget.org key secret (presumably currently defined in the organization)

@yhmo @weianweigan who would be a good person with permissions to create the Github organization and configure it appropriately?

Milvus 2.3.0 introduced support for change data capture (CDC). From the release notes:

Change Data Capture (CDC) is a widely used functionality in databases, typically employed in scenarios such as active/standby data synchronization, incremental data backup, and data migration. For more detailed information on CDC, please refer to https://github.com/zilliztech/milvus-cdc.

Describe the solution you'd like
Ensure that the final assembly is signed. This would prevent consumers who may have code analysis rules requiring signing to remove CS8002 suppressions in their projects

For FLAT index, the server ignores all the extra parameters such as "nprobe=10", the client can send an empty json object "{}" to server side.

List of bugs, all of them are pretty simple:

1. It should be possible to declare primary key on VarChar field
2. InsertAsync does not set NumRows which causes: error="num_rows(0) should be greater than 0"
3. CreateIndex ignores IndexName and always creates indexes with the default name
4. SearchParam.Create ignores param parameter and does not accept expr
5. It is not possible to create Scalar Index (only Vector indexes are allowed)
6. DescribeIndex behavior does not match API: empty index name is supported by API and returns all indexes for a collection (instead of a default name)

PR is coming soon.

Describe the bug

When following along with the Quick Start using 2.2.2-preview.1, it appears that MilvusClient cannot determine which MilvusClient to use.

"CS0121: The call is ambiguous between the following methods or properties: "MilvusClient.MilvusClient(string, int, string?, string?, CallOptions, ILoggerFactory?) and MilvusClient.MilvusClient(string, int, string?, string?, CallOptions, ILoggerFactory?)

Potential problem:

There doesn't appear to be any difference in the code between the gRPC and the Resftul client within the "Quick Start" guide. It is exactly the same code. I can't see a way to differentiate, since you do separate these out into different headers - I would have expected a difference?

You also call the client "client" but then refer to it as "milvusClient" - I only mention so that if you look at it and it needs updating you can update that part too.

Please do not think this is a criticism, it is not, I was really happy when I found this repo

I also got it working on the 2.2.1-alpha.7 version, as that is in the notebooks that you provide, so I know that Milvus is up and running and I can do a successful health check.

To Reproduce
Steps to reproduce the behavior:

Copy and paste the Quick Start code into a console application.

Basically the SDK cannot connect to collections on https://cloud.zilliz.com. It says auth check failure, please check username and password are correct.

public const string Host = "in01-XXXXXXXXX.aws-us-west-2.vectordb.zillizcloud.com";
public const int Port = 19543;

connect.UseHttps = true;

Exception Details

Grpc.Core.RpcException: Status(StatusCode="Unknown", Detail="auth check failure, please check username and password are correct")
   at Grpc.Net.Client.Internal.HttpClientCallInvoker.BlockingUnaryCall[TRequest,TResponse](Method`2 method, String host, CallOptions options, TRequest request)
   at Grpc.Core.Interceptors.InterceptingCallInvoker.<BlockingUnaryCall>b__3_0[TRequest,TResponse](TRequest req, ClientInterceptorContext`2 ctx)
   at Grpc.Core.ClientBase.ClientBaseConfiguration.ClientBaseConfigurationInterceptor.BlockingUnaryCall[TRequest,TResponse](TRequest request, ClientInterceptorContext`2 context, BlockingUnaryCallContinuation`2 continuation)
   at Grpc.Core.Interceptors.InterceptingCallInvoker.BlockingUnaryCall[TRequest,TResponse](Method`2 method, String host, CallOptions options, TRequest request)
   at Milvus.Proto.Milvus.MilvusService.MilvusServiceClient.Query(QueryRequest request, CallOptions options) in /Users/hieu/sources/repos/milvus-sdk-csharp/src/IO.Milvus/obj/Debug/net7.0/MilvusGrpc.cs:line 2030
   at IO.Milvus.Client.AbstractMilvusGrpcClient.Query(QueryParam requestParam, Nullable`1 callOptions) in /Users/hieu/sources/repos/milvus-sdk-csharp/src/IO.Milvus/Client/AbstractMilvusGrpcClient.cs:line 1337

var searchResults = await milvusClient.SearchAsync(
            MilvusSearchParameters.Create(collectionName, "text_vectors", search_output_fields)
                .WithVectors(search_vectors)
                .WithOutputFields(new List<string>() { "text" })
                .WithConsistencyLevel(MilvusConsistencyLevel.Strong)
                .WithMetricType(MilvusMetricType.L2)
                .WithTopK(topK: (int)topResults)
                .WithParameter("nprobe", "10")
                .WithParameter("offset", "5"));

Trying to extract field "text". the only way I found is: searchResults.Results.FieldsData.Where(fd => fd.FieldName == "text").ToList().First().ToGrpcFieldData().Scalars.StringData.Data[0], which is extremely nasty.

The overall tabular encapsulation in this SDK (getting a flat table and not a simple enumrable object list) is really hard to work with.
Instead of getting a clear object that represents an object in the db, I need to iterate all results and build them myself, manually.

The group_by_field search parameter should be supported as per docs here: https://milvus.io/docs/single-vector-search.md#Grouping-search

num_rows(0) should be greater than 0

DescribeIndex, FlushAsync etc fail with auth errors.
Pull request is created:

#16

Milvus 2.3 was released recently, we would like to upgrade our version due to the GPU support.
There are some breaking changes, so the C# SDK must also be updated. Is the support for the new version currently in the works? It would help us immensely to know whether we can expect the SDK to be updated in a week, a month, or later.

Important .NET version is implemented differently than the Java version, sometimes you can refer to the java version code and sometimes it doesn't work, you may need to check the .NET SDK source code to determine how to use it.

You should use DataType.FloatVector in Field Definition, but use BinaryVectorField when inserting.
Field<List<float>> is not worked.

    public static readonly FieldType TextVector = FieldType.Create("TextVector", DataType.FloatVector, new Dictionary<string, string>()
    {
        ["dim"] = TextVectorDim.ToString(),
    }, false);

List<Field> fields = new();
fields.Add(Field.Create(TextSpanCollectionDef.Id.Name, idArray));
fields.Add(new BinaryVectorField()
{
    FieldName = TextSpanCollectionDef.TextVector.Name,
    Datas = vectorArray.ToList(),
});

these folling code is not work for float vector:

fields.Add(Field.Create(TextSpanCollectionDef.TextVector.Name, book_intro_array));

         fields.Add(new Field<List<float>>()
        {
            FieldName = TextSpanCollectionDef.TextVector.Name,
            Datas = book_intro_array.ToList(),
        });

---- full codes ----

class TextSpanCollectionDef
{
    public const string CollectionName = "test";


    public static readonly FieldType Id = FieldType.Create("Id", DataType.VarChar, new Dictionary<string, string>()
    {
        ["max_length"] = "40",
    }, true);

    public const int TextVectorDim = 512;
    public static readonly FieldType TextVector = FieldType.Create("TextVector", DataType.FloatVector, new Dictionary<string, string>()
    {
        ["dim"] = TextVectorDim.ToString(),
    }, false);
    
    public static FieldType[] FieldDefs = new FieldType[]
    {
        Id,
        TextVector
    };

    static TextSpanCollectionDef()
    {
    }
}

#nullable enable
using System;
using System.Text.Json.Serialization;
using System.Threading.Tasks;
using Antlr4.Runtime.Misc;
using IO.Milvus.Client;
using IO.Milvus.Grpc;
using IO.Milvus.Param;
using IO.Milvus.Param.Collection;
using IO.Milvus.Param.Dml;
using IO.Milvus.Param.Index;
using IO.Milvus.Param.Dml.Field;
using Xunit.Abstractions;

namespace Xxxxx;

public class MilvusTest
{
    private readonly ITestOutputHelper _testOutputHelper;

    public MilvusTest(ITestOutputHelper testOutputHelper)
    {
        _testOutputHelper = testOutputHelper;
    }

    [Fact]
    public async Task MilvusConect()
    {
        var milvusClient = new MilvusServiceClient(ConnectParam.Create(host: "127.0.0.1", port: 19530));

        milvusClient.DropCollection(TextSpanCollectionDef.CollectionName);
        var createCollectionResult = milvusClient.CreateCollection(CreateCollectionParam.Create(TextSpanCollectionDef.CollectionName, 1, TextSpanCollectionDef.FieldDefs));
        _testOutputHelper.WriteLine(createCollectionResult.ToString());

        Random ran = new Random();
        List<string> idArray = new();
        List<List<float>> vectorArray = new();
        for (int i = 0; i < 2000; ++i)
        {
            idArray.Add(i.ToString());
            List<float> vector = new();
            for (int k = 0; k < TextSpanCollectionDef.TextVectorDim; ++k)
            {
                vector.Add(ran.Next());
            }

            vectorArray.Add(vector);
        }

        List<Field> fields = new();
        fields.Add(Field.Create(TextSpanCollectionDef.Id.Name, idArray));
        fields.Add(new BinaryVectorField()
        {
            FieldName = TextSpanCollectionDef.TextVector.Name,
            Datas = vectorArray.ToList(),
        });
        var insertRet = await milvusClient.InsertAsync(InsertParam.Create(TextSpanCollectionDef.CollectionName, "", fields));
        _testOutputHelper.WriteLine(insertRet.ToString());
    }
}

Docs

Specifically, replaced the MilvusFixture collection fixture with an assembly feature (docs). Aside from cleaning up the code, this will allow all test classes to run in parallel.

option csharp_namespace = "IO.Milvus.Grpc";

Watting for milvus-proto update.

这是我之前获取图片向量的方式 | This is how I got the image vector before

我该怎么结合 milvus-sdk-csharp 来使用，有demo吗

How should I combine [milvus - SDK - csharp] (https://github.com/milvus-io/milvus-sdk-csharp) to use, have a demo

Is your feature request related to a problem? Please describe.
Hi C# team. Thanks for the contribution and seems that the C# client almost catch up everything for Milvus 2.3(Milvus 2.4 is in release candidate but should be put into production soon.)

I'm proposing to put C# SDK into the one of the official supported SDKs and put into our document.

Describe the solution you'd like

1. We need help for converting all the samples in the document into C#.
2. Maybe some CI/CD to make sure milvus upgrade didn't broke the consistency.

Describe alternatives you've considered
Milvus maintainer team and doc team is glad to offer any help if necessary
Thanks for the awesome contribution!

Milvus 2.3.0 introduced support for Cosine metrics. From the release notes:

The Cosine Metrics is widely regarded as the standard method for measuring the distance between vectors, particularly in Large Language Models (LLMs). With the release of Milvus 2.3.0, cosine metrics are now natively supported. As a result, users no longer need to quantize vectors for IP (Inner Product) metrics.

Current client API does not allow you to specify field schema for Array types.

I propose extending the C# API to include support for defining array field schemas, which would allow developers to specify the type of elements in the array along with constraints such as capacity and maxLength. Below are proposed methods to add to the API for defining array fields with both varchar and scalar types:

FieldSchema.CreateVarcharArray("field_id", capacity: 200, maxLength: 255, description: "Array of varchars"),
FieldSchema.CreateArray<float>("field_id", capacity: 200, description: "Array of scalar types")

Milvus 2.3.0 introduced support for ScaNN. From the release notes:

Milvus now includes support for FAISS' FastScan, which has demonstrated a 20% performance improvement compared to HNSW and a 7-fold increase compared to IVF-FLAT in multiple benchmark tests. ScaNN, an index-building algorithm similar to IVF-PQ, offers a faster index-building process. However, it's important to note that using ScaNN may result in a potential loss of precision and therefore requires refinement using the raw vectors.

I deployed Milvus on a remote server and called API using SDK【Grpc.Core.RpcException:“Status(StatusCode="Internal", Detail="Error starting gRPC call. HttpRequestException: Requesting HTTP version 2.0 with version policy RequestVersionOrHigher while unable to establish HTTP/2 connection.", DebugException="System.Net.Http.HttpRequestException: Requesting HTTP version 2.0 with version policy RequestVersionOrHigher while unable to establish HTTP/2 connection.")”】

Hi,
I got a compiler error when trying to create a string field
Field.Create("test","firsttest")
return TDATA canot be nullable
Thanks in advance

https://milvus.io/docs/within_range.md#Conduct-a-Range-Search

i have a eroor when i use the same example in the document(book example )
Milvus.Client.MilvusException
HResult=0x80131500
Message=ErrorCode: UnexpectedError Reason: attempt #0: fail to get shard leaders from QueryCoord: collection=443943362921669896: collection not loaded: unrecoverable error: fail to search on all shard leaders
Source=Milvus.Client
StackTrace:
at Milvus.Client.MilvusClient.d__342.MoveNext() at Milvus.Client.MilvusCollection.<SearchAsync>d__181.MoveNext()
at Program.<>c__DisplayClass0_0.<<

Look Image

Same as the document

Package version ： 2.2.2-preview.6

See https://milvus.io/docs/bulk_insert.md

Milvus 2.4 has added beta support to sparse float vector to allow efficient storing, indexing and searching of such vectors. The main targeted use case is to support learned sparse models such as SPLADE and statistics based methods such as BM25.

We need to add sparse float vector to C# SDK as well. See: milvus-io/milvus#29419 (comment) for more details.

Thanks a lot for the efforts!

Our tests currently rely on an externally available Milvus instance. This makes it a bit harder to contribute, and also complicates our CI.

We can instead use testcontainers.net; note that there already are testcontainer modules for Milvus, but currently only for

@yhmo
I found that a action failed because of a missing field.
In c# project,we use nuget as default package manager. There is a github workflow named NugetPublish.I use it to push package.
There is a secret field used in NugetPublish.yml,but i cannot find it in this repos's settings area.
How could i set this field just like this or you can help me to set it?

Describe the solution you'd like
Would like the C# SDK for Milvus to have Tracing/HealthChecks/Metrics. For .NET Aspire, the components are desired to have all of these 'by default' for consumers. .NET Aspire is introducing hosting support for Milvus and wrapping the Milvus C# Client in a component and would like these to be exposed more. See: https://github.com/dotnet/aspire/tree/main/src/Components