Code Monkey home page Code Monkey logo

pymilvus's Introduction

Milvus Python SDK

version Supported Python Versions Downloads Downloads Downloads

license Mergify Status Static Badge

Python SDK for Milvus. To contribute code to this project, please read our contribution guidelines first. If you have some ideas or encounter a problem, you can find us in the Slack channel #py-milvus.

Compatibility

The following collection shows Milvus versions and recommended PyMilvus versions:

Milvus version Recommended PyMilvus version
1.0.* 1.0.1
1.1.* 1.1.2
2.0.* 2.0.2
2.1.* 2.1.3
2.2.* 2.2.15
2.3.* 2.3.7
2.4.* 2.4.0

Installation

You can install PyMilvus via pip or pip3 for Python 3.8+:

$ pip3 install pymilvus
$ pip3 install pymilvus[model] # for milvus-model

You can install a specific version of PyMilvus by:

$ pip3 install pymilvus==2.3.7

You can upgrade PyMilvus to the latest version by:

$ pip3 install --upgrade pymilvus

FAQ

Q1. How to get submodules?

A1. The following command will get the protos matching to the generated files, for protos of certain version, see milvus-proto for details.

$ git submodule update --init

Q2. How to generate python files from milvus-proto?

Before generating python files, please install requirements in requirements.txt

A2.

$ make gen_proto

Q3. How to use the local PyMilvus repository for Milvus server?

A3.

$ make install

Q4. How to check coding styles?

A4.

make lint

Q5. How to fix the coding styles?

A5

make format

Q6. How to run unittests?

A6

$ pip install ".[test]"
$ make unittest

Q7. zsh: no matches found: pymilvus[model] in mac, how do I solve this?

A7

$ pip install "pymilvus[model]"

Documentation

Documentation is available online: https://milvus.io/api-reference/pymilvus/v2.4.x/About.md

Developing package releases

The commits on the development branch of each version will be packaged and uploaded to Test PyPI.

The package name generated by the development branch is x.y.z.rc, where is the number of commits that differ from the most recent release.

  • For example, after the release of 2.3.4, two commits were submitted on the 2.3 branch. The version number of the latest commit of 2.3 branch is 2.3.5.rc2.

  • For example, after the release of 2.3.4, 10 commits were submitted on the master branch. The version number of the latest commit of master branch is 2.4.0.rc10.

To install the package on Test PyPi, you need to append --extra-index-url after pip, for example:

$ python3 -m pip install --extra-index-url https://test.pypi.org/simple/ pymilvus==2.1.0.dev66

License

Apache License 2.0

pymilvus's People

Contributors

bigsheeper avatar bosszou avatar congqixia avatar cqy123456 avatar cydrain avatar czs007 avatar dependabot[bot] avatar filip-halt avatar fishpenguin avatar godchen0212 avatar jaime0815 avatar jingkl avatar jinhai-cn avatar junjiejiangjjj avatar locorichard avatar longjiquan avatar mrpresent-han avatar notryan avatar simfg avatar smellthemoon avatar wayblink avatar weiliu1031 avatar xiaocai2333 avatar xige-16 avatar xuanyang-cn avatar xupeng-sh avatar yah01 avatar yamasite avatar yhmo avatar zhengbuqian avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

pymilvus's Issues

change Indextype

  1. change IndexType 1 to FLAT
  2. fix find_vector_in_files ranges param bug

update new milvus interface

new milvus version add some new interface and change some original interface name,. It occured that pymilvus cannot run with newest milvus docker server

[MS-243] topk=10000, nq=1000 200vectors sdk costs too much

200万向量, nq=1000, topk=10000
milvus从接到请求到查询完成消耗大约10秒, 而使用python sdk则消耗了50秒,有多余40秒的消耗在sdk这边

enhancement and fix bugs

  1. should add utils.py module and optimize some dataclass
  2. type of file_ids in searh_in_files api should be fixed up
  3. Prepare after Prepare should not raise exceptions

[MS182] query ranges param control

MS-182


预期结果:参数检查,并返回参数错误信息

...
        query_ranges = [(get_current_day(), get_last_day(2))]
        status, result = connect.search_vectors(table, top_k, query_vecs, query_ranges=query_ranges)
>     assert not status.OK()
E       AssertionError: assert not True
E        +  where True = <bound method Status.OK of Status(code=0, message='Search Vectors successfully!')>()
E        +    where <bound method Status.OK of Status(code=0, message='Search Vectors successfully!')> = Status(code=0, message='Search Vectors successfully!').OK

Status need to enhance

  • Status should be able to quick-check SUCCESS
  • Status should be related to Thrift Exception
  • TransportExceptions should be separated from ApplicationExceptions
  • Log message should be more understandable

Some bugs need to be fixed

  • MS-165 default ip wrong
  • MS-166 default ip wrong
  • MS-167 default ip wrong
  • MS-168 default ip wrong
  • MS-169 default uri wrong
  • MS-170 change repeatly connect error to status code

has_table interface need to implement

New interface has_table should be implemented in C++/Python SDK in release 0.3.0

This function will return if the table exists in Milvus DB by given table_name.

When there are too many vectors, they should still be able to print out

Even when there are too many vectors, these objects should be able to print out PART of the vectors, including:

  1. TopKQueryResult
  2. add_vectors return value ids

expected pirnt out:

ids = [1111111111111,
       1111111111112,
       ...
       1111111118888,
       1111111118889]

 TopKQueryResult=
 [ [QueryResult(id=000000001, distance=0.1234567678) ,
    ...
    QueryResult(id=000000001, distance=0.1234567678) ],

                           ...
                           ...
                           ...
                  
  [QueryResult(id=000000001, distance=0.1234567678) ,
    ...
   QueryResult(id=000000001, distance=0.1234567678) ] ]

Use gRPC

default to using thrift, can change to grpc by param

[MS-174] table_name None bug create table successfully

_____ TestTable.test_create_wrong_table_name_null ________________________________________________________

self = <test_table.TestTable object at 0x7fb65c466eb8>, connect = Status(code=0, message='Connected')

def test_create_wrong_table_name_null(self, connect):
    '''
    target: test create table with no table_name
    method: create table with table_name: null
    expected: create status return not ok
    '''
    table_name = None
    param = {'table_name': table_name,
             'dimension': dim,
             'index_type': IndexType.FLAT,
             'store_raw_vector': False}
    status = connect.create_table(param)
  assert not status.OK()

E AssertionError: assert not True
E + where True = <bound method Status.OK of Status(code=0, message='Table None created!')>()
E + where <bound method Status.OK of Status(code=0, message='Table None created!')> = Status(code=0, message='Table None created!').OK

test_table.py:96: AssertionError

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.