yegoroff / plainelastic.net Goto Github PK

HI Alexander,

You put a sample of mapping on the gist. Can you please put a sample for indexing of those classes probably there are company address and contact classes. this could really be helpful for those who are new like me

When doing a lookup to see if an index exists, ES send back a 404 or 200 response code (http://www.elasticsearch.org/guide/reference/api/admin-indices-indices-exists/). The current implementation of a HEAD request does not provide any way to get data sent back in this type of request.

I was searching around and I couldn't find anything about aliases:

http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indices-aliases.html

I need that functionality to create and manage aliases, isn't implemented yet or I'm not searching where I should?

Like match_all function in Elasticsearch, I want to fetch all the data from elasticsearch to my datagridview using Searchcommand. I only get 10 results but not whole data. what query should I use?

This is an awesome library that you created. One of the major things it is missing is bulk Index operations.

Hi,
I am trying to query elastic search as shown below.

string command = new SearchCommand("customer_index", "Customer");
string result = connection.Post(Commands.Search("customer_index", "Customer"), jsonQuery);

jsonQuery is a param and the value passed to is

{
"query":
{
"bool":{
"should":[
{"match": {"name" : "Dave"}}
]
}
}
}

the result always returned is
{"took":2,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":0,"max_score":null,"hits":[]}}

not sure how to fix it. I tried the sample queries you mentioned using query builder but the result always same.

Could you please help me to fix it?

Regards,
Sony Arouje

Not sure if this is a bug, or if I'm doing this wrong...

If I try to create a sort script (http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-sort.html#_script_based_sorting) using this format:

intermediate.Sort(s => s.Script(
"doc['testfield'].value * testparam",
"number",
"{'testparam': 1.1}",
SortDirection.asc
));

The resulting query has incorrect quotes around the doc which means parsing failed:

"sort": [{ "_script": { "script": "doc["testfield"].value * testparam","type": "number","params": {"testparam": 1.1},"order": "asc" } }]

It seems to be changing the single quotes to double quotes and I get an error from elasticsearch about unexpected token 't'

I have to implement customized sorting option in query. What is the best method to do it, Should I go on and on like this https://gist.github.com/chanchalcode/7689829 or is there any nice way to edit just sort option and order (desc or asc) option based on request.

Some of my dependent libraries are using a higher version of Json.Net compared to what is referenced by PlainElastic.Net. Thus I am unable to PlainElastic as it does not compile.

This is due to Json.Net changing frequently. Similar issue at https://github.com/danielcrenna/tweetsharp/issues/17

Would love to see support for the highlighting API.

Delete command returns DeleteResult object with ".ok = false" when the ES actually return result like this:

HTTP/1.1 200 OK
Content-Type: application/json; charset=UTF-8
Content-Length: 21

{"acknowledged":true}

The case is when I'm trying to delete the whole index. Workaround is to check "result.acknowledged" in addition to "result.ok".

Also, this is running against elasticsearch 1.1.1.

Probably related to this: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/_return_values.html

ok has been removed from all the responses

The following code seem to produce wrong json for the ES as it is missing facet name for the facet_filter.

        Facets<ESDocument> facets = new Facets<ESDocument>();
        facets.Terms(t => t.FacetName("brand").Field("brand"));

        FacetFilter<ESDocument> filter = new FacetFilter<ESDocument>();
        filter.Range(r => r.IncludeLower(false).IncludeUpper(true).From("Lower").To("Upper"));
        facets.FacetFilter(f => filter);

        builder.Facets(f => facets);

        json = builder.ToString();

...
},
"facets": {
"brand": {
"terms": {
"field": "brand"
}
},
"facet_filter": {
"range": {
"include_lower": false,
"include_upper": true,
"from": "Lower",
"to": "Upper"
}
}
}
...

There is also no way in API to define a simple filter so it produces the json like this (not counting .Custom()):
...
"facets" : {
"brand" : { "terms" : {"field" : "brand"} },
"color" : { "terms" : {"field" : "color"} },
"price" : {
"filter" : {
"bool" :
{
"should" :
{
"range" : {
"price_usd_default" : { "to" : 100 }
}
},
...

Hello guys, I believe that the Script function of Sort.cs does not generate the right JSON.

Current:
var expression = "'_script' : {0}, 'type': {1}, 'order': {2}, 'params': {3} " ......

It should be (I think):
var expression = "'_script' : { 'script':{0}, 'type': {1}, 'order': {2}, 'params': {3} }" ......

based on http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-sort.html#_script_based_sorting.

I do not know if I missing something.
Many Thanks,
Ioannis

When I post this:

PUT /manu/dataset/_mapping
{
   "dataset": {
      "type": "object",
      "enabled": true,
      "dynamic": "strict",
      "properties": {
         "type": {
            "type": "string",
            "analyzer": "keyword"
         }
      }
   }
}

generated by

MapBuilder<Dataset>()
  .RootObject(typeName: "dataset",
    map: r => r
    .Enabled(true)
    .All(a => a.Enabled(true))
    .Custom("\"dynamic\": \"{0}\"", "strict")
    .Properties(pr => pr
    .String("type", f => f.Analyzer(DefaultAnalyzers.keyword))).Build();

ES 1.2 returns this error

org.elasticsearch.index.mapper.MapperParsingException: Root type mapping not empty after parsing! Remaining fields: [type : object]

whilst it was working well with ES 1.1

Now this works

PUT /manu/dataset/_mapping
{
   "dataset": {
      "type": "object",
      "enabled": true,
      "dynamic": "strict",
      "properties": {
         "type": {
            "type": "string",
            "analyzer": "keyword"
         }
      }
   }
}

So probably the method RootObject() should be changed to remove the "type": "object" in root object type.

I looked through the examples and didn't find any using Common Terms. http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-common-terms-query.html

Do you or will you be supporting that?

The SearchCommand can contain multiple sort parameters, however API only allows adding one sort.

We don't want to index our entire objects, because less than 10% of it is "searchable". So we take the object and build json from scratch. This worked well for us, and was faster than creating another object and serializing it. Because of that, we're left with json (which is perfect for IndexCommand). I'm curious if we could get the BulkBuilder to take this raw json for each indexable item... or if the builder.Index command could support it in lieu of the current object which it deserializes.

Several of the other modern clustered technologies we use allow for the definition of a cluster when connecting, which is then either balanced or simply failed over if a connection is lost. Is there any way this could be done with elastic connections? An array of connections ordered by priority?

This doesn't follow the contract of plain elastic, and renders everything but the query string, which obviously breaks the query.

I'm evaluating your client for my company and have found one ElasticSearch feature that's missing: the scroll endpoint (at the bottom of the page).
http://www.elasticsearch.org/guide/reference/api/search/scroll/

Ideally, this new feature would add a new ScrollCommand class that would accept a _scroll_id. In addition, SearchCommand would have to support extracting the _scroll_id field for when SearchType is set to SearchType.scan.

While I would be ecstatic if one of you pushed a fix for this, feel free to let me know if you would like my assistance. Time permitting, I would be happy to create a patch to add this support in the coming months. Thanks for taking the time to look over this request!

HI,
I am new to Plainelastic. I am very much confused that how we can put parent child indexing. I need a simple and brief example.

I was trying to use the parent method while creating the child index. I first created the parent index then passed its id to the child index as you wrote in the documentation. but it generates the error that the index specified as parent doesnot exists.

Please help me

Hello,

Could someone point me to download the PlainElastic.Net client dll for the target .NET framework 3.5 please.

trying to implement Elasticsearch in SharePoint 2010.

Regards,
Shannu.

There is no update command specified. Neither in PlainElastic.Net.Commands, not can you do a ElasticConnection.Update or something like this. If you call POST or PUT command on any object/document, it gets replaced completely, that is the _source field gets replaced with the one you specified in the POST/PUT command.

As specified in

http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/partial-updates.html

you must issue a

_update

at the end of url so that it knows that we are partially updating the document and not completely replacing it, I have found no support for this.

I am able to perform a search successfully in my project, in order to get total number of results count I am trying to access the "hits" property which is unavailable in a vb.net project but if I do the same same in "c#" I am able to access it.
C# object browser view for SearchResult

Vb.net object browser view for SearchResult(of T)

Can you please help me out.

Couldnt see anything in the driver that would support a call such as http://localhost:9200/_cluster/nodes which would return some node information for that cluster.Is this supported in the driver and how?

Please can you provide Facet support for DateHistogram queries.

I am using the plainelastic version 1.1.53.0. I am unable to use the geo_distance and date_range aggregates of elasticsearch. I am working around by using custom options instead. Though I would prefer to use their power instead of custom option.

Also I am unable to access the aggregation field after deserializing the result. I only get the option to use facets here and not aggregations.

var serializer = new JsonNetSerializer();
var facets=serializer.ToSearchResult<>(result).facets;

Problem with special characters, with you use query_string with value "16/16" for example the elastic throw exception. I look for the documentation http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-query-string-query.html#_reserved_characters

I saw QueryString source code and the value don't escape this especial characters

The JSON returned from a DELETE request looks like this (version 20.6 of ES):

{
"ok":true,
"found":true,
"_index":"test-inmemory-docrepo_2013-04-16t15-33-59-812z_es_item",
"_type":"video",
"_id":"testcusomer/testservice/video(1234)",
"_version":2
}

This currently fails deserialization to DeleteResult - at least with the Json serializer I'm using. Haven't tried it yet with Json.NET.

I'm trying to build a query with a nested filter, but the NestedFilter class doesn't have a Filter method. Am I missing something or is this an oversight?

I've pushed a commit f316055 with the function added, that seems to work in my specific query but I'm new to the ElasticSearch DSL so I don't know if that's appropriate in all cases.

Pull request with the commit: #62

It would be great to have aggregations support -
http://www.elasticsearch.org/guide/en/elasticsearch/reference/master/search-aggregations.html
http://www.elasticsearch.org/blog/1-0-0-beta2-released/#aggs

I need a sample to create a Multifield in mapping

I am having trouble implementing nested and-or filter comprising many fields. Suppose I have field 1,2,3,4,5,6 and I want to implement filter like 1 and 2 and {3 or 4 or 5} and 6. Nothing is working for this kind of situation.

There's an easy enough work-around for us where we can map old versions forward in our app.configs, but we could save all of the build warnings by either updating which newtonsoft you require to >= 5.0.3 (tons of performance updates) or by simply marking it as "Specific Version": false in the reference properties of the project.

Not sure if this is brand new with 0.90.2, but this is the version I'm now using.

The response from a search request returned the following json:

{
"took": 4,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 3,
"failed": 2,
"failures": [
{
"index": "test-inmemory-093d8fe9-rule",
"shard": 3,
"status": 503,
"reason": "No active shards"
},
{
"index": "test-inmemory-093d8fe9-rule",
"shard": 4,
"status": 503,
"reason": "No active shards"
}
]
},
"hits": {
"total": 1,
"max_score": 1,
"hits": [
{
"_index": "test-inmemory-093d8fe9-rule",
"_type": "rule",
"_id": "rule1",
"_score": 1,
"_source": {
"_id": "rule1",
"Name": "testme"
}
}
]
}
}

So obviously my cluster isn't healthy, but I get a deserialization error as ShardsResult doesn't have a corresponding 'failures' property in it.

Just need to rename it to OperationException

Hi,

I think it could be a good idea if multi_match were integrated to the library. It's a very usefull feature of Elasticsearch and I think it's bad it's not (yet) in the library.

Thanks,
Loïc Wenkin

Please provide the constructor for a connection URL which has no port.
Actually we have configured elasticsearch on EC2 instance and our engineer gave us the url http://50.16.247.185/es now how we can put this url to ElasticConnection's constructor?
We need to set a port less URL

elastic/elasticsearch#1558

Need to add an option to Sort class.

        var query = new QueryBuilder<ElasticName>()
            .Query(q => q
                .Filtered(fd => fd
                    .Filter(f => f
                       .Missing(m => m.Field(col => col.first_name))
                    )
                )
            )
            .BuildBeautified();          

Returns an empty string. Same with Exists in place of Missing. Current workaround:

        var query = new QueryBuilder<ElasticName>()
            .Query(q => q
                .Filtered(fd => fd
                    .Filter(f => f
                        .Custom(@"{""exists"": {""field"": ""first_name""}}")
                    )
                )
            )
            .BuildBeautified();

It seems like Missing and Exists result in an empty query string wherever they're placed. I tried a constantScore function with those functions nested and had the same issue.

var isb = new IndexSettingsBuilder().CustomPart("test", "abc");

gives:

{
    "index": {
        test
    }
}

expected:

{
    "index": {
        "test": "abc"
    }
}

I get this when i ran this source code,
Step 1- implemented mongo db
Step 2- Installed Elastic Search
Step 3-Installed Plugin MongoAttached & Mongo river.....Note-I Don't whether it is Installed

            .Facets(facets => facets
                .Terms(t => t
                    .FacetName("NamesPerDatabasePerCategory")
                    .Fields("category.raw", "database_name.raw")
                )
            )

Results in a query without a Facets section. Replacing the argument in Fields with an array, or proper Funcs produces the same error (facet-less query).

Like my other issue, workaround using .Custom in place now. Would also much prefer the power of Aggregations to be adopted in PlainElastic

ES no longer returns the ok bool in responses, instead the success or failure of the response is depicted by the status code / status entry in bulk results.

Without this functionality the library is largely unusable as we can no longer check for command success.

Could you please provide an example on how to make a range facet with QueryBuilder? I was able to use the QueryBuilder with regular term facets, but I'm having trouble figuring out how to add the Range facets. Here's what I have so far:

var query = new QueryBuilder<ElasticCollection>()
            .Query(q => q
                .MatchAll()
            )
            .Facets(facets => facets
                .Range(r => r.FacetName("startRange").Field("start_year").Ranges())

but I'm stuck on how to specify the ranges in this format.

Also, is there a way to have multiple fields in the range facet, so that I can get a count of all items that either field A or field B within the range?

I ran into an issue today which is on a kind of unlucky use case. To reduce concern, with this issue I did not experience any data loss.

When using the following snippet of code

 var bulkCommand = new BulkCommand(searchableIndex, typedEntities.Key);
 var bulkJsons =
                    new BulkBuilder(serializer)
                    .PipelineCollection(entities, (builder, entity) => builder.Index(entity, id: Utility.GenerateId(entity)))
                    .JoinInBatches(batchSize: 100);

The following error is thrown when entities.length is greater than batchSize and entities.length MOD batchSize equals 0. So for this case if there are 1000 entities this issue occurs, but 1001 is fine.

PlainElastic.Net.OperationException: {"error":"Failed to de
rive xcontent from org.elasticsearch.common.bytes.BytesArray@0"}

The reason for this, is I suspect that the Pipeline is providing an Empty array as the last list to come out of its enumerable.

As a side bar, thank you for maintaining a really excellent library. Up until now its done everything I could want and been really effective at doing it.

Recently, I have had several issues connected to exceeding request timeout:

Failure to complete bulk indexing operation. at ElasticSearchEngine.EsIndexBuilder.BulkIndexRequest(String requestContent, String documentType)
at ElasticSearchEngine.EsIndexBuilder.IndexDatabaseTable(Int32 tableIndex, IDataReader reader, String messageSuffix, Int32 messageModule)
at ElasticSearchEngine.EsIndexBuilder.IndexData()
The operation has timed out.. Stack trace: at PlainElastic.Net.ElasticConnection.ExecuteRequest(String method, String command, String jsonData)
at PlainElastic.Net.ElasticConnection.Put(String command, String jsonData)
at ElasticSearchEngine.EsIndexBuilder.BulkIndexRequest(String requestContent, String documentType)
The operation has timed out.. Stack trace: at System.Net.ConnectStream.Read(Byte[] buffer, Int32 offset, Int32 size)
at System.IO.StreamReader.ReadBuffer()
at System.IO.StreamReader.ReadToEnd()
at PlainElastic.Net.ElasticConnection.ExecuteRequest(String method, String command, String jsonData)

After looking closely into the code, I have discovered that there is no way to set the HttpWebRequest timeout, which is default to 60 sec. Could it be possible to specify timeout when creating ElasticConnection object?

Hi,

First of all, nice project, well rounded. You work helped me a lot on my dev around ES.

Secondly, I would like to know if you are planning on an implementation of the UPDATE operation on the bulk API call ? This feature would be very helpful for me.

Here the quote from ElasticSearch website:

The possible actions are index, create, delete and update. index and create expect a source on the
next line, and have the same semantics as the op_type parameter to the standard index API
(i.e. create will fail if a document with the same index and type exists already, whereas index will
add or replace a document as necessary). delete does not expect a source on the following line,
and has the same semantics as the standard delete API. update expects that the partial doc,
upsert and script and its options are specified on the next line.
-- http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/docs-bulk.html

Thanks,
Benjamin

I can't seem to find support for the Geo Distance Range filter. Is this implemented?

http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-geo-distance-range-filter.html#query-dsl-geo-distance-range-filter