Code Monkey home page Code Monkey logo

Comments (6)

jiminoc avatar jiminoc commented on August 18, 2024

Looks like in DoCommand a bulk request will give a 200 status code and return the errors that came back from the bulk request so this code doesn't take that into consideration

httpStatusCode, body, err = req.Do(&response)

    if err != nil {
        return body, err
    }
    if httpStatusCode > 304 {

from elastigo.

jiminoc avatar jiminoc commented on August 18, 2024

I have an update to the BulkSend function which gets errors put on the channel. Example code below, let me know if you want it as a 2.0 pull request. (I'd move the struct out of the function, just there for brevity of code example).

The idea is that the bulk response is 200 OK but contains a JSON object that indicates how many errors that were returned. You have to unmarshall the JSON blob and check to see if the errors == true. This correctly now puts an error message on the ErrorsChannel.

// This does the actual send of a buffer, which has already been formatted
// into bytes of ES formatted bulk data
func BulkSend(buf *bytes.Buffer) error {

    type responseStruct struct {
        Took   int64                    `json:"took"`
        Errors bool                     `json:"errors"`
        Items  []map[string]interface{} `json:"items"`
    }

    response := responseStruct{}

    body, err := api.DoCommand("POST", "/_bulk", nil, buf)

    if err != nil {
        BulkErrorCt += 1
        return err
    }
    // check for response errors, bulk insert will give 200 OK but then include errors in response
    jsonErr := json.Unmarshal(body, &response)
    if jsonErr == nil {
        if response.Errors {
            BulkErrorCt += uint64(len(response.Items))
            return fmt.Errorf("Bulk Insertion Error. Failed item count [%d]", len(response.Items))
        }
    }
    return nil
}

from elastigo.

schleppy avatar schleppy commented on August 18, 2024

You might consider using

return fmt.Errorf("Bulk Insertion Error... [%d]", len(response.Items))

from elastigo.

jiminoc avatar jiminoc commented on August 18, 2024

updated based on schleppy's comment

from elastigo.

jiminoc avatar jiminoc commented on August 18, 2024

@mattbaird looks like you added the code example in commit:
dbd553d

want me to close this issue?

from elastigo.

mattbaird avatar mattbaird commented on August 18, 2024

yes please.

from elastigo.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.