Code Monkey home page Code Monkey logo

Comments (3)

keif888 avatar keif888 commented on June 2, 2024

It is possible, and involves changes to around 6 functions, creation of a couple, and form modifications.
And then the regression testing to ensure nothing is broken.

I understand the use case of attempting to control the amount of CPU that the hashing will consume.

FYI.
Currently Auto will use cores - 1, IF there are more than 5 output columns in the component.
If there are less than 5 output columns, then the overhead of multiple threads exceeds the gain's that it provides.

from ssismhash.

matsremman avatar matsremman commented on June 2, 2024

Wait, so - output columns, not the number of input columns to hash.. I've misunderstood this then - if so then a single thread is just fine, as I'm just using 1 column per component anyways.

from ssismhash.

keif888 avatar keif888 commented on June 2, 2024

If you have a wide input (100 odd columns for example), and are generating 15 hash output columns (for example) from that, then On and Auto will improve the hashing performance. This is because 15 byte arrays are being created, and then hashed, for each input row. That can be done in parallel, to improve performance (a lot if you have enough cores).

If you have all of those columns into a single hash, then Auto will choose single thread, as it has to put all the columns into a single byte array and then hash that result.

The component has to work row by row, as it's a sync non blocking component.

It sound's like you don't need the enhancement.
And that I might need to improve the documentation on how Auto and On help performance...

from ssismhash.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.