Code Monkey home page Code Monkey logo

senzing-elixir's Introduction

Senzing Elixir NIF

Main Branch Module Version Total Download License Last Updated Coverage Status

Elixir NIF for Senzing© Entity Resolution API.


Sustema Logo

This library was developed and is provided at no cost by Sustema AG, a company dedicated to fostering innovation and supporting the open-source community.


Senzing Entity Resolution API

Senzing software makes it easy and affordable to add advanced entity resolution capabilities to your enterprise systems and commercial applications.

The Senzing API provides highly accurate data matching and relationship detection to improve analytics, insights and outcomes with no entity resolution experts required.

You can be up and running in minutes and deploy into production in weeks.

https://senzing.com/entity-resolution-buyers-guide/

Installation

To be able to run this package, Senzing has to be installed by following the Linux setup guide: https://docs.senzing.com/quickstart/quickstart_api/

Especially make sure to load all environment variables as described in the Configure Environment section.

The package can be installed by adding senzing to your list of dependencies in mix.exs:

def deps do
  [
    {:senzing, "~> 0.1.0"}
  ]
end

To start the engine, add the following worker to your Application or Supervisor:

{Senzing.G2.ResourceInit, mod: Senzing.G2.Engine, prime: true}

Dirty IO Nif

Senzing calls take more time than the BEAM schedulers like. They are therefore run on Dirty IO schedulers. It is recommended to raise the amount of schedulers so that more calls can happen in parallel. Set the following option to start 50 instead of 10 dirty IO schedulers: +SDio 50.

Docs

senzing-elixir's People

Contributors

maennchen avatar dependabot[bot] avatar

Stargazers

avatar

Watchers

avatar

senzing-elixir's Issues

Update Error Codes

The error codes in sdk/python/senzing/G2Exception.py (line 132 onwards) are more up to date than the help article currently used.

Approach Senzing about Feature / Attribute Management

To be able to cleanly provide #3, we need functions to manage attributes & features. G2Config currently does not expose any features related to that. The feature is present in G2ConfigTool, but seems to manipulate the config json by hand.

Fix CI Object Linker

Error:

Error: 21:58:00.556 [error] loading module Elixir.Senzing.G2.Config.Nif {:load_failed, ~c"Failed to load NIF library /home/runner/work/senzing-elixir/senzing-elixir/_build/dev/lib/senzing/priv/lib/Elixir.Senzing.G2.Config.Nif: 'libG2.so: cannot open shared object file: No such file or directory'"}

build.zig:

  // ...
  lib.addObjectFile(.{ .path = "/opt/senzing/g2/lib/libG2.so" });
}

/opt/senzing/g2/lib/libG2.so exists as expected.

Build works locally, only CI seems to have trouble.

Use Go Helpers for Nif

In sdk/c/gohelpers/golang_helpers.h are some new headers that are simpler to work with than the C ones. (resize_pointer / free Et. al.)

Properly Test Redo Processing Functionality

I don't know how to trigger a redo process efficiently. Therefore the functionality is tested by creating a large number of records. This should be replaced with a more direct & faster way to trigger redo processes.

Extracted from #8

Telemetry Integration

  • Track Calls including duration
  • Create Telemetry Poller based matrics for Engine.stats/0.

Implement API surface

  • G2ConfigMgr
    • init
    • addConfig
    • getConfigList
    • getConfig
    • getDefaultConfigID
    • setDefaultConfigID
    • replaceDefaultConfigID
    • Cleanup
      • destroy
  • G2Config
    • Configuration object management
      • init
      • create
      • save
      • load
    • Datasource management
      • listDataSources
      • addDataSource
      • deleteDataSource
    • Configuration object cleanup
      • close
      • destroy
  • G2Engine
    • Initialization
      • init
      • initWithConfigID
      • reinit
      • destroy
      • primeEngine
      • getActiveConfigID
      • exportConfig
      • getRepositoryLastModifiedTime
    • Flags - Generic Flag Handling
    • Adding Records
      • addRecord
      • addRecordWithInfo
      • addRecordWithReturnedRecordID
      • addRecordWithInfoWithReturnedRecordID
    • Replacing Records
      • replaceRecord
      • replaceRecordWithInfo
    • Reevaluating
      • reevaluateRecord
      • reevaluateRecordWithInfo
      • reevaluateEntity
      • reevaluateEntityWithInfo
    • Redo Processing
      • countRedoRecords
      • getRedoRecord
      • process
      • processWithInfo
      • processRedoRecord
      • processRedoRecordWithInfo
    • Deleting Records
      • deleteRecord
      • deleteRecordWithInfo
    • Getting Entities and Records
      • getRecord
      • getEntityByRecordID
      • getEntityByEntityID
      • getVirtualEntityByRecordID
    • Searching for Entities
      • searchByAttributes
      • searchByAttributesV3
    • Finding Paths
      • findPathByEntityID
      • findPathByRecordID
      • findPathExcludingByEntityID
      • findPathExcludingByRecordID
      • findPathIncludingSourceByEntityID
      • findPathIncludingSourceByRecordID
    • Finding Networks
      • findNetworkByEntityID
      • findNetworkByRecordID
    • Why
      • whyRecords
      • whyEntityByRecordID
      • whyEntityByEntityID
    • Why (not)
      • whyEntities
    • How
      • howEntityByEntityID
    • Reporting
      • exportCSVEntityReport
      • exportJSONEntityReport
      • fetchNext
      • closeExport
    • Cleanup
      • purgeRepository
    • Statistics
      • stats

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.