verdan / pyatlasclient Goto Github PK

View Code? Open in Web Editor NEW

This project forked from jpoullet2000/atlasclient

6.0 1.0 5.0 256 KB

Apache Atlas Python Client

Home Page: https://pyatlasclient.readthedocs.io/en/latest/

License: Apache License 2.0

Python 99.70% Batchfile 0.11% Shell 0.19%

atlas apache-atlas atlasclient pyatlasclient atlas-python

pyatlasclient's Introduction

This repository is no longer maintained in favour of Official Apache Atlas' python client: https://pypi.org/project/apache-atlas/

Apache Atlas Client in Python

Apache Atlas client in Python. Only compatible with Apache Atlas REST API v2.

Based on the awesome work done by Poullet in atlasclient

Free software: Apache Software License 2.0
Documentation: https://pyatlasclient.readthedocs.io

Get started

    from atlasclient.client import Atlas
    client = Atlas('<atlas.host>', port=21000, username='admin', password='admin')
    client.entity_guid("<guid>").status
    params = {'typeName': 'DataSet', 'attrName': 'name', 'attrValue': 'data', 'offset': '1', 'limit':'10'}
    search_results = client.search_attribute(**params) 
    for s in search_results:
        for e in s.entities:
            print(e.name)
            print(e.guid)

Features

Lazy loading: requests are only performed when data are required and not yet available
Leverages Python's Data Classes for Glossary.
Resource object relationships: REST API from sub-resources are done transparently for the user, for instance the user does not have to know that it needs to trigger a different REST request for getting the classifications of a specific entity.

TODO features

allow multiprocessing
Implement Caching
Apply Data Classes to all entity types. For now only Glossary endpoints are using it.

Credits

This package was created with Cookiecutter and the audreyr/cookiecutter-pypackage project template.

pyatlasclient's People

Contributors

Stargazers

Watchers

Forkers

bolkedebruin mgorsk1 0x4ec7 dechoma pdefusco

pyatlasclient's Issues

entity_bulk drops classificationNames

I have setup dev Atlas instance with ~20 table entities. Every entity has attached single classification. When using basic_search every entity contains correct classificationNames attribute (single entry list). When using subsequent entity_bulk classificationNames is empty (for the same entity).

results = self.driver.search_basic(**full_params)

search_chunk = extract_entities(results)
print(search_chunk[0].classificationNames)  # every entry in search_chunk has classificationNames filled out

if relationships:
    guids = [table.guid for table in search_chunk]

    if guids:
        bulk_collection = self.driver.entity_bulk(guid=guids, ignoreRelationships=False)

        for collection in bulk_collection:
            search_chunk = list(collection.entities())
            classifiations = search_chunk[0].classificationNames

            if classifiations:
                print(classifiations)
            else:
                print('classifications empty')

below results of print statements

['storage_node']
classifications empty

Relationship method does not work

I have been trying out client.relationship.create(data=entity_def) and it is not sending the correct payload.

i followed the layout based on
https://pyatlasclient.readthedocs.io/en/latest/usage.html#create-a-new-relationship-between-entities

in my logs i see this as the request body
DEBUG:atlasclient.client:Request body: {"relationship": {}, "data": {"typeName": "...", "end1": {"typeName": "...", "uniqueAttributes": {"qualifiedName": "..."}}, "end2": {"typeName": "...", "uniqueAttributes": {"qualifiedName": "..."}}, "attributes": {"displayRank": 1}}}

Where as it should have just put everything inside "data" as the payload itself, not under a "data" json key.

I went into the library and updated the create method to directly put whatever the data parameter is, and it successfully worked. I think both create and update are incorrect in this regard.

atlas client does not support oidc authentication

Hello to all contributors, nice job.

I'd like to use this client to write a little hook for my local atlas. But my atlas uses oidc token to authenticate users.

But your atlas client only supports login password authentication. Is there any way to integrate the oidc auth in your next release?

Flaky test

tests/test_models.py::TestDiscoveryREST::test_search_attribute_get can fail when executing with pytest --randomly-seed=1234.

base url in glossary objects may be permanently overwritten

data = {
    "qualifiedName": "DummyGlossary",
    "name": "DummyGlossary",
    "shortDescription": "Short Description of Glossary 'DummyGlossary'",
    "longDescription": "Long Description of Glossary 'DummyGlossary",
}

try:
    terms = driver.glossary.fetch_terms(glossary_guid='nonexistingguid')
except:
    pass
glossary_entity = driver.glossary.create(**data)

method driver.glossary.fetch_terms changes permanently url and this lead to 500 error
HTTP request failed for POST http://localhost:21000/api/atlas/v2/glossary/nonexistingguid/terms: Internal Server Error 500:

Traceback (most recent call last):
  File "/Users/dominik/projects/ing/atlas_defs/test.py", line 18, in <module>
    glossary_entity = driver.glossary.create(**data)
  File "/Users/dominik/projects/ing/atlas_defs/venv/lib/python3.8/site-packages/atlasclient/base.py", line 303, in create
    model.create(**kwargs)
  File "/Users/dominik/projects/ing/atlas_defs/venv/lib/python3.8/site-packages/atlasclient/events.py", line 28, in replacement
    return method(self, *args, **kwargs)
  File "/Users/dominik/projects/ing/atlas_defs/venv/lib/python3.8/site-packages/atlasclient/base.py", line 716, in create
    self.load(self.client.post(self.url, data=data))
  File "/Users/dominik/projects/ing/atlas_defs/venv/lib/python3.8/site-packages/atlasclient/client.py", line 170, in request
    handle_response(response)
  File "/Users/dominik/projects/ing/atlas_defs/venv/lib/python3.8/site-packages/atlasclient/exceptions.py", line 210, in handle_response
    raise cls(**kwargs)
atlasclient.exceptions.ServerError: HTTP request failed for POST http://localhost:21000/api/atlas/v2/glossary/nonexistingguid/terms: Internal Server Error 500: There was an error processing your request. It has been logged (ID 9e9b5150143d521a).