Comments (1)
lhoss
commented on June 2, 2024
We have the same issue, even before using Nifi 2.x , because of our custom nifi build version ('1.19.1-p2'), which also contains not just simple sem-ver numbers :
Would be great to have this fixed and allow (almost) any version strings
Full stacktrace , using Datahub 0.13.1 (CLI v 0.13.1.2):
"cli": {
"cli_version": "0.13.1.2",
"cli_entry_location": "/tmp/datahub/ingest/venv-nifi-2b9c1ab97dc6cd7f/lib/python3.10/site-packages/datahub/__init__.py",
...
[2024-05-14 13:47:57,852] ERROR {datahub.entrypoints:201} - Command failed: Invalid version: '1.19.1-p2'
Traceback (most recent call last):
File "/tmp/datahub/ingest/venv-nifi-2b9c1ab97dc6cd7f/lib/python3.10/site-packages/datahub/entrypoints.py", line 188, in main
sys.exit(datahub(standalone_mode=False, **kwargs))
File "/tmp/datahub/ingest/venv-nifi-2b9c1ab97dc6cd7f/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
return self.main(*args, **kwargs)
File "/tmp/datahub/ingest/venv-nifi-2b9c1ab97dc6cd7f/lib/python3.10/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
File "/tmp/datahub/ingest/venv-nifi-2b9c1ab97dc6cd7f/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/tmp/datahub/ingest/venv-nifi-2b9c1ab97dc6cd7f/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/tmp/datahub/ingest/venv-nifi-2b9c1ab97dc6cd7f/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/tmp/datahub/ingest/venv-nifi-2b9c1ab97dc6cd7f/lib/python3.10/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "/tmp/datahub/ingest/venv-nifi-2b9c1ab97dc6cd7f/lib/python3.10/site-packages/datahub/telemetry/telemetry.py", line 454, in wrapper
raise e
File "/tmp/datahub/ingest/venv-nifi-2b9c1ab97dc6cd7f/lib/python3.10/site-packages/datahub/telemetry/telemetry.py", line 403, in wrapper
res = func(*args, **kwargs)
File "/tmp/datahub/ingest/venv-nifi-2b9c1ab97dc6cd7f/lib/python3.10/site-packages/datahub/cli/ingest_cli.py", line 201, in run
ret = loop.run_until_complete(run_ingestion_and_check_upgrade())
File "/usr/local/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
return future.result()
File "/tmp/datahub/ingest/venv-nifi-2b9c1ab97dc6cd7f/lib/python3.10/site-packages/datahub/cli/ingest_cli.py", line 185, in run_ingestion_and_check_upgrade
ret = await ingestion_future
File "/tmp/datahub/ingest/venv-nifi-2b9c1ab97dc6cd7f/lib/python3.10/site-packages/datahub/cli/ingest_cli.py", line 139, in run_pipeline_to_completion
raise e
File "/tmp/datahub/ingest/venv-nifi-2b9c1ab97dc6cd7f/lib/python3.10/site-packages/datahub/cli/ingest_cli.py", line 131, in run_pipeline_to_completion
pipeline.run()
File "/tmp/datahub/ingest/venv-nifi-2b9c1ab97dc6cd7f/lib/python3.10/site-packages/datahub/ingestion/run/pipeline.py", line 405, in run
for wu in itertools.islice(
File "/tmp/datahub/ingest/venv-nifi-2b9c1ab97dc6cd7f/lib/python3.10/site-packages/datahub/ingestion/api/source_helpers.py", line 171, in auto_workunit_reporter
for wu in stream:
File "/tmp/datahub/ingest/venv-nifi-2b9c1ab97dc6cd7f/lib/python3.10/site-packages/datahub/ingestion/api/source_helpers.py", line 253, in auto_browse_path_v2
for urn, batch in _batch_workunits_by_urn(stream):
File "/tmp/datahub/ingest/venv-nifi-2b9c1ab97dc6cd7f/lib/python3.10/site-packages/datahub/ingestion/api/source_helpers.py", line 391, in _batch_workunits_by_urn
for wu in stream:
File "/tmp/datahub/ingest/venv-nifi-2b9c1ab97dc6cd7f/lib/python3.10/site-packages/datahub/ingestion/api/source_helpers.py", line 184, in auto_materialize_referenced_tags
for wu in stream:
File "/tmp/datahub/ingest/venv-nifi-2b9c1ab97dc6cd7f/lib/python3.10/site-packages/datahub/ingestion/api/source_helpers.py", line 91, in auto_status_aspect
for wu in stream:
File "/tmp/datahub/ingest/venv-nifi-2b9c1ab97dc6cd7f/lib/python3.10/site-packages/datahub/ingestion/source/nifi.py", line 1048, in get_workunits_internal
self.process_provenance_events()
File "/tmp/datahub/ingest/venv-nifi-2b9c1ab97dc6cd7f/lib/python3.10/site-packages/datahub/ingestion/source/nifi.py", line 966, in process_provenance_events
for event in events:
File "/tmp/datahub/ingest/venv-nifi-2b9c1ab97dc6cd7f/lib/python3.10/site-packages/datahub/ingestion/source/nifi.py", line 674, in fetch_provenance_events
provenance_response = self.submit_provenance_query(
File "/tmp/datahub/ingest/venv-nifi-2b9c1ab97dc6cd7f/lib/python3.10/site-packages/datahub/ingestion/source/nifi.py", line 734, in submit_provenance_query
older_version: bool = self.nifi_flow.version is not None and version.parse(
File "/tmp/datahub/ingest/venv-nifi-2b9c1ab97dc6cd7f/lib/python3.10/site-packages/packaging/version.py", line 54, in parse
return Version(version)
File "/tmp/datahub/ingest/venv-nifi-2b9c1ab97dc6cd7f/lib/python3.10/site-packages/packaging/version.py", line 200, in __init__
raise InvalidVersion(f"Invalid version: '{version}'")
packaging.version.InvalidVersion: Invalid version: '1.19.1-p2'
from datahub.
Related Issues (20)
- Error when running protobuf as IDE with master branch.
- Quoted searches with underscores return non-exact matches HOT 1
- dbt cloud ingestion failing HOT 11
- The confluent images are not a configureable part of the compose file
- Superset ingestor is inefficient to ingest large amount of data HOT 1
- DataProcessInstance delete doesn't work (crash)
- Add SASL_SSL Support to Kafka Setup in Docker HOT 1
- When ingesting from multiple source with siblings, all siblings should be visible under 1 entity.
- datahub-upgrade process aborts after 8 hours.
- After connecting airflow, the platform of trino operator's in/outlet is changed to presto. HOT 2
- Ingestion for superset failed on LDAP HOT 1
- Ingestion from UI runs indefinetely
- Ingestion via UI of BigQuery metadata HOT 1
- Unable to restrict content of a dataset using policies HOT 1
- Data Product owner type in Datahub CLI read incorrectly
- Does DataHub support column-level lineage from Kafka contract to tables?
- I found that when searching for assets globally on the homepage, I tried to delete the owner information in bulk but it was not deleted
- Neo4j non exists function
- Users with Admin role cannot view Access Tokens
- Ownership info is missing for independent looks in Looker
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from datahub.