Code Monkey home page Code Monkey logo

Comments (3)

glennmatthews avatar glennmatthews commented on September 24, 2024

Probably the fix will be something like recursing through the dict returned by self.diff.dict() and explicitly converting all values to str. Alternately, it might be desirable to enhance DiffSync's Diff.dict() method to take a parameter like json_safe_values=True which would force it to do the appropriate conversions at the time the dict is initially constructed?

from nautobot-app-ssot.

jakubkrysl avatar jakubkrysl commented on September 24, 2024

I hit similar issue with the same cause when using UUID for RelationshipAssociation.

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/nautobot_ssot/jobs/base.py", line 327, in run
    self.sync_data()
  File "/usr/local/lib/python3.10/site-packages/nautobot_ssot/jobs/base.py", line 165, in sync_data
    self.execute_sync()
  File "/usr/local/lib/python3.10/site-packages/nautobot_ssot/jobs/base.py", line 93, in execute_sync
    self.source_adapter.sync_to(self.target_adapter, flags=self.diffsync_flags)
  File "/usr/local/lib/python3.10/site-packages/diffsync/__init__.py", line 546, in sync_to
    target.sync_from(self, diff_class=diff_class, flags=flags, callback=callback, diff=diff)
  File "/usr/local/lib/python3.10/site-packages/diffsync/__init__.py", line 524, in sync_from
    result = syncer.perform_sync()
  File "/usr/local/lib/python3.10/site-packages/diffsync/helpers.py", line 326, in perform_sync
    changed |= self.sync_diff_element(element)
  File "/usr/local/lib/python3.10/site-packages/diffsync/helpers.py", line 366, in sync_diff_element
    changed, modified_model = self.sync_model(src_model=src_model, dst_model=dst_model, ids=ids, attrs=attrs)
  File "/usr/local/lib/python3.10/site-packages/diffsync/helpers.py", line 441, in sync_model
    self.log_sync_status(self.action, status, message)
  File "/usr/local/lib/python3.10/site-packages/diffsync/helpers.py", line 454, in log_sync_status
    self.logger.info(message, status=status.value)
  File "/usr/local/lib/python3.10/site-packages/structlog/stdlib.py", line 148, in info
    return self._proxy_to_logger("info", event, *args, **kw)
  File "/usr/local/lib/python3.10/site-packages/structlog/stdlib.py", line 213, in _proxy_to_logger
    return super()._proxy_to_logger(method_name, event=event, **event_kw)
  File "/usr/local/lib/python3.10/site-packages/structlog/_base.py", line 203, in _proxy_to_logger
    args, kw = self._process_event(method_name, event, event_kw)
  File "/usr/local/lib/python3.10/site-packages/structlog/_base.py", line 160, in _process_event
    event_dict = proc(self._logger, method_name, event_dict)
  File "/usr/local/lib/python3.10/site-packages/nautobot_ssot/jobs/base.py", line 234, in _structlog_to_sync_log_entry
    self.sync_log(
  File "/usr/local/lib/python3.10/site-packages/nautobot_ssot/jobs/base.py", line 215, in sync_log
    SyncLogEntry.objects.create(
  File "/usr/local/lib/python3.10/site-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/django/db/models/query.py", line 453, in create
    obj.save(force_insert=True, using=self.db)
  File "/usr/local/lib/python3.10/site-packages/django/db/models/base.py", line 739, in save
    self.save_base(using=using, force_insert=force_insert,
  File "/usr/local/lib/python3.10/site-packages/django/db/models/base.py", line 776, in save_base
    updated = self._save_table(
  File "/usr/local/lib/python3.10/site-packages/django/db/models/base.py", line 881, in _save_table
    results = self._do_insert(cls._base_manager, using, fields, returning_fields, raw)
  File "/usr/local/lib/python3.10/site-packages/django/db/models/base.py", line 919, in _do_insert
    return manager._insert(
  File "/usr/local/lib/python3.10/site-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/django/db/models/query.py", line 1270, in _insert
    return query.get_compiler(using=using).execute_sql(returning_fields)
  File "/usr/local/lib/python3.10/site-packages/django/db/models/sql/compiler.py", line 1415, in execute_sql
    for sql, params in self.as_sql():
  File "/usr/local/lib/python3.10/site-packages/django/db/models/sql/compiler.py", line 1358, in as_sql
    value_rows = [
  File "/usr/local/lib/python3.10/site-packages/django/db/models/sql/compiler.py", line 1359, in 
    [self.prepare_value(field, self.pre_save_val(field, obj)) for field in fields]
  File "/usr/local/lib/python3.10/site-packages/django/db/models/sql/compiler.py", line 1359, in 
    [self.prepare_value(field, self.pre_save_val(field, obj)) for field in fields]
  File "/usr/local/lib/python3.10/site-packages/django/db/models/sql/compiler.py", line 1300, in prepare_value
    value = field.get_db_prep_save(value, connection=self.connection)
  File "/usr/local/lib/python3.10/site-packages/django/db/models/fields/__init__.py", line 842, in get_db_prep_save
    return self.get_db_prep_value(value, connection=connection, prepared=False)
  File "/usr/local/lib/python3.10/site-packages/django/db/models/fields/__init__.py", line 837, in get_db_prep_value
    value = self.get_prep_value(value)
  File "/usr/local/lib/python3.10/site-packages/django/db/models/fields/json.py", line 93, in get_prep_value
    return json.dumps(value, cls=self.encoder)
  File "/usr/local/lib/python3.10/json/__init__.py", line 231, in dumps
    return _default_encoder.encode(obj)
  File "/usr/local/lib/python3.10/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/local/lib/python3.10/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/usr/local/lib/python3.10/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type UUID is not JSON serializable

I do not have the UUID in attrs defined in base.py, instead I add it during update based on diff data, because data source has no access to Nautobot IDs.

The thing is RelationshipAssociation requires either UUID or the Nautobot model itself (which results in TypeError too), so the only way to have RelationshipAssociation in SSoT is to process given data after validated_save() to convert any value to string.

from nautobot-app-ssot.

jmcgill298 avatar jmcgill298 commented on September 24, 2024

I believe that #140 will also address this:

In [1]: sync = Sync.objects.last()

In [2]: import datetime

In [3]: sync.diff
Out[3]: 
{'site': {'AMS01': {'+': {'slug': 'ams01',
    'description': '',
    'region_name': 'Netherlands',
    'status_slug': 'active'}}}}

In [4]: sync.diff["data"] = datetime.date.today()

In [5]: sync.validated_save()

In [6]: sync.diff
Out[6]: 
{'site': {'AMS01': {'+': {'slug': 'ams01',
    'description': '',
    'region_name': 'Netherlands',
    'status_slug': 'active'}}},
 'data': datetime.date(2023, 6, 27)}

In [7]: sync.refresh_from_db()

In [8]: sync.diff
Out[8]: 
{'site': {'AMS01': {'+': {'slug': 'ams01',
    'description': '',
    'region_name': 'Netherlands',
    'status_slug': 'active'}}},
 'data': '2023-06-27'}

from nautobot-app-ssot.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.