Comments (6)
@greut you probably want "very liberal MARC" see #26
from dojson.
@greut I don't think we should remove unknown fields, actually this means that the translation for your data model is incomplete, therefore your tests should fail.
I think the question here is whether or not the liberal MARC21 implementation should add to the output JSON all the unknown fields. IMHO it should.
Just a side note, AFAIK the subfield $9
is used as a custom subfield, like 9xx
or 69x
, so in theory the definition of this subfield should go in your DoJSON package. We have a couple of cases like this now on CDS, i.e. https://github.com/CERNDocumentServer/cds-dojson/blob/master/cds_dojson/marc21/fields/default/bd7xx.py#L85
from dojson.
Thanks @egabancho, lemme invoke the MARC lord @blixhavn. The silent removal looks like a way to be in trouble.
from dojson.
Similarly to @egabancho I think the test should fail in this case (and not drop the field silently), because the input record is not compatible with the desired schema.
There are basically two solutions: either (1) stick to standard MARC that does not seem to allow $9
in that field(?), or (2) amend wanted schema to allow the presence of $9
, going towards as "liberal" MARC schema as the given library instance needs.
@aw-bib can advise about whether $9
has any specific convention meaning across multiple fields in the MARC standard?
from dojson.
@tiborsimko how easy could be to amend/edit the schema for a particular purposes. E.g. the 9xx
fields that are used internally by invenio.
If you provide a liberal MARC, people will use this one because loosing data is bit of a no-go.
from dojson.
The best example for 9xx so far is @egabancho's customisations for CERN-specific 9xx fields. I fully agree with you that loosing data is not acceptable, hence we should return an error, not silently ignore those "extra" fields. The error will hopefully prompt the site admins to amend the local schema rules to fit local cataloguing practices... or else switch to a liberal schema iike #26 that is very permissive on the input side. (At the price of loosing some precise JSON-Schema-generated editing forms.) Many sites have been calling for more permissive schema, see long discussion in #23, so perhaps we should get to implementing it sooner rather than later?
from dojson.
Related Issues (20)
- global: incompatible change released as patch version HOT 3
- Extreme CPU usage after updating to versions later than 1.0.1 HOT 10
- When reporting missing rules, pass original blob to exception handler
- GroupableOrderedDict riases KeyError on empty values
- utils: force_list returns a tuple HOT 5
- utils: filter_values() is not GroupableOrderedDict aware
- utils: eval(repr(GroupableOrderedDict(...)) != GroupableOrderedDict(...)
- filter away empty fields/subfields after input HOT 6
- utils: deepcopy() doesn't work on GroupableOrderedDict
- RFC Passing original blob to rules HOT 5
- Post-do functions HOT 6
- 044 (NR) is treated as repeatable
- Duplicate fields in bibliographic MARC21 json schemas
- RFC is 'Undo' still experimental HOT 3
- global: make new release
- utils.marc21.create_record does not support partial records HOT 1
- tests: remove python 2.7 support and fix broken tests
- release 1.4.1
- global: migration to github-actions from travis
- Fix Pycodestyle and Pydocstyle errors
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dojson.