What I did now is just adding markdown-files with some automatically parsable structure (hopefully) to cldf in the folder ontology. The question is: are there any style guides for this kind of endeavour? We should define some relations, etc., of which I am aware, but I don't know how to code them in a most principled way, etc.

I am still somewhat confused what to make of multiple comma-separated forms in the word list Value column. Let's say my source says

[de] Uncle: Onkel, Ohm/Oheim

What would be the natural way to put an entry like that in CLDF, where some forms are obvious variants of each other and other forms are not obviously related?

(1)

ID,Language_ID,Parameter_ID,Value,Form
1,stan1295,1327,"Onkel,Ohm,Oheim",Onkel
1,stan1295,1327,"Onkel,Ohm,Oheim",Ohm
1,stan1295,1327,"Onkel,Ohm,Oheim",Oheim

(2)

ID,Language_ID,Parameter_ID,Value,Form
1,stan1295,1327,"Onkel,Ohm,Oheim",Onkel
2,stan1295,1327,"Onkel,Ohm,Oheim",Ohm
3,stan1295,1327,"Onkel,Ohm,Oheim",Oheim

(3)

ID,Language_ID,Parameter_ID,Value,Form
1,stan1295,1327,"Onkel",Onkel
2,stan1295,1327,"Ohm,Oheim",Ohm
2,stan1295,1327,"Ohm,Oheim",Oheim

(4)

ID,Language_ID,Parameter_ID,Value,Form
1,stan1295,1327,"Onkel",Onkel
2,stan1295,1327,"Ohm,Oheim",Ohm
3,stan1295,1327,"Ohm,Oheim",Oheim

(5) Something else?

All properties used in tables specified by CLDF should have URIs in the CLDF Ontology - possibly suitable subPropertyOf existing properties in other vocabularies. This will make sure, our tools can recognize these properties unambiguously and vice versa other tools cannot mistake these properties.

This should include ID, forcing a format similar to the one for column names (in addition to adding a primaryKey property implicitely). To mark IDs as URIs, referencing entities defined outside of the dataset, an appropriate aboutUrl should be used.

Proposal

Concerning slicing as used here:

This is yet another "tab vs. comma" discussion, as there are at least two different approaches to counting: first the slicing from Python/C, where the counter starts at zero and the end is not included (basically you are counting boundaries here), and second the indexing/subsetting from Matlab/R, where the counter starts at one and the end is included (basically you are counting elements here).

Please note: I am perfectly aware that the two approach are equivalent, and I also know that this is very much a question of what you are used to. The crucial point is which approach will more quickly lead to misunderstanding when somebody uses the data without reading the documentation.

I think it important to realize that the CLDF is not a programming language, but a data description. So, it should be easily understandable for people outside a particular coding framework. I think that the 'element-counting' Matlab/R approach is much easier to grasp intuitively than the 'boundary-counting' Python/C approach.

If you have seven elements (with six explicit boundaries coded in the data), and you want to refer to the third element, then using "3" seems more natural than using "2:3". The same if you want to refer to elements 1 through 4, then "1:4" seems more natural than "0:4" (the more so as the boundary at "0" does not exist in the data).

So: my proposal is to not use "slicing" but "element counting" in the CLDF.

The subSequence property isn't used anywhere and should be obsolete and replaced by specific slice properties refereing to a sequence property.

Addresses #54

Components could be specified as orthogonal aspect to modules. I.e. pieces of specification which can be reused across modules. Candidates include:

• examples in IGT
• forms (could be re-used by dictionary, wordlist, and potentially even structure-dataset)

In terms of architecture, components could just be Table descriptions - as opposed to modules, which are TableGroup descriptions.

To keep the schema more consistent, the default column name for the glottocode property should be Glottocode.

Addresses #54

May I point you to a more profound way to deal with tabular data?
Core idea: column storage with support for sparse data.
Step one: define anchor points for the most granular units in your corpus.
Step two: define nodes for all subsets of anchor points that you want to annotate.
Step three: annotate in the form of mappings from nodes to feature values.
Every feature is 1 column of data, implicitly linked to the set of anchor points and nodes.
I have used this model extensively for the Hebrew Bible (400,000 anchor points, 1,000,000 extra nodes, 20,000,000 feature values), and it enables very flexible explorative data processing, e.g. in a Jupyter notebook.
By isolating columns, it becomes easier to produce them in a distributed fashion, and share them easily.
Have a look at the model

The specs contain several inconsistencies and apparent mistakes.

• You specify nowhere what columns are required generically (Word lists have 4, but readme states “While the file may contain any number of columns”, while implicitly also making at least ID and Language_ID required)
• There are half sentences: “Value: the word form, the main value, for the given language and the given concept (may have different degrees of ” (wordlist.md) – I think I saw another one, but can't retrace it right now.
• structure_dataset.md has a word list example even though it speaks of “(often typological)” values, and the description on multi-dimensional features is very short

Currently we set doublequote = False:
https://github.com/glottobank/cldf/blob/58a3e8b831bb0a3aa926d4811a5932509b0c545b/modules/StructureDataset/StructureDataset-metadata.json#L9-L13
AFAICT, calc basically uses the same default export format as Pythons default so-called csv.excel dialect with these settings (doublequote = True).

Also note that escapechar handling seems to be broken in Python (https://bugs.python.org/issue12178).

I have named regions in my database, and I think that might be a data type worth including in future CLDF releases. Macroareas (https://github.com/glottobank/cldf/blob/ff33e40dbccabaf1191f08eb8f486a13343a6f5b/terms.rdf#L272) are then an obvious, semantically well-defined subtype of named areas. Countries could be another type of named area.

While that type of data can be programatically extracted from the geo-coordinates using some set of covering polygons, it does have value in itself, I think.

rather than the under-specified slice property.

See also #49

We might want to curate a list of known prefixes for namespaces to allow for shorter IDs. E.g. with a mapping like

{
    'gl_ref': 'glottolog.org/resource/reference/id/',
    'gl_lang': 'glottolog.org/resource/languoid/id/',
}

we could write Source_ID as gl_ref:1234.

See also http://www.w3.org/2001/tag/doc/qnameids-2004-03-17#sec-archrec

Since we already have two more name-like attributes for each property, csvw:name and the local name in the CLDF namespace, an additional rdfs:label seems superfluous, considering that the other names are fairly human readable as well.

Addresses #54

We should think about some sort of component model tying together the core format with domain-specific components. Components that come to mind are

• Interlinear glossed text (IGT)
• cognate judgements

Needed for lexibank.

Like the Source column in the FormTable, the *_source columns in *CognateTable should accept lists of references as values, too.

I just generated an ER diagram for my dataset, and I noticed that there are no connections between the BorrowingTable's Form_ID columns and the FormTable. This is because no ForeignKey relations are defined in
https://github.com/glottobank/cldf/blob/aeba7c6c3df849495aea45f0a1f7c376b93579e0/components/borrowings/BorrowingTable-metadata.json

Currently, a weird hack involving the csvw:name of reference properties is used to determine the component which is referenced. This should be replaced by explicitly specifying the referenced component in the Ontology. With this in place, we should also regularize the csv:name of reference properties (see #54 ).

the more I think about it and the more I talk about it with colleagues, it becomes evident that cognacy needs to expanded by the aspect of partial cognacy. In Indo-European, we have only spurious accounts, but in Tukano, Sino-Tibetan, and many other families, compounds are not just some little noise but constitute the basis of the lexical structure of the languages (30% in the nouns of Chinese dialects, but also pervasive in other families, like Tupi-Guarani).

Now, for partial cognacy, there is a straightforward representation. If we have words like "she-goat" and "he-goat", we can assign two IDs, that would render the stuff as "1 2" and "1 3". Note that we have cases in which we have the opposite order in the same language family, that is, we have "she-goat" in langauge A and "he-goat" in language B. Here, we would write "1 2" and "2 1". Note also that we need to keep the information in one cell, since we want to store the order of the elements, and keep track of it for the alignmetns, which ideally would only be made for identical parts of the compounds. So we would have one alignment for cogid 1 and one for cogid 2, etc. In a file, we could represent this in a column by keeping the order of the original tokens, but adding a separator for morphemes:

TOKENS         COGID  ALIGNMENT           
sh e + g oa t       1 2         sh e + g oa t
h e + g oa t         1 3         h e + g oa t
g oa t + h e         3 1         g oa t + h e

So the alignment is no longer immediately visible when looking at the text-file, but automatically accessible and it can be displayed with tools that offer the service of comparing the cogid column with the column of segments (TOKENS). We loose transparency for those who want to read text-files, but we gain power for representing cognacy truthfully.

Furthermore, the format is consistent with unique-ID representations of cognacy:

• in a strict version, we require identity of compound structure, which is nothing else than a check for identical strings in the cogid-representation which gives multiple numerical IDs in one cell separated by a space.
• in a loose version, we can make a connected component analysis of the underlying network and label the respective items as cognate, which share direct or indirect links with other items

My most recent idea was, that this should essentially be the basis, since it is so frequent in many language families, and also useful where it is less frequent (but never underestimate the degree of partial cognacy, also for IE languages, once you start being strict with morphemes!). And since those datasets where partial cognacy doesn't play a role would still simply show one ID for a cognate set, it would be largely consistent with the rest.

But in danger of repeating myself here: we cannot ignore the problem of handling partial cognates. They are just too massively present in too many languages, and can also soon be fully handled by automatic cognate detection algorithms (provided, morpheme boundaries are known: I have recent highly promising results for lingpy).

In lexibank, we have an extra file for cognate sets, with foreign keys (Word_ID). It seems like this can be specified in a straightforward way in CSV metadata. However, there are a few open questions:

1. we put cognate sets in an extra file to be able to handle different sources for cognate sets (e.g., one automatic version vs. one non-automatic version, which would be two extra columns in a flat csv in LingPy-format), but this somehow feels strange, as we would have the potential to assign one word to multiple cognate sets, but we would only allow this if the source is the same? Somehow, too much depends on the source as being the discriminating factor here, as with two different series of cognate judgments in one file, the only way to distinguish cognate sets would be to look at the source when evaluating and hoping that no word_id with identical source is assigned to multiple cognate sets (which would be the fuzzy annotation which is often used in ABVD and difficult to interpret)

2. in partial cognates, we assign one word form to multiple cognate sets, and the order is important, so in principle, we could either put one word_id in multiple rows, one for each partial cognate set, but we would have to report the index, or we accept that the morpheme structure of the "Segments" follows the order of the list of cognate-identifiers in the partial cognate set (which is space-separated).

Apart from the fact that I'd like to preserve some base-functionality to re-use the columns in flat csv files, if cognate sets do not require specific source annotation, it seems we need to add at least a Cognate_Set_Type column indicating whether a cognate set is a) fuzzy (ABVD-style, assigning one word ID to multiple cognate sets), b) unique, or c) partial.

But I'm having some trouble in making this a clear-cut example now that would reflect all things neatly, specifically, since the fuzzy, unique, partial distinction would require that based on that distinction the content of the field Cognate_Set changes (string vs. list, or integer vs. list in case of LingPy/EDICTOR).

I noticed that I accidentally created a table without specifying the datatype for some column.
Is that valid?

CLDF does not say anything, cldf validate lets it pass, but pycldf.db assumes that column.datatype.base is defined.

The core standard says about Metadata-free conformance:
“To do so, the dataset must follow the default specification for the appropriate module regarding […] CSV dialect”, but the Wordlist module does no specify a default CSV dialect. Does that mean I should assume the dialect defaults given in http://w3c.github.io/csvw/metadata/#dialect-descriptions?

Cross-linguistic datasets often contain examples as interlinear glossed text (IGT) according to the Leipzig Glossing Rules (LGR). While this kind of data clearly can be modelled as tabular data, thus would fit into csv files, this format would probably go against the idea of having formats that can be easily edited by hand. To satisfy this requirement, a format that groups examples as blocks of aligned IGT constituents might be better suited. One candidate for such a format is the Toolbox variant exported by ELAN:

\utterance_id ...
\ELANBegin 5.708
\ELANEnd 8.974
\ELANParticipant
\utterance <text in source language>
\gramm_units <morphemes split by whitespace>
\rp_gloss <glosses split by whitespace>
\ft <translation>
\comment

This is a follow-up on a lexibank issue, showing that we need to think about useful formats to represent correspondence sets in cldf.

• fill in CHANGELOG.md
• draft release 1.0.0
• archive on zenodo

An explicit list of use cases should help in motivating design decisions and making an evaluation of a proposed standard easier.

http://glottolog.org/meta/glossary#macroarea

Proposal

I would like to propose a new module ParallelText for the CLDF, which uses two components: forms (i.e. FormTable) and a new component for functional equivalents (FunctionalEquivalentsTable).

The structure of parallel texts are very similar to wordlists in that there are parallel instances ("translations") for the same content, so they basically can be encoded using a FormTable.

The analysis of a parallel text is somewhat similar to the analysis of cognates, in that parts of the strings are to be aligned. However, these alignments are not cognates, but functional equivalents. Also, the alignments are mostly not nicely linear as with sound alignments, so it is mostly not possible to solve the multialignment by inserting gaps (there is also a lot of many-to-many alignments). Because of this I would like to propose a new component, the FunctionalEquivalentsTable (any improvements on this cumbersome name are welcome!).

The FunctionalEquivalentsTable encodes the parts from the different parallel texts that are functionally equivalent. In its most general form, this table is a list of counts/slices from the parallel texts (separated by spaces) that are aligned into FunctionalEquivalentSets.

{
    "url": "functionalEquivalents.csv", 
    "dc:conformsTo": "TODO", 
    "tableSchema": {
        "columns": [
            {
                "name": "ID", 
                "required": true, 
                "propertyUrl": "http://cldf.clld.org/v1.0/terms.rdf#id", 
                "datatype": {
                    "base": "string", 
                    "format": "[a-zA-Z0-9_\\-]+"
                }
            }, 
            {
                "name": "Form_ID",
                "required": true, 
                "propertyUrl": "http://cldf.clld.org/v1.0/terms.rdf#formReference", 
                "datatype": "string"
            }, 
            {
                "name": "Slice/Count", 
                "required": true, 
                "propertyUrl": "http://cldf.clld.org/v1.0/terms.rdf#slice", 
                "datatype": {
                    "base": "string", 
                    "format": "\\d+(:\\d+)?"
                }, 
                "separator": ","
            }, 
            {
                "name": "FunctionalEquivalentSet", 
                "required": true, 
                "propertyUrl": "TODO", 
                "datatype": "string"
            }
        ]
    }
}

See for example this multialignment of Germanic bibles (Note that the format here is no yet in accordance with the CLDF). Also nicely illustrated there is the non-linearity of the alignments, e.g. in this visualisation.

Proposal

Given a column in a CSV-file that uses secondary separators, sometimes these separated parts will be aligned, i.e. each string has the same number of chunks, and the chunks have some kind of equivalent status in each row. Such an multi-alignment is for example used in the component cognates.

In such a situation I would like to be able to refer to such a column to add extra information to such a column. This might be solved by adding a alignmentAnnotationTable to the CLDF.

• representation in standard orthography (STANDARD)
• mark columns as uninteresting for further analysis, because the data is not consistently encoded (IGNORE)
• mark sets of columns as "identical", which is necessary in case of metathesis (MERGE)
• mark sets of columns as "complex segment", which happens in situations like a multi-alignment of Germanic school: It is probably best to align two columns for the onset here (with languages having /sk/ or /sx/), but in some languages these two columns are merged into one (e.g. /ʃ/). (COMPLEX)
• etcetera

Technically it not problematic to make such annotations using column counting/slicing, for example like this:

{
    "url": "alignmentAnnotation.csv", 
    "dc:conformsTo": "TODO", 
    "tableSchema": {
        "columns": [
            {
                "name": "ID", 
                "required": true, 
                "propertyUrl": "http://cldf.clld.org/v1.0/terms.rdf#id", 
                "datatype": {
                    "base": "string", 
                    "format": "[a-zA-Z0-9_\\-]+"
                }
            }, 
            {
                "name": "Form_ID", 
                "required": true, 
                "propertyUrl": "http://cldf.clld.org/v1.0/terms.rdf#formReference", 
                "datatype": "string"
            }, 
            {
                "name": "Slice/Count", 
                "required": true, 
                "propertyUrl": "http://cldf.clld.org/v1.0/terms.rdf#slice", 
                "datatype": {
                    "base": "string", 
                    "format": "\\d+(:\\d+)?"
                }, 
                "separator": ","
            }, 
            {
                "name": "AnnotationKind", 
                "required": true, 
                "propertyUrl": "TODO", 
                "datatype": "string"
            }
            {
                "name": "AnnotationContent", 
                "required": false, 
                "propertyUrl": "TODO", 
                "datatype": "string"
            }
        ]
    }
}

Examples of the kind of information that I would like to add to each column are the following (see for example here, though note that this is not according to CLDF yet!).

With the benchmark database of phonetic alignments and the benchmark data for cognate detection there are two large datasets which provide phonetically segmented data. The question, also with respect to the idea that we might link individual sounds to phoible, is now: to which degree covers phoible the sounds in those and maybe also other datasets?

By carrying out this small analysis, we would have a better estimate on how difficult it is to use things like phoible for reference in our future endeavours (or whether we need to come up with an alternative system of tracking sounds ourself, that, eventually, may built upon phoible).

One further important point on mapping things to phoible is, that we should not underestimate the problems of IPA-variation. IPA has, for example, versions for /ts/ both as single character, as two characters, and as two characters with a combining character. There are also two versions of g, different ways of describing tone, and all other kinds of problems. Now, in order to use phoible as a reference, we need to come up with a mapping of all these variants to the respective phoible symbols, and preferably also with a mapping of non-standard IPA variants, like /th/ for aspirated t, which we will frequently encounter in the lexical data. This could turn out to be quite tedious, but also useful. Anyway, the starting point is to check to which degree we find agreement between the above-mentioned databases and phoible, but potentially also Paul's sound comparison databases.

which can be used by all modules.

... again, this is mostly to make things simpler.

We can annotate motivation of compound structures now, but we can't annotate word families yet. I am thinking of things like "walk" vs. "walker", etc., which are as a default best handled as directed graphs, where a given word form is annotated by adding a source, a relation, and a target. Source and target serve as identifiers for the node in the network.

This may seem similar to the compound motivation we use and which I gave examples for in #21, but it is essentially different in so far as the relation between source and target form may not be linear (think of Umlaut, Ablaut, ellipsis, etc.). So the source form defines an origin of the derivation, which is then rendered as node-ID in the directed network. We could ignore the target if we just use the Word_ID, but I think that the user-defined language-internal IDs are more easy for annotation, also in terms of readability.

As a rule, validation of these relations would require derivation rules, which are usually pretty language-specific and cannot really be handled cross-linguistically. But ideally, an application would have code to derive potential targets.

Working examples for this will be produced soon in CALC, but this is probably nothing we need to consider for the first publication of the CLDF specs.

Allow for a second separator inside csv-cells (the semicolon is already proposed as such for the Source column). It makes files more powerful, but also more complex. If we decide on a second-level separator, then I would propose a highly unusual UTF-8 symbol, e.g. \u2005 (FOUR-PER-EM SPACE) or \u204F (REVERSED SEMICOLON).

Media files should be referenced via URI, which would allow to even include them directly using data URIs.

In addition to the MediaTable component, we'd need a mediaReference property in the ontology.

The CLDF name property - if assigned to a column in a media table - is interpreted as filename, thus can be used to derive a filename extension, etc. alternatively, this can be derived from the return value of Dataset.get_row_url(...).

Proposal

Depending on the data, it is sometimes easier to add multiple datasets into a single file, and use a specific column to mark the different datasets. For example, one might either make a different alignment file for each cognate set, or one might combine many different alignment into one file and add a column like "COGNATE ID". Both options have their use-cases. For example, in dialect data with thousands of cognates for a specific meaning it often maked more sense to make separate files for each cognateset.

It is of course even possible to mix and match those two options, i.e. splitting data into different files in some situations, but using internal separation into groups in other files.

I could not find any mentioning of this in https://www.w3.org/TR/tabular-metadata/ (maybe it is too trivial?). It might make sense for CLDF to specify either:

• to always merge files with identical structure into one file (which might lead to very large files: I have actual use-cases that easily lead to files of multiple GB)
• to always split files into subsets (which might lead to very many files: I have use cases that lead to ten-thousands of files)
• allow both approaches and specify for the many-files usage to make sub-directories in a CLDF-directory named according to the component, e.g. a sub-directory cognates with many files for the different cognatesets. (which I would propose)

Note that this decision for the CLDF is independent of the practical work with those files, as it should be trivial to either merge multiple files into one or to split a large files into many separate files.

Should I add something about this to the section "Data Files" in the readme.md?

strapless may be a good choice to style the website.

mostly to simplify access, but also to enforce a common intuition.

I am preparing a paper on BEASTling for submission to PLoS One, who have a software paper category. I plan to mention that BEASTling supports the CLDF file format (as part of a general theme in the paper of pushing the merits of sharing linguistic data by referring to languages using standard identifiers (ISO or Glottocode), storing data in simple, standardised formats (CSV, and CLDF in particular), and publishing data under permissive copyright licenses (CC in particular). How should I best cite the CLDF concept? Should I ideally describe it as part of Glottobank? Or part of CLLD? Is CLLD formally affiliated with MPI-EVA? Is there a public URL for CLDF?

Questions

Within strings in a cell in a CSV-file, it is possible to use specific symbols to specify chunking of the string. The explanation in the FAQ is either badly formulated or confused. Or maybe I am confused :-).

The given example there is the following:

A third level of structured content can be achieved for cells of datatype string by specifying a regular expression which the values of a cell must match. E.g. the following column specification

{
    "name": "segments",
    "separator": "+",
    "datatype": {
        "base": "string",
        "format": "([\\S]+)( [\\S]+)*"
    }
}

This will make sure cell content of the form `abc def+geh` can be split into the nested lists `[['abc', 'def'], ['geh']]`.

In my understanding, this describes the plus sign as a secondary separator, and within each of the substrings there is an optional space (which is not defined as a separator in this simple example yet, it is just implicitly assumed in the regex). Just this declaration would analyse the string abc def+geh into ['abc def', 'geh'].

Actually, for the a tertiary separator to work, it would need something like this (I doubt that this is formally correct: please improve if necessary!):

{
    "name": "primary segments",
    "separator": "+",
    "datatype": {
        "base": "string",
        "format": "([\\S]+)( [\\S]+)*"
    }
    {
        "name": "secondary segments",
        "separator": " ",
        "datatype": "string"
    }
}

With this description the string abc def+geh will be split into the nested lists [['abc', 'def'], ['geh']].

When I'm not confused, I'll slightly reformulate the FAQ.

I'm going through the ontology, a few comments, please advice:

• the name Concepticon_ID for a Concept Set: I think we should only use the _ID for Reference properties. So maybe this property should be of dc:type='reference-property'?
• both Sound Sequence and Subsequence have the name Segments. Is that intentional?
• the description of Source is the only one that describes the secondary separator in the text.
• the name glottocode is the only one without capitalisation :-).

Overall, there are three names attached to each property: the rdfs:label, the csvw:name and the identifier in the rdf:about. In very many cases these three are identical (except for regular camelCase/noSpace/Underscore replacements). However, there are exceptions and I find that confusing. Some are clearly problematic, but others seem to be unnecessary shortcuts.

• Problematic: ISO 639-3 code: this label is rendered as iso639P3Code, apparently following lexvo.org. The csvw:name is Iso, which I find strange. I would propose:

  • rdfs:label ISO 639 part 3 code
  • rdf: about iso639P3Code
  • csvw:name ISO_639_Part_3_Code

• Probably difficult, because already in use are these mismatches:

  • rdfs:label Lexical Unit, but csvw:name Form. I think the label is too restrictive, and a label Form would be much more suitable.
  • rdfs:label Sound Sequence, but csvw:name Segments. I would urge to decide for one or the other. There is no reason to confuse people even more ;-).

• Unnecessary:

  • rdfs:label Primary Text: why not consquently csvw:name Primary_Text
  • rdfs:label Analyzed Word: why not consquently csvw:name Analyzed_Word
  • rdfs:label Translated Text: why not consquently csvw:name Analyzed_Text
  • rdfs:label Motivation Structure: why not consquently csvw:name Motivation_Structure
  • rdfs:label Prosodic Structure: why not consquently csvw:name Prosodic_Structure

• there is an almost regular replacement of Reference into _ID for the csvw:name, but why not

  • Meta-language Reference -> Meta_Language_ID (currently: Language_ID_Meta)
  • Source Form Reference -> Source_Form_ID (currently: Form_ID_Source)
  • Target Form Reference -> Target_Form_ID (currently: Form_ID_Target)

• Finally: the link between label Concept Set and csvw:name Concepticon_ID is confusing. Can we come up with something better here?

To make it possible to annotate columns in CLDF data files unambiguously, we should provide an RDF vocabulary (along the lines of the WGS84 example) defining properties which can be used as propertyUrl annotations in the metadata file.

This should include:

• http://cldf.clld.org/terms#glottocode
• http://cldf.clld.org/terms#iso-639-3
• http://cldf.clld.org/terms#concepticon-conceptset

This just took me some time to figure out, and left me with quite some confusion, so maybe it would be good to change this:

The names of columns in the description of the tables (e.g. formTable) are linked to properties in the ontology in terms.rdf. For clarity, I would propose to use the ontology names as column names to avoid confusion.

For example, in formTable:

• "name" = "Form" can simply be changed to "name" = "LexicalUnit"
• "name" = "Segments" can simply be changed to "name" = "soundSequence"

As a general rule-of-thumb, we can keep _ID instead of Reference, so we use "name" = "Language_ID" instead of the literal copy from the ontology, which would be "name" = "languageReference".

Of course these are just names, and they have to be explicitly referenced in the meta-data of each dataset, so you can change it to whatever you want. However, in the description of the CLDF I think it is better to make this easier to grasp immediately.

If you agree, I'll make a pull request with such changes

Followup on cldf/pycldf#50 from the formal side:

The specs state that

A dataset can be CLDF conformant without providing a separate metadata description file. To do so, the dataset must follow the default specification for the appropriate module regarding

• file names
• CSV dialect
• column names
  exactly. Thus, rather than not having any metadata, the dataset does not specify any; and instead falls back to using the defaults, i.e. "free" as in "beer" not as in "gluten-free".

A close reading of that seems to imply that additional columns outside those defined in the corresponding module .json should not be permitted.

However, several other bits of the specs (including the Wordlists.md with its talk about optional columns) give me the impression that additional columns should be permitted, and cldf validate does not complain, either.

• http://doi.org/10.5281/zenodo.886179