cdli-gh / framework Goto Github PK

From @epageperron on August 12, 2017 13:56

In order to make like data more compact and to facilitate data linking, we have to reinstate entries for retired P nos and join old revisions into main table and flag them as such.

The conception part should be taken into account while re-designing the database

• Modify the artifacts table to include new fields: composite, physical, retired / redirected, old revision
• Migrate data from FM to MariaDB
• Implement a check of this table when P no. is is invalid + warn user
• On delete artifact, create retired entry and prompt user for comment

Copied from original issue: cdli-gh/cdli.network#2

Summary

Rewrite the collections download module to integrate in the new framework and not using a ftp client on archival

Other links or relevant information

Collections managers (museum staff) can at this time login into the CDLI and download all images for all of their artifacts or part of it, based on the search query they do. The site creates an archive file with the required images on the archival server, shares it on a CuteFTP server through a unique link that is sent by e-mail to the collections manager.

• This module needs to be fully rewritten in cakephp
• Use Python for backend requirements
• The file to download should be accessed by the user from the production server, not archival
• Add ptm and eps files to collections downloads

Roadmap Data

🗓 Start Date:

🗓 Expected Date:

💪 Label:

📈 Progress (0-1):

See Gantt: http://cdli-dev.org/gantt/Framework/

From @epageperron on February 16, 2017 5:56

The problem comes from the forum access module
https://www.drupal.org/node/2789033

Copied from original issue: cdli-gh/cdli1#8

In order to have a site basis to test new implementations, we have to install a fresh Drupal8 copy and Cakephp in a virtual host on the dev server.

• Create a separate virtual host for this new dev installation (Until production is ported to virtual, the old and new dev instances must cohabit on the dev server)
• Deploy Drupal 8
• Deploy Cakephp
• Configure both so they can work side by side (.htaccess and respective config files)
• Document the configuration changes and save a copy of config files on Github

All customized files that do not contain sensitive information should be uploaded to Github in the Framrwork repository. The apps themselves should be replaced by a placeholder file with an indication of the version to use. On Gitlab, the whole install can be pushed.

🗓 Start Date: 2017-11-12

🗓 Expected Date: 2017-12-11

💪 Label: wp

📈 Progress (0-1): 0.3

Summary

Document catalogue management in FileMaker

Other links or relevant information

Roadmap Data

🗓 Start Date:

🗓 Expected Date:

💪 Label:

📈 Progress (0-1):

See Gantt: http://cdli-dev.org/gantt/Framework/

Summary

Implementing controller and view requirements to display the proper data in the individual artifact view

Other links or relevant information

Fist consult with @amanbis to make sure all elements required are accounted for ad revise the list below :

Types of images / visual rendering :

• JPG (base, detail, lineart)
• SVG
• PTM (through the RTI viewer)
• 3D (through 3D viewer)

SVG:
"That would want to either allow user to expand or swish out size of the offered svg line art, or to hard define a rendering resolution that might be something like 4x enlargement of original. svg files in CDLI should be restricted to _l.svg and _ld.svg. They as a rule were processed to lead _l.tif files and thus the primary purpose of this addition would be to make them individually downloadable (aside from svg online renderings being much superior to their rasterized cousins).:" -Bob

Data from fields with links to their relational data :

• Material
• Site
  ...

Roadmap Data

🗓 Start Date:

🗓 Expected Date:

💪 Label:

📈 Progress (0-1):

See Gantt: http://cdli-dev.org/gantt/Framework/

From @epageperron on August 12, 2017 11:58

Set up a document section specifically for agreements and contracts, for all partners

Copied from original issue: cdli-gh/cdli.network#1

We should have a list of keywords that when punched in in search redirect to specific pages. Examples : known objects, periods, places.

Migrate the Drupal data, views and functionalities from the current Drupal 7 install in production to Drupal 8, on the Drupal 8 dev install

🗓 Start Date: 2017-11-25

🗓 Expected Date: 2018-05-04

💪 Label:

📈 Progress (0-1): 0.3

Summary

Create a unified download page on cake

Other links or relevant information

Create a node type download in Drupal with fields for title, description and license type
Set up a folder which will contain automatically generated data packages with a structure of one folder per set + readme file that contain fields for title, description and license type
Create a view on cake that first displays all nodes type download then all automatically generated data packages as a neat and seamless list

the list should also show the date of creation or modification of the file, the file name and the size of the file.

Or find another solution to cater to all types of users inputting the data (editors and machines) and data packages

Roadmap Data

🗓 Start Date:

🗓 Expected Date:

💪 Label:

📈 Progress (0-1):

See Gantt: http://cdli-dev.org/gantt/Framework/

Summary

In order to keep the FM interface while also modernizing the database structure, we have to investigate the implementation of FM as a direct interface to MySQL OR an intermediary model where there is tables replication on the FM server. Extended tests will be needed to check the speed of populating FM views and doing searches.

This task is only about finding and testing the appropriate solution. Full implementation of the solution will happen only after DB design.

Tasks

• Read about FileMaker and MySQL
• Research about scenarios for using FM as the direct interface and MySQL as the master database
• Connect FM with MySQL using External Data Source connection (ODBC connection)
• Test the FM client and MySQL ESS connection
• Connect FM Pro to the FM server and connect the server to the MySQL database using ESS
• Test the connection (also test for speed with relationships and large tables)

Roadmap Data

🗓 Start Date: 09-23-2017

🗓 Expected Date: 01-19-2018

💪 Label: wp

📈 Progress (0-1): 0.8

See Gantt: http://cdli-dev.org/gantt/Framework/

Git versions of the daily open data bundle are accumulating on 156, can we have the history kept only on github?

From @epageperron on August 4, 2017 2:9

When the dockerization and virtualization of the CDLI are complete, tested and stable (#71 ) a mirror of the install must be prepared, tested, installed at partners's servers then released to production after local testing.

• Prepare an agnostic mirror package
• Thoroughly test deployment and crash recovery on dev
• Prepare instructions on setting up local configuration (deployment and crash recovery)
• Test & deploy at Compute Canada
• Test & deploy at the MPIWG
• Test & deploy at Oxford

🗓 Start Date: 2018-01-08

🗓 Expected Date: 2018-02-01

💪 Label: WP

📈 Progress (0-1): 0

Copied from original issue: cdli-gh/mtaac_work#4

From @epageperron on September 2, 2017 13:40

Prepare an adapted display for ATF of texts containing tabular data

Copied from original issue: cdli-gh/cdli.network#10

Am redoing the routing based on security advice from @nomoon

Develop a friendly filtering system that:

• hides unavailable options/ only shows available options in the data subset currently viewed, eg when looking at texts from a city, only periods represented in that city should be visible
• makes it easy to add and remove filters

Maybe using Ajax with a fall back to reload the whole page for those with JS disabled. (think accessible)

Summary

In order to make the CDLI Platform more efficient, we need to completely redesign the database, optimizing it based on the data.

Design a fully relational model to store catalogue, textual data and new data from MTAAC.

Tasks

• Explore CakePHP3 by doing the tutorials (w1)
• Design a draft of the catalogue portion of the database (w2)
  ( Checkout Cakephp3 Database conventions here https://book.cakephp.org/3.0/en/intro/conventions.html
  • For this, first review https://docs.google.com/spreadsheets/d/1FyYuZJlyFbd-NdWKyGXasY1iS4h4JKaTfPactMVn6Ok/edit#gid=0 with @epageperron
  • Second, use PHPMyAdmin recommendations tool to get a better sense of the repeating text data
  • Create the main table for artifacts
  • Prepare relational tables
  • Set up indexes and keys
• Review and correction of the draft after consultation with @epageperron, @greek-geek, @starlordphr @zarpanitu and @englundrk @amanbis (w2/3)
• Design bibliographic section + relationships (w3)
  • Include a World Cart field for OCLC # (eg http://www.worldcat.org/oclc/89291834 ---> 89291834 )
  • See if http://tronprog.blogspot.ca/2010/05/phpbibliography-10-using-cakephp.html can be updated, tunkated and adapted
  • Or this https://libraries.io/packagist/audiolabs%2Fbibtexparser or https://libraries.io/packagist/renanbr%2Fbibtex-parser ...
• Design corpus tool section if required (w3) (#)
• Script a data migration tool from old to new schema (w4)
• Bake and review the cakephp3 model layer to connect with the database (w4)

Other links or relevant information

This task is dependant on 2 things:

1. the FM Challenge (#16)
2. The requirements of a corpus analysis tool

After this issue is done, we will have a full functional first draft of the DB but we might still have to make later adjustments to the schema and thus also to the conversion script and the model layer of Cake.

Other notes :
Don't forget to take into account revisions and retired P nos.
Store CDLI-CoNLL version of ATF
Bool version consistent or not
Bool uncertain for period and provenience

Everything that will have a uri should have a specific field
Rdb2rdf for the LOD: http://www.rdb2rdf.org/

Roadmap Data

🗓 Start Date: 10-25-2017

🗓 Expected Date: 02-09-2018

💪 Label: wp

📈 Progress (0-1): 0.3

See Gantt: http://cdli-dev.org/gantt/Framework/

From @epageperron on September 13, 2017 16:35

inclusion of http://cdli.ucla.edu/?q=cuneiform-digital-library-preprints in the general Drupal management system for pubs. That would give

0001.0 Huber, Peter J. On the Old Babylonian Understanding of Sumerian Grammar 2015/09/03 PDF
0001.1 Huber, Peter J. On the Old Babylonian Understanding of Sumerian Grammar 2016/04/01 PDF
etc.

with only author names and PDFs hyperlinked, PDFs like other series in ../files/publications rather than in ../pubs/cdlp.

Example : https://www.mpiwg-berlin.mpg.de/resources/preprints

RKE

Copied from original issue: cdli-gh/cdli.network#11

From @epageperron on August 12, 2017 14:4

After implementing the most efficient way to manage dates in the database model #3

• Have search leverage the chronology system
• Prepare views for browsing based on Chronology
• Prepare display filters for search results

Copied from original issue: cdli-gh/cdli.network#5

Summary

Other links or relevant information

Roadmap Data

🗓 Start Date: 2017-11-28

🗓 Expected Date: 2018-02-02

💪 Label:

📈 Progress (0-1): 0.2

See Gantt: http://cdli-dev.org/gantt/Framework/

From @epageperron on January 24, 2017 16:51

Bidirectional text on the same line :
https://www.w3.org/International/articles/inline-bidi-markup/#whattodo

Steps to fix :

• Check length of first line, if equal or more than the actual characters of line 1 in P429858, turn a switch to true OR use P no as css id
• when switch is true, perform these :
  -insert a line break before each translation
  -don't indent the translation
• add a class "long-translit" to the translation spans
• add a span around language indice and give it a class named with the language indice

css :

• make long-translit (or with P no css id) class display:block so we can play with text alignment
• create a class for languages indices of right to left texts and apply whatever needed from link at top of this message to arrange text

Copied from original issue: cdli-gh/cdli1#1

From @epageperron on February 1, 2017 18:47

Investigate the 502 timeout and implement fix

Copied from original issue: cdli-gh/cdli1#4

Because FM 11 is crashing on the last version of OSX, we have to migrate the current server to FM 16.

• Test that the FM 14 converted db works as well as the FM 11 version
• Test the production workflow on FM server 16 with the FM 14 client in a dev environment, check if csv export marches csv export from FM 11
• Buy FM 16 server and client, download both Mac and Windows installs and archive!
• Reinstall and update 155, and install FM 16 there OR on virtual Windows server
• Test the workflow including updating MySQL with a dump from FM 16
• Retire 152

From @epageperron on August 13, 2017 12:18

• Show _e.jpg underneath the default image

Copied from original issue: cdli-gh/cdli.network#8

From @epageperron on February 20, 2017 0:11

Looks fine at 125%, overlaps at 100% on Chrome and icedove on Linux

Copied from original issue: cdli-gh/cdli1#9

From @epageperron on March 31, 2017 16:37

prepare a node type for agade posts
prepare a view to display nodes chronologically & according to title tag
process from e-mails

Copied from original issue: cdli-gh/cdli1#12

From @epageperron on February 1, 2017 19:5

Copied from original issue: cdli-gh/cdli1#6

• Install an authentication and access module for Cakephp that uses MFA
• Create the access levels and double check the users
• Document how to implement access check

🗓 Start Date: 2018-01-29

🗓 Expected Date: 2018-02-05

💪 Label:

📈 Progress (0-1):

Perform a rsync dry run to compare source and copy

Summary

Other links or relevant information

Roadmap Data

🗓 Start Date:

🗓 Expected Date:

💪 Label:

📈 Progress (0-1):

See Gantt: http://cdli-dev.org/gantt/Framework/

Lots of links in the web CDLI tablet app are hard links, remove protocol and domain.
cdli/cdlitablet/

Find a module, install and test it.
Integrate with our access system.

🗓 Start Date:

🗓 Expected Date:

🏷 Label: test

📈 Progress (0-1): 0

Summary

Convert the EPS files we have to SVG
Display SVG images along other artifact images

• Find a command line tool that properly handles the conversion of EPS to SVG without weird intermediary conversion to bitmap and that creates a true SVG output.

• Prepare a one liner to process a folder of eps files

Other links or relevant information

Roadmap Data

🗓 Start Date:

🗓 Expected Date:

💪 Label:

📈 Progress (0-1):

See Gantt: http://cdli-dev.org/gantt/Framework/

Summary

• Convert contents of http://doc.cdli-dev.org/ to private cakephp pages
• Write the documentation about the gitlab account for the private repository to the cdli software

🗓 Start Date:

🗓 Expected Date:

💪 Label:

📈 Progress (0-1):

See Gantt: http://cdli-dev.org/gantt/Framework/

Check file permissions and access to remove all backup errors

From @epageperron on March 3, 2017 17:43

split search string on + except if between | | or { }

Copied from original issue: cdli-gh/cdli1#11

Summary

Other links or relevant information

Roadmap Data

🗓 Start Date:

🗓 Expected Date:

💪 Label:

📈 Progress (0-1):

See Gantt: http://cdli-dev.org/gantt/Framework/

From @epageperron on March 3, 2017 17:28

Search count seems to take into account only some instances of search string in texts.
eg : "SZU+LAGAB" => 28,571 instances
counting them manually => 31,000+

Copied from original issue: cdli-gh/cdli1#10

From @epageperron on August 13, 2017 12:18

• Show _e.jpg underneath the default image

Copied from original issue: cdli-gh/cdli.network#8

This tool should process an ATF file and get rid of common problems, and add a space at the end of each line.

Summary

including the thumbnail with share message in one unique message

Roadmap Data

🗓 Start Date:

🗓 Expected Date:

💪 Label:

📈 Progress (0-1):

See Gantt: http://cdli-dev.org/gantt/Framework/

From @epageperron on March 3, 2017 17:43

split search string on + except if between | | or { }

Copied from original issue: cdli-gh/cdli1#11

From @epageperron on August 27, 2017 16:39

Copied from original issue: cdli-gh/cdli.network#9

From @epageperron on February 1, 2017 19:5

Copied from original issue: cdli-gh/cdli1#6

From @epageperron on February 20, 2017 0:11

Looks fine at 125%, overlaps at 100% on Chrome and icedove on Linux

Copied from original issue: cdli-gh/cdli1#9

Summary

• Using cakePHP's bake tool, generate the controllers and views for each of our data tables
• Make sure all objects can be properly viewed, created and edited
• Make sure an individual object display has all required information and links displaying properly
• Make sure the list of artifacts display has all required information and links displaying properly

Roadmap Data

Should start as soon as the model layer is ready from #15

🗓 Start Date: 2018-02-09

🗓 Expected Date: 2018-02-12

💪 Label:

📈 Progress (0-1):

See Gantt: http://cdli-dev.org/gantt/Framework/

Summary

Prepare the new search engine for CDLI. Consult first with @amanbis for an overview of the requirements of the search forms. Result display enhancements ( eg filtering results, changing columns order etc etc will be managed in another task)

Other links or relevant information

• Inventory Cakephp implementation of the usual search tools and choose the best one
• Build a form for basic search
• Build a form for advanced search

Roadmap Data

🗓 Start Date: 2018-02-09

🗓 Expected Date: 2018-02-23

💪 Label:

📈 Progress (0-1):

See Gantt: http://cdli-dev.org/gantt/Framework/

cdli-wide replacement of all "http://cdli.ucla.edu” hyperlinks with "https://cdli.ucla.edu"