jasata / utu-vm-site Goto Github PK

Simple web service for Turku University's Course Virtualization project. Students can download course-specific virtual machines and teachers can upload and manage them.

License: MIT License

CSS 7.70% HTML 20.24% JavaScript 37.39% Python 34.68%

virtual-machines lecturer ova education-platform educational-institutions

utu-vm-site's Introduction

University of Turku - Course Virtualization

2020-01-10: Version 1.0 release (and first publication) by the end of January 2020.

A website for university's Course Virtualization project. Serves both students (get hypervisor installed and download course specific virtual machine images) and teachers (learn how virtual machines could help running a course, instructions on how to create an virtual machine image, how to get help doing it, uloading and publishing made images).

See also:

Static version of this project at github.io.
Wiki pages for technical and design notes.

Design Goals

Simplicity (in implementation, site structure, usage...). And where not achievable, good and clear documentation.
Minimal dependencies (see "Third-party components").
Small scope (Only the essentials discussed in the site. More detailed information will be provided in form of PDF documents).
Two-step principle. All actions (primary use cases) should be accomplished in no more than two clicks. One to choose the relevant subpage, another to open a folding information container (if necessary).
Integration-friendly Architecture. Basically meaning that the backend is a REST API and none of the client pages are dynamically parsed.

Primary use cases (in order of importance)

Student retrieves course-specific virtual machine image
Student comes to learn how to set his or her personal system up to run virtual images
Student comes to resolve an issue and/or report it
Lecturer uploads a new course-specifc virtual machine
Lecturer comes to learn about this tool and its benefits
Lecturer seeks technical documentation on how to create new virtual images
Lecturer comes to seek assistance / help

Server

Original idea about separate virtual server has been dropped in favor of expanding ftdev resources and reserving an additional DNS name for it (https://vm.utu.fi). Or in other words, vm.utu.fi is a virtual host in ftdev.utu.fi (virtual server).

Functional Specifications

Publishes a download list (in the downloads -page) of available .OVA files.
OVA metadata will be a combination of extracted XML from OVA and uploader specified information.
Download statistics will be based on webserver log files, which will be generated into (some kind of) graph/report nightly and made available in separate statistics page.
Uses UTU SSO authentication, but restricts itself to UTU UID only. No other Personally Identifiable Information (PII) is handled or stored. (NOTE: Students have no PII of any kind stored).

GDPR

IT-Services will be consulted about UID as PII data. If UID alone is also considered to enough to bring this solution under GDPR regulations, solution model where all UID's are one-way hashed (both in client's session storage and teacher database table) will be proposed.

2020-01-10 update: In a meeting, it has been said that storage of UTU SSO UID (for teachers) is allowable as long as this is clearly indicated in the data privacy statement, along with the description of its usage, purpose for storing it, expected lifetime (when and/or under what conditions it will be cleaned out) and providing a way to exercice all the rights granted by GDPR (which in our case, is a communique to support email address).

Items That Need Technical Solutions

Integration of distribution builder - as a separate Build Manager agent, providing a REST API interface.
HTML5 File API based Flow.js needs server side implementation in Python, since nothing even remotely acceptable exists.
Statistics that satisfy not only our curiosity, but also justify the existence of this service when the time comes (in a year or two) that this can no longer be considered to be a education development project (ergo, would need to be adopted by IT-Services).

Third-party components

Frontend

Bootstrap 4.3.1
JQuery 3.4.1
Font Awesome 4.7.0 (because that is the last free version)
Datatables 1.10.20 for download page tables
CardTabs 1.0 for tabulated content
JSONForm for VM details edit
Flow.js v.2.13.2 for HTML5 File API transfers

Backend

Nginx ver.1.14.0+
Python 3.6.8 (Ubuntu Server 18.04.3 LTS, offered by IT-Services)
Flask 0.12.2 (2017-05-16!! Current version: 1.1.2) & UWSGI 2.0.15+
SQLite3 3.22.0 (2018-01-22)

utu-vm-site's People

Contributors

Stargazers

Watchers

Forkers

skatans apmak3 koivunen visioninhope

utu-vm-site's Issues

Install facts verification

Action points to be distributed in the next project meeting... Will be added here once handed out.

Index header shadow effect

Needs the same as the subpages

Web Accessability Requirements

https://intranet.utu.fi/index/webaccessibility/Pages/default.aspx

Enforced at 23.09.2020 at the latest. Validate and correct.

Setup does not ensure /etc/nginx/sites-available having correct paths

Error message reveals that the path is incorrect - not what the instance/applicatioin.conf dictates:
[2020-09-13 12:15:10,843] ERROR in File: File '/var/www/downloads/FreeDOS 1.2.ova' does not exist!

api.File.download() class method needs to be reviewed and possibly fixed.

Upload mechanism

Find a way to use Flow.js or fork and modify it to suit our needs.

Requirements are:

transmission-fault tolerant upload (checksums for chunks)
support dran'n drop
support classical "open file" dialog
limit upload to one file at the time

Disable login/logout button until it has read its state

Apply deactivated in the inner HTML inject.

Cleaner cron.job

Create daily cron job to clean out failed uploads. Use "last modified" on {UPLOAD_DIR} files for files that are modified over 24 hours ago (successfully uploaded are published and moved to an accessible directory and will not be located in {UPLOAD_DIR}).

Change last link of the footer into "Privacy" and create a page for it.

Create a page that discusses personal information, privacy and GDPR. Consult IT-Services. Remember to update the link on every page.

Add 7z suffix to allowed list

Simple enough, will be done soon.

Footer is a mess, in small view ports...

No explanations needed, just reduce view port width and observe.

Flex'ing FOOTER as a solution?

Lisence page for teachers

Note that this needs to come from the REST API so that it is controlled by the SSO authentication.

Front page redesign

Should not display teacher menu when not viewed as a teacher.

Api package Exceptions lose base class

Unknown cause, Breaks error reporting.
When fixed, also add an optional Exception subclass attribute for SQL sentences.

Teacher Help Content

Mostly document linking..

Delegate "Creating Virtual Machine Image" to create.html.
Delegate "Installing Course Specific Software" to create.html.
Evaluate the need for FAQ - one can always be added later, if needed.

In the Teacher VM management view

The cursor should change into hand when going over the list of VMs.

Automatic purge of _columns from result sets

Implement automatic purge of columns that have a name beginning with '_' -character. These columns are considered "hidden" or meta-data, something that is not supposed to be delivered to the clients.

Statistics

Probably requires a cron job to process Nginx logs.

Wanted statistics:
download counts for each image, over weeks (1 - 52/53). Count unfinished downloads separately.

DO NOT IMPLEMENT BEFORE UPLOAD SOLUTION HAS BEEN COMPLETED AND TESTED! Download might also be changed into HTML5 API implementation, and if so, then the statistics need to be maintained and recorded into a database.

Leave this until the download mechanism is truly fixed!

"Right off the bat" -questions (limited to these)
- Do I need this? (only if your course uses virtualization - check your Moodle or talk to your teacher, if you are unsure)
- Is it a security / privacy risk?
- Can I remove it when I don't use it anymore? (link to help page)
Brief intro (with visual aid/diagram)
- What does it do? (fundamental basics )
- Most basic vocabulary (just enough to manage with the install.html page)
Virtualization BIOS setting
- Generic instructions
  -> proceed to (link to) install.html

Add Debian "answers" file for teachers that do their own VMs

Consult Ari, Timo or Tuisku for a file, which I believe they have already created.
Put this up and available, document it into teacher page(s).

Re-style header positioning

Header element should be anchored as:

|  UTU-logo  [scroll-spy ]                                             Login/Logout  |

Publish / webform solution

Create API endpoint api/publish.

GET ?file=<identifier|name> (prepare for publish)
- Test file integrity (for .OVA) ... do nothing for .IMG?
- Extract manifest XML (for .OVA) ...do nothing for .IMG?
- Insert extracted information to database (leave published datetime column empty to signify prepared, but not yet published file).
- HTTP Response JSON containing all (viewable) table columns.
  Solve metadata issue...
- Browser generates an editable form and allows user to modify data. User "Submit" initiates POST into the same endpoint
POST {"file": "identifier|name", ... } (commit publish)
- Check each field for correctness, return errors accordingly.
- Update row with new data (include value for published column!), return SQL errors if any.
- Move file to proper location with other accessible images and .OVAs.
- Return success to client, which confirms success to user, and closes the view on OK.

SHA1 cron job

Background task that looks for files without SHA1 checksum and calculate them.