Legend Background

On November 20, 2019, at FINOS' flagship conference, the Open Source Strategy Forum, FINOS Platinum member Goldman Sachs announced its intention to open source two closely related products into FINOS:

Its internally developed logical modeling language, referred to within the bank as "PURE", and
A workbench of tools, including a visual modeling platform, collectively referred to as "Alloy", created and used within Goldman Sachs to build, design, and execute PURE models

The PURE language, together with four modules - Studio, SDLC, Engine, and Shared - were open sourced into FINOS on October 19, 2020. Concurrent to their open sourcing, they were collectively given a new, unified name - "Legend". The choice of the Legend name reflects how for Goldman Sachs the language and platform have been a guide - a legend - for the bank's data strategy.

Shared Public Instance of Legend

A shared, external instance of Legend deployed on AWS within FINOS' infrastructure is available at https://legend.finos.org/studio. The primary purpose of this instance is to provide a shared modeling environment for the FINOS community and its projects. An additional purpose of this instance is to provide a place where people can try out Legend for themselves, check out its tools and features, and see what a Legend model looks like. This instance of Legend was the same instance that was used by the pilot group to model in Phase 1 (see Roadmap section below for more info on our phases).

To request an account on the shared instance, use the form at the bottom of https://www.finos.org/legend. You will need a GitHub account and, if want to be able to create and edit models, be covered by a Contributor License Agreement with FINOS (otherwise a read-only account will be created).

Note that the features available in the shared, external Legend instance within FINOS infrastructure largely map to the five modules open sourced to date. Features that are not yet available include model-to-database mappings and the execution of queries as services. As more modules are open sourced, more features will be activated in the shared instance.

Installation | Running a Legend Instance Locally | Deploying an Instance of Legend

Information on running and installing your own instance of Legend locally, on premise, or in a cloud provider can be found [here].(https://legend.finos.org/docs/getting-started/installation-guide).

Code (and Model) Repositories

GitHub and GitLab

The Legend platform and project makes use of both GitHub and GitLab.

GitHub

GitHub, specifically the FINOS GitHub organization, is the distributed source and version control management system used for Legend code.
- The Legend GitHub repos, like all FINOS repos, are configured to make use of the FINOS Open Developer Platform tooling.
GitHub infrastructure is what's used to do Legend builds (e.g., of Legend JARS, which are then distributed through Maven Central)

GitLab

GitLab is used for source control of models "under the hood" within the Legend SDLC module. As such the models, written in the Legend language, are stored in GitLab repositories.

Repositores

Code Repositories (GitHub)

This repository (finos/legend) is the parent repository for the project. It is used principally for overall Legend documentation and issue tracking. The code itself for the 5 modules open sourced thusfar can be found in the following 5 child repositories:

Module Specific Issues

Module specific issues are opened, maintained, and prioritized within the modules specific issue list (e.g., https://github.com/finos/legend-studio/issues for Legend Studio). Issues can be aggregated from multiple reposistories as needed through FINOS organization level GitHub kanban boards("Projects").

Model Repositories (GitLab)

Legend models developed on the external shared instance, including those built during the Phase 1 pilot described below, are themselves open sourced code and available in the following GitLab repositories:

FX Options extensions to the CDM: https://gitlab.legend.finos.org/legend-pilot/cdm
Commodities Reference Data: https://gitlab.legend.finos.org/legend-pilot/commonrefdata

Note on model access in GitLab Currently to access (and fork, etc.) these GitLab model repositories directly, you will need an account on the Legend shared instance (an account is also required to model and access these models through the Studio UI). An account may be requested through the account request form at the bottom of https://www.finos.org/legend. In the future we plan to migrate these model repositories to public repos on GitLab.com, after which an account will no longer be needed to access and contain the repositories that contain the models.

Contributing

To learn more about how to get involved with Legend and contribute to it, see the CONTRIBUTING.md file or the Contribute section of the docs site.

Development setup

(Coming Soon)

Roadmap

Phase 0 (October 2019 - January 2020) - COMPLETED

Announced the intention to open source PURE/Alloy (now named Legend) at the Open Source Strategy Forum
Deployed an external instance of Legend to AWS within FINOS infrastructure, initially for use in Phase 1 (see below) for pilot participants to use to do shared modeling.
Commenced internal code evaluation and preparation for open sourcing

Phase 1 (January 2020 - September 2020) - COMPLETED

Opened the external instance of Legend Studio deployed in Phase 0 to a pilot group of financial institutions, composed of FINOS members plus several other institutions that expressed an interest in participating in the pilot, for them together with Goldman Sachs and FINOS to evaluate the use of Legend to do shared, collaborative modeling. In this release, Studio supported a limited set of functionality, including the creation of data models, model-to-model mappings, constraints and derivations
During the pilot phase, modelers collaborated on the Legend external instance to build shared models. Modeling occurred in two work streams:
- An FX workstream of the the pilot group extended and expanded the FX Option model of the Common Domain Model (CDM), developed by the International Swaps and Derivatives Association (ISDA). The Averaging Model developed during the pilot has since been approved by the ISDA Architecture Review Committee and released in the CDM (see 2.82.8).
- The pilot group also created a Commodity Reference Data workstream, to work on commodities data, especially in the context of a Fixed vs Float swap, including defining a payout model to be proposed into the CDM.
Other areas that the pilot group have discussed as potential areas to model together in Legend include Environmental, Social and Governance (ESG) data as well as potentially continuing some of the work started by the FINOS Financial Objects program on RFQ in the context of interest rate swaps.
Performed code readiness work to extract Legend code from GS infrastructure, and sufficiently abstract and generalize code base as needed. Revised code base to reflect new name (note that some references to old "PURE" and "Alloy" name still exist). GS Engineering, together with FINOS and Linux Foundation DevOps teams, performed code scanning and remediations. Moved Goldman Sachs's daily production builds of Legend to FINOS infrastructure and public GitHub repos.

Phase 2 (October 2020 - December 2020)

On 2020.10.19 made underlying code bases to five Legend modules available as open source under an Apache 2.0 license
Announced new name (Legend) and rebranded
Identify "good first issues" for developers new to Legend
Begin accepting pull requests of code from external (non-GS) developers.
Conduct office hours for external developers interested in getting involved with Legend

Phase 3 (Q1 2021)

Make additional functionality available in Studio, including relational execution, support for flat data sources (e.g., csv) and automated code generation (e.g., JSON schema, Protobuf, AVRO).
Transition shared model repositories from hosted GitLab instance (gitlab.legend.finos.org) to gitlab.com.

Phase 4 (Q3 2021)

Open source Cube UI for user-friendly tabular data exploration.

Phase 5 (Q4 2021)

Open source Query UI to search and access modeled data.

Phase 6 (Q4 2021)

Open source Services UI. This will allow managing of production data services (APIs) for repeatable and controlled data access.

Documentation

Legend Documentation Site (legend.finos.org)

The documentation site is built using Docusaurus based on the markdown files in the docs folder.

Feel free to suggest areas for documentation improvement by raising an issue

This Repository - GitHub.com (github.com/finos/legend)

This repository (github.com/finos/legend) hosts:

The underling files for the Legend Documentation site (composed in markdown file format and stored in the docs/ folder)
The shared issues list used for task management to track project activity around documentation, deployments, and the overall effort associated with open sourcing Legend
These issues are organized in various kanban boards, including the overall project kanban.

FINOS Hosted GitLab Repository (gitlab.legend.finos.org)

A separate hosted Gitlab repository, gitlab.legend.finos.org hosts shared models that were developed by the Legend project pilot team.

gitlab.com (https://gitlab.com/finosfoundation) [Not Yet Used]

In the future we intend to migrate the models repositories from the hosted GitLab repository to https://gitlab.com/finosfoundation. For now, this is unused but will eventually be the long-term home for public models built using Legend.

Find out more

Review the existing materials

Raise an issue

If you cannot find the answer to your question in the above materials, please raise an issue at https://github.com/finos/legend/issues

Join the upcoming Legend meetings

Legend Office Hours

Legend Office Hours are hosted by experts from Goldman's Securities and Engineering divisions to answer the Community's questions about Legend. The next Office Hours are scheduled for Thursday, December 10th, 11:30am ET.

Legend General Meeting in January 2021

Legend General meetings are your opportunity to get updates on Legend, provide feedback on the product and the roadmap, see demos, and ask questions. The next general meeting is scheduled for January 28, 2021.

License

Distributed under the Apache License, Version 2.0.

SPDX-License-Identifier: Apache-2.0

stjordanis / legend Goto Github PK

legend's Introduction