Public Policies and Schemas about guardian HOT 10 CLOSED

As per @sergmetelin the Mindset we need to have is that Hedera is the Source of Truth. Every single record needs to go to Hedera first. Everything needs to be attested by the Root Authority. All of this needs to be recorded on the Hedera Consensus Service.

As per @sergmetelin , we should use IPFS for now. Everything is public and all SDK’s are available.

Definition Of Done is in the Schemas, there needs to be post a link to the Schema in the topic after we post the Schema on IPFS, we also post a Topic Message. Topic needs to be an auditable log of everything that the Root Authority does. Currently, there is no access to the linked Schema, therefore it is not auditable. Root Authority needs to create a schema, the schema is posted to IPFS, then the ID is posted to the topic. Every Root Authority action needs to be recorded under that topic, every schema, every credential. Policy has its own topic, and Root Authority has its own topic. From the moment the Root Authority came in, the Guardian should be tracking ALL of the events. Its important to have Hedera as a Source of Truth in disaster recovery (DR) situations (Shawn needs to add this as another Issue)

DOD – A Hedera Improvement Proposal, new Features, Frameworks, all in a Public manner needs to be issued describing how this works. This may be an addition to HIP 28 or a HIP 28 Updated. A flag describing how the Policy Engine works. Creating a HIP will be an entirely new Issue, @envisionblockchainpm to create as an Action Item.

from guardian.

We need to gather more requirements from Hedera on this ticket during todays Grooming.

from guardian.

Next step is to have a discussion on this as per this mornings Scrum.

from guardian.

Next Step for this is to have a meeting with developers, Serg and for Andrey to have a Review Document completed prior to this meeting. Shawn to get Serg's timing on meeting next week for another Discovery Session,

from guardian.

Next step is on Andrey to reach out to Serg for a one on one Tech Talk

from guardian.

@sergmetelin Upon deeper analysis I have concerns that using IPFS here provides any real advantages. I'll summarise them here, and hopefully we can discuss them either in a meeting of asynchronously in the ticket.

There are three options of enabling storage of information on IPFS:

• running your own nodes
• paying a service provider such as Pinata for hosting content in IPFS
• a combination of the two above

To host data on IPFS it requires running you own server, and saving content onto it. By default this information would be only available on this one server (not replicated), and only for the duration while the application/node is running. There is a way of making the data replicate which involve costs (Filecoin), or the data only gets replicated when someone else from elsewhere needs to access the file, and I believe for a limited time.

Instantiating and operating an IPFS node is not a trivial task, and is likely to provide a barrier to entry comparable to, if not more severe, using an unfamiliar API. Introducing IPFS technology will complicate Guardian solution architecture. It is much easier and cheaper to achieve geo-replicated and backed-up storage using commercial cloud providers services.

For demo/test environment running your own node seems to be the only option since costs and account setup etc for just with storing test/demo data seems to be unacceptable.

The alternative is to use Pinata service offering for IPFS access. This is simpler since no technical operations are required, and simpler to implement because Pinata offers an API for storage and retrieval. However it involves costs (if the usage goes beyond the very small volume of 'free tier') and complexities of setting up and maintaining accounts. It also introduces dependency on the external commercial service provider.

I propose we reassess the benefits of using IPFS, and consider alternatives such as for example introducing a solution based on a new set of Guardian APIs for discovering and accessing published schemas and policies on the instance. Each each instance of the Guardian would have a well-known API access point which can be accessed in various ways.

from guardian.

As of today, Andrey is discussing this with @sergmetelin .

from guardian.

The outcome of the conversation is the following design which is an MVP for the public availability of schemas and policies only:

• Local Guardian instance includes a DB which contains drafts and 'cached' published policies and schemas
• When a schema or a policy version is published it becomes immutable and is uploaded onto the IPFS via nft.storage API. NFT Storage guarantees the permanence of the uploaded info on IPFS.
• The ID of the published schema/policy is posted into the corresponding Hedera topic.
• This should work with the versioning functionality as per #46

There will be another ticket to do with the 'discoverability' of assets (Policies, Schemas) and their 'import' into a Guardian instance for DR and other reasons.

There is a broader ticket #324 that covers all the work we are going to do to decentralise Guardian, beyond to just publishing schemas/policies on IPFS.

Please note that published data need to be verifiably immutable as per #162.

from guardian.

Next step:

• @Pyatakov design session with development for the functionality

from guardian.

Alex has this been Designed and have you reviewed with the Development Team?

from guardian.