latticexyz / mud Goto Github PK

Currently you can use Type.String and defineStringComponent to handle large numbers like uint256s client-side, but this means that you have to convert them to bigints any time you want to do arithmetic with them, and then convert them back to a string if you want to save the new value. Having the ability to get and set bigints directly to/from the component would be nicer.

This is mostly in an idea phase rather than a well-formed proposal. Hoping to start a discussion around use cases and potential solutions.

There are many cases in which you might want to know when certain actions are being taken by players, that may or may not result in component updates, e.g. a player attacks another player and misses.

We have a system call stream, but it's just a different view on component updates (grouped by system calls), so it depends on a component updates being emitted for the system call to propagate to clients.

This also means that you have to infer what happened based on component updates, essentially duplicating logic on the client that already exists in the contracts (source of truth). "Attack system was called, and player health lowered, so they must have landed a hit and taken damage." We could expand the system call stream to include all system calls, but it still requires inferring behavior in the client, and streaming this data from the RPC node isn't as efficient as filtering on events.

For cases where you might want to "listen" to things like a "missed attack", you can create/update an arbitrary component to get it into the system call stream, and then check that no damage was done (or however you model your combat system), but this feels more like a workaround and adds a lot more gas.

I'd like to propose we have some generic Solidity event type that you can emit from systems (and/or is emitted automatically on execute calls) that can represent named actions. Then expose an action stream so that UIs can trigger animations, effects, etc. We can create some patterns that help guide folks away from one-off modeling and preferring ECS where possible, but I think the cases above just can't be modeled well in our current setup with system call stream, etc.

Right now entity ids (which are uint256 on the contracts) are stored as hex strings on the client. We need to convert entity ids to strings to use them as key in objects/maps. But converting to hex wastes memory, because javascript stores strings with 2 bytes per character, but hex only utilizes 16 characters. Instead we should use a base64 string encoding for entity ids on the client and convert it back to hex/bigint just before sending it to the contract.

Upon adding the mud noise package, this error appears. This is because ABDKMath64x64 isn't a dependency of noise despite using it in Perlin.sol

Preface:
I thought about this when making solecslib. It wasn't very relevant there however.
Thinking about this issue led me to discover the bug in #286, and what I suggest here depends on the fix there.
I'm not actually sure this is an issue, curious about your take on it.

The issue:
RegisterSystem takes an ID as-is, which is fine when not many people use MUD, but might have problems for adopting it as a general development framework.

1. Hurts extensions by argument bloat. You won't notice this with e.g. CoordComponent, that's too abstract to need an id. But if one were to e.g. make a reusable StaminaSystem and StaminaComponent, the system would have to take staminaComponentId as an argument. Otherwise a hardcoded ID means a world can only have 1 deployer use this stamina extension.
2. Front-running security risks (or at least developer experience issues). If you make your IDs public before registering them in a world (e.g. via a github repo), then someone might register the ID before you. It's fine if you are deploying a new world. It's not fine if you're developing for an existing world.

The basic fix I almost tried out before running into #286 was namespacing an id with the deployer address. That would be baked into RegisterSystem, it already even receives msgSender, but only uses it for owner check atm. Owner check would be obsolete, ownership becomes a role for just managing the component, not deploying it.

• This lets anyone reuse ids by just having different deploy addresses
• Allows immutable ids by burning the deployer
• And the owner can still be unburned for contract-determined admin stuff
• But also allows weird cases where the component owner isn't related to the component deployer

The obvious new difficulty: To use a component you need to pass in both id and deployer address.
For most current projects/use-cases this can actually be solved easily:

function namespaceId(address deployer, uint256 id) pure returns (uint256) {
  return keccak256(abi.encode(deployer, id));
}

// this is the new function
function getAddressById(IUint256Component registry, address deployer, uint256 id) view returns (address) {
  uint256 namespacedId = namespaceId(deployer, id);
  uint256[] memory entities = registry.getEntitiesWithValue(namespacedId);
  require(entities.length != 0, "id not registered");
  return entityToAddress(entities[0]);
}

// this is the modified old function
function getAddressById(IUint256Component registry, uint256 id) view returns (address) {
  address owner = OwnableStorage.owner();
  getAddressById(registry, owner, id);
}

To access owner in a free function you need diamond storage for it, which I coincidentally add in #265.
Changing a system's owner becomes dangerous however, potentially breaking the system if the dev didn't know about this namespacing thing. (To circumvent that a system could store the deployer too?)

Anyways, adds complexity, not sure it's a good idea

in definePhaserConfig users should be able to override render params (like isPixelArt)

I have had trouble understanding how these concepts work in relation to the rest of the client -- documentation on Mud.dev would be useful

I want to make the scroll wheel zoom in and out but I'm unable to using PhaserX because the wheel property isn't exposed 😢

We should have a way for the current owner to update the owner. We also might consider inheriting from Ownable in one of the standard set of solidity libraries (OpenZeppelin, etc.)

Originally posted by @holic in #244 (comment)

I have two use cases currently:

1. ZK proof verifier contracts auto-generated by snarkjs from circom circuits. They are just standard solidity contracts with a verifiyProof function.
2. Poseidon hash function contracts (with various input counts), which actually take the form of raw bytecode that is generated by a circomlibjs function. Currently I am copying this bytecode directly into a PoseidonSystem and then deploying it with an assembly create() call in the system constructor.

For both of these cases, I think it makes the most sense to store the resulting contract addresses in an AddressComponent, but I am wondering what the best way is to include the creation of these contracts into the mud deployment process, or non-standard use cases like this more generally?

While I was working on create-mud, I noticed an npm install failed due to missing peer deps.

I see you have added --forgeOpts as a flag to mud test, but I'm struggling to get it to work.

mud test -vv --forgeOpts '--mt testFiringArea' fails, along with mud test -vv --forgeOpts '--match-test testFiringArea'., mud test -vv --forgeOpts --mt testFiringArea, etc. What is the proper syntax here?

TLDR: abi.decode has stack overflow errors when the data it decodes is too complex (even though the data isn't really that complex). I would recommend either setting the optimizer to false by default or providing an alternate method of execution that doesn't involve abi.decode when passing in complex structs.

In one of my systems, I am passing in an array of Action structs as follows:

    Action[] memory actions = abi.decode(arguments, (Action[]));

The struct is organized as follows:

struct Action {
  uint256 shipEntity;
  bytes[2] metadata;
  ActionType[2] actionType;
}

This gives me the following error:
CompilerError: Stack too deep. Try compiling with --via-ir (cli) or the equivalent viaIR: true (standard JSON) while enabling the optimizer. Otherwise, try removing local variables. When compiling inline assembly: Variable headStart is 1 slot(s) too deep inside the stack.

To fix it, I set optimizer to false in foundry.toml.

IMO there should be a workaround when you want to use complex structs as system parameters or the optimizer should be off by default

Right now forge errors get kinda swallowed. Forge itself emits some errors, but we continue with the script as if it succeeded. Then the script fails because it can't find forge's deploy artifacts.

See:

forge build -o ./out
[⠊] Compiling...
[⠢] Compiling 80 files with 0.8.13
[⠆] Solc 0.8.13 finished in 38.86ms
Error:
Compiler run failed
error[6275]: ParserError: Source "../../../mud/node_modules/@solidstate/contracts/contracts/access/ownable/Ownable.sol" not found: File not found. Searched the following locations: "/Users/kevin/Projects/latticexyz/emojimon/packages/contracts".
 --> /Users/kevin/Projects/latticexyz/mud/packages/solecs/src/Ownable.sol:4:1:
  |
4 | import { Ownable as SolidStateOwnable } from "@solidstate/contracts/access/ownable/Ownable.sol";
  | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^



error[6275]: ParserError: Source "../../../mud/node_modules/@solidstate/contracts/contracts/access/ownable/OwnableStorage.sol" not found: File not found. Searched the following locations: "/Users/kevin/Projects/latticexyz/emojimon/packages/contracts".
 --> /Users/kevin/Projects/latticexyz/mud/packages/solecs/src/Ownable.sol:5:1:
  |
5 | import { OwnableStorage } from "@solidstate/contracts/access/ownable/OwnableStorage.sol";
  | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^



error[6275]: ParserError: Source "../../../mud/node_modules/@solidstate/contracts/contracts/interfaces/IERC173.sol" not found: File not found. Searched the following locations: "/Users/kevin/Projects/latticexyz/emojimon/packages/contracts".
 --> /Users/kevin/Projects/latticexyz/mud/packages/solecs/src/interfaces/IERC173.sol:4:1:
  |
4 | import { IERC173 } from "@solidstate/contracts/interfaces/IERC173.sol";
  | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^



error[6275]: ParserError: Source "../../../mud/node_modules/@solidstate/contracts/contracts/access/ownable/Ownable.sol" not found: File not found. Searched the following locations: "/Users/kevin/Projects/latticexyz/emojimon/packages/contracts".
 --> ../../node_modules/@latticexyz/solecs/src/Ownable.sol:4:1:
  |
4 | import { Ownable as SolidStateOwnable } from "@solidstate/contracts/access/ownable/Ownable.sol";
  | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^



error[6275]: ParserError: Source "../../../mud/node_modules/@solidstate/contracts/contracts/access/ownable/OwnableStorage.sol" not found: File not found. Searched the following locations: "/Users/kevin/Projects/latticexyz/emojimon/packages/contracts".
 --> ../../node_modules/@latticexyz/solecs/src/Ownable.sol:5:1:
  |
5 | import { OwnableStorage } from "@solidstate/contracts/access/ownable/OwnableStorage.sol";
  | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^



error[6275]: ParserError: Source "../../../mud/node_modules/@solidstate/contracts/contracts/interfaces/IERC173.sol" not found: File not found. Searched the following locations: "/Users/kevin/Projects/latticexyz/emojimon/packages/contracts".
 --> ../../node_modules/@latticexyz/solecs/src/interfaces/IERC173.sol:4:1:
  |
4 | import { IERC173 } from "@solidstate/contracts/interfaces/IERC173.sol";
  | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^



Cleaning output directory (./abi})
Error: ENOENT: no such file or directory, scandir './out'
    at readdirSync (node:fs:1438:3)
    at getContractsInDir (/Users/kevin/Projects/latticexyz/mud/packages/cli/dist/commands/deploy-contracts.js:216299:37)
    at filterAbi (/Users/kevin/Projects/latticexyz/mud/packages/cli/dist/commands/deploy-contracts.js:216313:21)
    at generateTypes (/Users/kevin/Projects/latticexyz/mud/packages/cli/dist/commands/deploy-contracts.js:216417:5)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async generateAndDeploy (/Users/kevin/Projects/latticexyz/mud/packages/cli/dist/commands/deploy-contracts.js:217374:5)
    at async Object.handler (/Users/kevin/Projects/latticexyz/mud/packages/cli/dist/commands/deploy-contracts.js:217464:70) {

the verbosity flag in mud test sometimes doesn't work, and sometimes it does. It seems forge may cache the preferred verbosity level.

For example:
After running the command yarn test -v, the script executes:

For now, my workaround is stating verbosity in the --forgeOpts param instead.

I am confused about how to use setComponent for PhaserX objects.

The problem: I want to initialize a set of values for an object that will granularly update later. I see a couple ways of doing this:

1. Create a new phaserX object component for each phaser attribute and update each individually in each update.
   My concern is this is a lot of extra code, especially for phaser objects with many attributes
2. Use one phaserX object component throughout the codebase for a single phaser object. Whenever any phaser object attribute needs to be updated, the same phaserX component is updated each time.
3. Update the same phaser object attribute across different phaserX object components.
   I'm not sure if this will cause re-render issues as different phaserX components will conflict.

Which of these, if any, would you recommend?

Currently, defineRxSystem in [recs](https://github.com/latticexyz/mud/blob/main/packages/recs/src/System.ts) does not return the rxjs Subscription object, but only registers the unsubscribe to world.registerDisposer()

This prevents the ability to granularly unsubscribe from systems, which is useful in many situations, like building a game with different game modes, for example.

Easy possible fix: Return Subscription object. Current workaround:

function defineRxSystemUnsubscribable<T>(
  world: World,
  observable$: Observable<T>,
  system: (event: T) => void
): Subscription {
  const subscription = observable$.subscribe(system);
  return subscription;
}

export function defineComponentSystemUnsubscribable<S extends Schema>(
  world: World,
  component: Component<S>,
  system: (update: ComponentUpdate<S>) => void,
  options: { runOnInit?: boolean } = { runOnInit: true }
): Subscription {
  const initial$ = options?.runOnInit
    ? from(getComponentEntities(component)).pipe(toUpdateStream(component))
    : EMPTY;
  return defineRxSystemUnsubscribable(
    world,
    concat(initial$, component.update$),
    system
  );
}

Credit for this proposal goes to @holic

Current situation

• We're currently distinguishing between EntityID and EntityIndex on the client. EntityID is the entity's uint256 value converted to a hex string, EntityIndex is a client only number referring to the entity's index in the client World's entities array.
• The reason for this split is that we want to use entity ids as keys in our client components, but storing the raw hex string would cause a lot unnecessary memory usage (as the full id string would be duplicated in every component it is used as a key)
• However, this split frequently causes confusion and converting back and forth between index/id is inconvenient and requires access to the World object

Proposal

• Use JavaScript's native Symbol feature instead of a local mapping from id to index
• Symbols use a native global registry to map from number or string to symbol, and can be used to efficiently index objects/maps

Usage

// ------- NEW -------
function entityToSymbol(entity: string) {
   return Symbol.for(entity);
}

function symbolToEntity(entitySymbol: symbol) {
   return Symbol.keyFor(entitySymbol);
}

// ------- OLD -------
function entityToIndex(entity: string, world: World) {
   return world.entityToIndex.get(entity);
}

function indexToEntity(entityIndex: number, world: World) {
   return world[entityIndex];
}

Note how this is a pure util and doesn't require access to the world object. We could go one step further and automate the conversion from entity to symbol and back within all MUD utils / components so devs never have to think about it.

const a = Symbol("0x00")
// undefined

const b = Symbol("0x00")
// undefined

const c = Symbol.for("0x00")
// undefined

const d = Symbol.for("0x00")
// undefined

a == b
// false

a == c
// false

c == d
// true

Symbol.keyFor(a)
// undefined

Symbol.keyFor(c)
// '0x00'

• Remove project setup assumptions from MUD CLI
• Move LibDeploy.ejs / codegen for deployment to MUD instead of individual project repos

Some files still use Unlicense even though all of MUD is MIT licensed. We should update all license declarations to reflect that.

this one's probably better explained with a visual:

$ mud call-system --systemId ConfirmTrade --args ...
EntityID 91
| Component | Value |
| --------- | ----- |
| State | 'Complete' |
| Requester | 19 |
| Requestee | 27 |

EntityID 92
| Component | Value |
| --------- | ----- |
| State | 'Confirmed' |
| TradeLogID |  91 |

EntityID 97
| Component | Value |
| --------- | ----- |
| State | 'Confirmed' |
| TradeLogID |  91 |

This could probably be achieved now by defining and calling a System that takes in an entity ID as input, but integrating this into the call-system workflow would really help tighten the feedback loop on any manual testing. Imagining either a config file that keeps track of entity IDs to report on

$ mud config add watched-entities 91,92,97
$ mud call-system --systemId ConfirmTrade --args ...
EntityID 91
| Component | Value |
...

or a flag for listing those entities

$ mud call-system --systemId ConfirmTrade --args ... --entities 91,92,97

MUD (especially recs) started out building heavily on mobx as foundation for reactivity, but at some point switched to rxjs for performance reasons. However, mobx is still used in some places of the codebase, leading to "glue code" to convert mobx observable into rxjs observables and vice versa. We should remove all usages of mobx (eg replace them with rxjs's BehaviorSubject) to be able to remove mobx as a dependency.

I updated MUD and got a fun new error, so the world contract is undefined. Even so I am told that contract deployment was successful.

It the CLI already supports calling a system using system[Id|Address], --args and --argTypes. Extending that functionality, a "natural language" input with some argument parsing would significantly smoothen that experience.

e.g.

$ mud call-system --??? mud.System.ConfirmTrade(uint: {charID}, uint: {tradeLogID})

in this example the ConfirmTrade system is called using uint inputs charID and tradeLogID, as specified by the type casting. taking this one step further, we could imagine:

$ mud call-system --??? mud.System.ConfirmTrade({charID}, {tradeLogID})

type inference could be achieved (when none is provided) by relying on the abi jsons found in contracts/src/abi/ or an equivalent source generated during build. using that same source we could probably achieve something like:

$ mud call-system --??? ConfirmTrade({charID}, {tradeLogID})

as an ideal format for calling systems through the CLI

Inspired by #361 and the hardhat config

Currently deploy.json is a bit arcane in what it expects, and json is very strict with " everywhere.
Having a ts config with an attached type seems like it'd offer better devex and make deploy.ts easier to extend with more options without making those options super obscure.
Also a ts config partially solves the repeated writeAccess problem since you can use variables for similar arrays.

Naturally deploy.json and deploy.ts can coexist;
some other things to consider:

• mud uses ts anyways, deploy.js is not needed
• deprecating deploy.json might make sense, at least in the sense of not using it in tutorials, templates etc

https://hardhat.org/hardhat-runner/docs/guides/typescript
basic hardhat.config.ts:

import { HardhatUserConfig } from "hardhat/config";
import "@nomicfoundation/hardhat-toolbox";

const config: HardhatUserConfig = {
  solidity: "0.8.17",
};

export default config;

Currently I am using defineNumberComponent to store entities and I cast the component value to an EntityIndex. Having a built-in defineEntityIndexComponent in the std-client makes code more legible and adds some type safety.

Frequent updates of coordinate states are not possible to implement in a smart contract. Since updating a smart contract requires gas, I'm curious how mud implements recording and updating character coordinates.

ISystem is hardcoded in LibDeploy.ejs and LibDeploy.sol

so running mud test gives error

Compiler run failed
error[7407]: TypeError: Type contract MyPayableSystem is not implicitly convertible to expected type contract ISystem.
  --> src/test/LibDeploy.sol:84:14:
   |
84 |     system = new MyPayableSystem(world, address(components));
   |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Reported by @emersonhsieh

Minimal reproduction case:

https://github.com/emersonhsieh/mud-minimal-test
https://github.com/emersonhsieh/mud-minimal-test/blob/main/packages/client/src/App.tsx

Currently, the call-system on the CLI requires multiple sources of inputs (callerPrivateKey and world) that could be inferred during local testing. Would be dope if the two flag arguments could be saved to a config file during deployment and defaulted to when calling the command without those parameters.

Is it possible to add phaser groups to PhaserX? I'm currently manipulating multiple sprites as a single group, and it would be nice to be able to store them as a single object in the objectPool.

I am getting odd behavior in my test suite -- my msg.sender value changes when I call a system, which breaks my test. This wasn't happening before I updated MUD to v31.1 -- any ideas why this is occurring?

testSpawn.t.sol:
    console.log("msg sender in spawn test:", msg.sender); // -> 0x1804c8
    spawnSystem.executeTyped("Jamaican me crazy", Coord(1, 1));

execute(bytes memory arguments) within SpawnSystem.sol:
    (string memory name, Coord memory location) = abi.decode(arguments, (string, Coord));
    console.log("msg sender in spawn system:", msg.sender); // -> 0x416C4

We've had a bunch of related questions and would be nice to point folks to a source of truth for the system call stream.

When a transaction, called in the client by

systems["system.xxx"].executeTyped(...)

cause a revert, an error is thrown:

What I would like is to be able to catch this error on the app-level.

Followup from #232

Add support when using the Streaming service. Currently only supports events over RPC.
Relevant RFC for Streaming service support: https://www.notion.so/latticexyz/System-Call-Events-in-Stream-Service-b04601c721c644859e890e2cee984c17

This is a note for other devs:

If you set a contract value in Solidity as anything greater than a uint32, the client will receive that value as a hex string regardless of what component type (Type.Number) you give it when you create a component in createNetworkLayer.ts.

If you want a uint256 component to be treated as a number client side, you need to set its value in createNetworkLayer.ts to be Type.String, then manually convert it to a number (assuming the value is <= uint32 ) when it's used in the client.

As a general practice, we recommend using uint32 -> Type.Number and uint256 => Type.String.

Problem description

Systems encapsulate logic. It would be nice if it was possible to use systems as modular building blocks to create higher level systems, but when a system is called from another system, any access control logic related to msg.sender fails, because the calling system is the new msg.sender.

A common workaround is to encapsulate logic in libraries instead, use those libraries as building blocks and use systems only as entry points to the application. However, this comes with two issues:

1. It is hard to keep track of which components the libraries need write access to. Write access control to components happens on the level of systems, not libraries, therefore every system using a library needs to be granted write access to the components the library writes to.
2. This workaround works for development, when the deployer of the components and systems is the same, so systems can be granted write access to any component. But it is not possible for third party developers to use the logic encapsulated in libraries as building blocks for their own higher-level systems (because they're lacking write access to the required components).

Example of a situation where it would be nice to use systems as building blocks, but we run into the issue of msg.sender being different:

• A MoveSystem has access to a PositionComponent and implements the rules of movement in the world (eg. the entity to be moved needs to be owned by the system caller (msg.sender), and the target position needs to be adjacent to the current position)
• Imagine a third party developer wants to extend the World by adding a PathSystem, which batches calls to MoveSystem to move multiple steps in one transaction. In an ideal world PathSystem could just call MoveSystem multiple times with valid steps, thus respecting the existing rules and adding a higher level system.
• But the call from PathSystem to MoveSystem fails, because msg.sender is PathSystem and not the owner of the entity to be moved.

Approaches

1. access control via tx.origin

• use tx.origin instead of msg.sender -> not recommended because it can lead to phishing vulnerabilities and discriminates against smart contract wallets.

2. Subsystems

• Proposed in #268, see #319 (comment)
• Downsides to this approach
  • Development issues are solved, but it's still not possible for third party developers to use systems as building blocks to build higher level systems. (Which is intentional in this approach, because subsystems are supposed to implement partial logic that might assume permission checks to be done by the parent system).

3. Global entry point

• Another rough idea that was floating around was to add a global entry point to the World contract.
• The entry point function could store the caller in a temporary variable on the World, before calling the requested systems, and finally resetting the caller variable to some placeholder value.
• Systems could use this global caller variable for access control instead of msg.sender, thereby allowing systems to call other systems while preserving the initial caller.
• Downsides to this approach
  • This approach feels similar to using the generally frowned upon tx.origin for access control, with the exception of supporting smart contracts wallets, and maybe less phishing potential because the entry point call has to be explicit (instead of the possible use of fallback functions with tx.origin).
  • This approach would add a small gas overhead to every entry point call (around 5800 if the global caller variable is never set to 0 but another non-zero placeholder value).
  • With the ability to upgrade systems it might be possible to trick users into calling a malicious system, which then calls another system with full permissions of the user.

And also the corresponding AddressArrayComponent and AddressArrayBareComponent to std-contracts.

Call me pedantic, but the capitalization of ID vs Id is confusing me as it is different in various places throughout the codebase.

For example, ComponentDevSystem uses componentId and chainId is the typical usage but otherwise ID is capitalized in almost every case.

Just a little thing -- I personally prefer Id instead of ID

I have begun to use defineComponentSystem because the component within its system's update param is strictly typed. defineSystem's update param has generic Component schema.

However, the update param in defineSystem includes the update's type, whereas defineComponentSystem doesn't. IMO it makes sense for defineComponentSystem to include the update's type so users can handle different update types differently without needing to check if update.value [0] is undefined.

Note: I’ll use keccak(a,b,c) in this proposal to refer to keccak256(abi.encode(a,b,c))

Problem description

This proposal is a solution for multiple problems:

1. Session wallets:
   Currently, we mainly use burner wallets to reduce the friction of manually approving each transaction. However, this approach is not suitable for storing valuable or permanent assets because the private key of the burner wallet is stored in the browser's local storage and is therefore vulnerable to phishing or loss. The current alternative is to use regular wallets such as MetaMask, which drastically increases the friction of using tx-heavy applications because each tx must be approved. We need a way to store valuable and permanent assets in a main wallet, but interact with the application via a temporary session wallet.
2. Modular systems:
   Described in more detail in #319. Short summary: Currently, systems can only serve as an entry point into the application, but it is inconvenient to use systems as stateful libraries or modules to build other higher level systems.

Proposal description

On a high level, the idea is to create a mechanism to allow any address to approve any other address to call certain systems on its behalf.

It is inspired by ERC20’s approve / transferFrom pattern, @dk1a’s Subsystem proposal (#319 (comment) / #268) and @dk1a's comment about approvals here: #319 (comment)

Changes to World contract

We add an ApprovalComponent and ApprovalSystem to the MUD World contract.

• ApprovalComponent

  • EntityID: keccak(grantor, grantee) (for generic approvals) or keccak(grantor, grantee, systemID) (for approvals restricted to certain systems)

  • Value: Approval

    struct Approval {
       uint128 expiryTimestamp;
       uint128 numCalls;
       bytes args;
    }

    • (Side note: To save gas when verifying approvals, expiryTimestamp and numCalls could be bitpacked into a single storage slot. To save even more gas, one bit in this bitpacked slot could be reserved as a flag to represent whether the args value is empty or non-empty.)

• ApprovalSystem

  • Has write access to ApprovalComponent
  • Functions:

    • setApproval(address grantee, uint256 systemID, Approval memory approval)

      • Creates a new approval from msg.sender as grantor to grantee. To create a generic approval, systemID can be set to 0.

    • reduceApproval(address grantor, address grantee, bytes args)

      • Throws if no valid approval is found, reduces numCalls if applicable (see below pseudo code for details)

        function reduceApproval(address grantor, address grantee, bytes memory args) public {
        	// Check for a generic approval
        	(Approval memory approval, bool found) = getApproval(grantor, grantee);
        	
        	// Return successfully if a valid approval is found
        	if(found && approval.expiryTimestamp > block.timestamp) return;
        
        	// Check for a specific approval
        	uint256 systemID = getIdByAddress(msg.sender); // Only the approved system can reduce the approval's numCalls value
        	(approval, found) = getApproval(grantor, grantee, systemID);
        
        	// Trow if no approval is found
        	require(found, "no approval");
        
        	// Throw if expiry timestamp is in the past
        	require(approval.expiryTimestamp > block.timestamp, "approval expired");
        
        	// Throw if numCalls is 0
        	require(approval.numCalls > 0, "no approved calls left");
        
        	// Throw if args exists and doesn't match approved args
        	require(approval.args.length == 0 || approval.args == args, "args not approved");
        
        	// Reduce numCalls, unless it's MAX_UINT128 to support permanent approvals
        	if(approval.numCalls != MAX_UINT128) {
        	     approval.numCalls--;
        	     setApproval(grantor, grantee, approval);
        	}
        }

    • revokeApproval(address grantee) / revokeApproval(address grantee, uint256 systemID)

      • Removes the approval for the given grantee / systemID

    • More functions like checkApproval can be added, but are less relevant for this proposal

Changes to Systems

Every system implements its logic in an internal _execute(address from, bytes memory args) function.

The base System contract implements the following functions:

• execute(bytes memory args): executes the system as msg.sender

  function execute(bytes memory args) public {
  	return _execute(msg.sender, args);
  }

• executeFrom(address from, bytes memory args): checks if msg.sender has approval from from to call the system, then executes the system as from

  function executeFrom(address from, bytes memory args) public {
  	ApprovalSystem.reduceApproval(from, msg.sender, args);
  	return _execute(from, args);
  }

• executeInternal(address from, bytes memory args): checks if msg.sender has approval from address(this) to call this system. If so, the call is trusted and the system is executed as from

  function executeInternal(address from, bytes memory args) public {
  	ApprovalSystem.reduceApproval(address(this), msg.sender, args);
  	return _execute(from, args);
  }

Applications

Session wallets

• At the beginning of each session, the user gives a generic (or restricted) approval with expiryTimestamp to a burner wallet.
• The client calls all systems via their executeFrom functions as the burner wallet, with from set to the main wallet. All assets are owned by the main wallet.
• After the session, the main wallet can revoke the approval, or the approval runs out after the expiryTimestamp

Contract interaction

• Similar to ERC20’s approve / transferFrom pattern, approvals can be used for atomic interactions with contracts, like
  1. User approves a contract to call a TransferSystem once with a certain argument to transfer ownership of a certain item to the contract.
  2. User calls the contract’s swap function, which calls TransferSystem on the user’s behalf to transfer the item to the contract, and then calls TransferSystem on its own behalf to transfer another item from the contract to the user. (Note how this is not possible if TransferSystem only uses msg.sender for access control.)

First party modular systems / using systems as stateful libraries

• Developer wants to modularize movement logic into a MoveSystem, and use it as building block in a PathSystem and CombatSystem (since it’s the same developer, there is full trust in PathSystem and CombatSystem)
• MoveSystem can create a permanent generic approval for PathSystem and CombatSystem, to allow both to call MoveSystem's executeInternal function. (Just like a library, except component access control happens on the level of MoveSystem instead of PathSystem/CombatSystem)
• This use-case and usage is very similar to the approach explored in @dk1a’s subsystems #268
• Creating these permanent generic approvals between first party systems can be automated via the MUD CLI / deploy process.

Third party modular systems

• Assume a third party developer wants to create a new higher level system by combining multiple existing lower level systems.
• They can use the lower level system’s executeFrom functions to do so.
  • This requires users to explicitly approve the higher level system to call the lower level systems on the user’s behalf (with optional restrictions), which is intended to prevent systems from being able to call any system on behalf of users.
  • The approval process can be a default client pop-up integrated into MUD.
• Example:
  • There is a MoveSystem deployed by an unknown developer.
  • A third party developer wants to add a new PathSystem to automate pathfinding and travelling larger distances.
  • PathSystem declares MoveSystem as dependency, client automatically shows a pop up for user to give necessary permissions the first time user attempts to call PathSystem.
    • User calls RegisterSystem.setApproval(PathSystem, MoveSystemID) to allow the PathSystem to permanently (or with restrictions) call MoveSystem on user’s behalf.
  • From now on, PathSystem can call MoveSystem.executeFrom on behalf of the user to automate movement.

We shouldn't train folks to pass around their private key in scripts, URLs, etc. so we should make it easier to use our CLI commands with a .env file. For example, auto detecting if DEPLOYER_PRIVATE_KEY is set and using that in deploy-contracts.

In TS, if I use getComponentValue when the value is a typescript object, it returns a reference to the component value. This means I can update the component values without using setComponent. This creates issues when I want to compare the old update and the new update in a createUpdateSystem function, because the before and after update values are identical. To me it makes sense to pass the component by value, this way the only way to alter the object is through setComponent.

For example (please ignore terrible code):

  const selectedActions = getComponentValue(SelectedActions, ship) || {
    actionTypes: [ActionType.None, ActionType.None],
    specialEntities: ["0" as EntityID, "0" as EntityID],
  };

    if (index == -1) {
      const unusedSlot = selectedActions.actionTypes.indexOf(ActionType.None);
      if (unusedSlot == -1) return;
      selectedActions.actionTypes[unusedSlot] = action; // <- this line actually updates this slot in selectedActions.
      selectedActions.specialEntities[unusedSlot] = cannonEntity;
    } else {
      selectedActions.actionTypes[index] = ActionType.None;
      selectedActions.specialEntities[index] = "0" as EntityID;
    }
    setComponent(SelectedActions, ship, {  // <- This component update will return identical before/after values because the component was already updated above.
      actionTypes: selectedActions.actionTypes,
      specialEntities: selectedActions.specialEntities,
    });

Running this in an empty directory:

> npx @latticexyz/cli create mmm
Need to install the following packages:
  @latticexyz/cli
Ok to proceed? (y) 
npm WARN tarball tarball data for ds-test@git+https://github.com/dapphub/ds-test.git#9310e879db8ba3ea6d5c6489a579118fd264a3f5 (null) seems to be corrupted. Trying again.
npm WARN tarball tarball data for forge-std@git+https://github.com/foundry-rs/forge-std.git#6b4ca42943f093642bac31783b08aa52a5a6ff64 (null) seems to be corrupted. Trying again.
npm WARN tarball tarball data for ds-test@git+https://github.com/dapphub/ds-test.git#9310e879db8ba3ea6d5c6489a579118fd264a3f5 (null) seems to be corrupted. Trying again.
npm WARN tarball tarball data for forge-std@git+https://github.com/foundry-rs/forge-std.git#6b4ca42943f093642bac31783b08aa52a5a6ff64 (null) seems to be corrupted. Trying again.
npm WARN tarball tarball data for ds-test@git+https://github.com/dapphub/ds-test.git#c7a36fb236f298e04edf28e2fee385b80f53945f (null) seems to be corrupted. Trying again.
npm WARN tarball tarball data for ds-test@git+https://github.com/dapphub/ds-test.git#c7a36fb236f298e04edf28e2fee385b80f53945f (null) seems to be corrupted. Trying again.
npm WARN tarball tarball data for forge-std@git+https://github.com/foundry-rs/forge-std.git#37a3fe48c3a4d8239cda93445f0b5e76b1507436 (null) seems to be corrupted. Trying again.
npm WARN tarball tarball data for forge-std@git+https://github.com/foundry-rs/forge-std.git#37a3fe48c3a4d8239cda93445f0b5e76b1507436 (null) seems to be corrupted. Trying again.
npm WARN tarball tarball data for memmove@git+https://github.com/brockelmore/memmove.git#d577ecd1bc43656f4032edf4daa9797f756a8ad2 (null) seems to be corrupted. Trying again.
npm WARN tarball tarball data for memmove@git+https://github.com/brockelmore/memmove.git#d577ecd1bc43656f4032edf4daa9797f756a8ad2 (null) seems to be corrupted. Trying again.
npm WARN ERESOLVE overriding peer dependency
npm WARN While resolving: @latticexyz/[email protected]
npm WARN Found: ds-test@undefined
npm WARN node_modules/ds-test
npm WARN 
npm WARN Could not resolve dependency:
npm WARN peer ds-test@"https://github.com/dapphub/ds-test.git#c7a36fb236f298e04edf28e2fee385b80f53945f" from @latticexyz/[email protected]
npm WARN node_modules/@latticexyz/solecs
npm WARN   @latticexyz/solecs@"^1.33.1" from @latticexyz/[email protected]
npm WARN   node_modules/@latticexyz/cli
npm WARN ERESOLVE overriding peer dependency
npm WARN While resolving: @latticexyz/[email protected]
npm WARN Found: forge-std@undefined
npm WARN node_modules/forge-std
npm WARN 
npm WARN Could not resolve dependency:
npm WARN peer forge-std@"https://github.com/foundry-rs/forge-std.git#37a3fe48c3a4d8239cda93445f0b5e76b1507436" from @latticexyz/[email protected]
npm WARN node_modules/@latticexyz/solecs
npm WARN   @latticexyz/solecs@"^1.33.1" from @latticexyz/[email protected]
npm WARN   node_modules/@latticexyz/cli
npm WARN ERESOLVE overriding peer dependency
npm WARN While resolving: @latticexyz/[email protected]
npm WARN Found: memmove@undefined
npm WARN node_modules/memmove
npm WARN 
npm WARN Could not resolve dependency:
npm WARN peer memmove@"https://github.com/brockelmore/memmove.git#d577ecd1bc43656f4032edf4daa9797f756a8ad2" from @latticexyz/[email protected]
npm WARN node_modules/@latticexyz/solecs
npm WARN   @latticexyz/solecs@"^1.33.1" from @latticexyz/[email protected]
npm WARN   node_modules/@latticexyz/cli
npm ERR! code ENOENT
npm ERR! syscall open
npm ERR! path /home/nazar-pc/.npm/_cacache/tmp/git-cloneDw2u84/package.json
npm ERR! errno -2
npm ERR! enoent ENOENT: no such file or directory, open '/home/nazar-pc/.npm/_cacache/tmp/git-cloneDw2u84/package.json'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent 

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/nazar-pc/.npm/_logs/2023-01-27T14_40_01_845Z-debug.log
Exit code 254

I see some of the failed commits do exist, no idea what is happening here, but it is a bit frustrating.

> node --version
v18.13.0
> npm --version
7.24.2

the DecodedSystemCall type is missing message sender:

export declare type DecodedSystemCall<T extends {
    [key: string]: Contract;
} = {
    [key: string]: Contract;
}, C extends Components = Components> = Omit<SystemCall<C>, "updates"> & {
    systemId: keyof T;
    args: Record<string, unknown>;
    updates: DecodedNetworkComponentUpdate[];
};

It would be useful for this to be exposed

I'm getting this error message when I try to commit:

.husky/pre-commit: 2: .: Can't open .husky/_/husky.sh

So i can't make improvement PRs :'(

Note: this proposal contains a lot of pseudo code and some of the core aspects of the proposal are contained in the code comments - don't skip over it

Table of contents

• Abstract
• Issues with current approach
• Design goals
• Core storage management library
  • Illustration of data model
  • Pseudo-code implementation with more details
  • Notes
• Wrapping typed libraries
  • Pseudo-code implementation with more details
  • Usage examples
  • Notes
• Framework (aka World)
  • Pseudo-code implementation with more details
  • Usage example
• Further work / extensions
• Acknowledgements

Abstract

This proposal addresses a couple of issues that come with MUD’s current approach to on-chain data modelling and state management.

We propose to move away from individual component contracts to store application state, and instead create a core spec and library for on-chain for data modelling and storage. The core library can be used in any contract to store state in a MUD compatible way and emit MUD compatible events for general-purpose indexers. (The core library doesn’t implement access control.)

We then use this core library to create a framework to add storage access control and the ability for third party developers to register new data pools and mount new contracts (similar to the current World contract).

Issues with the previous approach

• Currently, state is organised into separate components to manage access control and implement functions with typed parameters and typed return values (since Solidity doesn’t support generic types)

  • The component contracts call a function on a central World contract to register their update
  • The components use solidity’s abi.encode under the hood, which leads to unnecessarily high gas costs (because abi.encode reserves one 32 byte word per struct key)

• Currently, developers have to opt-in to MUD’s entire framework to benefit from conceptually independent features like general purpose indexers, instead of being able to upgrade their existing codebases

• Currently, developers have to manually define their components' schemas using a DSL, which is not intuitive for Solidity developers and leads to easy to miss bugs (when the defined schema doesn’t match the abi.encoded value)

• Currently, developers using MUD have to implement a lot of “boilerplate” code to read or write component values compared to setting vanilla Solidity storage variables

  • Current MUD:

    PositionComponent position = PositionComponent(getAddressById(components, PositionId));
    position.set(0x00, Position(1,2));

  • Vanilla Solidity:

    positions[0x00] = Position(1,2);

• Currently, MUD is limited to the ECS pattern (Entity-Component-System), requiring every piece of data to be associated with a single uint256 id. This makes some data modelling harder than desired, for example using a composite key (consisting of two values)

  • The current workaround is to create composite entity ids by using keccak(entity1, entity2), but this approach obfuscates the used entity ids and is cumbersome to work with

The following is a proposal to address all of the issues above and more.

Design goals

• A way to store introspectable structured data on-chain
  • Introspectable = data schema can be retrieved on-chain, so default general-purpose indexers are possible
• General-purpose indexers by default
  • Events notifying indexers about every state change
  • On-chain schema, so indexers know how to interpret state changes
  • SQL-compatible data modelling on-chain, so indexers can benefit from decades of SQL research
• Dynamic schemas / ability to add more schemas after the core contract has been deployed
  • This is important to enable “autonomous worlds” where third party developers can add data packets and systems to an application
• As little gas-overhead compared to the most efficient custom way of storing data on-chain as possible
• As little “third party code managing core state” as possible. As much as possible should be done by the core library
• The best developer experience possible (at least as good as working with native solidity structs/mappings)
• Splitting up the core storage problem from the framework problem
  • This allows more people to develop tools integrating with the core storage method, without having to opt-in to the framework

Core storage management library

• Implements logic to store and access data based on registered schemas
• Implements update events
• “Untyped” - uses bytes everywhere - typing is responsibility of wrapping libraries (see below)
• Any contract can implement the IMudStore interface / extend the MudStore base contract to become compatible with a large chunk of MUD’s toolchain, like general-purpose indexers
• Data is organised by table and index into the table, where the index can be a tuple of multiple bytes32 keys
  • This is a superset of ECS: In ECS, “Components” correspond to tables, and “Entities” are indices into the table. In this proposal, we allow to use tuples as index into the table, allowing more complex relationships to me modelled (and making the data model more similar to a relational database). However, single keys are still possible, so ECS is still possible.
  • The tuple of keys used to index a table is emitted as part of the event, so it can be picked up by indexers and we don’t have to rely on hacks like hashed composite entities anymore.

Illustration of data model

// Illustration of data model:
// Assume we want to index with two keys: A, B

keys: (A, B)
valueSchema: (X, Y, Z)

conceptually:
{
	[A1]: {
		[B1]: {
			X1,
			Y1,
			Z1
		},
		[B2]: {
			X2,
			Y2,
			Z2
		}
	},
	[A2] { ... }
}

-> translates into relational database:
| A  | B  | X  | Y  | Z  |
| -- | -- | -- | -- | -- |
| A1 | B1 | X1 | Y1 | Z1 |
| A1 | B2 | X2 | x2 | Z2 |
| ...

-> translates to on-chain:
mapping(
	keccak(A1,B1) => {X1, Y1, Z1},
	keccak(A1,B2) => {X2, Y2, Z2}
)

Pseudo-code implementation with more details

// Solidity-like pseudo code
// omitting some language features for readability
// eg using keccak(a,b,c) for keccak256(abi.encode(a,b,c))
// or omitting memory, public, pure etc

enum SchemaType {
	UINT8,
	..., // in steps of 8, so 32 total
	UINT256,
	INT8,
	..., // in steps of 8, so 32 total
	INT256,
	BYTES1,
	..., // in steps of 1, so 32 total
	BYTES32,
	BOOL,
	ADDRESS,
	BYTES,
	STRING, // until here we have 100 types
	BIT, // we could add a native "bitpacking" type using the same approach described below
	<T>_ARRAY // everything above as an array - until here we have 202 types
	// 54 more slots to define more types and keep SchemaType a uint8
}

// A table schema can have up to 32 keys, so it fits into a single evm word.
// (Schemas above 32 keys are definitely an anti-pattern anyway)
// Working with unnamed schemas makes the core library simpler; naming keys is the job of wrapping libraries
type Schema = SchemaType[32];

// Interface to turn any contract into a MudStore
interface IMudStore {
	event StoreUpdate(bytes32 table, bytes32[] index, uint8 schemaIndex, bytes[] data);
	function registerSchema(bytes32 table, SchemaType[] schema);
	function setData(bytes32 table, bytes32[] index, bytes[] data);
	function setData(bytes32 table, bytes32[] index, uint8 schemaIndex, bytes data);
	function getData(bytes32 table, bytes32[] index) returns (bytes[] data);
	function getDataAtIndex(bytes32 table, bytes32[] index, bytes32 schemaIndex) returns (bytes data);
	function isMudStore() returns (bool); // Checking for existence is this function is sufficient for consumers to check whether the caller is a MUD store (this could potentially be turned into eip-165 in the future)
}

library MudStoreCore {
	// Note: the preimage of the tuple of keys used to index is part of the event, so it can be used by indexers
	event StoreUpdate(bytes32 table, bytes32[] index, uint8 schemaIndex, bytes[] data);
	constant bytes32 _slot = keccak("mud.store");
	constant bytes32 _schemaTable = keccak("mud.store.table.schema");

	// Register a new schema
	// Stores the schema in the default "schema table", indexed by table id
	function registerSchema(bytes32 table, SchemaType[] schema) {
		// Optional: verify the schema only has one dynamic type at the last slot, see note 1 below
		setData(_schemaTable, table, Convert.encode(schema));
	}

	// Return the schema of a table
	function getSchema(bytes32 table) returns (SchemaType[] schema) {
		bytes value = getData(_schemaTable, table);
		return Convert.decodeUint8Array(value);
	}

	// Check whether a schema exists for a given table
	function hasTable(bytes32 table) returns (bool) {
		return getData(_schemaTable).length > 0;
	}

	// Update full data
	function setData(bytes32 table, bytes32[] index, bytes[] data) {
		// Optional: verify the value has the correct length for the table (based on the table's schema)
		// (Tradeoff, slightly higher cost due to additional sload, but higher security - library could also provide both options)

		// Store the provided value in storage
		bytes32 location = _getLocation(table, index);
		assembly {
			// loop over data and sstore it, starting at `location`
		}

		// Emit event to notify indexers
		emit StoreUpdate(table, index, 0, data);
	}

	// Update partial data (minimize sstore if full data wraps multiple evm words)
	function setData(bytes32 table, bytes32[] index, uint8 schemaIndex, bytes data) {
		// Get schema for this table to compute storage offset
		SchemaType[] schema = getSchema(table)[];

		// Compute storage location for given table, index and schemaIndex
		bytes32 location = _getLocation(table, index);
		uint256 offset = _getByteOffsetToSchemaIndex(schema, schemaIndex); // Simple helper function
		assembly {
			// set data at the computed location (location + offset)
		}
	
		// Emit event to notify indexers
		emit StoreUpdate(table, index, schemaIndex, [data]);
	}

	// Get full data
	function getData(bytes32 table, bytes32[] index) returns (bytes[] data) {
		// Get schema for this table
		// Compute length of the full schema
		// Load the data from storage using assembly
		// Split up data into bytes[] based on schema
		// Return the data as bytes[]
	}

	// Get partial data based on schema key
	// (Only access the minimum required number of storage slots)
	function getDataAtIndex(bytes32 table, bytes32[] index, bytes32 schemaIndex) returns (bytes data) {
		// Get schema for this table
		// Compute offset and length of this schema index
		// Load the data for this schema index from storage using assembly
		// Return the data as bytes
	}

	// Compute the storage location based on table id and index tuple
	// (Library could provide different overloads for single index and some fixed length array indices for better devex)
	function _getLocation(bytes32 table, bytes32[] index) returns (bytes32) {
		return keccak(_slot, table, index);
	}

	// Simple helper function to compute the byte offset to the given schema index based in the given schema
	function _getByteOffsetToSchemaIndex(schema, schemaIndex) returns (uint256) {
		// Sum `getByteLength(schemaType)` for every schema index before the given index
	}

	// Simple helper function to return the byte length for each schema type
	// (Because Solidity doesn't support constant arrays)
	function _getByteLength(SchemaType schemaType) returns (uint8) {
		// Binary tree using if/else to return the byte length for each type of schema
	}
}

// A helper library to convert any primitive type (+ arrays) into bytes and back
library Convert {
	// Overloads for all possible base types and array types
	// Encode dynamic arrays in such a way that the first 2 byte are reserved for the array length = max arr length 2**16 (to help decoding)
	function encode(uint256 input) returns (bytes);

	// Decoder functions for all possible base types and array types
	function decodeUint8Array(bytes input) returns (uint8[]);

	...
}

Notes

1. If we only allow one dynamic array type per table schema, encoding/decoding/storing partial data gets much simpler and cheaper (the dynamic array type always has to come last in the schema)
   • cheaper because only one storage access to get the schema, instead of additional storage access to get the length of each dynamic array. Also, dynamic array types anywhere else but at the last schema slot would shift all remaining schema values (even non-dynamic ones), so modifying partial data would be much more expensive (worst case as expensive as modifying the full data) - we could save developers from having to think about this in their model by restricting schemas to one dynamic type that has to come last.

Wrapping typed libraries

• While Solidity doesn’t support generic types, we can autogenerate libraries to set/get typed values based on user defined schemas to emulate the experience of working with a generically typed core library.
• The libraries encode typed values to raw bytes and vice versa to improve developer experience (in theory devs could call the core functions manually but devex would suck)
• The library detects whether the call comes from within a MudStore (eg if the contract using the library is called via delegatecall from a MudStore) or if the msg.sender is a MudStore (eg if the contract using the library is called via call from a MudStore) and automatically switches between writing to own storage using the core library and calling the respective access controlled methods on the calling MudStore.

Pseudo-code implementation with more details

// Solidity-like pseudo code
// omitting some language features for readability
// eg using keccak(a,b,c) for keccak256(abi.encode(a,b,c))
// or omitting memory, public, pure etc

// ----- Example of an auto-generated typed library for a Position table -----

// -- User defined schema and id --

bytes32 constant id = keccak("mud.store.table.position");

struct Schema {
	uint32 x;
	uint32 y;
}

// -- Autogenerated schema and library --

library PositionTable {

	// Detect whether the call to the system was done via delegatecall or a regular call
	// to switch between writing to own storage and using access controlled external storage functions
	// (see note 1. below)
	function isDelegateCall() internal returns (bool) {
		(bool success, bytes memory data) = address(this).call(
			abi.encodeWithSignature("isMudStore()")
		);

		return success && abi.decode(data, (bool));
    }

	// Register the table's schema
	// (used to compute data length when returning values from core lib and for input validation)
	function registerSchema() {
		// Autogenerated schema based on schema struct definition
		SchemaType[2] schema = [SchemaType.UINT32, SchemaType.UINT32];

		// Call core lib or wrapper contract to register schema
		if(isDelegateCall()) {
			MudStoreCore.registerSchema(id, schema);
		} else {
			MudStore(msg.sender).registerSchema(id, schema);
		}
	}

	// Set the full position value
	function set(uint256 entity, uint32 x, uint32 y) {
		bytes[] data = [
			Convert.encode(x),
			Convert.encode(y)
		];

		// Set the data via core lib or wrapper contract
		if(isDelegateCall()) {
			MudStoreCore.setData(id, entity, data);
		} else {
			MudStore(msg.sender).setData(id, entity, data);
		}
	}

	// Offer both syntax for convenience
	function set(uint256 entity, Schema data) {
		set(entity, data.x, data.y);
	}

	// Set partial schema values
	function setX(uint256 entity, uint32 x) {
		// Set the data via core lib or wrapper contract
		if(isDelegateCall()) {
			MudStoreCore.setData(id, entity, 0, x);
		} else {
			MudStore(msg.sender).setData(id, entity, data);
		}
	}

	function setY(uint256 entity, uint32 y) {
		// Set the data via core lib or wrapper contract
		if(isDelegateCall()) {
			MudStoreCore.setData(id, entity, 1, x);
		} else {
			MudStore(msg.sender).setData(id, entity, data);
		}
	}

	// Get the full position value
	function get(uint256 entity) returns (Schema) {
		// Get data via core lib or wrapper contract
		bytes[] data = isDelegateCall() 
			? MudStoreCore.getData(id, entity)
			: MudStore(msg.sender).getData(id, entity);
			
		return Schema(
			Convert.decodeUint32(data[0])),
			Convert.decodeUint32(data[1]))
		);
	}

	// Get partial schema values
	function getX(uint256 entity) returns (uint256) {
		// Get data via core lib or wrapper contract
		bytes data = isDelegateCall()
			? MudStoreCore.getData(id, entity, 0)
			: MudStore(msg.sender).getData(id, entity);

		return Convert.decodeUint32(data);
	}

	function getY(uint256 entity) returns (uint256) {
		bytes data = isDelegateCall()
			? MudStoreCore.getData(id, entity, 1)
			: MudStore(msg.sender).getData(id, entity);
		return Convert.decodeUint32(data);
	}
}

Usage examples

// Usage examples from within System:
PositionTable.set(0x01, 1, 2);

PositionTable.set(0x01, {x: 1, y: 2});

PositionTable.set({entity: 0x01, x: 1, y: 2});

PositionTable.setX(0x01, 1);

Schema position = PositionTable.get(0x01);

uint32 x = PositionTable.getX(0x01);

Notes

1. We want to be able to detect deletegatecall in the storage library called in the system
   • If the system is called via delegatecall, it means it can write to storage using MudStoreCore directly without having to call functions with access control on a MudStore contract. This saves (700 call base gas + x calldata gas + y access control check gas) per storage operation
   • To detect delegatecall inside of a library, we can check if this has the isMudStore() function
     • since systems don’t implement their own isMudStore function, if this supports isMudStore, it means the current context is a MudStore and we can use libraries directly (this could be turned into something like ERC165’s supportsInterface)
     • This approach is cheaper than alternatives like setting a temporary storage variable (5k gas to temp store, 2.1k to read from the system)

Framework (aka World)

// Solidity-like pseudo code
// omitting some language features for readability
// eg using keccak(a,b,c) for keccak256(abi.encode(a,b,c))
// or omitting memory, public, pure etc

// `MudStore` base contract implements all view functions from IMudStore (getData, ...)
// that don't require access control checks.
// World contract extends `MudStore` and implements access control for write methods (`setData`) 
contract World is MudStore {
	error World_TableExists();

	function registerSchema(bytes32 table, SchemaType[] schema) {
		// Require unique table ids
		if(MudStoreCore.hasTable(table)) revert World_TableExists();
		
		// Register schema
		MudStoreCore.registerSchema(table, schema);

		// Set table's owner in owner tab
		// (OwnerTable uses auto-generated typed helper table like `PositionTable` described above)
		OwnerTable.set({ index: table, owner: msg.sender });
	}

	function setData(bytes32 table, bytes32[] index, bytes[] data) {
		// TODO: Require caller to have permission to modify table
		//       (access control details tbd)

		// Set data
		MudStoreCore.setData(table, index, data);
	}

	// Register a new system
	// -> Anyone can call this method, but only World owner can pass DELEGATE mode
	//    - DELEGATE systems are called via delegatecall and have access to all storage
	//    - AUTONOMOUS systems are called via call and modify storage via access controlled `setData` method
	function registerSystem(
		address contractAddress,
		string contractName,
		string functionSig,
		ExecutionMode mode) {
		// TODO: if mode is DELEGATE, require msg.sender to be World's owner
		
		// TODO: check if contract name is already registered
		//       - if so, require msg.sender to be owner
		//       - else, register contract name and set msg.sender as owner

		// TODO: check if function signature already exist for the given contract
		//       - if so, this is an upgrade
		//           - require msg.sender to be system's owner
    //           - and if the given system is a DELEGATE system, require World's owner to be system's owner
		//             (to prevent upgrades to DELEGATE systems in fully autonomous mode)
		
		// Compute the selector to use to call this system via the fallback() entry point
		// using the format <contractName>_<functionSig>()
		// NOTE: this is slightly simplified - in reality we have to remove the `address _from` parameter
		//       from the function signature because it will be automatically populated by the World based on `msg.sender` (see notes above)
		bytes4 worldSelector = bytes4(keccak(abi.encodePacked(contractName, "_", functionSig)));
		
		// Register World selector with contract address
		SystemTable.set({
			index: bytes32(worldSelector),
			addr: contractAddress,
			selector: bytes4(keccak(functionSig),
			mode: mode
		});
	}

	// TODO: Set approval (see general approval pattern discussion in mud#327)
	function approve( ... ) { ... }

	// The fallback function is used for consumers to call system functions
	// with proper types. We can generate an ABI for the World contract based
	// on registered systems.
	// The function selector is generated in `registerSystem` (see above)
	fallback() external payable {
		// Find system based on function selector
		SystemTableEntry system = SystemTable.get(msg.sig);
		
		if(system.mode == ExecutionMode.DELEGATE) {
			// TODO: If system is DELEGATE system, populate the _from parameter with msg.sender,
			//       forward the call via `delegatecall`, and return any value.
			//       This is almost equivalent to EIP2535 (diamond pattern), except from
			//       using `_from` instead of `msg.sender`
		} else {
			// TODO: If system is an AUTONOMOUS system, populate the _from parameter with msg.sender
			//       forward the call via `call`and return any value.
			//       The called system will use access controlled `setData` methods of this contract.
		}
		

	}
}

// ----- Example of a move system -----

contract MoveSystem {
	// System can trust the `move` function will only be called via a `MudStore` contract (in our case World)
	// and must therefore use the _from parameter instead of msg.sender. (Note: this requires something like the "general access pattern" (#327) to be in place)
	// Since system doesn't have any internal state, it doesn't have to check whether the call actually comes from a `MudStore`
	// (because state will always be modified in the calling contract and the call fails if it doesn't come from a MudStore)
	function move(address _from, bytes32 _entity, Position _position) public {
		// Check if the `_from` address owns the given entity
		require(OwnerTable.get(_entity) == _from, "only owner can move entity");
		
		// Set the new entity's new position value
		PositionTable.set(entity, position);
	}
}

Further work / extensions

Table migrations

• For a persistent world it is plausible that table schemas need to be upgraded from time to time. How could this be implemented in this proposal?
  • We could add an additional signature for setData and getData that includes a uint16 version parameter
  • MudStoreCore._getLocation includes the version to get the storage location hash
  • If the version parameter is omitted, it is set to 0 by default
  • To increase a table’s version, a “migration” has to be specified (how to interpret the original data with the new schema). This migration is used to generate a typed access library using the new schema, which calls setData with an incremented index value and the new schema, and implements the migration in the getter functions.

Acknowledgements

• This proposal is based on many internal discussions with and ideas by @ludns, @holic, @Kooshaba and @authcall
• Generating libraries to improve developer experience and allow typed access to tables is based on an idea by @FlynnSC
• Registering contracts as “facets” and calling them via a fallback method, as well as using delegated storage is based on Nick Mudge, "EIP-2535: Diamonds, Multi-Facet Proxy," Ethereum Improvement Proposals, no. 2535, February 2020. [Online serial]. Available: https://eips.ethereum.org/EIPS/eip-2535.
• Using diamond storage to improve the developer experience and gas efficiency of MUD is based on ideas by @cha0sg0d, @0xhank and @dk1a