Code Monkey home page Code Monkey logo

ipvm-wg / homestar Goto Github PK

View Code? Open in Web Editor NEW
200.0 200.0 23.0 20.32 MB

Homestar is the individual node that makes up the Everywhere Computer network (similar to how IPFS Kubo, Iroh, Nabu, or other implementation nodes make up the IPFS network). It's written in Rust and is designed for performance and composability.

Home Page: https://docs.everywhere.computer/homestar/what-is-homestar/

License: Apache License 2.0

Rust 56.62% Nix 0.77% Dockerfile 0.21% WebAssembly 42.34% JavaScript 0.06% Shell 0.01%
compute fission ipvm rust wasm wit workflows

homestar's People

Contributors

avivash avatar bgins avatar bmann avatar dependabot[bot] avatar euamcg avatar expede avatar hugomrdias avatar marcoieni avatar mriise avatar quinnwilton avatar release-plz-ipvm-wg[bot] avatar walkah avatar zeeshanlakhani avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

homestar's Issues

Support expressive input types

In the current proof of concept, we only have support for i32s. This needs to be generalized to a much broader set of types, at least those available in IPLD. This both needs a clear ABI (e.g. C conventions) and/or support for the Wasm component model & WIT.

RUSTSEC-2020-0168: mach is unmaintained

mach is unmaintained

Details
Status unmaintained
Package mach
Version 0.3.2
URL fitzgen/mach#63
Date 2020-07-14

Last release was almost 4 years ago.

Maintainer(s) seem to be completely unreachable.

Possible Alternative(s)

These may or may not be suitable alternatives and have not been vetted in any way;

See advisory page for additional details.

Metering

Summary

We want to be able to calculate and properly meter fuel & memory usage on wasm functions within a workflow, dependent on workflow settings.

Goals

  • Test and showcase fuel decreasing over a computation workflow
  • Test and showcase memory decreasing over a computation workflow

ETA: 2023-09-30

Node Bootstrapping: Sharing information for coordination

Summary

Nodes need to bootstrap with information and share that information with other homestar nodes in a peer group. This opens up distributed scheduling and coordination.

Components

  • What to capture for the capability matrix per node:
    • System metrics I: (available) CPU, memory, disk, etc
    • System metrics II: (available) GPU
  • Gossiping information to a subset of peers, a "peer group", on a windowed interval.
  • TBD

ETA: 2024-02-19

Self-hosted version/deployment

Summary

We want to be able to run a Homestar (runtime) node and deploy it as a binary and a Docker container.

Goals

  • Docker build (homestar/server start)
  • *-nix binary
  • Deployment workflows/examples

ETA: 2023-09-30

RUSTSEC-2022-0081: json is unmaintained

json is unmaintained

Details
Status unmaintained
Package json
Version 0.12.4
URL maciejhirsz/json-rust#205
Date 2022-02-01

Last release was almost 3 years ago.

The maintainer is unresponsive with outstanding issues.

One of the outstanding issues include a possible soundness issue.

Possible Alternative(s)

The below list has not been vetted in any way and may or may not contain alternatives;

See advisory page for additional details.

Compute: Include & Index resolved values within Task Invocation

Summary

"Instruction" Invocations are content-addressed as given, which can/will typically include promise pipelined CIDs that must be resolved during execution. However, once resolved (before the actual execution of the function itself), we don't currently have a way to equate the resolved instruction (i.e. resolved with values) with the one given CIDs initially.

Essentially, the instructions given CIDs should be treated the same as those substituted with values from the output of those other task/instruction CIDs.

This will involve tracking pre/post-deferred instruction Cids for lookup on receipt checking.

Discovery

Dependency

  • Discovery has to be completed before #88 and #89

ETA: 2023-11-03

Distributed Runtime + Scheduler

Summary

  • Failure Handling, Traces
  • Virtual Resiliency
    • a mechanism in a (possibly distributed) programming and
      execution environment, typically employing a log, which exploits the replayably
      deterministic nature and serializilbility of an application to automatically mask
      failure.

ETA: 2023-12-31

Task Permissions: (UCAN validation) on workflows and tasks

Summary

We want to leverage Ucans to authorize and validate task invocations and workflow execution. The current Ucan invocation spec has this at the task level, but we'd direct this information from workflows at the top level, pushing information into individual tasks, and use it at the receipt level as well.

Components

  • TBD

ETA: 2023-12-20

RUSTSEC-2021-0060: `aes-soft` has been merged into the `aes` crate

aes-soft has been merged into the aes crate

Details
Status unmaintained
Package aes-soft
Version 0.6.4
URL RustCrypto/block-ciphers#200
Date 2021-04-29

Please use the aes crate going forward. The new repository location is at:

<https://github.com/RustCrypto/block-ciphers/tree/master/aes>

AES-NI is now autodetected at runtime on i686/x86-64 platforms.
If AES-NI is not present, the aes crate will fallback to a constant-time
portable software implementation.

To force the use of a constant-time portable implementation on these platforms,
even if AES-NI is available, use the new force-soft feature of the aes
crate to disable autodetection.

See advisory page for additional details.

Workflow Parser

Summary

For the MVP, Workflows will be rendered as JSON and need to be loading, parsed, and then executed---really queued for execution. Eventually, we'll allow for other input formats, and #103 for generating workflow configuration and translation.

Goals

  • Make sure workflows are convertible to JSON and back.
  • Run multiple workflows over CLI.
  • Load workflows from CLI .json files.

ETA: 2023-08-15

RUSTSEC-2021-0059: `aesni` has been merged into the `aes` crate

aesni has been merged into the aes crate

Details
Status unmaintained
Package aesni
Version 0.10.0
URL RustCrypto/block-ciphers#200
Date 2021-04-29

Please use the aes crate going forward. The new repository location is at:

<https://github.com/RustCrypto/block-ciphers/tree/master/aes>

AES-NI is now autodetected at runtime on i686/x86-64 platforms.
If AES-NI is not present, the aes crate will fallback to a constant-time
portable software implementation.

To prevent this fallback (and have absence of AES-NI result in an illegal
instruction crash instead), continue to pass the same RUSTFLAGS which were
previously required for the aesni crate to compile:

RUSTFLAGS=-Ctarget-feature=+aes,+ssse3

See advisory page for additional details.

Affinities

Dependencies

  • This task has to be completed after #84 and before #89

ETA: 2023-12-31

DSLs / Tooling for writing tasks and using workflows

Summary

The current MVP version will run workflows via configuration, i.e. JSON, YAML, etc. We need to investigate and build a suite of libraries and tools to generate workflows for composing tasks, task-types, etc.

ETA: 2024-03-31

Bidirectional IPLD from/to WIT interpreter

Summary

IPLD <=> WIT interpreter for executing functions within Wasm components.

Flexible Types (not part of IPLD proper)

  • record(s)
  • flag(s)
  • option(s)
  • result(s)
  • variant(s)
  • enum(s)
  • union(s) (unions have been ๐Ÿช“'ed!)
  • tuple(s)

ETA: 2023-10-20

Link/CID AOT Validation

Summary

For deterministic wasm, we should verify all links as part of the ahead-of-time "compilation" of workflows before any execution takes place. We should fail the workflow and return errors for all CIDs that do not exist at the time of execution.

If links do exist on the network but are not local, we should stream in the necessary (IPLD) blocks over the network to make the execution feasible, which is related to #78.

This does not involve the coordination piece of who should actually run the workflow, but only at the point when a node is deemed ready to execute it.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.