sof3 / dynec Goto Github PK

Is it meaningful to have dynamically created archetypes?

Currently isotope components work like "dynamic columns" for an entity, but they waste a lot of memory for entities that don't use all isotope components if they use storage::Vec for storage.

If we allow isotope archetypes, entities that use a particular isotope component can be abstracted out.

The problem with this approach is, in that case, we have multiple archetypes with the same combination of components anyway. In that case, why not just store them together?

Therefore, it seems isotope archetypes don't really make sense, since the root problem of isotope components is the need for a dynamic number of components without creating new entities.

Leaving this issue open for future reference.

Add an API to quickly drop all components in a storage. This is implemented by setting all bitflags to 0, which is much faster than calling set(None) one by one.

This is implemented by (1) filling new entities like we do with simple storages, and (2) filling default values when a new discriminant is instantiated.

It is supposed to be cross-checked with generation::WeakStore, but apparently it is not done anywhere.

entity::Weak should not implement entity::Ref directly. It should expose a separate function to use as entity::Ref:

impl<A: Archetype> entity::Weak<A> {
    pub fn try_as_ref(&self, store: impl generation::WeakStore<A>) -> Option<entity::TempRef<'_, A>>;
}

If a system s1 requests an entity creator for archetype A, for each component/global T where s1 requests write access to T and T may own a Entity<A>, for any other system s2 where s2 reads/writes T, s2 must execute strictly before s1 within a cycle (i.e. there must exist a partition P where s2 executes before P and s1 executes after P, or other systems P[i] and s[i] such that s2 < P[i] < s[i] < P < s1.

Alternatively, when s2 requests access to T, it must declare maybe_uninit(A) (or #[dynec(maybe_uninit(A))] in #[system] macro syntax), where the system author acknowledges that some Entity<A> values may be uninitialized.

Things we are interested in benchmarking:

• Scheduler initial planning: how long does it take to initialize the planner?
• Scheduler throughput: how many empty, mutually exclusive systems can we dispatch per second?
• Entity creation: how many entities (with one dummy component) can we create per second?
• Entity deletion: how many entities (with zero or one deferred finalizer component) can we delete per second?
• Chunked component access: throughput of i32 sum += delta over all entities (with randomly generated holes)
• Same as above, but for isotope components
• Random component access: throughput of flow graph simulation
• parallel accessors

While auto initializers make sense for simple components because they are required for modular initialization for comp::Must, isotope lazy initializers do not really serve any good purpose other than increasing implementation complexity. They can totally be implemented in userland without getting built into the isotope declaration.

In particular, it is very inconsistent that isotope storages return Some for nonexistent lazy-initialized isotopes but subsequent iterators do not yield the value.

Currently storage refers to two different concepts. In the public API, "storage" only refers to the actual data structure that holds the components (i.e. VecStorage and storage::Tree), but internally it is also used to refer to the storage wrapper types for simple and isotope components (i.e. storage::Simple and storage::IsotopeMap). We should rename the latter to something else like "storage wrapper" etc.

thread 'world::tests::test_entity_create_and_delete' panicked at 'already borrowed: BorrowMutError', src/entity/ealloc.rs:555:27

This is because both arguments require calling ealloc_shard_map.get(TypeId::of::<A>()).borrow().

We do not need to allow concurrent mutable access to the same archetype, and EntityIterator does not read the output of EntityCreator, so the immutable part could be extracted out.

• Maintain a generation_store: Vec<Geneneration> for each archetype, initialized as zero.
• Every time an entity is allocated, increment the Generation by 1.
• This buffer is readable to all systems during online mode, because it is used for verifying validity of entity::Weak.
• When an entity::Weak is created from a entity::Entity, retrieve the generation from generation_store. This means entity::Entity::weak(&self) also requires an argument for accessing the store.

Add a test to ensure that, if the same system adds two equal partitions as dependencies, it should have no error if the dependencies are of the same direction.

Dependencies in different directions are expected to panic during cycle detection.

Currently comp::Map uses a BTreeMap, where remove_single is a slow operation. Checking for duplicates is probably unnecessary anyway, and isotopes could have been split to a separate field instead of mixing with simple components.

Maintain a deletion_flag: Vec<bool> and finalizer_count: Vec<AtomicUsize> for each archetype.

Entity creation

• Count the components that are finalizers, stored to finalizer_count
• Initialize deletion_flag as false.

Entity deletion flagging

• Offline mode: accepts an entity: impl entity::Ref, get the raw ID and drop(entity).
• Online mode: accepts an entity: impl entity::Ref, get the raw ID, push the raw ID to a sharded queue, process in batch after join.
• During offline/join: If finalizer count is zero, proceed to "Entity deletion". Otherwise, set deletion_flag to true to wait for trigger from finalizers.

Component update

If a component is removed and the deletion flag for its entity is true, the entity is queued for finalizer test after join.

Entity deletion

• Check the refcount of the entity, ensure that the internal refcount is the only reference.
• Loop through all storages, drop the component from the storage if exists.
• Deallocate the entity ID.

No panic message is shown. Panic handler is not working. Process does not abort. Other worker threads deadlock waiting for the task to complete.

Worker thread management logic is found in https://github.com/SOF3/dynec/blob/a40afddf40e00e4ab014e16540f82e0eb4f0d473/src/world/scheduler/executor.rs

Supports enums and single-field structs in which the only field U satisfies T: xias::SmallInt<usize>, usize: xias::SmallInt<T>.

This is equivalent to "flushing" in other ECS frameworks. This allows entities created/deleted in scheduler set A to take effect before systems in scheduler set B execute, thus allowing more flexibility with entity creation partition guarantees.

Even though we obtain &mut map[k1] and &mut map[k2] for distinct keys k1 != k2 and we only access their interior, it is still unsound to obtain the latter by taking a &mut map first, because BTreeMap does not guarantee not to e.g. rebalance the tree as we call map.get_mut (although it has no practical reason to).

The correct approach would be to use SyncUnsafeCell and/or splitting slices.

Currently, worker threads poll tasks by acquiring a mutex on the scheduler planner. This may not scale very well if there are many systems that execute very quickly, or if there are many cores fighting for the same lock.

If both A and B are optional dependencies, even though the initializer of B depends on A, no error should be generated even if A is missing without initializers.

Currently component accessor takes &mut self, which makes parallelism not possible.

Allow users to change the implementation of entity allocator in the Archetype implementation.

If near hints are not required, the fastest implementation is actually to use a FILO stack storing all available entity IDs.

Users may store entity references in system local states. We should visit references from those places as well.

This is currently only possible through calling t.id() == u.id(), but the entity ID is supposed to be more low-level.

The Storage implementors contain a lot of unsafe code, but they are severely lacking tests, greatly increasing the risk of UB.