Decrease VM size

remove cpp exception handling and remove CONFIG_COMPILER_CXX_EXCEPTIONS idf option
look into smaller memory pages (eWASM)
reduce size of protobuf code

Out-of-Things Multiverse Debugger

Features to add:

Add SIMD

Add WebAssembly SIMD instructions proposal: https://github.com/WebAssembly/simd/blob/main/proposals/simd/SIMD.md

Overview of new opcodes: https://github.com/WebAssembly/simd/blob/main/proposals/simd/NewOpcodes.md

Full list of current opcodes: https://webassembly.github.io/spec/core/appendix/index-instructions.html?highlight=opcode

Spectests that fail because of this missing proposal:

Add wrapper lambda that ignores parameters

This function shoud replace fn with

(topic,payload)=>{fn()}

WARDuino/examples/assemblyscript/lib/warduino.ts

Lines 48 to 50 in f2ea06a

    
           export function interrupt_on(pin: u32, mode: u32, fn: () => void): void { 
        
               subscribe_interrupt(pin, fn, mode); 
        
           }

MQTT unsubscribe should only unsubscribe if no more callbacks exist

The MQTT unsubscribe function removes the callback from the callback list, but there may be multiple callbacks for the same topic. As long as there are callbacks left, the subscription should remain.

WARDuino/src/Primitives/primitives.cpp

Lines 709 to 731 in f2ea06a

    
           def_prim(mqtt_unsubscribe, threeToOneU32) { 
        
               uint32_t topic_param = arg2.uint32; 
        
               uint32_t topic_length = arg1.uint32; 
        
               uint32_t fidx = arg0.uint32; 
        
               const char *topic = 
        
                   parse_utf8_string(m->memory.bytes, topic_length, topic_param).c_str(); 
        
               Callback c = Callback(m, topic, fidx); 
        
               CallbackHandler::remove_callback(c); 
        
               bool ret = mqttClient.unsubscribe(topic); 
        
           #if DEBUG 
        
               Serial.print("Unsubscribed to "); 
        
               Serial.println(topic); 
        
               Serial.flush(); 
        
           #endif 
        
               pop_args(2); 
        
               pushInt32((int)ret); 
        
               return true; 
        
           }

Note: I did not test this and could be entirely wrong, to me the code seems to not handle this case correctly

Send callback mapping from drone to emulator

recvCallbackmapping interrupt
dumpCallbackmapping interrupt

Fix watchdog timer

We should fix this properly. That way we can also remove the sdkconfig file from the repository.

Update smartlamp.ts example

I am unable to connect to Wifi when using the smartlamp.ts example.
However if I change the following function:

function until_connected(connect: () => void,
connected: () => boolean): void {
while (!connected()) {
wd.delay(1000);
connect();
}
}

into

function until_connected(connect: () => void,
connected: () => boolean): void {
while (!connected()) {
connect();
wd.delay(5000);
}
}

the program connects to Wifi.

Add proxy version of primitives

Fix GitHub Actions

Update the GitHub Actions to the new project structure
Add ESP-IDF to GitHub Actions
Fix compilation on macos and windows

Quality of Life changes

Requested features:

README in platforms
Make platform files into actual templates
Update upload script

Document and test debugger API

We must make sure we know immediately when changes might break the VS Code plugin.

Add a complete reference for the debugger API
Add CI tests to verify the VM implements this API correctly

`subscribe_interrupt` requires the use of a memory page and table

I'm not sure whether this is an issue or not but I rather mention it just in case.

I noticed that to correctly use primitive subscribe_interrupt the executed Wasm needs to use a memory page and provide the function that needs to be used as callback as a table index.

I can understand why we need the memory page since this is the location where MQTT payloads get stored but are we 100% sure that compilers will always generate Wasm that uses at least one memory page? And also is there a reason why we need to pass the callback identifier through the table? Can the function identifier of the callback simply be passed as an argument to subscribe_interrupt?

Update `--help` option

Add Spec Tests to Warduino

https://github.com/kanaka/wac#running-webassembly-spec-tests

Minimise code size for IDF

https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-guides/performance/size.html

AssemblyScript example optimized version fails

Trying to run the AssemblyScript example without the mqtt fails. (= programming demo example)

When the busy loop is empty, binaryen optimizes away callback functions. It eliminates the funcref table and passes the function's own index to the subscribe_interrupt primitive.

    while (true) {    
        wd.sleep(5); // Sleep for 5 seconds    
    }

The resulting index is obviously out of range since there is no table. This causes an unhandled exception whenever the VM wants to resolve an event.

It seems users of Asyncify have run into similar problems: WebAssembly/binaryen#4484

Source-level debugging?

Debugging wasm opcode execution doesn't seem very useful. Is source-level debugging available? If yes, are there any docs on this?

Add latest WASM proposal support

100 files from the official spectests fail due to changes from newly standardized WASM proposals:

Fix Schrödinbug: reflashing + incomplete callback synchronisation

OOP doesn't seem to work anymore:

Reflashing undoes init code: set_pinmode, subscribe_interrupt, ... So no events are generated from the device.
Callbackmapping only synced when fetching the snapshot. So either (A) the drone can receive events but the emulator has an empty callbackmapping or (B) the drone cannot receive events (see 1) while the emulator has a correct callbackmapping.

Support multiple modules

Tasks:

Move stack pointers from Module to WARDuino class

Spectests that fail because of this missing proposal:

block.wast

Once WARDuino reaches a breakpoint, it is not possible to resume the execution with a RUN interrupt

Once WARDuino reaches a breakpoint, sending a RUN interrupt to WARDuino does not resume the computation, but instead WARDuino gets stuck at the currently reached bp.

The reason is the if-statement responsible to determine whether the current pc_ptr contains a breakpoint (see interpret function file instructions.cpp line nr. 1490). When you resume computation after reaching a bp, the if statement re-evaluates the pc_ptr to be a bp and thus pauses the execution once again. I guess this is not desired behavior?

If needed I can make a pull request for this.

Push debugging

json interface:

{
   "topic":"topic string",
   "payload":"payload as a string"
}

How to support push OOP debugging when device is in production mode?

Add pull push debugging unit tests

Refactor code so an emulated version can also be a drone
Test connection
Test proxy calls (check return value)
Test push debugging with fake events in drone

Version 0.2.2-dev

0.2.2-dev: Stable EDWARD

The #47 pull request works, but there are a lot of outstanding issues before it can be merged into master.

Code smells

remove strict wifi socket requirement
Remove #ifndef ARDUINO as much as possible
Separate proxy and proxysupervisor code. Use clearer names.
#70

Bugs

Major features

Minor features

#60
#88

Plugin

Related issues in the plugin:

fix breakpoints
implement step back until start

Investigate benchmark slow-down

After adjusting the benchmarks for the new project structure, and adding edward benchmark (c85bc8e), the benchmarks are 5 to 6 times as slow.

Add ESP-IDF implementation of primitives

Primitives are now implemented in separate implementation files for each platform with one common header.

We should add an implementation file for ESP-IDF.

Add dronify message

Instead of reflashing a drone version of the Arduino.ino file to the board after we pull the debugsession, we should add a dronify or zombify message which simply tells the device to turn into a drone.

Remove Arduino-socket.ino

Tests for debugger functions

Full emulated testing of out-of-place

Our goal is to allow the full VM to be tested with the emulated version.
The most challenging part is testing OOP with the emulator because a lot of inherited code is microcontroller specific.

Things to fix in order:

Replace SocketServer.h with abstract and uniform socket interface, the same used for remote debugging. (deprecates SocketServer.h)
Implement functionality of deprecated SocketServer for emulated VM
Enable OOP code in emulated VM (can become a drone with the dronify message)
Add code to CLI to control drone: answer proxy calls and send events

This will allow for manual testing of OOP with 2 emulated VMs. As part of #53 we can add automated API tests in a next step.

Add tests for debugger API

Missing detachInterrupt?

Are you missing a detachInterrupt at unsubscribe_interrupt primitive?

If so, I'll provide a PR.

step does not advance when bp reached

Since the solution on issue 5, the "step" interrupt no longer advances after reaching a breakpoint . The cause is the following if

WARDuino/instructions.cpp

Line 1498 in 3f349ed

if (m->warduino->isBreakpoint(m->pc_ptr) &&

when a breakpoint is reached and you step the branch condition evaluates to true.

Off by one error with breakpoints

When debugging larger files, the offset reported by warduino is sometimes wrong (off by one)

The cause is probably somewhere here:

WARDuino/WARDuino.cpp

Lines 314 to 327 in 07cf2b6

    
           uint8_t *pos = bytes; 
        
           word = read_uint32(&pos); 
        
           debug("Magic number is 0x%x\n", word); 
        
           ASSERT(word == WA_MAGIC, "Wrong module magic 0x%x\n", word); 
        
           word = read_uint32(&pos); 
        
           ASSERT(word == WA_VERSION, "Wrong module version 0x%x\n", word); 
        
           // Read the sections 
        
           uint8_t *bytes_end = bytes + byte_count; 
        
           while (pos < bytes_end) { 
        
               uint32_t id = read_LEB(&pos, 7); 
        
               uint32_t section_len = read_LEB_32(&pos); 
        
               uint8_t *start_pos = pos; 
        
               debug("Reading section %d at 0x%p, length %d\n", id, pos, section_len); 
        
               switch (id) {

WARDuino/WARDuino.cpp

Lines 335 to 339 in 07cf2b6

    
           // TODO: make use of these 
        
           uint32_t memorysize = read_LEB_32(&pos); 
        
           uint32_t tablesize = read_LEB_32(&pos); 
        
           (void) memorysize; 
        
           (void) tablesize;

@carllocos could you supply 2 files: one with correct offset, one with a wrong offset

WASI support

We should look into supporting WASI.

Remove unnecessary files

On branch feat/wood there are a lot of temporary and unnecessary files.
We should clean the repo before merging #47.

Remove the following files:

Interrupt payload needs to end with a space and a new line

I'm writing a python client to WARDuino and noticed that when sending an interrupt through the socket, the interrupt payload has to end with a space and a new line in order to be processed by WARDuino: '60 \n' works fine but '60\n' or '60 ' does not work.

However, when sending interrupts through Telnet the space is not needed i.e., it suffices to type '60' and press enter.

Block type collision

Block types may be expressed in one of two forms; (1) as a type index of an existing function type or (2) as a value type.¹

Currently the VM only supports the latter. (see WARDuino.cpp#L57)
However, to avoid collision with the type indices, the value types are encoded as negative numbers.
In WARDuino we only use unsigned integers.
This means that simply adding type indices to the switch statement in the get_block_types function, like we did in commit fc5c1df, means collisions are technically possible.

How would we best resolve this?

¹ https://webassembly.github.io/spec/core/valid/types.html?highlight=block#block-types

Split includes

Memory synchronisation during OOP

Changes to the memory are not forwarded to the drone.
This means that calling primitives with pointers (start + length of string for example) may fail, as the memory is not kept in sync.

Reproduce

Start OOP
Change memory
Call a primitive that uses the altered memory

	export function interrupt_on(pin: u32, mode: u32, fn: () => void): void {
	subscribe_interrupt(pin, fn, mode);
	}

	def_prim(mqtt_unsubscribe, threeToOneU32) {
	uint32_t topic_param = arg2.uint32;
	uint32_t topic_length = arg1.uint32;
	uint32_t fidx = arg0.uint32;

	const char *topic =
	parse_utf8_string(m->memory.bytes, topic_length, topic_param).c_str();

	Callback c = Callback(m, topic, fidx);
	CallbackHandler::remove_callback(c);

	bool ret = mqttClient.unsubscribe(topic);

	#if DEBUG
	Serial.print("Unsubscribed to ");
	Serial.println(topic);
	Serial.flush();
	#endif

	pop_args(2);
	pushInt32((int)ret);
	return true;
	}

	uint8_t *pos = bytes;
	word = read_uint32(&pos);
	debug("Magic number is 0x%x\n", word);
	ASSERT(word == WA_MAGIC, "Wrong module magic 0x%x\n", word);
	word = read_uint32(&pos);
	ASSERT(word == WA_VERSION, "Wrong module version 0x%x\n", word);
	// Read the sections
	uint8_t *bytes_end = bytes + byte_count;
	while (pos < bytes_end) {
	uint32_t id = read_LEB(&pos, 7);
	uint32_t section_len = read_LEB_32(&pos);
	uint8_t *start_pos = pos;
	debug("Reading section %d at 0x%p, length %d\n", id, pos, section_len);
	switch (id) {

	// TODO: make use of these
	uint32_t memorysize = read_LEB_32(&pos);
	uint32_t tablesize = read_LEB_32(&pos);
	(void) memorysize;
	(void) tablesize;

topllab / warduino Goto Github PK

warduino's People

Contributors

Stargazers

Watchers

Forkers

warduino's Issues