altid / libs Goto Github PK

Currently, libs/server provides an underlying framework for a complete server, monitoring several services. It only needs to provide a simple listener with our main file handles synthesized to the client

Prior to the reorganization, unread buffer counts would clear when a client moved to that buffer. We have to reimplement that feature

We have fuzzing implemented on our Lexer, but we still need to write testing for all other types

Describe the bug
The fuzzing test is designed to run many passes with arbitrary data, in an attempt to break the parser. We managed to do that, so now we can investigate where that occurred.

To Reproduce
Currently unknown

Expected behavior
lexer should always error or return EOF, in this case it did neither

Currently, we have servers that sit in front of services; for example
irc <--> 9pd <--> client

This works, but is entirely unneccessary, if irc can broadcast over 9p itself. The benefit of the middleware server is very low compared to the complexity overhead it introduces

Currently the notifications file is not deletable, so the notifications for a given channel never clear.

Currently, Tabs just shows a list of open buffers for a given service. Ideally, we would add a suffix that shows the current unread count for a buffer

#altid [32]
#mychan [12]
#linux [49893]

A cleaner-still way to do configs would be to send a struct in to config.New/config.Mock, and have the library populate each of the struct entries automatically.

For example,

func main() {
    mytype := struct {
        // The entry is used for the tuple key for a service
        // The following would create a service=test name=mything port=564 entry, using defaults for port
        Name string `Select username`
        Port int
    }{username, 564}

   myconf, err := config.New(mytype, "test", false)

Any value that wasn't located in the ndb would either interactively prompt the user for a value, with a given default; or use the default outright.

If a struct tag, that is the Select username bit above exists, it would be used in the prompt to a user for their username.

$ Select username (default [halfwit])

If there is no struct tag, it would simply notify the user that it was using the default value, and they could set it in their configs

$ using port=564

This facilitates a few things. when we add a config option, we could alert the user logging either the default value being used, or exiting when the config option was mandatory with the canonical run myserv -conf to add this interactively

Bytes and String currently output the Args of a given command as an array, "[arg, arg]" and should output as a simple space-separated list. "arg arg"

Used with Notifications, for example

• found in store/internal/ramstore, *File.(Delete)

The normal logstore feed files are working correctly, and they are backed by a real file

• Possibly related to #68

Currently we do not send a correct EOF when we change the buffer on feed reads; we just return "ErrFileClosed". We should be returning io.EOF

The fs library expects a readCloser that is attached to either real or in-memory files as part of the public API

The TestWriters test in ctl_test.go currently errors occasionally on macOS, upon system restart the issue has been hard to reproduce.
The failure indicated that pushTabs in mock_ctl.go was not appending correctly to a map

Segfault on reading feed file

Calling close on a nil file: c.f. https://github.com/altid/libs/blob/main/service/internal/listen9p/9p.go#L261

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x2 addr=0x0 pc=0x10437d274]

goroutine 27 [running]:
github.com/altid/libs/service/internal/listen9p.(*client).getFile(0x140000a2080, {0x14001802226?, 0x140006079e8?})
        /Users/halfwit/go/pkg/mod/github.com/altid/[email protected]/service/internal/listen9p/9p.go:261 +0x1c4
github.com/altid/libs/service/internal/listen9p.(*client).stat(0x14001810070?, {0x14001802226?, 0x0?})
        /Users/halfwit/go/pkg/mod/github.com/altid/[email protected]/service/internal/listen9p/9p.go:198 +0xc4
github.com/altid/libs/service/internal/listen9p.(*client).walk(...)
        /Users/halfwit/go/pkg/mod/github.com/altid/[email protected]/service/internal/listen9p/9p.go:208
github.com/altid/libs/service/internal/listen9p.(*Session).Serve9P(0x140001ae320, 0x14001810070)
        /Users/halfwit/go/pkg/mod/github.com/altid/[email protected]/service/internal/listen9p/9p.go:156 +0x894
github.com/halfwit/styx.(*conn).handleTattach.func2()
        /Users/halfwit/go/pkg/mod/github.com/halfwit/[email protected]/conn.go:363 +0x30
created by github.com/halfwit/styx.(*conn).handleTattach
        /Users/halfwit/go/pkg/mod/github.com/halfwit/[email protected]/conn.go:362 +0x5ec

Currently, libs/fs creates a file store for all of the client-facing files: status, title, input, etc
This leaks potentially secure data on the systems; and while things like chroots or jails can assure nothing goes out, it's prone to disk corruption. Additionally, simple data requests results in expensive syscalls

Changing libs/fs to use in-memory store, with a simple file descriptor for the main content to read and write data as it becomes available/is created should assure we are more performant, and more secure. (The main content will only create a real file if logging is enabled. This will reuse the logic to read and write from log files when available)

In the config library, we want to be able to instantiate a repl to create a configuration on the fly, for a particular service. To do so, we would can output an error on not finding configs for a service, such as

$ myservice 
> unable to find config entry for myservice. Please add one, or run 'myservice --create-conf' to do so interactively

And then call the underlying Configurator when it's invoked that way, as a library feature.

We currently have no auth protocols, unless we borrow from Styx or libauth; but it would be beneficial to have reusable styles of authentication that we can leverage on many systems, regardless of whether they are able to run a factotum

Both Reads and Writes create these objects fairly frequently. Use a Pool to attempt to mitigate allocations

Currently, the whole tag itself is the prompt. To allow future growth, we should use a prompt tag value, such as follows

conf := &struct{
    Name string                `altid:"Username to use"`
    Port   int                 `altid:"Port to use"`
    SSL    bool                `altid:"no_prompt"`
    Log    types.Logdir        `altid:"Directory for cached buffers"`
    Listen types.ListenAddress `altid:"IP address for Altid server,omit_empty"`
}{"default name", 1234, false, "none", "none"}

This follows roughly with json, xml, and other packages that use struct tags to aid in semantic parsing. Currently the most useful idiomatic tag would be omit_empty as above, when it's only useful for a run with -conf to set a prompt, and never marshalled directly for a service. (And we could have none match to empty, for marshalling purposes). Additionally, no_prompt could be handy in the -conf stage when defaults are supposed to be accepted.

Current markdown formatters are specific to altid, for bold/strike/emphasis, and there's no bold+emphasis type.

When a service using this library fails to find a config file, it should error with -conf, and in the createConfig step it should make the dir.

Something along the lines of, "Read" for arbitrary files, or "Title" "Status" "Ctl" and related for well-known files.

Move input to be handled internally:

• allows us to close it cleanly
• log to errors
• Reduces boilerplate

Move context internally

• Cleans up implementations

Describe the bug
When the parser encounters a single tag, such as in the string test-string it will not correctly send a related error

Expected behavior
When an improperly formatted input string is, we want an error returned, logically within a services Input.Handle function

It may be beneficial to have it outside of the markdown handling specifically, as it will likely grow well beyond the scope

9pd looks for an events file on start, so it can tail for events. Currently, services don't automatically create this - we should do it in Start or Listen ideally.

In order to make clean tests throughout, it would be beneficial to have a type we can call that won't actually log to files

Write a simple test to ensure that duplicate entries error correctly, ambiguous entries error correctly, and start working on coupling

Is your feature request related to a problem? Please describe.
Currently, error logging is done via status messages, in the services themselves; this would standardize the process

Describe the solution you'd like
Any errors which occur in a given file service would log to errors, from client input errors to service errors; and be served up to the client along with the rest of the regular files

Describe alternatives you've considered
Previously, we used services' setting a status for error. This would require bespoke client parsing to be useful

In store/internal/ramstore, the Open method for the Dir struct is messy and potentially giving incorrect handles. Refactor with strings.Split(dir, os.PathSeparator) or similar to get each piece of the path, and build out the maps dir[item[0]].dir[item[2]].file[item[3]] for example

The output of tabs is currently not correct

• Truncation seems to be incorrect
• Occasionally, buffers are not shown

Currently, mock and defaults only provide a valid writecloser. They will need to be modified to also provide a valid ReadCloser for our uses

We want a library feature for mocking input as well, for testing clients

input, err := NewMockInput(h Handler, buffer string, debug bool, reqs chan string) (*Input, error)

Writes on reqs constitute file writes to input, and errs will be made accessible through
a method call, Input.Err() on both normal and Mock input variants; should they be needed.

Required: Handle(), Run(), Quit(),

The html lib will be a place where a lot of changes happen, as we improve. It's important to have everything be very well tested to streamline the development process.

Especially for fs, markup, and html

Currently, client activity simply resets the unread count for tabs, which is beneficial in bookkeeping the sessions. It would be further beneficial to forward a callback from the services themselves, such that they could update their respective endpoints for Discord or Matrix, for example, to show that content has been read.