Code Monkey home page Code Monkey logo

sumi's Introduction

Sumi

A multi-user issue tracking system built with Yew frontend and actix-web backend.

This is a project for learning. The app is in a working state, but it is relatively unstyled, does not yet display meaningful errors, and has little input verification.

Features

  • Create, edit, close, re-assign tickets
  • Ticket list with sorting, filtering, and pagination
  • Enter notes for a ticket
  • Ticket event tracking (re-assignments, status changes, etc)
  • Nested-document style wiki
  • Markdown support for ticket descriptions, notes, and wiki
  • Editing conflict handling with revision history
  • Multi-user support
  • Local authentication with Argon2 hashing and salting
  • REST-style API
  • Dark/light theme
  • Localization support

Future Features

Down the pipe

These are some features I'm looking to add:

  • Custom dashboard
  • Projects
  • Tasklists for tickets
  • Show number of notes and tasks for a ticket in ticket list
  • Show description preview for a ticket on ticket list
  • Add On Hold status for tickets
  • Time tracking
  • Custom ticket tags
  • Recycle bin for deleted items
  • Contacts
  • Access levels and disabled users
  • Test coverage
  • More logging and error handling

Pipe-dream

These are some features I'd like to see but will not get to at this stage:

  • E-mail integration (updates, submit via-e-mail)
  • Scheduled tickets
  • Issue submission portal
  • Asset tracking
  • Reporting
  • Optional OIDC authentication
  • Custom Fields

Quickstart with Docker

  1. Generate and place certificates in ./certificates/ folder (cert.pem & key.pem). Sample command to generate self-signed cert:
openssl req -x509 -newkey rsa:4096 -nodes -keyout key.pem -out cert.pem -days 365 -subj /CN=localhost
  1. Set your environment variables in .env (refer to .env.sample).

  2. Build the Docker image

docker build -f Dockerfile -t vgwidt/sumi .
  1. Modify docker-compose.yml as needed and run:
docker-compose up -d

Setup Development Environment

Install Dependencies

Install Rust

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

Install diesel_cli dependencies:

  • Linux
sudo apt-get install libpq-dev
  • Windows Install PostgreSQL (don't need to include DB server if not using that for the instance). Add the following environment variables:
PQ_LIB_DIR = C:\Program Files\PostgreSQL\14\lib
PATH = C:\Program Files\PostgreSQL\14\bin

If you tried to build or run using cargo before adding the environment variables, run cargo clean. If in Windows and you get a Non-UTF-8 output error, install English language pack for VC build tools.

Windows: libintl-9.dll which ships with EDB is broken, get libintl-8.dll and put in debug folder (diesel-rs/diesel#2947)

Install diesel_cli

cargo install diesel_cli --no-default-features --features postgres

Install Docker (optional for quick PostgreSQL setup)

sudo apt -y install apt-transport-https ca-certificates curl gnupg2 software-properties-common
curl -fsSL https://download.docker.com/linux/debian/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
echo "deb [arch=amd64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/debian $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list
sudo apt update
sudo apt install -y docker-ce docker-ce-cli containerd.io
sudo systemctl status docker

Install target for wasm

rustup target add wasm32-unknown-unknown

Install trunk

cargo install trunk

Other dependencies

cargo install --locked wasm-bindgen-cli

If running from Windows, you need to manually create the symlink. Run the following from the backend folder:

mklink /D dist ..\frontend\dist

Finally, create .env file in project root. Refer to .env.sample for required variables.

Note, in a clean Linux environment using Docker for Redis without reverse proxies where everything runs on the same server, localhost does not work. Use 127.0.0.1 instead.

Build

Run ./run.sh (Linux, set to executable with chmod +x ./run.sh) or .\run.bat (Windows). This will execute the database migration, use trunk to build the frontend, then run the backend with cargo.

Getting Started

Default login is admin/password

sumi's People

Contributors

vgwidt avatar

Stargazers

 avatar 三米前有蕉皮 avatar Ian Thomas avatar Michael Cheng avatar da2n avatar cjschneider2 avatar Dr. med. Jan Schiefer avatar Ahiru avatar

Watchers

 avatar

sumi's Issues

Inline ticket editing

I prefer to have a sort of inline editing style, where items can be changed from the ticket description page. The title and description won't necessarily be fully inline, but would at least look like what the document page looks like now.

Markdown checkboxes don't render as HTML

- [ ] test should show as:

  • test

Since this isn't part of the CommonMark spec, pulldown_cmark doesn't implement this by default. Need to use Options (https://docs.rs/pulldown-cmark/latest/pulldown_cmark/struct.Options.html).

We'll also need to allow input types of checkboxes in sanitizer (allowing type attribute as checkbox and checked attribute).

It would be nice to allow a user to tick a checkbox without being in edit mode and update the raw text in the database, but for now we'll just add the disabled attribute to every checkbox when parsing the markdown to html.

Limit ticket and document title length

Title lengths are not limited in the front or backend (lengths should match). Should the returned length also be limited for the ticket list to allow for longer titles?

Frontend cannot handle logout API response

Frontend expects a body in the response, but the backend does not send one for the logout handler, causing an error on logout:

Failed to parse response: Err(reqwest::Error { kind: Decode, source: Error("EOF while parsing a value", line: 1, column: 0) })

Implement preferred timezones

All times are currently displayed in UTC. Allow timezone to be set and saved to user preferences. user_preferences table already has timezone (TEXT) field that can be used.

Set secondary sort to ticket_id

There is no way to customize secondary sort in the front or backend yet, so just default the secondary sort to ticket_id.

Cannot update ticket that has no assignee

Updating an unassigned ticket without changing assignee results in a 500 error:

insert or update on table "tickets" violates foreign key constraint "fk_ticket_assignee"

Improve time tracking

Notes can hold time spent, which is the only means of tracking time for a ticket. There is no way to edit the time or the date which the time should apply to, and there is no use of this time anywhere else (reports, ticket summary).

We need to decide if it is appropriate to store the time in a note, or if time entry should be separated from a note. Keeping time as an optional field would be simple, but adding a separate table for time tracking might allow for more flexibility and easier querying for reports. It would make it easier to add fields as well such as start/end time.

Once the data structure is decided and editing is implemented, I'd like to use this to get started on reports.

Custom fields

I find myself needing a "Location" field, but adding it as a default field may not be appropriate. I could add it as a toggable field, but I think it might be better to just add custom fields. Either way, we need a system in which the user can customize these values.

Database minimum requirements

Create a table that lists the custom fields:

CREATE TABLE ticket_custom_fields (
    id SERIAL PRIMARY KEY,
    field_name TEXT NOT NULL,
    field_type TEXT NOT NULL, 
    is_select BOOLEAN NOT NULL,
    select_values TEXT[]
);

field_type would allow for type checking (i.e. int, text). is_select would make it a select input only. select_values would be default values. If is_select is false, but there are select_values, it would be a combobox.

Then, we need a table to store the actual field data for each field for each ticket:

CREATE TABLE ticket_custom_field_data (
    id SERIAL PRIMARY KEY,
    ticket_id INTEGER REFERENCES tickets(id) ON DELETE CASCADE,
    custom_field_id INTEGER REFERENCES custom_fields(id) ON DELETE CASCADE,
    field_value TEXT
);

Considerations

  • Display priority should be allowed to be set
  • Allowing for customizing element size may be necessary
  • Should we allow it to be set as a default column in the ticket list, or leave that to user preferences?
  • Should all custom fields be added to the ticket filter section?
  • A lot of care will be needed for type checking and handling of changes or deletions of fields

Implement Due Dates

To start, this would just involve adding a due_date field to the tickets table (timestamp), a field in the ticket edit page, a display on the summary page and an entry on the ticket table. Conditional highlighting would be nice for tickets near or past the due date.

This will require some opinionated decisions for now on when to highlight, default due dates, etc, but I'd like to eventually allow these to all be configurable.

Handle editing conflicts

Some type of conflict handling is needed. I think for the purpose of this app, optimistic locking is the best option. I planned to implement some level of version control anyways.

Improve Docker builds

Docker build speeds can be sped up significantly.

  • Cache dependencies
  • Eliminate cargo cache clearing commands
  • Use pre-compiled binaries (i.e. diesel
  • Move binary to a slim Debian image (depends on ability to build diesel binaries, until then use rust image)
wget -qO- https://github.com/thedodd/trunk/releases/download/v0.16.0/trunk-x86_64-unknown-linux-gnu.tar.gz | tar -xzf-
      chmod +x trunk
      cp trunk ~/.cargo/bin/

Separate Dockerfiles are not necessary either, especially once caching is properly utilized.

Add title filtering to ticket list

Add optional title field to the ticket filter struct, add the additional query in the backend handler, and add an input box to the ticket list.

Implement versioning

I'd like to setup versioning for tickets, notes, and documents. Separate tables would be made for each. I imagine the columns would be:
revision_id (pk), document_id (fk), content, created_at, created_by. Maybe a serial for revision number?

Related to #18

Shared

  • New ticket_revisions and document_revisions table
  • New Response type that contains error message on failure and typed data on success (c1daa47)
  • Component to ask for user action when conflict detected
  • Enum for Response message types (good opportunity to introduce shared/common)
  • Component to view revision history (use diff for display)
  • Handlers to retrieve version history

Tickets

  • Use new Response type
  • New ticket_events table
  • Backend logic to compare provided revision time with current revision time
  • Ensure backend uses a single variable for time when the same time is updated for multiple fields
  • Update "updated_at" for any changes, update "revision" only for description.
  • Backend to create events when fields other than description are changed
  • Display events in note list (sort between notes by using creation date)
  • Only send updated fields to backend when updating

Documents

  • Use new Response type (c1daa47)
  • Backend logic to compare provided revision time with current revision time
  • Only update "updated_at" for content (Backend set in b5de478, but frontend still sends content even if it isn't changed, resulting in revision)
  • Display warning if there is a new revision between opening and clicking edit
  • Only send updated fields to backend when updating
  • Backend: skip creating revision and don't update timestamp if content is the same

Priority system makeover

I originally liked the idea of having a simple opinionated priority system, separate from using tags. While tags would allow more flexibility, they may require more upfront customization, where the existing system with hard-coded levels provides a simple starting point. It could be extended fairly easily to allow custom priorities and ordering. It would also be different from tags in that you can only assign one priority to each ticket. But when tags are added, we could blur the lines, with a separate section for selecting the "priority" tag if desired.

My thoughts at this point are to create a new priorities table, where the priority of a ticket links to a unique priority. Then there would be an interface to create new priorities where the order could be changed and colors specified for display. This would also allow for other customization like automatic due dates, alerts, etc.

Add ability to mark a note as resolution

I've already added a resolution column to the tickets table which references note_id. A note could be marked as the resolution when creating a note (checkbox), or in the actions menu of the note. The note marked as resolution would be highlighted. It could be shown at the top, or there could be an object in the ticket details area that could be clicked to shown the resolution.

Consider URL parameters for ticket list

URL parameters would provide navigational advantages. We can grab the TicketFilterPayload values with two lines of code:

let location = yew_router::prelude::use_location().unwrap();
let params = location.query::<TicketFilterPayload>();

Then navigator.push_with_query or navigator.replace_with_query can be used to update the URL:

navigator.push_with_query(
  &AppRoute::Home,
  &TicketFilterPayload {
      assignee: filter.assignee.clone(),
      status: filter.status.clone(),
      page: filter.page.clone(),
      per_page: Some(value),
      sort_by: filter.sort_by.clone(),
      sort_order: filter.sort_order.clone(),
  },
);

It gives us a URL that looks like this with all options:

https://localhost:9080/?assignee=ddf8994f-d522-4659-8d02-c1d479057be6&status=Open&page=1&per_page=30&sort_by=ticket_id&sort_order=asc

and some:

https://localhost:9080/?status=Open&page=1&per_page=34&sort_by=ticket_id&sort_order=asc

push_with_query pushes a navigable entry, whereas replace does not.

I've been thinking about implementing it for a while, but am still undecided. How useful is navigation history for any of the filtering or sorting options? Could it instead be a nuisance?

Use Ticket Events on Frontend

Questions:

  • Do we want to sort events in-between notes or show them in a separate section? I prefer to sort between notes.
  • Do we want to create "pseudo-events" (ones that don't exist in the database) to be displayed for things like the ticket being opened? Or should that be an actual event?

Session error

Hi,

I'm trying to run it on MacOS, but with no success. Database connection is working properly. Login is successful, but session is not working. I also checked the Redis server, it works fine, the user key is inserted. What could cause the problem? How can I try.

Screen Shot 2023-01-02 at 22 02 13

Add tasklists

Tickets should be able to have tasklists associated with them with a list of items to do. Github Issues does something interesting in that it looks for all checkboxes in the original issue posting. I think that would be nice, but I also believe a task list should be more dynamic and that the description of a ticket generally won't change much. I think it would be cool to have an immediately editable task list section in a ticket where tasks can be managed. This will require a new table for tasks.

Improve authentication

A few essentials:

  • Add password confirmation to new user page
  • Add helpful error messages for input validation
  • Refactor NewUser and Settings duplicate code
  • Add username requirements
  • Increase password requirements (do we want to make anything optional?)
  • Add basic rate limiting to login
  • Ensure login form cannot be abused to determine any existential information

I've gained a bit more Rust and web development knowledge since building the authentication system some months ago, so I'll attempt a thorough review along with some research to ensure that it is solid.

This will not include anything major like OIDC, 2FA, token auth, etc.

Add custom view settings

I'd like to start implenting custom view settings. A JSON column should be added to user_preferences that contains view settings handled by the app.

Add ability to sort ticket list

I'd rather add sorting sooner than later. However, how it works would depend on pagination. As is, there is no pagination, so sorting could easily be done on the frontend. But if pagination is added, the sorting would need to be handled on the backend.

I also need to consider adding multi-level sorting. I could allow sorting by priority then sort the rest by ticket # by default, but I can see wanting to sort first by priority then by last updated as being useful among other patterns.

I'm considering adding simple front-end sorting in the meantime, where the sorting would be done by clicking a table header. For that, some neat little arrows: ↓↑

Implement search

Search needs to be added for tickets and documents.

IIRC postgres searching can be a bit difficult with CJK languages. Plugins are needed in order to separate words. I may just do a simple like/wildcard search for now, though it will come at a performance cost.

Language does not support use of variables

i18n strings will need to support the use of variables. The following does not work because word order is different in languages like Japanese:

format!("{} {} {}", actor_display, language.get("updated ticket status to"), event.event_data)

Upgrade to yew 0.21 and yew-router 0.18

Need to update use_effect_with_deps and use_future_with_deps to remove "_deps" and re-order signatures. Stylist-rs also needs an update, we should be able to just bump that when it is made available.

Also getting the following error:

error[E0277]: the trait bound `DelayedFormat<StrftimeItems<'_>>: ToHtml` is not satisfied
   --> frontend\src\components\ticket_list.rs:475:39

Implement Dashboard

The home page currently just implements the ticket list component. I'd like it to instead be a dashboard where modules. A module might be something like "My High Priority Tickets".

The ticket list will be made into its own route.

Trunk build fails with version 0.17

Using Trunk version 0.17 results in the following error when running build from workspace root directory;

2023-07-24T07:04:56.732533Z  INFO fetching cargo artifacts
2023-07-24T07:04:57.491615Z ERROR  error
error from HTML pipeline

Caused by:
    0: error from asset pipeline
    1: cargo artifacts not found for target crate
Error: error from HTML pipeline

Caused by:
    0: error from asset pipeline
    1: cargo artifacts not found for target crate
The command '/bin/sh -c trunk build -d dist ./frontend/index.html --release' returned a non-zero code: 1

For now, I'll lock the Docker file to using trunk version 0.16.0.

Remove CC0 Logo

I used a CC0 logo I liked a placeholder, but I don't plan to continue development on this, so I'd rather pull the logo to prevent any confusion.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.