Code Monkey home page Code Monkey logo

kindmetrics's Introduction

Kindmetrics

Kindmetrics is an strictly privacy focused web analytics for your website.

Thanks to being built on Crystal, Kindmetrics is very small when compiled (~10MB) and is very fast. Faster than Elixir, Rails or java. It use very little cpu and memory and could be deployed on old machines with linux as distro.

It is built based on following ideas:

  • Simplicity
  • Privacy for both you and your visitors
  • GDPR compliant
  • NO COOKIES

And as a service we follow these ideas:

  • Only EU-based services used, except for DO but the servers will be on EU soil
    • If we find a better managed kubernetes service in EU than DO, with managed db, we will move.
  • No CDN for APIs, If we add cdn for the assets files it will be an EU-based one - but for now we won't.
  • We shall always force privacy-features like DNT.

And for the technical stand, we follow these ideas:

  • Small footprint (both in size and traffic)
  • As less third party dependent as possible
  • Focus on one thing only: Analytics
  • No SPA here. we focus on backend with some javascript.

Technical stuff

Kindmetrics is built on:

  • Lucky framework, on Crystal language
  • Tailwind css
  • Stimulus.js
  • Apexchart
  • DataMap
  • Litepicker
  • Clickhouse
  • PostgreSQL

Setting up the project

If you want to run this project for dev, like when you want to fix bugs or new features, you have to install Crystal language. You can find more info about crystal at https://www.crystal-lang.org - You can use crenv, asdf or similar. Check the details on https://luckyframework.org/guides/getting-started/installing#1-install-crystal

When that is done, you have to setup the lucky project:

  1. Install required dependencies
  2. Update database settings in config/database.cr
  3. Run lucky kind.clickhouse
  4. Run script/setup
  5. Run lucky dev to start the app

Tests

Kindmetrics have some tests, even if they could be more. But you can run them by:

crystal spec

And the code should be in /src and the tests in /spec

Deployment

You can find the deployment instructions in deployment.md file

Contribute

I am all open for any help I can get with this project.

Fork this repo, add push your changes to a new branch and create an pull request. I will review when I can. if they follow the aboves ideas and standpoints and don't go against my plans for future, which can be seen in the issues I will approve. Just add tests. Even if I have been bad on that I am trying to be better.

kindmetrics's People

Contributors

confact avatar dependabot[bot] avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

kindmetrics's Issues

Email report not sent

It seems the email report didn't get sent out for weekly or monthly.

Have to check it up.

Heatmaps

I guess it's an awesome feature and people love it.
It should be an optional feature though because people who don't need it, shouldn't be suffering performance-wise.

There are also JS libraries that are doing this alreadyand they even have wonderful live demos (i.e. simpleheat demo, webgl-heatmap blog post, heatmap.js demo or this PoC). Of course there are also some SO threads, like this or this that give good insights.

Maybe it might be a killer feature that helps to gain visibility against the other analytic application.
Because apps like PostHog, Plausible, Ackee, Fanthom and Shynet doesn't seem to have it.
Countly. and Matomo have it only as a premium feature.

Heatmap-768x442

EDIT: Maybe it would also help to have a look how OpenWebAnalytics implemented this.

Don't save user-agent

User agent can be seen as personal information for some. Let's not save it and also use it for the one-way hash for more accuracy.

Take out the website out from the app

move the marketing website, with the marketing pages and future blog out from the app and maybe move the application to the app. domain.

It should be pretty easy, and it is good to do this asap.

domains dropdown not working

Domains dropdown, when logged in, won't show domains for subpages, only add domain when clicking on it.

Let's add domains on the subpages as well.

Clickhouse support

I saw that you are using Postgres for now. I thought that it might be of interest for you that Yandex created a database exactly for analytics data for their service Yandex.Metrica. It is called ClickHouse and uses Apache 2.0 license.

It is probably easy to integrate something like this in the beginning, otherwise it might be difficult at a later point (i.e. like it happened for Countly).

There are also a few crystal shards already: clickhouse.cr, clickhouse-table and clickhouse-crystal.

Fix better device support

Now when a device type is not found, it returns an empty string or nil.

Let's add a fallback using screen width to determine device type.

Make domain sharable

Make to metrics can be public with a sharable link.

Maybe using a token or a slugified url.

upgrade Lucky to 0.23.0

Lucky 0.23.0 handles sessions and cookies better and has some nice things.

Let's upgrade lucky to 0.23.0 soon.

redesign

I have thought of some redesign to make it more clean and better structure of the subpages of an domain metrics.

What I will do:

  • clean up header and navbar
  • Add subnavbar for domains
  • Change colors a bit to make it more clean

Add up or down from prior period

Add so you can see how much up or down it have gone since last period in percentage

Like:
β–² 35%

or

β–Ό25%

ANd show with colours if it is good or bad. Like for bounce, it is good if it goes down when the rest is bad if it goes down.

Add session

Add support for sessions so you can get:

  • length user spent on the page
  • bounce rate

And more things later, like:

  • average page visits
  • Average sessions per user

Add more details to page details

Current there are

  • unique visitors
  • page views
  • bounce rates
  • top referrers

And maybe it would be great to add some of the overview details also to the page details (i.e. countries and devices).

Also things like load time and session time would be interesting.

Here's an example from Shynet (IMHO it's not a page detail view but I guess you get the idea):
service

I'm not quite sure what "Duration" and "Load time" mean here.
I guess it would also be interesting to have

Ignore bots from the stats

Check if the client is a bot and don't save it as a pageview.

Maybe save it but don't count it on pageview.

See current users live

See amount of users that are on the website live.

Should be able to see that through sessions without length set.

Recode track.js file

Recode it to be more flexible and prepare it to handle events.

Idea is to load it and be able to call window.kindmetrics.track('sign_up') to send custom events.
Pageview should work as usual.

Able to delete domain and user

Should be able to delete a domain in settings.

Same goes to the user, then all data will be removed, including domains and events.

write thousand as `*k`

When total counts are above thousand, let's type it as 2.3k instead of 2321

It makes it more readable.

Clean up referrer

Decide what to do with the referrer that is same domain:

Either:

  • hide it
  • Show it as (direct)

Support for own subdomain

Add support to be able to host the track.js code on own subdomain.

Maybe even let it host the API there as well for extra based.

Add today and custom periods

Have a today as period option where you can see the hours so far have gone today.

Also, add a custom period support where you can pick to and from dates.

Device analytics/mobile analytics

It's probably good to have such things on the roadmap for later.
The current device overview doesn't seem to provide details like resolution and (i'm not sure how useful this is) something like carriers etc.

1_countly-mobile-analytics-dashboard

A general screen resolution overview would probably be nice as well.
This is how it looks in Ackee:

Screenshot_20200702_114708

Fix url in email

The URL in the confirmation email is wrong and is sent as an empty <a> tag.

It becomes <a>click here</a>

Probably because I use UserConfirmations::Show.url(token: @user.confirmed_token) instead of UserConfirmations::Show.with(token: @user.confirmed_token).url

Email reports

Build support to send email reports, weekly (Mondays) or monthly. (first day of a new month)

These will be summaries the previous period with all the data we show on the dashboard in a nice way.

Should be able to add an email address to send to. So people don't need to share accounts or something.

Clean up the classes

Clean up the classes, remove duplicated code.

Make an parent class for action and pages with prepared methods and params.

track language preferences

Browsers also store and send language preferences and I guess it would make sense to track & show this as well.

This is how it looks in Ackee:
Screenshot_20200702_115006

Add metrics for domain

Add the most common metrics for analytics.

  • Unique visitors
  • Pageviews
  • Top Referrer
  • Top pages visited (your own pages)
  • Countries
  • Devices
  • Browsers
  • OS

Add payments

Add payments for plans

It will be one of these:

  • Stripe
  • Paddle

Plans rules:

  1. 9 EUR? - 50k page views per month
  2. 15 EUR? - 200k page views per month
  3. 39 EUR? - 1milj page views per month
  4. QUOTE? - 1milj+ page views per month

Fix duplicate referrers

Now referrer can get duplicated

One is the name and one is the domain of the referrer.

As example:

  • Reddit
  • reddit.com

Probably some bug in referer-parser.

Add a stronger privacy logic

Adding a bit stronger privacy on how identifying users.

Now we can see when a user come back after any time, 1 hour, 1 day, 1 week.

We should add a timeout to the user_id so we can not identify the user after he comes back.

My idea is to set a temporary user_id that is the same as we have now but gets removed after 30 minutes of inactivity, and then it will only have this time-based user_id.

we need it temporary to map a unique visitor for that moment.

Add support for mediums

We can add support to for medium.

It will be gathered from:

  • utm_medium
  • referer-parser's medium

in that way, you can group them.

Move all metrics to a central class

Making it easier to handle params for all queries.

Every query won't be called, but it will use common filtering/where like periods, domain_id scoping and stuff.

Make DNT logic optional(?)

I refreshed a few times but I didn't got anything.

EDIT: Ah, I might have forgotten about the DNT setting! I guess that was the reason then?

Will DNT lead to a request in general?
I'm not sure how compliant this is but would it make sense to have a minimal request that counts non-tracked requests? Like an API call that just increases a number?

Filter metrics by path?

Hi, just found about Kindmetrics. Looks nice.

Any plans to allow filtering metrics by path? (eg click on /foo and view pageviews and referrers for that specific URL)

Add Api

Adding API for at least the data on the dashboard. Giving more value and more "ownership" to people over their data.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.