Code Monkey home page Code Monkey logo

social4git's Introduction

social4git: Non-federated decentralized social media

Briefly, this is a decentralized version of Twitter that does not use federation.

Every user stores their application state in a git repo they own and control. No other infrastructure is necessary!

You use the application using a client. We have a command-line client for now.

Install and use

All you need to do is read this short walkthrough.

You are not alone

We keep a directory of users who want to be followed. Feel free to grant their wish.

Going forward

This is a proof-of-concept, but quite capable at it.

There's a lot we hope to materialize soon:

  • mobile client
  • liking
  • verifiable posting
  • track a Twitter timeline
  • analytics
    • who are my followers
    • what is the impact of my posts

Who's behind this

social4git is the little sister of gov4git. Both projects are a collaboration between @protocollabs, @MSFTResearch and @RadxChange, focused on creating tools for digital democracy.

Help is welcome

We welcome open-source contributions. The biggest item on our TODO list is to build a mobile app for iOS and Android. Please, contact us if you are up to the task.

Stay informed

Follow us on Twitter @maymounkov and @social4git.

social4git's People

Contributors

petar avatar chrisbergeron avatar

Stargazers

Nikolaus Schlemm avatar Dwight Spencer (denzuko@mastodon.social) avatar  avatar Jason Carver avatar Chad Nehemiah avatar  avatar  avatar Hossein Zeinali avatar Luis Oala avatar corey james avatar Dharmi Kumbhani avatar Yash Atreya avatar Saurabh Chaturvedi avatar Vipul Ved Prakash avatar Mike Chung avatar  avatar Michael S. Manley avatar Wing Kwok avatar Stewart McGinnis avatar sanmh avatar Andrii Tarykin avatar Maxime avatar Jack Merrill avatar  avatar Alexandr Zahatski avatar neuroplastic avatar Thai Thien avatar LU XIUYUAN avatar Ins avatar Arran Ubels avatar  joker@home:~$ whoami avatar Austin avatar Victor Andritoiu avatar  avatar  avatar  avatar  avatar Borna Bešić avatar 姚文强 avatar Zero avatar Masanori Ogino avatar Nikola Rusakov avatar  avatar familyboat avatar  avatar Constantin Rack avatar ZheNing Hu avatar Clément Charmet avatar Thomas Nicholson avatar Bryan Guffey avatar  avatar Brian Redfern avatar Rahul Kulkarni avatar  avatar Jake Levine avatar Yukai Huang avatar Jeron Wong avatar Jeff Kingyens avatar Dist Bit avatar Balamurugan Soundararaj avatar  avatar Salvatore Gentile avatar Nikolay Kolev avatar Andrew McElroy avatar Burney Hoel avatar Pavel avatar Árni Björnsson avatar  avatar Vít Falta avatar  avatar karie avatar Patrick Smith avatar Tom Waterman avatar Adrien avatar Krzysztof Sinica avatar Seth avatar Mikael Karon avatar Callum Galbreath avatar Asier avatar Mahesh Narayanamurthi avatar Ian Channing avatar Bora Alp Arat avatar Lucas Barbosa avatar  avatar Martin Bergström avatar Ertuğrul avatar Julian Xhokaxhiu avatar Richard Silver avatar Kishore Debnath avatar Josh Haglund avatar Andrew Pynch avatar Doug Holton avatar R. Cooper Snyder avatar Rodolfo avatar josephedward avatar Mukunda Modell avatar Vineeth Voruganti avatar Raimon Grau avatar stagas avatar Jonny Dubowsky avatar

Watchers

Abhik Khanra avatar Fabio S. avatar  avatar  avatar

social4git's Issues

social4git sync: malformed unpack status: 0024unpack index-pack abnormal exit

Running social4git sync occasionally (!) produces an error:

malformed unpack status: 0024unpack index-pack abnormal exit

This is flaky behavior. Rerunning sync a few times tends to fix the problem.
The error comes from the github servers.

It is currently unclear whether this issue is due to:

  • a bug in go-git (the git client library we use)
  • a non-standard corner-case behavior on the github git servers

gossip social graph

objective:

  • users must learn a meaningful portion of the ambient social graph

approach:

  • piggyback gossip of the social graph on top of sync operations
  • apply random adaptive subsampling to keep space complexity under a given limit

I'd like to join the public directory

Your social4git public repo URL

https://github.com/neuroplastic/neuroplastic.social4git.public.git

Your intro

From: https://github.com/neuroplastic/neuroplastic.social4git.public.git Time: 2023-03-30 08:01:37.057429 +0000 UTC Link: social4git-https://github.com/neuroplastic/neuroplastic.social4git.public.git?post=20230330080137_2d57018c46f3c72e42b73b322a32b35dac087bb76665604ac8728c0236894836_afca3b1bd2d73dacd1d636dfbd3d25b94c16a04d37a06661d52611b265c2cc30 Hei from Finland! I'm Myles Byrne (neuroplastic). I started using the semantic web when i helped convert the card catalog of the oldest library in the US (Redwood Athenaeum) to digital in the mid-90's. I started working towards decentralised digital democracy in the 00's when, while working for an e-democracy startup headed by Clinton's press sec. Mike McCurry called Grassroots, i discovered that it was actually an advanced astroturfing system. It was mostly used by corporations like PG&E to manipulate its customers into influencing legislation. Over the 10's i worked on leading projects in Finnish digital health data ecosystem. I tried to get Finland to use Tim Berners-Lee's Solid to provide healthdata pods for national and intl. patients, and academic pods for researchers. I worked for Inrupt in '22, and realised what the Gov4Git whitepaper stated in Jan. '23: Solid is too caught up in the gordian hairball of enterprise SAAS to reach its intended publics. I'm here to integrate health, wellness, and education, using RDF, with exactly the type of system gov4git and soc4git is prototyping. It's been a great journey from Markus Sabadello's Project Danube, to Michiel De Jong's unhosted, to Ruben Verborgh's personal RDF websites, to my own pod project P2PScience, to Solid. But now is the moment for the p2p web to take its place as the realisation of the dream of universal real democracy.

fetch followed repositories in parallel

when running the sync command, social4git fetches the followed user's repositories sequentially.
this should be done in parallel with a simple cap on concurrent fetches.

show/sync fail

I am trying to run it and running into this:

post and follow work fine, but sync / show fail.

MacBook-Pro-2:~ vipulprakash$ social4git show repository not found goroutine 1 [running]: runtime/debug.Stack() /usr/local/go/src/runtime/debug/stack.go:24 +0x64 [github.com/gov4git/lib4git/must.mkErr](http://github.com/gov4git/lib4git/must.mkErr)(...) /Users/vipulprakash/go/pkg/mod/[github.com/gov4git/[email protected]/must/must.go:16](http://github.com/gov4git/[email protected]/must/must.go:16) [github.com/gov4git/lib4git/must.Panic](http://github.com/gov4git/lib4git/must.Panic)(...) /Users/vipulprakash/go/pkg/mod/[github.com/gov4git/[email protected]/must/must.go:20](http://github.com/gov4git/[email protected]/must/must.go:20) [github.com/gov4git/lib4git/must.Errorf](http://github.com/gov4git/lib4git/must.Errorf)({0x104f59cc0, 0x1400022e3c0}, {0x104d51b67?, 0x140001d9610?}, {0x140001d95e8?, 0x104f5da28?, 0x1a?}) /Users/vipulprakash/go/pkg/mod/[github.com/gov4git/[email protected]/must/must.go:24](http://github.com/gov4git/[email protected]/must/must.go:24) +0x50 [github.com/gov4git/lib4git/must.Assertf](http://github.com/gov4git/lib4git/must.Assertf)(...) /Users/vipulprakash/go/pkg/mod/[github.com/gov4git/[email protected]/must/must.go:31](http://github.com/gov4git/[email protected]/must/must.go:31) [github.com/gov4git/lib4git/git.PullOnce](http://github.com/gov4git/lib4git/git.PullOnce)({0x104f59cc0, 0x1400022e3c0}, 0x1400022f050, {0x140001b6500, 0x35}, {0x140002170d0, 0x1, 0x1}) /Users/vipulprakash/go/pkg/mod/[github.com/gov4git/[email protected]/git/sync.go:135](http://github.com/gov4git/[email protected]/git/sync.go:135) +0x2d4 [github.com/gov4git/lib4git/git](http://github.com/gov4git/lib4git/git).(*clonedCacheProxy).pull(0x14000244600, {0x104f59cc0, 0x1400022e3c0}) /Users/vipulprakash/go/pkg/mod/[github.com/gov4git/[email protected]/git/cache.go:147](http://github.com/gov4git/[email protected]/git/cache.go:147) +0x144 [github.com/gov4git/lib4git/git](http://github.com/gov4git/lib4git/git).(*Cache).clone(0x140001b5ba0, {0x104f59cc0, 0x1400022e3c0}, {{0x140001b6500?, 0x140001b5ba0?}, {0x104d52f6d?, 0x1052fe9e0?}}, 0x0) /Users/vipulprakash/go/pkg/mod/[github.com/gov4git/[email protected]/git/cache.go:77](http://github.com/gov4git/[email protected]/git/cache.go:77) +0x188 [github.com/gov4git/lib4git/git](http://github.com/gov4git/lib4git/git).(*Cache).CloneOne(0xf000?, {0x104f59cc0?, 0x1400022e3c0?}, {{0x140001b6500?, 0x1056a0a68?}, {0x104d52f6d?, 0x140001d9888?}}) /Users/vipulprakash/go/pkg/mod/[github.com/gov4git/[email protected]/git/cache.go:57](http://github.com/gov4git/[email protected]/git/cache.go:57) +0x30 [github.com/gov4git/lib4git/git.CloneOne](http://github.com/gov4git/lib4git/git.CloneOne)({0x104f59cc0, 0x1400022e3c0}, {{0x140001b6500?, 0x140001d9918?}, {0x104d52f6d?, 0x1052cf0a0?}}) /Users/vipulprakash/go/pkg/mod/[github.com/gov4git/[email protected]/git/clone.go:33](http://github.com/gov4git/[email protected]/git/clone.go:33) +0x64 [github.com/social4git/social4git/proto.FetchFollowedPostsMonth](http://github.com/social4git/social4git/proto.FetchFollowedPostsMonth)({0x104f59cc0, 0x1400022e3c0}, {{{0x140001b6480, 0x5}, {0x140001b6488, 0xa}, {0x140001e1290, 0x25}}, {0x140001b64c0, 0x37}, ...}, ...) /Users/vipulprakash/go/pkg/mod/[github.com/social4git/[email protected]/proto/show_followed.go:151](http://github.com/social4git/[email protected]/proto/show_followed.go:151) +0x90 [github.com/social4git/social4git/proto.FetchFollowedLatestPostsMonth](http://github.com/social4git/social4git/proto.FetchFollowedLatestPostsMonth)({0x104f59cc0, 0x1400022e3c0}, {{{0x140001b6480, 0x5}, {0x140001b6488, 0xa}, {0x140001e1290, 0x25}}, {0x140001b64c0, 0x37}, ...}) /Users/vipulprakash/go/pkg/mod/[github.com/social4git/[email protected]/proto/show_followed_latest.go:19](http://github.com/social4git/[email protected]/proto/show_followed_latest.go:19) +0xd4 [github.com/social4git/social4git/social4git/cmd.glob..func7](http://github.com/social4git/social4git/social4git/cmd.glob..func7)(0x1052c7b40?, {0x104d52121?, 0x0?, 0x0?}) /Users/vipulprakash/go/pkg/mod/[github.com/social4git/[email protected]/social4git/cmd/show.go:55](http://github.com/social4git/[email protected]/social4git/cmd/show.go:55) +0x308 [github.com/spf13/cobra](http://github.com/spf13/cobra).(*Command).execute(0x1052c7b40, {0x1052feb80, 0x0, 0x0}) /Users/vipulprakash/go/pkg/mod/[github.com/spf13/[email protected]/command.go:920](http://github.com/spf13/[email protected]/command.go:920) +0x5b0 [github.com/spf13/cobra](http://github.com/spf13/cobra).(*Command).ExecuteC(0x1052c6ce0) /Users/vipulprakash/go/pkg/mod/[github.com/spf13/[email protected]/command.go:1044](http://github.com/spf13/[email protected]/command.go:1044) +0x35c [github.com/spf13/cobra](http://github.com/spf13/cobra).(*Command).Execute(...) /Users/vipulprakash/go/pkg/mod/[github.com/spf13/[email protected]/command.go:968](http://github.com/spf13/[email protected]/command.go:968) [github.com/social4git/social4git/social4git/cmd.Execute](http://github.com/social4git/social4git/social4git/cmd.Execute)() /Users/vipulprakash/go/pkg/mod/[github.com/social4git/[email protected]/social4git/cmd/root.go:88](http://github.com/social4git/[email protected]/social4git/cmd/root.go:88) +0x28 main.main() /Users/vipulprakash/go/pkg/mod/[github.com/social4git/[email protected]/social4git/main.go:22](http://github.com/social4git/[email protected]/social4git/main.go:22) +0x38

How does “show --date” work?

social4git show --help says:

Flags:
      --date string   show posts from a UTC date in format MM/DD/YYYY

Does “from a UTC date” mean posts posted on that UTC date (from 00:00:00 to 23:59:59.999999 UTC)?

When I run social4git show --date 03/31/2023 I see posts ranging from 2023-03-24 to 2023-03-30, but none from yesterday, so I wonder if I take “from a UTC date” to mean something it doesn't (English is not my first language).

show --month and show --year don't show a thing (see #12).


Related: The MM/DD/YYYY date format is a US peculiarity. Can we have ISO-8601 dates (YYYY-MM-DD), too? Please.

Lessons learned from twtxt: discoverability, rate-limiting, …

Conceptually, social4git seems to be very similar to twtxt, and as such I expect social4git to suffer from at least some of the same weaknesses or inconveniences.

Before social4git takes off for real, I'd like to offer some of my experiences with twtxt:

Poor discoverability

Problem: I have installed a new piece of social software that looks exciting, now what? How do I find someone to follow? How do I become a part of a community or a “circle”? It is very discouraging to just sit there with an empty timeline, and have no idea how to populate it.

Initially, twtxt mitigated the problem by publishing a “we are twtxt” file, containing the nicknames and URLs of people who wanted to be discovered, and eventually people could learn about other participants from “retweets” or “mentions” in their timeline, but in the long run it didn't solve the problem.

A better attempt of discoverability was when twtxt started using the HTTP header User-Agent for announcing its own URL: in that way twtxt participants could learn about new people by looking for twtxt user-agents in their HTTP logs. Not all would benefit from this, but at least some would.

I stopped using twtxt in November 2020, so I cannot say where the protocol is today. They may have introduced other ways of discoverability since then, who knows.

My point is that if social4git is to proliferate and survive in the long run, it will need a way for participants to discover new people out there. Perhaps HTTP headers could be of some help to some, but I assume that many will host their repos on e.g. Codeberg or Microsoft GitHub, and HTTP headers will be inaccessible there.

Would it be possible for social4git to publish the followees in a separate branch (unless a followee is marked as private, of course) that can be fetched by followers and used as suggestions?

Rate-limiting

From twtxt I learned that even if each participant rarely published more than a handful of tweets daily, some followers seemed to think it was necessary to set up a cron job to fetch the whole damn timeline every few minutes (and many clients didn't bother to check the last modification date), or even every minute. Obviously this doesn't scale, and it also introduces a lot of annoying noise in the HTTP log.

Could each social4git instance publish some metadata about its owner and repo? One parameter could be a “crawl delay” that would serve as a suggestion to followers to not come back for more until after such a delay. That way remote won't be disturbed unacceptably often, even if somebody sets up a cron job to sync every minute.

Also, social4git should probably use exponential backoff if a “feed” is unavailable, rather than banging the remote end relentlessly. And if a git repo has been unavailable for more than a certain time, stop fetching it altogether and post a notice to its own owner that a feed has been disabled and may need manual inspection.

I'm out, but nobody seems to notice

As mentioned before, I stopped using twtxt back in November 2020 (i.e., almost 2½ years ago). I announced this to my feed well in advance, and also had myself removed from the “we are twtxt” directory. Yet even today people are still fetching my feeds (I had more than a handful). I have tried “404 Not Found” and “410 Gone” — each for extended periods — to no avail. I have tried banning with iptables, which has proved useless against dynamic IPs, and I have tried posting “This feed has been discontinued, please unsubscribe!” every minute, only to receive an angry email from a follower that I shouldn't fill their database with such nonsense. The problem seems to be that newcomers will use old twtxt files they find here and there, extract twtxt URLs from posts, and then add them as is to their following list whether remote is up or down, and no matter what the current contents of the feeds are.

My point is: There should be a way for a feed to announce that “This feed has been discontinued. Stop your git pull indefinitely. Now!”. Either explicitly (metadata) or implicitly (exponential backoff with an upper limit) — or both. Big players like Codeberg and MS GitHub are unlikely to notice useless knocking on their doors, but it sure becomes annoying on a small homeserver.

Please teach the social4git software to be considerate and polite. In other words: behave.

Edit / Delete / Expire

Not so much learned from twtxt, but still:

As I see it now, the social4git repo is ever growing (and I assume that new followers will initially have to fetch the entire repo), and I have no easy way to edit, expire or delete older posts. Expiring posts could be either per-post or per-repo (e.g., auto-delete all posts older than 1 year). Also: For how long should I keep other people's posts?

Or is social4git an append-only log?

Thanks for reading me.

Exits successfully, even after errors

My social4git has stopped showing posts. At all. Even posts it showed yesterday are no longer shewn. Since social4git exits with EXIT_SUCCESS (i.e., 0 on a Linux system), I didn't believe this to be an error. Instead I imagined that perhaps it would default to not show posts that are older than a couple of days.

However, when I ran it with --verbose, this is what I saw:

2023-04-01T15:24:53.796+0200	INFO	base/logging.go:61	github.com/gov4git/[email protected]/git/nocache.go:36 materializing repo https://codeberg.org/kas/kas.social4git.private.git on disk /tmp/user/1234/nonce-n5qngltsmzjc

open /tmp/user/1234/nonce-n5qngltsmzjc/post/2023/04/01: no such file or directory
goroutine 1 [running]:

It is true that the last two directories (…/04/01/) don't exist. There are some directories for March, though the posts are never shewn before social4git exits “successfully”.

And after this error, the “nonce” directory is never cleaned up. Instead it is just left to rot on the filesystem.


Version: latest (0.0.5, I believe)
Go version: go1.20.2 linux/amd64
OS: Linux x86_64

local aliases for users

  • users can define local aliases for other users' handles, e.g. chris -> github.com/...
  • users can be mentioned using @chris, @petar@chris, @petar@chris@george, etc.

robot.txt

add robot.txt containing advisory and other policies

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.