Currently, we use https://cla-assistant.io/ for checking that our CLA is signed. (Please note that we don't require copyright assignment.) CLA Assistant uses MongoDB and can be self-hosted: https://github.com/cla-assistant/cla-assistant

We should make MangoDB compatible enough for CLA Assistant to work and then self-host it.

https://cla.ferretdb.io

We don't know how many MongoDB users use decimal BSON type. Please vote for that issue if you need them.

We should provide a non-internal package that would allow Go developers to embed FerretDB functionality into their programs. They would use an official MongoDB Go driver to connect to an address (typically, but necessary, on localhost) that our code would listen on.

Let's put it into /ferretdb directory. Let's start with the most basic public interface:

• no way to configure mode (it should be "normal");
• no way to enable or configure logging (it should be disabled);
• no way to configure metrics (they should be disabled);
• etc.

To get a connection string (URI), they would call some method after starting the embedded instance.

We will add more functionality based on the community requests.

Let's not move some existing code into that package but copy the smallest possible amount of code and provide the smallest public API possible. Of course, we will still be able to make breaking changes in it before 1.0, but we should not do them lightly.

• There should be a testable example.

One possible implementation could be:

• configuration type Config struct { GoodName string }
• a single Run(ctx context.Context, config *Config) error function that runs embedded proxy until ctx is canceled.

See #733.

Support non-default collations per collection, view, index, operation.

https://www.mongodb.com/docs/manual/reference/collation/

We don't know how many MongoDB users need them. Please vote for that issue if you need them.

We could do it ourselves in Go code or use something like https://github.com/supabase/pg_jsonschema.

• #415
• #419

• $and
• $not
• $nor
• $or

PostgreSQL Docker image starts the temporary server, initializes it, then stops it. Waiting for port availability is not enough as we can hit the temporary server, and then the test would fail when the temporary server is stopped.

Instead, we should test with psql query like SELECT 1. Additionally, our psql line actually uses a UNIX socket, not a port; that should be fixed too.

listDatabases command should return sizes of each database (PostgreSQL schema) as well as PostgreSQL database total size on disk.

The latter is easy: just call SELECT pg_database_size(current_database()) and convert result. pg_database_size is documented there.

The former is only slightly harder. Let's call pg_total_relation_size on each table. Getting a table list is already implemented in the MsgListCollections method; small refactoring to remove duplication might be needed.

Tests should be added too.

See linked commit for code locations.

We should support limit: 0 for deleting all matching documents and limit: 1 for deleting only one. Since PostgreSQL does not support DELETE … LIMIT …, we should instead do a subquery:
DELETE FROM xxx WHERE _jsonb->_id IN (SELECT _jsonb->_id FROM xxx WHERE xxx LIMIT 1).

• Support exact matches (including fields order): db.collection.find({'value': {'foo': 'bar', 'baz': 42}})
• Support several ANDed conditions
• Add more tests for dot notation and deeply nested documents ({"foo.bar.baz": {"qux.qux2": 42}})

We might already support some of those points, but we should add integration tests (moving them from handler tests and dance), and maybe add more tests for weird cases and fix them.

Make pgx return UTC instead of Local.
Use UTC everywhere. Remove conversion to Local.

https://docs.github.com/en/repositories/releasing-projects-on-github/automatically-generated-release-notes

It should be possible to make FerretDB listen on a TLS port.

Let's add flags:

• --listen-tls to specify a listening address;
• --listen-tls-cert-file to specify a PEM file containing a certificate chain;
• --listen-tls-key-file to specify a PEM file containing an unencrypted private key;

Integration tests should have -target-tls boolean flag that would switch them to use insecure (accepting wrong hostname and invalid or self-signed certificates) TLS connections for both in-process and external target ports. For the in-process listener, a self-signed certificate (keep it in the memory) should be generated only once, not for each test.

Not in the scope:

• TLS over Unix sockets;
• client certificates;
• embeddable ferretdb package.

Create a simple release checklist in the repository.

• #38
• #1593
• #1693
• #1694

• Instead of using one global PostgreSQL connection pool, separate pools should be shared by all client connections with the same username and password.

• FerretDB should handle the client's authentication commands and use provided credentials to authenticate in PostgreSQL.

Is it in planning ?

SOMEONE FINALLY DID IT LEZGOOOOOO 🥭🥭🥭🥭🥭

It does not appear to work, and tests are missing.

So let's release it!

We should also remove text references to "MangoDB". See #2281

We want to support a subset MongoDB's OpLog that is enough for applications and frameworks tailing it, such as Meteor.
It is not a goal (yet) to make it good enough for replication.

See:

• https://github.com/meteor/meteor/blob/devel/packages/mongo/oplog_observe_driver.js
• https://github.com/meteor/meteor/blob/devel/packages/mongo/oplog_tailing.js
• https://github.com/meteor/meteor/blob/devel/packages/mongo/oplog_v2_converter.js

We should support the "old" format, with full $set for updates, not the new format with diff.

Let's track the number of incoming requests by message type, command, and response/error status. Expose them as Prometheus metrics on debug endpoint, and dump them on MangoDB termination. That will be useful for testing applications in proxy-diff mode.

• https://dev.to/yugabyte/open-sourcemongodb-api-to-yugabytedb-with-mangodb-proxy-22ka
• https://dev.to/yugabyte/embedded-ferretdb-with-yugabytedb-mongodb-api-on-distributed-sql-jbi

Should we start PostgreSQL and Yugabyte side-by-side? Should we use Docker Compose profiles? Should we expand envtool?

MongoDB abandoned its open-source roots, changing the license to SSPL - making it unusable for many open source and commercial projects.

SSPL is not OSI approved. ( https://opensource.org/node/1099 )

But the claim that SSPL licensed software is unusuable for many opensource / commercial projects is not true. MongoDB, or any SSPL licensed software is totally usable as-is for 99% projects. Only exception is where the database itself is provided as a service. It is usable even in that case too, but brings extra open sourcing requirements. That's all.

Claiming that MongoDB is not usable for "open source and commercial projects" is quite wrong IMO.

Please check.

(Also, thanks for the interesting project)

• Remove Documents field.
• Unexport Sections field.
• Add Document method that could be used to implement lazy reading in the future.

• #1699

Search mangoDB will go to : https://github.com/dcramer/mangodb

which is a fake useful MangoDB.

We can't even reach real git repo:
https://github.com/MangoDB-io/MangoDB

The project currently does not support the count function.

Scope {_id: <value>} where is ObjectID

• #896
• #897

https://www.mongodb.com/docs/manual/core/capped-collections/
https://www.mongodb.com/docs/manual/core/tailable-cursors/

Blocks #76.

Pushdown sorting too.

FerretDB should manage users by itself and not rely on the backend to do that:

• FerretDB should store credentials in the admin.system.users collection;
• FerretDB should enforce authentication (check passed username and password against stored) itself;
• FerretDB should not change the backend connection string (PostgreSQL URI, etc) it was started with;
• FerretDB should not create users in the backend.

Among other things, this feature is required for some authentication mechanisms like SCRAM-SHA-256, which in turn is needed for some applications and drivers.

Note that this issue covers authentication only; authorization is #174.

Links

SASL / SCRAM

• Overview: https://www.cockroachlabs.com/docs/dev/security-reference/scram-authentication
• SASL: https://www.rfc-editor.org/rfc/rfc4422.html
• SCRAM: https://www.rfc-editor.org/rfc/inline-errata/rfc5802.html
• SCRAM-SHA-256: https://www.rfc-editor.org/rfc/rfc7677.html

PLAIN

• https://cheatsheetseries.owasp.org/cheatsheets/Password_Storage_Cheat_Sheet.html
• https://www.ietf.org/archive/id/draft-ietf-kitten-password-storage-04.html
• https://www.rfc-editor.org/rfc/rfc9106.html
• https://pkg.go.dev/golang.org/x/crypto/argon2

• Update listCommands to include requiresAuth.
• Update connectionStatus to include authInfo.

• $eq
• $gt
• $gte
• $in
• $lt
• $lte
• $ne
• $nin

Right now, it is a bit of a mess.

• Make Error a pointer method.
• Document the contract.
• Actually use returned errors in conn.

We already build Docker images on CI for PRs and pushes to main, but not for tags.

• On every new git tag, we should build a Docker image with tag matching git tag without leading v, and also tag it as latest. See FerretDB/github-actions#130.
• We should support multiple names (ferretdb and ferretdb-dev).
• We should push that image to both https://github.com/FerretDB/FerretDB/pkgs/container/ferretdb and https://hub.docker.com/r/ferretdb/ferretdb.

See also #323.

Support transactions and the following commands:

• abortTransaction
• commitTransaction

Implicit transactions for inserts are covered by #200. See also #866.

Blocked by #153.

Please vote for that issue if you use MongoDB transactions.

Infinity, -Infinity, NaN

We should run our tests (both locally and on CI) with CockroachDB. There is no separate handler for it; instead, a regular pg handler should be used. We start CockroachDB side-by-side with PostgreSQL (and Tigris), don't try to replace the latter with the former.

Our changes in Taskfile(s) and CI configuration should be more or less copy&pasted. We might want to do small refactoring, but that's not required. We will revisit that at #85.

But changes in the envtool, if any, should be good; its refactoring is in scope.

DoD:

• tests could be run agains CockroachDB both locally (with task) and on CI;
• it should be possible to run FerretDB with CockroachDB with task run-cockroach.
• if we see that CockoachDB is not fully compatible, let's document what doesn't work.

There is room for query performance improvements.

Some of them will not require us to break strict compatibility with MongoDB. For example, we could move document sorting to a stored procedure that would do everything on the PostgreSQL side but (with a lot of logic and tests) can still be fully compatible with MongoDB. Or we could write a custom PostgreSQL operator class / data type / extension to do more work on the backend side.

Other performance improvements may require us to break strict compatibility while keeping query results sane (and even saner than MongoDB). For example, we could push down ORDER BY query part as is. It will not sort some things the same way as MongoDB, but maybe that's fine.

All that functionality should be under testing flags until the next major version.

Forgive me for being that guy, I know we have unix and mac covered, what about windows? 😄

This line deferences err:
https://github.com/MangoDB-io/MangoDB/blob/88bd1d8e1cce1eff22a62349ccb2da28c2c1ea88/internal/handlers/common/error.go#L62

But sometimes, such as when ErrNotImplemented is passed, such as by here:
https://github.com/MangoDB-io/MangoDB/blob/main/internal/handlers/handler.go#L144

it is nil, and thus a panic occurs, rather than an empty error message returned.

It should be possible to Query an Array. Research dot notation support for arrays. Add more tests.

As a result of this task we should also have a better estimation where dot notation is supported and where it isn't.

dvyukov/go-fuzz-corpus#3
golang/go#14750
golang/go#48298

Something like that might be needed in the future:

// check if j contains objects with duplicate keys that not handled by encoding/json yet:
// * https://github.com/dvyukov/go-fuzz-corpus/issues/3
// * https://github.com/golang/go/issues/14750
// * https://github.com/golang/go/issues/48298
{
	var o1 interface{}
	if err := json.Unmarshal([]byte(j), &o1); err != nil {
		t.Skip(err)
	}

	b, err := json.Marshal(o1)
	if err != nil {
		t.Skip(err)
	}

	var o2 interface{}
	if err := json.Unmarshal(b, &o2); err != nil {
		t.Skip(err)
	}

	if !reflect.DeepEqual(o1, o2) {
		t.Skip("j contains objects with duplicate keys that not handled by encoding/json yet")
	}
}

You might want to consider a rename.

https://github.com/dcramer/mangodb

Hi,

great project. I have heard that MongoDB team has a test suite, that is also used by Amazon and Azure Teams to test the coverage of MongoDB features.

Have you tested this and what is the current coverage of the feature set?

We need a Markdown linter for our docs. It should be packaged in a Dockerfile (unless it is written in Go; in that case, it should be in tools/go.mod) and invoked with some Makefile Taskfile target.

• https://github.com/markdownlint/markdownlint
• https://github.com/DavidAnson/markdownlint
• https://github.com/igorshubovych/markdownlint-cli
• https://github.com/remarkjs/remark-lint