Code Monkey home page Code Monkey logo

microsoft / extensible-storage-engine Goto Github PK

View Code? Open in Web Editor NEW
810.0 810.0 62.0 2.53 MB

ESE is an embedded / ISAM-based database engine, that provides rudimentary table and indexed access. However the library provides many other strongly layered and and thus reusable sub-facilities as well: A Synchronization / Locking library, a Data-structures / STL-like library, an OS-abstraction layer, and a Cache Manager, as well the full blown database engine itself

License: MIT License

C++ 94.05% C 2.28% CWeb 2.07% Assembly 0.30% SWIG 0.15% Modula-3 0.02% CMake 0.21% Perl 0.48% Monkey C 0.44%

extensible-storage-engine's Introduction

Extensible-Storage-Engine

A Non-SQL Database Engine

The Extensible Storage Engine (ESE) is one of those rare codebases having proven to have a more than 25 year serviceable lifetime. First shipping in Windows NT 3.51 and shortly thereafter in Exchange 4.0, and rewritten twice in the 90s, and heavily updated over the subsequent two decades after that, it remains a core Microsoft asset to this day.

  • It's running on 100s of thousands of machines and millions of disks for the Office 365 Mailbox Storage Backend servers
  • It's also running on large SMP systems with TB of memory for large Active Directory deployments
  • Every single Windows Client computer has several database instances running in low memory modes. In over 1 billion Windows 10 devices today, ESE has been in use in Windows client SKUs since Windows XP

ESE enables applications to store data to, and retrieve data from tables using indexed or sequential cursor navigation. It supports denormalized schemas including wide tables with numerous sparse columns, multi-valued columns, and sparse and rich indexes. ESE enables applications to enjoy a consistent data state using transacted data update and retrieval. A crash recovery mechanism is provided so that data consistency is maintained even in the event of a system crash. ESE provides ACID (Atomic Consistent Isolated Durable) transactions over data and schema by way of a write-ahead log and a snapshot isolation model.

The library provides many other strongly layered and, thus, reusable sub-facilities as well:

  • A synchronization and locking library
  • An STL-like data structures library
  • An OS abstraction layer
  • A Block / Cache Manager

All this is in addition to the full-blown database engine itself.

The version of source we post here will likely be a bit in advance of the version compiled into the latest Windows update. Therefore, the JET API documentation may be out of date with it.

Is this the JET database / engine?

No. Well ... it depends ... the question is not quite correct. Most people do not know that JET was an acronym for an API set, not a specific database format or engine. Just as there is no such thing as "the SQL engine", as there are many implementations of the protocol, there is no "JET engine" or "JET database". It is in the acronym, "Joint Engine Technology". And as such, there are two separate implementations of the JET API. This is the JET Blue engine implementation, see Notes in here. The origin of the colors have an an amusing source by the way. Most people think of the "JET engine" as JET Red, that shipped under Microsoft Access. This is not that "JET engine". We renamed to ESE to try to avoid this confusion, but it seems that the confusion continues to this day.

Future Plans

Continuous Commit Push

The code as of June 2021 is a single snapshot of the source code. We will be pushing commits from our internal repo as they are made individually, instead of as a periodic monolithic drop.

Tests

We released the unit tests, but we may in the future release more of the test code.

We will also be adding Azure pipelines to run the tests that are already present in this repo.

extensible-storage-engine's People

Contributors

2bitsalute avatar agmt5989 avatar evgenykotkov avatar microsoft-github-policy-service[bot] avatar microsoftopensource avatar msftbot[bot] avatar msftbrettshirley avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

extensible-storage-engine's Issues

Compatibility with windows 11

Hello,
I have just noticed that this code is not aligned with the version of esent.dll shipped with windows 11.
At least some files created with the version of esent.dll shipped with Windows 11 cannot be opened with this code.
Is there a plan to align the code here with what ships with Windows?
Thanks,
Claudio

Add sample starter code for a pure and safe read-only database reader application.

I have seen previous ESE database reader applications do some scary things in terms of their settings and behavior in ways that could be very disruptive or even create inconsistent DBs that the native application for that DB may not be able to recover from.

This work item respresents creating some boilerplate JET API start code, that would attach a clean database read only, and with no logging, and thus have no affect on the persisted ESE files. Such that the DB is not left inconsistent if the application should crash, nor have the possibility of creating a logical inconsistency for the native application that uses this DB.

Up-vote this issue if it is of interest.

Big treasure.

Happy to see microsoft opensource this good database engine.

Hope you guys continue opensource more and more project.

Very good move.
To open .for all developers.

We all can got benefit on your opensource codebase.

How to use debug esent.dll for easier debugging of your ESE application.

How to use debug esent.dll for easier debugging of your ESE application ... this would cover our "Error Trap", some registry parameters that are only used in debug, common Assert()s that might help you, etc. This really could only be helpful for developing your own private ESE applications (i.e. it would not help you understand or debug any other ESE-based windows services, such a AD or Exchange Server).

Up-vote this issue if it is of interest.

Debugging your ESE application (or even - ESE-based Windows services) with the ESE debugger extension.

Debugging your ESE application (or even - ESE-based Windows services) with the ESE debugger extension.

This could go on and on ... but to start we at least need to cover the basics of how to use windbg.exe/cdb.exe, what the ESE debugger extension is, and a few basic ESE debugger commands.

Then later we would cover more advanced ESE debugger commands, that accomplish specific purposes. You can inspect a great many things in a process with our debugger extensions - how our memory is being utilized, finding the longest running transaction, seeing if we have a hung IO operation (disk not returning), and many many more.

Feel free to respond to the issue to:

  • Suggest specific debug scenarios,
  • Or questions you might have of an active process.

This issue will be closed when basics and a few extra scenarios are documented, and I will spin anything left in comments into new issues.

Up-vote this issue if it is of interest.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.