tjweir / bnrpersistence Goto Github PK

Aaron Hillegass [email protected] Jan 29, 2009

After a few years of whining that Core Data could have been better, I thought I should write a persistence framework that points in what I think is the right direction. And this is it. One big difference? Objects can have ordered relationships. For example, a playlist of songs is an ordered collection. This is awkward to do in Core Data because it uses a relational database. Another big difference? It doesn’t use SQLite, but rather a key-value store called Tokyo Cabinet. BNRPersistence is not really a framework at the moment, just a set of classes that you can include in your project.

First, you need to download Tokyo Cabinet: 1978th.net/tokyocabinet/ (there is a sourceforge page, but the latest build seems to be on this site). In Terminal.app, untar the tarball and cd into the resulting directory. You want 64-bit library? In Terminal, set an environment variable for 64-bit builds:

export CFLAGS='-arch x86_64'

Configure, build, and install:

./configure && make && sudo make install

Now, you have a /usr/local/lib//usr/local/lib/libtokyocabinet.a that needs to linked into any project that uses these classes. (I usually use the “Add->Existing Frameworks” menu item to do this.)

You’ll also need to have /usr/local/include/ among your header search path. (See the Xcode target’s build info to add this.)

Now just add the classes in the BNRPersistence directory into your project. (If you copy them, you won’t get the new version when you update your git repository. This may be exactly what you want, but these are pretty immature, so I would suggest that you link to them instead.)

You’ve used Core Data? The BNRStore is analogous to NSManagedObjectContext. BNRStoredObject is analogous to NSManagedObject. There is no model, instead, like archiving, you must have two methods in you BNRStoredObject subclass. Here are the methods from a Playlist class:

- (void)readContentFromBuffer:(BNRDataBuffer *)d
{
  [title release];
  title = [[d readString] retain];

  [songs release];
  songs = [d readArrayOfClass:[Song class]
                   usingStore:[self store]];
  [songs retain];
}

- (void)writeContentToBuffer:(BNRDataBuffer *)d
{
  [d writeString:title];
  [d writeArray:songs ofClass:[Song class]];
}

So, BNRDataBuffers are like NSData, but they have methods for reading and writing different types of data. (It does byte-swapping so you can move the data files from PPC to Intel without a problem.)

(I certainly debate the idea of a model file that would replace these methods, but for now this is fast and simple. If you would rather have a model file than implement these methods, you are invited to write a BNRPersistenceModelEditor.)

All the instances of class will be stored in a single TokyoCabinet file, thus you will be saving to a directory containing one file for each class that you are storing.

To create a store, first you must create a backend. I have tried BerkeleyDB and GDBM, but I am quite enamored with Tokyo Cabinet right now. (If you would like to try the other backends, write me and I’ll send you the necessary classes.)

NSError *error;
NSString *path = @"/tmp/complextest/";
BNRTCBackend *backend = [[BNRTCBackend alloc] initWithPath:path
                                                   error:&error];
if (!backend) {
  NSLog(@"Unable to create database at %@", path);
  /* display error here */ 
}

Now that you have a backend, you can create a store and tell it which classes you are going to be saving or loading:

BNRStore *store = [[BNRStore alloc] init];
[store setBackend:backend];
[backend release];

[store addClass:[Song class]];
[store addClass:[Playlist class]];

(You must add the classes in the same order every time. The classID of a class (see BNRClassMetaData) is determined by the order)

Now to get a list of of the playlists in the store:

NSArray *allPlaylists = [store allObjectsForClass:[Playlist class]];

To insert a new playlist:

Playlist *playlist = [[Playlist alloc] init];
[store insertObject:playlist];

To delete a playlist:

[store deleteObject:playlist];

To update a playlist:

[store willUpdateObject:playlist];
[playlist setTitle:@"CarMusic"];

(Yes, this is a place where a model file would make the framework cooler: I could become an observer of this object and get notified when the value changed.)

(As a side-note, there is some experimental support for automatically updating the undo managed. Just give the store an undo manager. This also could be made better with a model file.)

To save all the changes:

BOOL success = [store saveChanges:&error];
if (!success) {
    NSLog(@"error = %@", [error localizedDescription]);
    return -1;
}

Each class in a store can have a version. This is kept in the BNRClassMetaData object for the class. You can reach this in your readContentFromBuffer method (because you have access to the store).

In your BNRStoredObject class, you can implement these two methods if you wish:

• (void)prepareForDelete;

• (void)dissolveAllRelationships;

prepareForDelete is where you implement your delete rule: When a song is deleted, for example, it needs to remove itself from any playlists it is in.

dissolveAllRelationships is a fix for a common problem. You close the document, but the objects in the document have retain cycles so they don’t get deallocated properly. In dissolveAllRelationships, your stored objects (the ones in memory) are being asked to release any other stored objects they are retaining.

In the directory, you will find TCSpeedTest and CDSpeedTest. These are command-line tools that compare the speed of some tasks in BNRPersistence (TCSpeedTest) and Core Data (CDSpeedTest)

Your mileage may vary, but I see:

Creating 1,000 playlists, 100,000 songs, and 100 songs in each playlist and saving (ComplexInsertTest): BNRPersistence is 10 times faster than CoreData

Reading in the playlist and getting the title of the first song in each playlist (ComplexFetchTest): BNRPersistence is 13 times faster than CoreData

Creating 1,000,000 songs and inserting them and saving (SimpleInsertTest): BNRPersistence is 17 times faster than CoreData

Fetching in 1,000,000 songs (SimpleFetchTest) CoreData is 2 times faster than BNRPersistence (BNRPersistence is single-threaded and CoreData has some clever multi-threading in this case. I think I can do similar tricks in BNRPersistence and catch up in this case. In either case, it is very, very fast. On my machine, fetching a million songs takes 6.2 seconds the first time and 3.9 seconds the second time.)

The first problem is that you need to compile TokyoCabinet for arm. I tried every configure trick I could come up with and then just created an Xcode static library project and dumped the source into the project. This project is in the repository.

When you link to the resulting static library, you will also need to link in libz, which is part of the iPhone SDK

My code is under the MIT license and Tokyo Cabinet is under the LGPL. I think this will enable you to use it how you want to use it. If something bad happens because of the code, you can’t sue us. And if you make changes to Tokyo Cabinet, I think you need to submit those changes to the author. But I’m not a lawyer…

I recognize that there is room for improvement here:

• The creation of a model-file architecture and editor

• Add Tokyo Distopia to make full-text search fast

• Use B+ trees to make attributes indexable

• Better automatic undo support

• Automatic syncing to a web service

• Easy hooks for QuickLook images and Spotlight metadata in BNRStoreDocument

• Hook it up to Tokyo Tyrant for non-local storage