impactrudia / objectify-appengine Goto Github PK

Not sure if it is a design decision to throw the NPE, but would vote for just 
ignoring null objects, or 
throwing a proper dedicated exception. The stacktrace shows the spot:

java.lang.NullPointerException
    at com.googlecode.objectify.ObjectifyFactory.getMetadataForEntity(ObjectifyFactory.java:219)
    at com.googlecode.objectify.impl.ObjectifyImpl.put(ObjectifyImpl.java:183)
    at de.fasthome.gae.ods.AdManager.putRealEstateAd(AdManager.java:80)
    at de.fasthome.server.UploadManager.doPost(UploadManager.java:261)

I clocked how much time the first call to register takes for a class and it 
takes about 200ms.  After that, each registration of a class takes 3ms to 
40ms for my classes depending on the size of the class.

I'm fine with a few hundred milliseconds, but on the home page one of the 
bullet points says, "Objectify does not impact application cold-start time, 
adding a few milliseconds at most."

Is there a way to speed this up?

From the documentation of type Objectify:

/**
 * <p>Create a typesafe query across all kinds of entities.</p>
 */
<T> Query<T> query();

/**
 * <p>Create a typesafe query across one specific kind of entity.</p>
 */
<T> Query<T> query(Class<T> clazz);

A test case (version 2.2.1):

int a = objectify.begin().query().countAll();
System.out.println( a);   // 0

int b = objectify.begin().query(Contact.class).countAll();
System.out.println( b );  // 100

I would expect--according to the documentation--that the first query result 
will give me at least 100.

So sadly List<Key<Object>> listKeys = objectify.begin().query().listKeys(); 
will result in an empty list too.

I would like to be able to configure my entity classes via an external file 
(something like the hibernate mapping files). The configuration with 
annotations is great but adds unnecessary dependencies on the Objectify 
framework. For example let's say I define the following class:

class Car {
    @Id String vin;
    String color;
    @Unindexed String unindexed;
}

If I want to use it outside of the GAE server application (for example in GWT 
or in Android or in whatever Java client that connects to the GAE server 
application) because of the @Unindexed annotation I have to include objectify 
as a dependency in the client application. Also because the @Id annotation I 
have to include another library. This may not seem so problematic in standard 
Java clients but on Android it does have a big impact on the size of the app.
The externalization of entity configuration to a different file would solve 
this problem. We could add this file during registration or the name of the 
mapping file could be implied from the name of the class.

ObjectifyFactory needs methods that turn Key<?> into a web safe string (and 
back).

What steps will reproduce the problem?
When using FetchOptions I've got exception 
{{{
Caused by: java.lang.IllegalArgumentException: interface 
com.googlecode.objectify.Objectify is not visible from class 
loader
    at java.lang.reflect.Proxy.getProxyClass(Proxy.java:353)
    at java.lang.reflect.Proxy.newProxyInstance(Proxy.java:581)
    at com.googlecode.objectify.DatastoreTimeoutRetryProxy.wrap(DatastoreTimeoutRetryProxy.java:30)
    at com.googlecode.objectify.ObjectifyFactory.maybeWrap(ObjectifyFactory.java:137)
    at com.googlecode.objectify.OPreparedQueryImpl$ToObjectIterable.<init>(OPreparedQueryImpl.java:122)
    at com.googlecode.objectify.OPreparedQueryImpl.asList(OPreparedQueryImpl.java:79)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at com.google.appengine.tools.development.agent.runtime.Runtime.invoke(Runtime.java:100)
    at com.googlecode.objectify.DatastoreTimeoutRetryProxy.invoke(DatastoreTimeoutRetryProxy.java:54)
    at $Proxy24.asList(Unknown Source)
}}}

Code sample
{{{
        Objectify ofy = ObjectifyService.begin();
        OQuery<MyClass> q = ObjectifyService.createQuery(MyClass.class);
        List<OKey<MyClass>> keys = ofy.prepareKeysOnly(q).asList(FetchOptions.Builder.withLimit(10));//ecxeption here 
!!!
}}}
but next code (without FetchOptions) works.
{{{
        Objectify ofy = ObjectifyService.begin();
        OQuery<MyClass> q = ObjectifyService.createQuery(MyClass.class);
        List<OKey<MyClass>> keys = ofy.prepareKeysOnly(q).asList();
}}}


What is the expected output? What do you see instead?
Must work

What version of the product are you using? On what operating system?
objectify-1.0.jar
Windows XP 32x
Eclipse Java EE IDE for Web Developers.Build id: 20090920-1017
Google Appengine 1.3.0 in Development Mode
Sun JDK 1.6u12




Please provide any additional information below.

As discussed in the following thread:

http://groups.google.com/group/objectify-appengine/browse_thread/thread/d3cde5bb
408f839

and proposed by Scott and Jeff, please implement an additional lifecycle event 
handling not within 
the entity class itself, but rather as an external EventListener.

The following reference shows the equivalent Hibernate design:

http://docs.jboss.org/hibernate/stable/entitymanager/reference/en/html/listeners
.html

"but don't forget that this is really just creating a OKey and calling get()!"

"an OKey" not "a OKey".

"However, when you lookup or reference your object, you do so by OKey." 
That should be "look up", not "lookup".

To make index entities easier to work with, the Query interface could use 
fetchParentKeys() and fetchParents() methods.

See this thread:

http://groups.google.com/group/objectify-
appengine/browse_thread/thread/233359c04d7c54e9

This issue should be revisited after GAE/Python exposes a cursor API which 
we can pattern on.

We need some sort of simple "getting started" project.

Annotation Reference is missing an @Embedded section.

What steps will reproduce the problem?
1.  Update HasArrays.java to include: 
        public int[][] ints2;
2.  Run the unit tests:  ant test
3.  Note testIntArrays() FAILS:
         ints2: java.util.ArrayList is not a supported property type.

Objectify v2.2.  

Hello,

I like you lightweight abstraction on top of the datastore. I was just about 
to start something similar for my own project.

Many people (including me) like to manager their build time dependencies with 
Maven so it would be great if you could provide a Maven repo for your SVN 
trunk. You may have a look at the provided example here: 
http://www.thewebsemantic.com/2009/04/11/your-very-own-google-code-maven-
repo/

Tobias Neef

What steps will reproduce the problem?
1. An empty database
2. query returns no result with 
3.
    OQuery<User> query = fact().createQuery(User.class);
    query.filter(column, param);
    User user = null;
    OPreparedQuery<User> pq = ofy().prepare(query);
    user = pq.asSingle();


What is the expected output? What do you see instead?
Expected returning null asSingle(), but we have a NPE:

java.lang.NullPointerException
    at
com.googlecode.objectify.OPreparedQueryImpl.asSingle(OPreparedQueryImpl.java:99)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.jav
a:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at
com.google.appengine.tools.development.agent.runtime.Runtime.invoke(Runtime.java
:100)
    at
com.googlecode.objectify.DatastoreTimeoutRetryProxy.invoke(DatastoreTimeoutRetry
Proxy.java:54)
    at $Proxy8.asSingle(Unknown Source)
    at com.miracle.server.dao.UserDao.getUser(UserDao.java:51)
    at com.miracle.server.pe.Login.doPost(Login.java:57)
        ......

What version of the product are you using? On what operating system?
Objectify v1.0

Please provide any additional information below.

I discovered a sneaky bug, which can happen due to a developer's mistake.

class Party {
@Id Long id
....
}

In one of my UI screens, I do a post of id (from the above Party), and on the 
server, that
'id' is received as a String.  Hence, the Party key can be created using String 
id.
Key<party> key = new Key(Party.class, id)
ofy.get(key)
I will never get the Party entity back, because the Party was creating using 
the long id.

The Keys constructed with 'String id' and 'long id' are different.

Can any error checking be done while creating the Objectify Key.

I'm using the Objectify 2.1.

In my User class, I store the hashed password as a byte[].  When I put it in 
the datastore, it is correctly stored as a blob.

When I try to load the User object back out I get this error:

java.lang.IllegalStateException: Cannot load non-collection value '<Blob: 40 
bytes>' into private byte[]

Is this a problem with how I have things set up, or a problem with Objectify?  
Anyone know how I could fix this?

Provides the ability to do case insensitive searching as described at 
http://gae-java-
persistence.blogspot.com/2009/11/case-insensitive-queries.html  

query.iterator() does not honor cursor constraint, even though 
query.fetch().iterator() does

It would be nice if there was possibility to transparently convert types 
not supported by Datastore to it's core types. 
For example, we have problem if we use BigDecimal. In datastore we can 
persist a number only as double or long, what can cause loss of precision. 
Converting BigDecimal to string will do the trick, but problem is that we 
have to write conversion logic in our model classes.

My vision for resolution of this problem:

Objectify defines a simple interface for value converter:

interface ValueConverter {
Object toDatastoreValue(Object v);
Object fromDatastoreValue(Object v);
}

We write simple converter for BigDecimal (we can write more sophisticated 
one which will prevent order)

class BigDecimalConverter {
Object toDatastoreValue(Object v) {
   return ((BigDecimal)v).toPlainString();
}
Object fromDatastoreValue(Object v) {
   return new BigDecimal((String)v);
}
}

We register our converter:

factory.register(java.math.BigDecimal.class, new BigDecimalConverter());

Now we simply use BigDecimal fields in our classes and Objectify does the 
rest.

Similar to Twig, allow relationship references to be defined as entity classes 
instead of Keys:

{{{
class SomeEntity {
   @Id long id;
   OtherEntity other;
}
}}}

The OtherEntity instance will contain only the Key information (id and parent) 
- there will be no separate fetch and thus the body of the entity will not be 
loaded from the database.

In order to populate this entity, a method will be added to Objectify:

Objectify.refresh(Object... entities)

Alternatively, the get() methods could be extended to take Object.

Beware, there are hazards to doing this - you now can have entity objects in 
your code which are uninitialized.

Automatic activation as per twig (or JDO) is not being considered at this time. 
 Too much magic with strong performance implications.

As you said in the wiki... 
I like managed relationships.. so I have added support for them. 

Lets say this:

class Person {
    @Id String name;
    Person friend;
}

now works exactly the same as this one:

class Person {
    @Id String name;
    Key<Person> friend;
}

The only difference is that in the first one the "friend" get fetched 
automatically when you get a 
Person, and when you put a Person with a friend != null, the friend has to be 
on the datastore (it 
doesn't put the child when you put the person), otherwise you get an "entity 
doesn't exists" 
exception.

No annotation needed. I made it to detect if the type is registered.
Collections also magically worked.

Please, review it for me, either if you won't include it to trunk.

All test passed and I have added a few more. Regards  

We need to create an @Indexed annotation. In the current incarnation this
is implied for all fields, but with @Embedded (+@Unindexed) it can be
disabled for a whole class, and object graph. Unfortunately that leaves us
no way to index a (set of, or single) property of the embedded object which
is marked with @Unindexed.

In addition, both of these annotations should be applicable for the type
and field. The annotation on the type (class-level) should be the default
for all fields in that class unless explicitly decorated.

appengine Link objects are not supported for serialization to GWT.  It would 
be great if you could add support for these.

In the Relationships section you say "Note that this is an inappropriate
use of the @Parent entity; if a car were to be sold to a new owner, you
would need to delete the Car and create a new one."  You could add
something explaining that the Google App Engine data store has the
restriction that it doesn't allow reparenting.  Or maybe add it further
below, where you tell about changing the @Parent key field doesn't do what
might be expected.

Near the end of the Transactions section: nontrasactional -> nontransactional

At the beginning you say "You can persist any of the core value types,
collections of the core value types".  You could add a remark saying that
this does not include Maps, for dunderheads like me who forget that Map
isn't a subinterface of Collection.

We should test and document the fact that Blobs and Texts go to the
end of list properties, no matter what their initial order (this only
impacts heterogeneous Lists).

http://groups.google.com/group/objectify-appengine/msg/eec47e21d631d13c

We should probably disable automatic String->Text conversions in
arrays/Collections

Running queries and seeing memory consumption go way up due to these 
classes not being GC'ed...

com.google.storage.onestore.v3.OnestoreEntity$PropertyValue 66,840,592
com.google.storage.onestore.v3.OnestoreEntity$Property          47,743,240

The number to the right is the heap size in memory.  Memory looks okay in 
com.googlecode.objectify, and in my package.  Scoping the app w/ JProbe.  

Perhaps it's due to DB connections not being closed, found this...

http://groups.google.com/group/google-appengine-
java/browse_thread/thread/afc98d9a853601d2

Wondering about the objectify connection session, and the lack of a close() 
call.  Perhaps the wiki could contain some info about this?  

Gonna need to get to the bottom of this soon, as my app can only run for a 
couple hours before running out of memory. 

Objectify v2.2. GAE v1.3.4.

In the Keys section you say "but for now what you need to know is that this
parent (which is often simply null)".  You could expand that parenthetical
remark and say "(which is often simply null, making it an unparented, root
entity)".

Could you publish the (time of writing) latest version (2.2.1) in your Maven 
repo ?

The last published version is 2.2 ...

Thanks in advance

What steps will reproduce the problem?
-----------------------------------------
1. Have embedded class structure similar to the intro wiki (i.e. 
EntityWithEmbedded, LevelOne, LevelTwo)

2. Modify LevelOne to be a generic class LevelOne<T>

3. Create interface class for LevelTwo (LevelTwoInterface). This is 
"extract interface" in eclipse Refactor menu.

4. Instead of LevelOne embed LevelTwo. Have it LevelOne<T> embed T 
class. This T class will eventually be set as LevelTwoInterface.

5. Create the classes hierarchy and store it. Storing is 
successful.

6. Retrieve EntityWithEmbedded and note that LevelTwo is null. 
EntityWithEmbedded and LevelOne is fine. 

Expected output? What do you see instead?
-----------------------------------------
1. Storing the embedded class hierarchy does not result to 
exception. If generic class can not be save, then at least I expect 
to see exception here telling me that I can not do this.

2. More importantly expected that generic class is embeddable and 
can be stored using objectify. This is not the case since when the 
entire hierarchy is retrieved LevelTwo object is return as null.

3. The wiki does not mention whether generic object can be stored 
or not. So this is a gray area.

What version of the product are you using? 
-----------------------------------------
Objectify 2.1
AppEngine 1.3.2 
GWT 2.0.3
Eclipse Build id: 20100218-1602

On what operating system?
-----------------------------------------
Ubuntu Karmic 

Please provide any additional information below.
---------------------------------------------------
Attached a sample eclipse+AppEngine+gwt workspace/project.

What steps will reproduce the problem?

- As in the introduction Querying section:

   Objectify ofy = ObjectifyService.begin();
   Car car = ofy.query(Car.class).filter("vin", "123456789").get();

- query as in ofy.query is missing or not defined 

What version of the product are you using? On what operating system?

   objectify 1.0.3 (latest) Appengine 1.3 GWT 2.0

What steps will reproduce the problem?
1.  Create serializable type MyType
2.  Use an OKey<MyType> as a return object in GWT-RPC
3.  Attempt to run  

What is the expected output? What do you see instead?
java.lang.Class<T> is not assignable to 
'com.google.gwt.user.client.rpc.IsSerializable' or 
'java.io.Serializable' nor does it have a custom field serializer 
(reached via 
com.googlecode.objectify.OKey<com.whatever.Myclass>)


What version of the product are you using? On what operating system?
GWT 2.0 on Ubuntu Linux

Please provide any additional information below.

Looks like the serialization of OKey is having trouble with it's Class<? 
extends T> field.

In lieu of using @Id/@Parent, entity pojos could be written with a Key<?>, 
Key, or String field annotated with @Key:

class Foo {
    @Key String key;
    ...
}

This would store a stringified version of the Key for the entity in the entity.

Queries would treat the key as a single component, just like the native 
datastore.

See issue 32.  You shouldn't be able to register a class with an @Embedded 
interface, only 
@Embedded concrete classes.

What steps will reproduce the problem?

1) Copy paste from the "batch" part of the Wiki on this site

Car porsche = new Car("2FAST", "red");
Car unimog = new Car("2SLOW", "green");
Car tesla = new Car("2NEW", "blue");
ofy.put(tesla, unimog, porsche);

2) Compile

What is the expected output? What do you see instead?

Expect to compile okay as written here in the Wiki: 
http://code.google.com/p/objectify-appengine/wiki/IntroductionToObjectify#Basic_
Operations:_Get,_Put,_Delete

But I get: "cannot find symbol method put(Car,Car,Car)"

What version of the product are you using? On what operating system?
Objectify 2.1 
Ubuntu 9.10

Please provide any additional information below.

What steps will reproduce the problem?
1. Export Javadoc from Eclipse
2.
3.

What is the expected output? What do you see instead?
The output on the console is:

cannot access com.googlecode.objectify.Objectify
bad class file: 
[path]/objectify-2.2/objectify-2.2.jar(com/googlecode/objectify/Objectify.class)
class file has wrong version 50.0, should be 49.0
Please remove or make sure it appears in the correct subdirectory of the 
classpath.
import com.googlecode.objectify.Objectify;


What version of the product are you using? On what operating system?
Mac OS X 10.5.8
Objectify 2.2


Please provide any additional information below.
Everything runs fine during dev/testing, but when I try to export the Javadoc 
it barfs up that 
error.

this makes unit testing life harder, especially given that you can get this 
non-interface from the Objectify core interface

It's was great to allow to convert a Objectify Key<?> to the DataStore Key
and reversibility for allow to be compatible with last code.

The name of the MemCache namespace which is declared in the 
CachingDatastoreService class is invalid from GAE SDK 1.3.4 onwards.  

This isn't a problem if you continue to use the deprecated setNamespace method 
on the MemcacheService object, but if you use the correct static method on the 
MemcacheServiceFactory to obtain a namespace-aware MemcacheService then it 
fails.

The error which is thrown is:

java.lang.IllegalArgumentException: Namespace 'Objectify Cache' does not match 
pattern '[0-9A-Za-z._-]{0,100}'.
    at com.google.appengine.api.NamespaceManager.validateNamespace(NamespaceManager.java:154)
    at com.google.appengine.api.memcache.MemcacheServiceImpl.<init>(MemcacheServiceImpl.java:163)
    at com.google.appengine.api.memcache.MemcacheServiceFactory.getMemcacheService(MemcacheServiceFactory.java:45)

From thread:
- @Expando
http://groups.google.com/group/objectify-appengine/msg/fa8ec94382ec11b2
- @RawEntity
http://groups.google.com/group/objectify-appengine/msg/5028988170f34269

What steps will reproduce the problem?
1. Register a class with no public no-arg constructor (this works)
2. Do a get() or query() using that entity
3. Get an exception like:
     [java] java.lang.RuntimeException: java.lang.IllegalAccessException:
Class com.googlecode.objectify.EntityMetadata can not access a member of
class com.foo.SomeEntity with modifiers "protected"
     [java]     at
com.googlecode.objectify.EntityMetadata.toObject(EntityMetadata.java:288)
     [java]     at
com.googlecode.objectify.OPreparedQueryImpl$ToObjectIterator.next(OPreparedQuery
Impl.java:166)
     [java]     at
com.googlecode.objectify.OPreparedQueryImpl.asList(OPreparedQueryImpl.java:79)
     [java]     at com.foo.model.DAO.getHistory(DAO.java:43)


What is the expected output? What do you see instead?

An exception during register(), not after register().

What version of the product are you using? On what operating system?

1.0.1

Please provide any additional information below.

I've attached a patch (against trunk@187) that throws an exception during
register() (and a test, too!)

What steps will reproduce the problem?
1.
2.
3.

What is the expected output? What do you see instead?


Please use labels and text to provide additional information.

What steps will reproduce the problem?
1. Create an ArrayList
2. Put n entities in the list
3. make n = 0
4. call put with the list as the parameter

What is the expected output? What do you see instead?
It is expected that no entites are stored and no exception is thrown, instead 
the datastore throws a NullPointerException:

java.lang.NullPointerException
    at com.google.appengine.api.datastore.dev.LocalDatastoreService.putImpl(LocalDatastoreService.java:491)
    at com.google.appengine.api.datastore.dev.LocalDatastoreService.put(LocalDatastoreService.java:440)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at com.google.appengine.tools.development.ApiProxyLocalImpl$AsyncApiCall.call(ApiProxyLocalImpl.java:305)
    at com.google.appengine.tools.development.ApiProxyLocalImpl$AsyncApiCall.call(ApiProxyLocalImpl.java:262)
    at java.util.concurrent.Executors$PrivilegedCallable$1.run(Executors.java:463)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.util.concurrent.Executors$PrivilegedCallable.call(Executors.java:460)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
    at java.util.concurrent.FutureTask.run(FutureTask.java:138)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:637)


What version of the product are you using? On what operating system?
Objectify 2.2.1 on the App Engine SDK 1.3.5

Please provide any additional information below.
The bug was found in a unit test of my program, using the datastore 
LocalServiceTestHelper. Don't know if that can be the problem.

Query<Trivial> q = ofy.query(Trivial.class).filter("someString =", "foo");
q.limit(20).cursor(cursor);
Trivial gotten = q2.get();
assert gotten.getId().equals(objectAfterCursor.getId());

Instead of calling Class.newInstance(), we should use reflection to
get a hold of the zero-arg constructor and use that to create new
instances. That way, we can call private zero-arg constructors. 

From thread here:
http://groups.google.com/group/objectify-appengine/msg/bf8d30ffa59d0f6e

"If another process modifies the your data before you commit, your
datastore operations will fail with a ConcurrentModificationException"

Spurious "the".

I created an entity that had an @Parent(so as to be part of a specific entity 
group) 
and was annotated with @Cached. This entity has a simple List member.
My test created the entity with O1 (O for Objectify).

The entity was read with O2 (O2 was a transactional Objectify) and an item was 
added to 
the List. Item was put and committed using O2.

The entity was read with O3 and the size of the List was printed. It remained 
zero.

I re-did the above using a non-trasnactional O2 and the size of the List when 
printed 
was 1.

I would have expected ObjectifyFactory's allocateIds(java.lang.Class<T> clazz, 
long num) to preallocate a contiguous range of unique ids within the namespace 
of the specified entity class.

It appears to allocated ids paying no attention to the namespace.

I have attached an updated AllocateTests for verification.

Key used to have a getKind(), but that was changed to
getKindClassName() for GWT serialization purposes. I'd like that try
and change that back, if I can work out how to still get it to work
with serialization. Using Kind instead of KindClassName seems more
orthogonal to the datastore Key's class. 

From http://groups.google.com/group/objectify-appengine/msg/bf8d30ffa59d0f6e

Before commit, needs to re-read the entity and check the counter has not been 
incremented.
As described here 
http://gae-java-persistence.blogspot.com/2009/10/optimistic-locking-with-
version.html