danielmewes / php-rql Goto Github PK

The current implementation of Queries is more verbose than necessary, as the conversion to protobufs is split between the constructor of the Query object and the getPBTerm() method. The latter is re-implemented in each query type.

While this might vaguely make sense if we later want to add for example __toString() methods to Queries, I think it mostly makes the code more error-prone and difficult to read.

We should provide a a default implementation for getPBTerm() either directly in Query or in a special subclass of Query. We would modify the constructor of that class to take the fields required to construct the protobuffer term, i.e. term type, args (array of Query), optArgs (associative array of Query) and datum (Datum object), where all but the first can be null.

You can create a secondary index without specifying an index function. Currently, we insert r\row($attribute) as the default index function if none was given by the user.

However it appears that we can simply omit the function when sending the request to the server, and the server will take care of selecting the default: rethinkdb/rethinkdb#855

I don't know if this makes any practical difference, but we should probably make use of the server-side default instead of specifying a client-side default, as this is what we do everywhere else.

While PHP-RQL 1.6 is fully compatible with RethinkDB 1.7, it doesn't support all of 1.7s new features. Update PHP-RQL.

We also have to update the API docs.

The following code seems to fail with:

Runtime error: Expected type OBJECT but found NULL.

r\table($table)->get($id)->replace( 
        function($doc) use ($id,$request) {
            return $doc->rDefault(array('id' => $id))->merge(array('test' => $request));
        }
    )->run($conn);

Judging from the stack trace below, it seems to be because of the check in the return function. I'm unsure on how to debug this further.

#0 /lib/php/rdb/connection.php(157): r\Connection->checkResponse(Object(r\pb\Response), 418578082, Object(r\Replace))
#1 /lib/php/rdb/connection.php(101): r\Connection->receiveResponse(418578082, Object(r\Replace))
#2 /lib/php/rdb/misc.php(35): r\Connection->_run(Object(r\Replace), NULL)
#3 /lib/api/rdb.api.php(112): r\Query->run(Object(r\Connection))
#4 [internal function]: appendToDB('test', '571e5bbb-dfee-4...')

Any ideas how to make this work? I'm stuck until I can find a work around for this... I would note that both $id and $replace are in the scope of the function and that replacing them with static values throws the same error...

We could adapt pb4php further to expose the same internal interface as the PHP-Protobuf compiler. That way, people could easily install PHP-RQL with pb4php for development purposes, without having to compile and install any binary extensions.
For production systems and where performance matters, they could then simply install the PHP-Protobuf extension and PHP-RQL would start using that one.

See the related issue rethinkdb/rethinkdb#1568

Right now PHP-RQL closes a cursor on the server when the Cursor object is destructed. This is probably fine in most use cases, but a bit implicit.

We should add a close() method the cursor objects, so you can close them explicitly without having to know about details of the PHP garbage collector.

There is currently no good way of finding out which version of PHP-RQL is installed. There should be.

The documentation is currently lacking behind. It does not mention getMultiple(), pattern-matching pluck(), r\json(), and the fact that ->attr() is deprecated in favor of getField()

We used to put all the protobuf code into a namespace r\pb. I fixed the compiles protobuf definition file manually to use this namespace. However we should adapt the compiler to generate the correct code automatically, to simplify future updates.

Adding the new ->profile method (added in 1.11) to the API docs fell through the cracks.
Fix that.

There is this new implementation of protobufs for PHP: https://github.com/allegro/php-protobuf

Currently, the encoding and decoding of protobufs is a significant bottleneck for the performance of PHP-RQL. PHP-protobuf seems to use a compiled extension, which could potentially make it much faster than our current adapted variant of pb4php.

#5 caused some significant changed to the code base.
We have to check everything for possible regressions. Ideally, this will happen in the form of adding loads of automated test cases to our test framework (compare #14 ).

If an error occurs in a query, the RethinkDB server sends us a backtrace. Currently we make no use of it.

I am not sure i am using this correctly, however when i run the command in the javascript console it appears to work the way i expect it to.

The issue is this: when attempting to filter on an array say something like: \r\row('years')->contains(2001) all rows get returned.

Reproduction of error and expected result:
** data **

{"name":"Tundra", "years":[2007,2008,2009]}

{"name":"prius", "years":[2007,2008,2009]}

Javascript from the web console (returns expected values)

//correct
r.db('cars').table('toyota').filter(r.row('years').contains(2009))
returns = [{"years":[2007,2008,2009],"name":"prius","id":"47af6cee-af03-4396-a95f-a00a2b365937"},{"years":[2007,2008,2009],"name":"Tundra","id":"8ba0ac80-673d-4846-a559-25fc9d600c17"}]

//correct
r.db('cars').table('toyota').filter(r.row('years').contains(2001))
returns = [];

php:

<?php
$this->rethink->useDb('cars');
$data = \r\table('toyota')->filter(array(\r\row('years')->contains(2008)))->run($this->rethink)->toNative();
print_r($data);
$data = \r\table('toyota')->filter(array(\r\row('years')->contains(2001)))->run($this->rethink)->toNative();
print_r($data);

/* output:
Array
(
    [0] => Array
        (
            [years] => Array
                (
                    [0] => 2007
                    [1] => 2008
                    [2] => 2009
                )

            [name] => prius
            [id] => 47af6cee-af03-4396-a95f-a00a2b365937
        )

    [1] => Array
        (
            [years] => Array
                (
                    [0] => 2007
                    [1] => 2008
                    [2] => 2009
                )

            [name] => Tundra
            [id] => 8ba0ac80-673d-4846-a559-25fc9d600c17
        )

)
Array
(
    [0] => Array
        (
            [years] => Array
                (
                    [0] => 2007
                    [1] => 2008
                    [2] => 2009
                )

            [name] => prius
            [id] => 47af6cee-af03-4396-a95f-a00a2b365937
        )

    [1] => Array
        (
            [years] => Array
                (
                    [0] => 2007
                    [1] => 2008
                    [2] => 2009
                )

            [name] => Tundra
            [id] => 8ba0ac80-673d-4846-a559-25fc9d600c17
        )

)
*/

RethinkDB 1.5 will introduce a few extensions to the query language:

1. shortcuts r\tableCreate() r\tableDrop() and r\tableList()
   rethinkdb/rethinkdb#46
2. JavaScript timeouts r\js(code, timeout)
3. table->getAll(key, indexName) to query secondary indexes
   rethinkdb/rethinkdb#602
4. optional secondary index parameters to table->between() and table->eqJoin()
5. Secondary index management commands table->indexCreate(), table->indexDrop() and table->indexList()
6. A query->info() command
   rethinkdb/rethinkdb#469
7. UPDATE A noreply flag to run()

There might be more changes that I've missed.

These changes are implemented for PHP-RQL in the 1.5 branch. However two things are left to do:

1. The new commands must be tested
2. The API documentation must be updated. RethinkDB seems to be working on overhauling some of their documentation, and I will try to merge back their changes into the PHP docs as soon as they are done.

When using multiple options (e.g. both primary_key and cache_size) with tableCreate, the value of the first option specified overwrites that of all remaining options.

Similar bugs existed with rDo when providing multiple arguments, groupBy when providing multiple attributes, orderBy when providing multiple attributes, and contains, pluck and without when providing multiple attributes.

This bug is already fixed in 084880e

Hi!

I have issue when i try to fetch data using :
$i = array(
'index' => 'userid'
);

            $o = r\table('smsq')->getAll($userid, $i)->run($this->conn);
            $arr = $o->toNative();

$arr contains empty array.

same thing when i use filter instead of getAll()

When running a groupedMapReduce in javascript there is lexicon variable scoping so the variables above are aviable so i can then have an inner map to use as a loop.

doc("rawr").keys().map(function(doc2) {
return [doc2, doc("rawr")(doc2)]
}).corecTo("object")

When i try it in php the doc var is NULL

Reading an empty string from the database triggers a driver error.
I already have a fix, and adding it here just for the record.

The problem is, that php4bb returns false when it tries to read a string of length 0. StringDatum then fails because false is not a string.

See https://groups.google.com/forum/#!topic/rethinkdb/d4WqBxNDaXQ
and http://php.net/manual/en/mbstring.overload.php

I don't think there is much we can do. This overloading just completely changes the semantics of the PHP string functions which we rely on for processing binary data.

What we probably should do is to check whether MB_OVERLOAD_STRING is set and to fail somewhat more gracefully with an error message

From #rethinkdb on IRC:
18:36 <hazardous> is there a way to set some kind of higher timeout ? i'm using php-rql and keep getitng this after a mintue 18:36 <hazardous> PHP Fatal error: Uncaught RQL Driver Error: 18:36 <hazardous> Unable to read from socket. Disconnected.

We need an option to change the timeout of the connection.

I already had this implemented once, but didn't push it because I couldn't verify that timeouts were actually working on PHP 5.3.2 (in fact I never get a timeout there). But that's probably just a bug of some sort in that specific version of PHP.

PHP doesn't like is_subclass_of() being called on non-object variables. Depending on your warning/error reporting level, it will print a warning if this happens.

Some calls to is_subclass_of() are prefixed by an @ to suppress this warning, but that is certainly not the right way to go (and it is currently inconsistently implemented). We should replace in the code as follows:
is_subclass_of($x, T) -> (is_object($x) && is_subclass_of($x, T))

Hi,

I'm using this query to get timesheet events that have occurred on a particular day.

$timesheet = r\table("timesheet_day")->filter(function($timesheet) {
    return $timesheet("arrive")->date()->eq(r\time(2013, 6, 20, 'Z'));
})->run($app->rethink)->toNative();

This works but is obviously static, so to test I defined a $year variable that I want to use, like this:

$year = 2013;
$timesheet = r\table("timesheet_day")->filter(function($timesheet) {
    return $timesheet("arrive")->date()->eq(r\time($year, 6, 20, 'Z'));
})->run($app->rethink)->toNative();

but i'm unsure how to pass the variable to the inline function, since it won't work like that. Is there some other way I need to do this?

I tried using rethinkdb data explorer and it works in JS without needing to pass the variable so I assume its a PHP limitation.

Felix.

I shut down the server while a PHP-based client was running.

I got a seemingly infinite sequence of the message

PHP Notice:  fwrite(): send of 8192 bytes failed with errno=32 Broken pipe in /home/ssd3/daniel/php-rql/php-rql/src/rdb/connection.php on line 277

but no exception was thrown. Somehow PHP-RQL did not seem to detect the failed write.

Compare https://github.com/rethinkdb/rethinkdb/releases

In Connection, we generate random query tokens. However we don't check if they collide with other currently active queries.
To change this, we first have to keep track of active queries. Then generate new tokens until we have found one that doesn't collide.

The API documentation suggests that it does (http://php-rql.dnsalias.net/api/#ph:transformations-order_by), but it actually does not use indexes.

Including #42, #35 and #40

We have some internal methods, that yet are public in the class definitions. For example the continue() / stop() methods in Connection should never be called directly (they are called internally by cursors).

We should either protect those through means of PHP language features, or make it clear which methods are considered to be callable by the user and which are not. For example we might prefix the names of the latter by underscores.

We need a set of automated tests that can be run against an actual server, and which check the values of the returned data.

I've done some performance optimizations in the branch performance2, but I wouldn't like to include them in the next release without making sure that they don't break anything.

I get shitloads of errors with protobuf installed:

Strict Standards: Declaration of r\pb\Datum_AssocPair::clear() should be compatible with ProtobufMessage::clear($position) in /Users/ianbytchek/Development/playground.local/deps/vendor/danielmewes/php-rql/src/rdb/php-protobuf_pb_proto_ql2.php on line 197

Fatal error: Class r\pb\Datum_AssocPair contains 1 abstract method and must therefore be declared abstract or implement the remaining methods (ProtobufMessage::reset) in /Users/ianbytchek/Development/playground.local/deps/vendor/danielmewes/php-rql/src/rdb/php-protobuf_pb_proto_ql2.php on line 197

After fixing the signature and adding the missing method I get:

Fatal error:  Call to undefined method r\pb\Term::setValue() in /Users/ianbytchek/Development/playground.local/deps/vendor/danielmewes/php-rql/src/rdb/php-protobuf_pb_proto_ql2.php on line 1068

What I did

I tried something like:

r\table('posts')->filter(r\rAnd(
    r\row('author')->eq('Michel'),
    r\row('category')->eq('Geek')
));

What I expected

I expected to get the posts filtered by those two conditions.

What happened

Error stating that the r\rAnd function doesn't exist: Fatal error: Call to undefined function r\rAnd()

Possible solution

Refactor the methods out of ValuedQuery so that they become functions in the r namespace. The methods could still exist if you'd call the new functions there.

Background information

• http://www.rethinkdb.com/docs/cookbook/javascript/#filtering-based-on-multiple-fields
• http://en.wikipedia.org/wiki/Prefix_notation

Hey,

First of all - love it! Been waiting for PHP driver since I've first learnt about RethinkDB. Would have tried to do something myself, but lack of time or knowledge were good barriers.. ;)

Anyway, to the point.

It would be really nice if you registered this driver as a package in Composer to spread the usage and make it so much easier to install / update the driver.

The API doc at http://dmewes.com/~daniel/php-rql-api/#ph
has the following problems:

• Parent and member method names are separated by the page scripts using a dot . , while for PHP it should be ->
• The namespace r is presented as if it was an object, which it is not in PHP-RQL
• Some of the argument lists are not currently overridden for PHP in the YAML files. For example pluck in PHP takes an array, but it is listed in the docs using a variable number of arguments.

Also, we are missing a general overview documentation.
For example we should outline somewhere how the r\Datum type works, that we can always convert to Datum explicitly and give it to the API calls (for example if you have a fully associative array, but you want it to be an ArrayDatum, not an ObjectDatum).
We should explain how native types get converted to r\Datum.
We should explain how to retrieve values out of Datum objects.
It should be explained what the limits of using function(...) {...} as a query argument are (e.g. only use RQL operations, you might have to call r\expr before returning etc.)

Some of our tests fail with RethinkDB 1.8. I believe the format that groupBy results are provided in by the server has changes.

If i try to fire off a large patch write lets say 20k objects it writes them to the database creatly but something breaks on getting the data back. I am not sure if its just getting the response or when i try to turn run toNative on it just locks up for a few minutes i end up cancling the script.

We have to document how to install everything.

toNative() should invert the effect of nativeToDatum(). This would make it easier to work with the results returned by queries.

Something like

array($doc('_source')('cpu')('loadAvg')->nth(0)->rDefault(0))

Retruned an error like this

PHP Parse error: syntax error, unexpected '(' expecting ')' in file /path/to

The work around was

$source = $doc('_source')
$cpu = $source('cpu')
$cpu('loadAvg')->nth(0)->rDefault(0)

Suppose we create an empty table named "foo" and run the following program:

<?
require 'rdb/rdb.php';

$conn = r\connect('localhost');

$x = 1387366123.83013;
$y = 1387366123.8301;
$doc = array("name" => "Barmuel", "email" => "[email protected]", "age" => $x);

$result = r\table('foo')->insert($doc)->run($conn);
echo $result;

$cur = r\table('foo')->filter(array("age" => $x))->run($conn);
echo "\n filter with inserted value returns\n";
foreach ($cur as $doc) print_r($doc->toNative());

echo "\n filter with the value 'y' returns \n";
$cur = r\table('foo')->filter(array("age" => $y))->run($conn);
foreach ($cur as $doc) print_r($doc->toNative());

echo "\n fin\n";

?>

Here's the output we get:

array('deleted' => 0, 'errors' => 0, 'generated_keys' => array('da87bd43-770a-4e22-b5d5-95cf49d77f41'), 'inserted' => 1, 'replaced' => 0, 'skipped' => 0, 'unchanged' => 0)
 filter with inserted value returns

 filter with the value 'y' returns 
Array
(
    [age] => 1387366123.8301
    [email] => [email protected]
    [id] => da87bd43-770a-4e22-b5d5-95cf49d77f41
    [name] => Barmuel
)

 fin

Suppose we then insert another document in the Data Explorer:

r.table('foo').insert({'name': 'Farmuel', 'email': '[email protected]', 'age': 1387366123.83013})

The program then produces this output:

array('deleted' => 0, 'errors' => 0, 'generated_keys' => array('d8f4961f-d49d-43f1-b5c7-4f5c1a15d231'), 'inserted' => 1, 'replaced' => 0, 'skipped' => 0, 'unchanged' => 0)
 filter with inserted value returns
Array
(
    [age] => 1387366123.8301
    [email] => [email protected]
    [id] => bec73d73-df52-4256-ba54-a693c748e0da
    [name] => Farmuel
)

 filter with the value 'y' returns 
Array
(
    [age] => 1387366123.8301
    [email] => [email protected]
    [id] => d8f4961f-d49d-43f1-b5c7-4f5c1a15d231
    [name] => Barmuel
)
Array
(
    [age] => 1387366123.8301
    [email] => [email protected]
    [id] => da87bd43-770a-4e22-b5d5-95cf49d77f41
    [name] => Barmuel
)

 fin

The filter then seems to work. It is clear that when we run an insert with the document having value $x, the server instead receives a document with value $y. However, we are capable of constructing a filter query that uses value $x.

I believe this is because we are converting the document to JSON and sending that to the server. I believe the conversion to JSON is improperly converting floating point values to strings in a way that discards information.

This bug was originally reported by @wojons.

The following does not work:

r\expr(array(
        new r\ObjectDatum(array())
    ));

...but it should.

This is obviously an oversight on my end. Nesting ReQL quries works for other types of r\Query, just not for r\Datum.

We should demonstrate that PHP with RethinkDB is the way to go if you want to run a scalable web application.

The official RethinkDB site has a few example applications for the official drivers here:
http://www.rethinkdb.com/docs/

However these are small applications. Nothing which would really demonstrate the power of RethinkDB, and nothing which would be motivation for an administrator to install RethinkDB just for the sake of these apps.

If we could port something like Wordpress or one of the major CMS to PHP-RQL, that would be pretty cool I think. People who are already running the software and want to increase scalability, durability and/or availability by means of clustering would have a hard time not to consider RethinkDB then. If it was just supported.

I have taken a quick look at the following so far:
Wordpress, Drupal, phpBB
However all of them seem to be very SQL-centric and porting would be a lot of work. There is a port of Drupal to use MongoDB https://drupal.org/project/mongodb which might give some ideas on what has to be done.

Does anyone have an idea for an application which would be relatively easy to port and is reasonably popular?
I am thinking of something that has a high-level DB abstraction layer, so we would only have to touch a relatively small piece of the code.

We have to add support for the options argument to indexCreate.

...for now: Using the examples from the API docs.

PHP-protobuf uses a binary PHP extension. We have to figure out how to install that with Composer.

We should have introductions, references and tutorials at
http://php-rql.dnsalias.net/wiki/