teamtnt / tntsearch Goto Github PK

For Example:

Indexing like this:
$indexer = TNTSearch::createIndex('book.index');
$indexer->query('SELECT id, title, category_id FROM books');
$indexer->run();

How I can search for title in specific category_id for example?

I would like to have a way to use another stemmer class like the one:
https://github.com/wamania/php-stemmer

It's possible?

From what I saw is fixed in the code the use of Stemmers:
$class = 'TeamTNT\TNTSearch\Stemmer\' . ucfirst(strtolower($language)) . 'Stemmer';

ps:sorry about my English

Thank you

Firstly, thank you for a great plugin.

It's very helpful.

I have everything set up but having one issue with search.

I have 2 emails in the database.

[email protected]
[email protected]

If I search for mandy, I only get back [email protected].

Why doesn't mandy2 appear ?

I have fuzziness set to true.
I also have searchBoolean.

Thanks.

Hi,

In case of multi server environment, the search can originate form any server ( www1, 2, 3..10 ).

It's not possible to have an index on every box.

How do we solve the issue of having storage_path on an external ( virtual ) server accessible to all ?

Thanks.

Hi,

I have found an issue when using the search/searchBoolean method where the phrase being searched was returning no ids despite the phrase being present in my db.

To replicate, please insert the following rows into the articles db with tests:

id | title  | article
11 | Eldred | What the foo
12 | Eldred | Oh my bar

Then add these assertions and run phpunit.

$res = $tnt->searchBoolean('eldred -bar');
$this->assertEquals([11], $res['ids']);

$res = $tnt->searchBoolean('Eldred -bar');
$this->assertEquals([11], $res['ids']);

The second assertion will fail. It appears linked to the phrase being used in addition to that phrase starting with a capital letter. I have managed to get around this issue by using strtolower.

Hey @nticaric, thanks for sharing this great package.

I have a big WordPress website that I want to improve the search of. How would I go about to integrate TNTSearch with the current website? Do I need a new DB for it or it would work fine with the current website?

Any direction on this would help. Thanks very much.

I LOVE this package. Thanks very much for putting it together.

I am using the Laravel driver and search multiple models in turn. I would like to then rank the results from all models by a score.

I presume from my digging around that this currently isn't possible? Or am I wrong?

One again, thanks for this. So simple and affective.

Impressive, good work.

Hi
thank you for this great package ...
I tested it for a sample table with 1 million row , this is the link for download the sample database :
http://www.sample-videos.com/sql/Sample-SQL-File-1000000rows.sql

I ran this query and it returned about 40000 rows but the package returned just 500 rows .
query : select * from user_details where first_name = 'david'

and this is my code :
$tnt->loadConfig($config);
$tnt->selectIndex("Full");
$res = $tnt->search('david', 1000000);
var_dump($res);

what is the problem ?

Could you store the indexes on Amazon S3? So multiple servers could access the same data.

I pretend to have something like this

    $whereRaw = "MATCH(name) AGAINST('+jame* +walk*' IN BOOLEAN MODE)";

As I have now against MySQL with the following results:

    James Walker
    Jamenie Walk
    Jameelah Walka
    Jamelle Walkama
    Jamei Walkde

Can TNTSearch do this?

Hi there,

How do I tell TNTSearch to ignore accents in its indexing and searching process?

María becomes maria

Thank you.

Hello!

I want when search keyword "[email protected]" only return result of "[email protected]" . Currently, It return results with "@example.com". How to configuration it.

Hi, I'm trying to follow the steps at the readme but there is no method run() or query() at TNTSearch(); Please fulfill the requirements to build the index. Thanks!

IMHO, it'd be better to leave this package as simply framework agnostic, and not include the Laravel service provider, and create a separate package like teamtnt/tntsearch-laravel which would require this package and additionally require illuminate/support (or whatever required Laravel stuff) with the appropriate version constraints, etc.

Use of undefined constant SQLITE3_INTEGER - assumed 'SQLITE3_INTEGER'

I use a php framework called Fat Free Framework and I intend to create an indexer for its own database storage called Jig. It is simple an active cursor implementation which stores data as JSON structures.

I stumbled across TNTSearch but found the documentation is lacking greatly. I've glossed over the code but I suspect it may take a few weeks to fully understand how to work with the code base.

Might anyone give me any hints on how I may tackle this project?

Hello! As I mentioned in one of my email lately, I'm very happy to use the package for the performance it gives my app and such but I've been having a couple of issues with it. I started implementing a search feature in Laravel app on a Mac. It worked fine, even with a few bugs, but it worked. I then switched to using a PC with Ubuntu 16.04 and since then the package stopped working. At first it said "PHP driver not found". I then installed php7.0-sqlite3. I have sqlite3 installed on the machine so trying to open the index files created from artisan works fine. I can do a "select * from wordlist" for example and I got some results. But it doesn't show anything in the application. I got empty results whenever I try.

What am I doing wrong?

• Regards for your awesome work

I'm having some issues with the ranking. Is there any way to tell the driver to prioritize results that contain a match in the title rather than in the body? Right now it seems both columns have equal weight and the items I'm looking for almost never show at the top.

Thanks!

Hello

IndexNotFoundException in TNTSearch.php line 65:
Index ./storage/users.index does not exist

I'm sure I have users.index file in storage directory. Am I missing something?

hello
i have tried to index arabic words, but the search not works
is there any configuration for utf-8 coding?

When I try to use the search method in multiple calls it does not return result after adding the value to the index.

For example:

public function test()
    {
        $this->loadConfig();

        if (!file_exists(storage_path('/app/events.index'))) {
            $this->tnt->createIndex('events.index');
        }

        $this->tnt->selectIndex('events.index');

        $str = 'test';

        $this->tnt->search($str);  // returns no results as expected

        $index = $this->tnt->getIndex();
        $index->insert(['id' => 1, 'name' => $str]);

        $this->tnt->search($str);  // should return 'test' as a result from the index but returns 'ids' array as null
    }

Hi,

I'm using TNTSearch with Scout.

I'm having a little issue with this:

$ php artisan scout:import App\\Models\\CarModel

sqlite> SELECT * FROM info WHERE key = 'total_documents';
total_documents|0

Instead

$ php artisan tntsearch:import App\\Models\\CarModel

sqlite> SELECT * FROM info WHERE key = 'total_documents';
total_documents|4496

I think the problem is with TNTIndexer's delete method, executed by the update method.

delete method doesn't check if the document exists before decrement total_documents.

    public function delete($documentId)
    {
        $rows = $this->prepareAndExecuteStatement("SELECT * FROM doclist WHERE doc_id = :documentId;", [
            ['key' => ':documentId', 'value' => $documentId]
        ])->fetchAll(PDO::FETCH_ASSOC);

        ...

        $total = $this->totalDocumentsInCollection() - 1;
        $this->updateInfoTable('total_documents', $total);
    }

tinker:

sqlite> SELECT * FROM info WHERE key = 'total_documents';
total_documents|4490

$ php artisan tinker

>>> use TeamTNT\TNTSearch\TNTSearch;
>>> $tnt = new TNTSearch;
>>> $tnt->selectIndex(storage_path("car_models.index"));
>>> $index = $tnt->getIndex();
>>> $index->delete(30557); <-- this document exists

sqlite> SELECT * FROM info WHERE key = 'total_documents';
total_documents|4489

>>> $index->delete(305512397); <-- this document doesn't exists

sqlite> SELECT * FROM info WHERE key = 'total_documents';
total_documents|4488

Possible fix

    public function delete($documentId)
    {
        ...

        if (sizeof($rows) > 0) {
            $total = $this->totalDocumentsInCollection() - 1;
            $this->updateInfoTable('total_documents', $total);
        }
    }

Thanks

Hey,

What are the fields table and hitlist table for in the database, they remain empty.

Also is there a way to save a title in the database so the search results have a title.

And third and last, my info table is filled with 'TeamTNT\TNTSearch\Stemmer\PorterStemmer' what is this for?

How to get the numbers of entry in spesefic index ?

Problem 1
- Installation request for teamtnt/tntsearch ^0.8.9 -> satisfiable by teamtnt/tntsearch[v0.8.9].
- teamtnt/tntsearch v0.8.9 requires ext-pdo_sqlite * -> the requested PHP extension pdo_sqlite is missing from your system.

To enable extensions, verify that they are enabled in those .ini files:
- /etc/php/7.0/cli/php.ini
- /etc/php/7.0/cli/conf.d/10-mysqlnd.ini
- /etc/php/7.0/cli/conf.d/10-opcache.ini
- /etc/php/7.0/cli/conf.d/10-pdo.ini
- /etc/php/7.0/cli/conf.d/15-xml.ini
- /etc/php/7.0/cli/conf.d/20-calendar.ini
- /etc/php/7.0/cli/conf.d/20-ctype.ini
- /etc/php/7.0/cli/conf.d/20-curl.ini
- /etc/php/7.0/cli/conf.d/20-dom.ini
- /etc/php/7.0/cli/conf.d/20-exif.ini
- /etc/php/7.0/cli/conf.d/20-fileinfo.ini
- /etc/php/7.0/cli/conf.d/20-ftp.ini
- /etc/php/7.0/cli/conf.d/20-gettext.ini
- /etc/php/7.0/cli/conf.d/20-iconv.ini
- /etc/php/7.0/cli/conf.d/20-json.ini
- /etc/php/7.0/cli/conf.d/20-mbstring.ini
- /etc/php/7.0/cli/conf.d/20-mysqli.ini
- /etc/php/7.0/cli/conf.d/20-pdo_mysql.ini
- /etc/php/7.0/cli/conf.d/20-phar.ini
- /etc/php/7.0/cli/conf.d/20-posix.ini
- /etc/php/7.0/cli/conf.d/20-readline.ini
- /etc/php/7.0/cli/conf.d/20-shmop.ini
- /etc/php/7.0/cli/conf.d/20-simplexml.ini
- /etc/php/7.0/cli/conf.d/20-sockets.ini
- /etc/php/7.0/cli/conf.d/20-sysvmsg.ini
- /etc/php/7.0/cli/conf.d/20-sysvsem.ini
- /etc/php/7.0/cli/conf.d/20-sysvshm.ini
- /etc/php/7.0/cli/conf.d/20-tokenizer.ini
- /etc/php/7.0/cli/conf.d/20-wddx.ini
- /etc/php/7.0/cli/conf.d/20-xmlreader.ini
- /etc/php/7.0/cli/conf.d/20-xmlwriter.ini
- /etc/php/7.0/cli/conf.d/20-xsl.ini
You can also run php --ini inside terminal to see which files are used by PHP in CLI mode.

I get this error when I try to install....
I checked my phpinfo and pdo_sqlite is enabled.

please I need help. Thank you

After build in your search into my user search I found a bug.

When searching for something and the unique identifier is 0
the algorythmus finds the data but the filter

https://github.com/teamtnt/tntsearch/blob/master/src/TNTSearch.php#L109

filters out the correct result. *Because key is 0 I guess.

Do you have any idea?

Work arround for me is remove the filter lines between L109 and L117.
Maybe the Collection map/filter does something wrong there?

Do you plan on adding Fuzzy Searching to this awesome project?

Hello! Thanks for your package. It's really great.

I have it setup in a Laravel 5.2 application. The main issue I have is with the Ajax search. Doing search with one letter is necessary for my app, just like Google. After setting the asYouType property, which by the way needs the initialization of the class otherwise it won't accept it, it returns results for one letter but the wrong way.

Say I have many posts that have the word yeah in their title, when I type 'y' it will just show on post. But 'yeah' returns all the posts. That's a big issue for me.

Please see what you can do for that. And again good job!

• Regards!

The old version has folders in small case, in the src which is causing errors.

The updated version if this package fixed that, but it isn't the one on Packagist.

When will the updated version be available?

Thanks.

Hello, i have a question regarding tntsearch and german umlauts on laravel 5.3. As i see the german umlauts i.e. ä,ö,ü,ß are not indexed. In the search form, i get no result for this. Do you have any suggestions how to config that? Thank you
regards Klaus

I'm having some ranking issues. When I use multiple words in the search string, the result containing all of the terms is not at the top. When using only one word the results are ok. How does searching with multiple terms work?

If I copy-paste the title of the article in the search box I would expect it to be the top result but it seems that it's always outranked by all articles containing only one of the words.

I'm having issues getting the index to automatically update. I followed the instructions outlined here:

http://tnt.studio/blog/searching-for-bobby-fisher-with-laravel-5

I added all 4 of the static functions to my model (I'm assuming the boot method is in the same model?). When I insert a new record into my database (using the DB facade), nothing seems to get indexed. Search never finds these new records, unless I manually run: php artisan .index.

At this point, I'm not sure if it's a bug…or if I added the boot method to the wrong model…or if it's something else. Any help would be much appreciated!

I'm very happy with this package!

If I understand correctly I need to re index after inserts\updated are made?

do I need to run this code on all the results or only on the updated results like

    $indexer = $tnt->createIndex('name.index');
    $indexer->query('SELECT id, article FROM articles where id = $last_inserted_article;');
    $indexer->run();

Testing

Hi there, first of all, thank's for this awesome project.

Is there a way to search for dash keywords, like this: 70-200. My app have a lot products like Canon 70-200, and it's not searching correctly.

When I run php artisan index:users, the command throws this error.

However, a file is still generated.

When I try and use TNTSearch::selectIndex("users.index");
I get a "could not find driver" error.

Could you please help me out?

1. whether supports Chinese search?
2. can i index some text which are not in database, but all text have ID.

thanks for your answer.

I have table where ID is uuid type (string). This library doesn't work when ID type is string. Can you make it?

Is there a way to search for a partial of a word not for the full word? For example to return "Apple" if I search for "App" or "ppl"?

First, thanks a lot for creating this package! :)

I just noticed a small issue - Laravel already has a dd function defined, but there is also one defined in this package and it takes precedence over Laravel's implementation. I would like to continue using Laravel's dd, since it uses the Symfony var_dumper which results in a much more readable output.

Is there any chance that the dd function supplied with this package could be renamed since it's not really part of any core functionality?

Is this a typo? or am I confused?
Shouldn't TeamTNT\TNTSearch\Support\Hihglighter be TeamTNT\TNTSearch\Support\Highlighter

Helllo,

I want to use this package in a Bengali website. Works fine with English characters. But doesn't index utf8 bin characters.

Is there anyway I can achieve it?

Regards

How would you go about indexing and searching relationships from like a eloquent query?

It finds results when searched by ID of indexed record.
How to avoid that?

Here is how I created the index:

$indexer = $this->tnt->createIndex('artists.index');
$indexer->query('SELECT id, name FROM artists');
$indexer->run();

Retrieving search results:

$this->tnt->selectIndex("artists.index");
$results = $this->tnt->search($keyword, 5);
$artists = Artist::whereIn('id', $results['ids'])->get();

I've been struggling to get this installed and working. Every time I work through an issue, a new one pops up. Here is my latest error:

[PDOException]
SQLSTATE[08006] [7] invalid connection option "localhostdbname"

TNTSearchServiceProvider.php: I'm using the defaults (except for the driver).

public function register()
    {
        $this->app['tntsearch'] = $this->app->share(function ($app) {
            $config = [
                'driver'   => 'pgsql',
                'host'     => env('DB_HOST', 'localhost'),
                'database' => env('DB_DATABASE'),
                'username' => env('DB_USERNAME'),
                'password' => env('DB_PASSWORD'),
                'storage'  => storage_path(),
            ];

            if (isset($app['config']['services']['tntsearch'])) {
                $config = $app['config']['services']['tntsearch'];
            }
            $tnt = new TNTSearch;
            $tnt->loadConfig($config);
            #dd($tnt);
            return $tnt;
        });
    }

When I dd($tnt) and run php artisan index:users, it displays the correct settings for my postgresql database.

I'm not sure why it's throwing this error. I've checked everywhere in the documentation, and don't see anywhere that indicates postgresql isn't supported.

Appreciate any help!

Hi, TNTSearch was working fine using the Facade, but I wanted to use fuzziness, so I tried the method in the docs:

use TeamTNT\TNTSearch\TNTSearch;

$tnt_search = new TNTSearch;

$tnt_search->selectIndex( 'notes.index' );
$tnt_search->fuzziness = true;

$search_results = $tnt_search->searchBoolean( $search, 1000 );

But I get an error about some info table not existing..?

at PDO->query('SELECT * FROM info WHERE key = 'stemmer'') in TNTSearch.php line 297

First I will like to say this is an amazing search extension! However, I having issues getting this to work on my digitalocean cloud server with laravel forge. The extension works perfectly on my local server (Homestead / vagrant) but onces I upload everything to the cloud, it shows no result.

here is var_dump of my local server Homestead:

And here is var_dump of my digitalocean server:

Here is a var_dump of my code section:

Somehow it seems not pass in the results from $tnt on my cloud side.. Please let me know if I'm doing anything wrong here. My guess is that this is a server configuration issue since it's working fine on my local server.

Hello,

I have a list of ~10k airports with additional data in a table. For example, I have a dynamic search input and want to select an airport in Berlin, if I request "b", I get one result "Vernon B. C." (a city in the USA), "be" gets me only "Nossi-Be" (Madagascar), "ber"/"berl"/"berli" — no results, and only full city name "berlin" finds and returns 6 Berlin airports. Same goes for every other query. I saw TNTSearch working very differently on the demo page, any ideas on what I might be doing wrong?

I'm indexing city names like so:

        $tnt = new TNTSearch;

        // I get "SQLSTATE[HY000] [14] unable to open database file"
        // error unless I load a configuration.
        $tnt->loadConfig([
            'driver'    => config('tntsearch.driver'),
            'host'      => config('tntsearch.host'),
            'database'  => config('tntsearch.database'),
            'username'  => config('tntsearch.username'),
            'password'  => config('tntsearch.password'),
            'storage'   => config('tntsearch.storage')
        ]);

        $indexer = $tnt->createIndex('airports.index');
        $indexer->query('SELECT id, city_name_ru, city_name_en FROM airports;');
        $indexer->run();

And here's my controller:

    public function searchCityWithAnAirport(Request $request)
    {
        $tnt = new TNTSearch;

        $tnt->loadConfig([
            'driver'    => config('tntsearch.driver'),
            'host'      => config('tntsearch.host'),
            'database'  => config('tntsearch.database'),
            'username'  => config('tntsearch.username'),
            'password'  => config('tntsearch.password'),
            'storage'   => config('tntsearch.storage')
        ]);

        $tnt->selectIndex('airports.index');
        $tnt->asYouType = true;
        $result = $tnt->searchBoolean($request->input('query'), 10);

        return $result;
    }

In case you'll have time to test it yourself, here's a Laravel migration file, and here's a CSV file with data.

How would I go about using the facade with delete? I'm getting undefined method currently?