piwikpro / plugin-sitemigration Goto Github PK

Migrate websites and website data between two Piwik installations

PHP 100.00%

plugin-sitemigration's Introduction

Piwik

This is the last snapshot of open source Piwik PRO, it's no longer maintained, so please don't use it. We've migrated to a new, proprietary platform, its cloud "Core" version is available for free here.

Description

Piwik is the leading Free/Libre open analytics platform.

Piwik is a full featured PHP MySQL software program that you download and install on your own webserver. At the end of the five minute installation process you will be given a JavaScript code. Simply copy and paste this tag on websites you wish to track and access your analytics reports in real time.

Piwik aims to be a Free software alternative to Google Analytics, and is already used on more than 1,000,000 websites. Privacy is built-in!

Mission Statement

« To create, as a community, the leading international Free/Libre web analytics platform, providing access to all functionality through open components and open APIs. »

Or in short:

« Liberate Web Analytics »

License

Piwik is released under the GPL v3 (or later) license, see misc/gpl-3.0.txt

Requirements

PHP 5.5.9 or greater
MySQL version 5.5 or greater, or MariaDB
PHP extension pdo and pdo_mysql, or the MySQLi extension.
Piwik is OS / server independent

Install

Upload piwik to your webserver
Point your browser to the directory
Follow the steps
Add the given javascript code to your pages
(You may also generate fake data to experiment, by enabling the plugin VisitorGenerator)

Quality Assurance

The Piwik project uses an ever-expanding comprehensive set of thousands of unit tests and hundreds of automated integration tests, system tests, JavaScript tests, and screenshot UI tests, running on a continuous integration server as part of its software quality assurance.

We use BrowserStack.com testing tool to help check the Piwik user interface is compatible with many browsers.

Security

Security is a top priority at Piwik. As potential issues are discovered, we validate, patch and release fixes as quickly as we can. We have a security bug bounty program in place that rewards researchers for finding security issues and disclosing them to us.

plugin-sitemigration's People

Contributors

Stargazers

Watchers

Forkers

saadouus sgiehl ravikannankr

plugin-sitemigration's Issues

Save data for later use

Would it be possible to have an option not to write data to a target db but to save as file for later use?

Migrating data to existing site

Using this plugin is it possible to migrate the data to an already existing site? What we want to do is point our tracking code to the new piwik install first and then migrate after so that we do not lose any data. Will this plugin overwrite/affect the existing data on the destination db? Or will it just prepend the visits?

Migration fails

We're trying to migrate some sites from Piwik 2.11.2 to another instance, unfortunately it fails even though manual mysql access works fine. Database schemas on both sides are complete. Any hints on what can be wrong - the tables on both DB use underscores instead of . but AFAIK this shouldn't be an issue?

root@status-status-dev-www1:/srv/piwik# ./console migration:site -v -H host -U migracja -N piwik  2
DEBUG SitesManager[2015-03-17 07:50:46] Db::fetchAll() executing SQL: SELECT idsite FROM piwik_site
DEBUG SitesManager[2015-03-17 07:50:46] Db::fetchAll() executing SQL: SELECT idsite FROM piwik_site
Please provide the destination database password: 
INFO SiteMigration[2015-03-17 07:50:48] Start transaction
INFO SiteMigration[2015-03-17 07:50:48] Migrating site config



  [Zend_Db_Statement_Exception]                                                         
  SQLSTATE[42S02]: Base table or view not found: 1146 Table 'piwik.site' doesn't exist  



Exception trace:
 () at /srv/piwik/libs/Zend/Db/Statement/Pdo.php:234
 Zend_Db_Statement_Pdo->_execute() at /srv/piwik/libs/Zend/Db/Statement.php:300
 Zend_Db_Statement->execute() at /srv/piwik/libs/Zend/Db/Adapter/Abstract.php:479
 Zend_Db_Adapter_Abstract->query() at /srv/piwik/libs/Zend/Db/Adapter/Pdo/Abstract.php:238
 Zend_Db_Adapter_Pdo_Abstract->query() at /srv/piwik/core/Db/Adapter/Pdo/Mysql.php:234
 Piwik\Db\Adapter\Pdo\Mysql->query() at /srv/piwik/libs/Zend/Db/Adapter/Abstract.php:575
 Zend_Db_Adapter_Abstract->insert() at /srv/piwik/plugins/SiteMigration/Helper/DBHelper.php:43
 Piwik\Plugins\SiteMigration\Helper\DBHelper->executeInsert() at /srv/piwik/plugins/SiteMigration/Migrator/TableMigrator.php:56
 Piwik\Plugins\SiteMigration\Migrator\TableMigrator->processRow() at /srv/piwik/plugins/SiteMigration/Migrator/TableMigrator.php:47
 Piwik\Plugins\SiteMigration\Migrator\TableMigrator->migrate() at /srv/piwik/plugins/SiteMigration/Migrator/Migrator.php:144
 Piwik\Plugins\SiteMigration\Migrator\Migrator->migrateSiteConfig() at /srv/piwik/plugins/SiteMigration/Migrator/Migrator.php:107
 Piwik\Plugins\SiteMigration\Migrator\Migrator->migrate() at /srv/piwik/plugins/SiteMigration/Commands/MigrateSite.php:97
 Piwik\Plugins\SiteMigration\Commands\MigrateSite->execute() at /srv/piwik/vendor/symfony/console/Symfony/Component/Console/Command/Command.php:253
 Symfony\Component\Console\Command\Command->run() at /srv/piwik/vendor/symfony/console/Symfony/Component/Console/Application.php:874
 Symfony\Component\Console\Application->doRunCommand() at /srv/piwik/vendor/symfony/console/Symfony/Component/Console/Application.php:195
 Symfony\Component\Console\Application->doRun() at n/a:n/a
 call_user_func() at /srv/piwik/core/Console.php:58
 Piwik\{closure}() at /srv/piwik/core/Access.php:454
 Piwik\Access::doAsSuperUser() at /srv/piwik/core/Console.php:59
 Piwik\Console->doRun() at /srv/piwik/vendor/symfony/console/Symfony/Component/Console/Application.php:126
 Symfony\Component\Console\Application->run() at /srv/piwik/console:27





  [PDOException]                                                                        
  SQLSTATE[42S02]: Base table or view not found: 1146 Table 'piwik.site' doesn't exist  



Exception trace:
 () at /srv/piwik/libs/Zend/Db/Statement/Pdo.php:228
 PDOStatement->execute() at /srv/piwik/libs/Zend/Db/Statement/Pdo.php:228
 Zend_Db_Statement_Pdo->_execute() at /srv/piwik/libs/Zend/Db/Statement.php:300
 Zend_Db_Statement->execute() at /srv/piwik/libs/Zend/Db/Adapter/Abstract.php:479
 Zend_Db_Adapter_Abstract->query() at /srv/piwik/libs/Zend/Db/Adapter/Pdo/Abstract.php:238
 Zend_Db_Adapter_Pdo_Abstract->query() at /srv/piwik/core/Db/Adapter/Pdo/Mysql.php:234
 Piwik\Db\Adapter\Pdo\Mysql->query() at /srv/piwik/libs/Zend/Db/Adapter/Abstract.php:575
 Zend_Db_Adapter_Abstract->insert() at /srv/piwik/plugins/SiteMigration/Helper/DBHelper.php:43
 Piwik\Plugins\SiteMigration\Helper\DBHelper->executeInsert() at /srv/piwik/plugins/SiteMigration/Migrator/TableMigrator.php:56
 Piwik\Plugins\SiteMigration\Migrator\TableMigrator->processRow() at /srv/piwik/plugins/SiteMigration/Migrator/TableMigrator.php:47
 Piwik\Plugins\SiteMigration\Migrator\TableMigrator->migrate() at /srv/piwik/plugins/SiteMigration/Migrator/Migrator.php:144
 Piwik\Plugins\SiteMigration\Migrator\Migrator->migrateSiteConfig() at /srv/piwik/plugins/SiteMigration/Migrator/Migrator.php:107
 Piwik\Plugins\SiteMigration\Migrator\Migrator->migrate() at /srv/piwik/plugins/SiteMigration/Commands/MigrateSite.php:97
 Piwik\Plugins\SiteMigration\Commands\MigrateSite->execute() at /srv/piwik/vendor/symfony/console/Symfony/Component/Console/Command/Command.php:253
 Symfony\Component\Console\Command\Command->run() at /srv/piwik/vendor/symfony/console/Symfony/Component/Console/Application.php:874
 Symfony\Component\Console\Application->doRunCommand() at /srv/piwik/vendor/symfony/console/Symfony/Component/Console/Application.php:195
 Symfony\Component\Console\Application->doRun() at n/a:n/a
 call_user_func() at /srv/piwik/core/Console.php:58
 Piwik\{closure}() at /srv/piwik/core/Access.php:454
 Piwik\Access::doAsSuperUser() at /srv/piwik/core/Console.php:59
 Piwik\Console->doRun() at /srv/piwik/vendor/symfony/console/Symfony/Component/Console/Application.php:126
 Symfony\Component\Console\Application->run() at /srv/piwik/console:27


migration:site [--skip-archive-data] [--skip-log-data] [-H|--db-host="..."] [-U|--db-username="..."] [-P|--db-password="..."] [-N|--db-name="..."] [--db-prefix[="..."]] [--db-port="..."] [-F|--date-from="..."] [-T|--date-to="..."] idSite

PDO exception about integrity constraint

After I called the console command I waited a few minutes. After that it aborted with the following error:

  [Zend_Db_Statement_Exception]
  SQLSTATE[23000]: Integrity constraint violation: 1048 Column 'idaction_url' cannot be null

  [PDOException]
  SQLSTATE[23000]: Integrity constraint violation: 1048 Column 'idaction_url' cannot be null

Using Piwik 2.9.1

Release SiteMigration publicly on Marketplace

The goal of this issue is to release SiteMigration plugin to the Marketplace.

Steps

Code review
Check Readme has sections, screenshots of the console command, and plugin.json
Distribute on Marketplace
Close this issue: matomo-org/matomo#1879
Publish new FAQ to point to this plugin eg. "How do I migrate a particular website in Piwik to a different Piwik server?"

SSL support for MySQL?

Does the plugin support using SSL for MySQL? I haven't seen how to configure it. If it doesn't, it should. I'm trying to use the plugin, but I'm reluctant to send MySQL login credentials and user data in cleartext across a network.

Data Missing After Site Migration

Hi,

We are trying to migrate sites from one Piwik instance to another( source db from windows server and target db in linux environment ) using the Site Migration plugin. We could find some mismatch in the data after migration. Please find the details below:

Current Piwik Version : 3.0.4
DB name : sourcedb and targetdb in mysql
we are using : site migration plugin
we have activated the site migration plugin in windows and ran the below command for site migration :

php ./console migration:site 1 --db-prefix piwik_ --db-username arun --db-password xxxxxxx --db-name target
destination database host : localhost
start tranasaction
migrating site config
loading existing actions
migrating log data -visits
migrating log data - link visit action
migrating log data - conversions and conversion items
migrating archive data
commit transaction .

the above command was migrated the from source db site id : 1 to siteid : 50 to our target db. In end of the migration we are getting commit transaction line.
(ending without any error ) .

But only piwik_site and piwik_archive_x and numeric table data were get populated into tragetdb from sourcedb..

Data are not populated in the below tables ..

piwik_log_visit
piwik_log_action
Piwik_log_link_visit_action

Can anyone please let me know what is the issue here.

Regards

Problem Migrating Site Data

I am executing the following:

/usr/bin/php5 /path/to/piwik/console migration:site 21 --skip-archive-data --date-from=2016-02-29 --date-to=2016-03-25 --db-prefix=piwik_ -v

The output given is:

INFO [2016-03-25 21:21:26] Start transaction
INFO [2016-03-25 21:21:26] Migrating site config
INFO [2016-03-25 21:21:26] Loading existing actions
INFO [2016-03-25 21:21:26] Migrating log data - visits
INFO [2016-03-25 21:21:26] Migrating log data - link visit action
INFO [2016-03-25 21:21:26] Migrating log data - conversions and conversion items
INFO [2016-03-25 21:21:26] Commit transaction
DEBUG [2016-03-25 21:21:26] Time taken: 0.04 sec
DEBUG [2016-03-25 21:21:26] Peak memory usage: 14.00 MB

So everything seems fine. When I check the target database only one record is ported across but there are records in the piwik_log_visit table between the dates specified for that site in the source DB. The actions for the imported record are imported fine. There is no conversion data which needs migrating. The Piwik version I am using is 2.16.0 and plugin version is 1.0.8.

Am I missing something obvious?

Migration fails on InvalidArgumentException

Using SiteMigrator (trunk version) we've encountered the error below. It is not possible to skip the failed record(s) using a skip-conversion-items option or so.
Since it is a large site, we already had to split it across multiple date ranges, loading it into different sites on the other end and then update the database, such as:

./console migration:site 2 --db-prefix=piwik_ --db-host=somehwere.com --db-username=someone --db-name=piwik --skip-archive-data --verbose --db-password='something' --date-from=2013-01-01 --date-to=2014-01-01

Suggested change:

Allow migration to continue even when some records fail, for instance up to maximum 1% or 1.000 or alike.

...
INFO [2016-02-14 12:10:06] Migrating log data - conversions and conversion items

  [InvalidArgumentException]
  Id 2453741 not found in Piwik\Plugins\SiteMigration\Migrator\TableMigrator

Exception trace:
 () at /srv/www/htdocs/secret.com/html/plugins/SiteMigration/Migrator/TableMigrator.php:70
 Piwik\Plugins\SiteMigration\Migrator\TableMigrator->getNewId() at /srv/www/htdocs/secret.com/html/plugins/SiteMigration/Migrator/ConversionMigrator.php:59
 Piwik\Plugins\SiteMigration\Migrator\ConversionMigrator->translateRow() at /srv/www/htdocs/secret.com/html/plugins/SiteMigration/Migrator/TableMigrator.php:54
 Piwik\Plugins\SiteMigration\Migrator\TableMigrator->processRow() at /srv/www/htdocs/secret.com/html/plugins/SiteMigration/Migrator/TableMigrator.php:47
 Piwik\Plugins\SiteMigration\Migrator\TableMigrator->migrate() at /srv/www/htdocs/secret.com/html/plugins/SiteMigration/Migrator/Migrator.php:204
 Piwik\Plugins\SiteMigration\Migrator\Migrator->migrateLogVisitConversions() at /srv/www/htdocs/secret.com/html/plugins/SiteMigration/Migrator/Migrator.php:114
 Piwik\Plugins\SiteMigration\Migrator\Migrator->migrate() at /srv/www/htdocs/secret.com/html/plugins/SiteMigration/Commands/MigrateSite.php:98
 Piwik\Plugins\SiteMigration\Commands\{closure}() at /srv/www/htdocs/secret.com/html/core/Piwik.php:758
 Piwik\Piwik::doAsSuperUser() at /srv/www/htdocs/secret.com/html/plugins/SiteMigration/Commands/MigrateSite.php:104
 Piwik\Plugins\SiteMigration\Commands\MigrateSite->execute() at /srv/www/htdocs/secret.com/html/vendor/symfony/console/Symfony/Component/Console/Command/Command.php:257
 Symfony\Component\Console\Command\Command->run() at /srv/www/htdocs/secret.com/html/vendor/symfony/console/Symfony/Component/Console/Application.php:874
 Symfony\Component\Console\Application->doRunCommand() at /srv/www/htdocs/secret.com/html/vendor/symfony/console/Symfony/Component/Console/Application.php:195
 Symfony\Component\Console\Application->doRun() at n/a:n/a
 call_user_func() at /srv/www/htdocs/secret.com/html/core/Console.php:79
 Piwik\{closure}() at /srv/www/htdocs/secret.com/html/core/Access.php:466
 Piwik\Access::doAsSuperUser() at /srv/www/htdocs/secret.com/html/core/Console.php:80
 Piwik\Console->doRun() at /srv/www/htdocs/secret.com/html/vendor/symfony/console/Symfony/Component/Console/Application.php:126
 Symfony\Component\Console\Application->run() at /srv/www/htdocs/secret.com/html/console:27

Running SiteMigration on target server

Unfortunately I can't expose the MySQL server on the target server to the world; but I can do so on the source server. Given this circumstances and having read the description of this plugin which states:

You must be able to connect to the Mysql server of the Target Piwik Server.
You must run the console command on the Piwik Server that data will be copied from.
https://plugins.piwik.org/SiteMigration

How shall I proceed?

Base table or view not found: 1146 Table 'piwik.site' doesn't exist

I am trying to migrate my piwik data from current Piwik to my Target Piwik, I am able to connect to my target piwik's mysql.

I am getting this error:
[Zend_Db_Statement_Exception]
SQLSTATE[42S02]: Base table or view not found: 1146 Table 'piwik.site' doesn't exist

Exception trace:
() at /var/www/piwik/libs/Zend/Db/Statement/Pdo.php:234
Zend_Db_Statement_Pdo->_execute() at /var/www/piwik/libs/Zend/Db/Statement.php:300
Zend_Db_Statement->execute() at /var/www/piwik/libs/Zend/Db/Adapter/Abstract.php:479
Zend_Db_Adapter_Abstract->query() at /var/www/piwik/libs/Zend/Db/Adapter/Pdo/Abstract.php:238
Zend_Db_Adapter_Pdo_Abstract->query() at /var/www/piwik/core/Db/Adapter/Pdo/Mysql.php:245
Piwik\Db\Adapter\Pdo\Mysql->query() at /var/www/piwik/libs/Zend/Db/Adapter/Abstract.php:575
Zend_Db_Adapter_Abstract->insert() at /var/www/piwik/plugins/SiteMigration/Helper/DBHelper.php:43
Piwik\Plugins\SiteMigration\Helper\DBHelper->executeInsert() at /var/www/piwik/plugins/SiteMigration/Migrator/TableMigrator.php:56
Piwik\Plugins\SiteMigration\Migrator\TableMigrator->processRow() at /var/www/piwik/plugins/SiteMigration/Migrator/TableMigrator.php:47
Piwik\Plugins\SiteMigration\Migrator\TableMigrator->migrate() at /var/www/piwik/plugins/SiteMigration/Migrator/Migrator.php:144
Piwik\Plugins\SiteMigration\Migrator\Migrator->migrateSiteConfig() at /var/www/piwik/plugins/SiteMigration/Migrator/Migrator.php:107
Piwik\Plugins\SiteMigration\Migrator\Migrator->migrate() at /var/www/piwik/plugins/SiteMigration/Commands/MigrateSite.php:97
Piwik\Plugins\SiteMigration\Commands\MigrateSite->execute() at /var/www/piwik/vendor/symfony/console/Symfony/Component/Console/Command/Command.php:257
Symfony\Component\Console\Command\Command->run() at /var/www/piwik/vendor/symfony/console/Symfony/Component/Console/Application.php:874
Symfony\Component\Console\Application->doRunCommand() at /var/www/piwik/vendor/symfony/console/Symfony/Component/Console/Application.php:195
Symfony\Component\Console\Application->doRun() at n/a:n/a
call_user_func() at /var/www/piwik/core/Console.php:79
Piwik\Console->Piwik{closure}() at /var/www/piwik/core/Access.php:456
Piwik\Access::doAsSuperUser() at /var/www/piwik/core/Console.php:80
Piwik\Console->doRun() at /var/www/piwik/vendor/symfony/console/Symfony/Component/Console/Application.php:126
Symfony\Component\Console\Application->run() at /var/www/piwik/console:27

What I am not able to understand is error says it is unable to connect with "piwik.site" where as all the tables I have are like "piwik_site".

Can someone pls look into this.

Additional tables not migrated

Hi,

I have two instances of Piwik: Instance 1 and Instance 2. I am trying to merge data from Instance 1 to Instance 2. Both instances have plugin which adds additional table to DB. While running migration:site data in this additional table on Instance 1 is not copied to Instance 2.

Take the local database settings as the default and override only the given settings

It would be good to have the settings of the piwik installation as default and override them as needed.

Use case: With many installations on one server, the only diferent settings are the database name and maybe the prefix.

Pull request: #18

When `--db-prefix` is different from the source db prefix, script fails

When target --db-prefix is set to a value different from the source db prefix, the script will not migrate the data because the sequence does not use the target db prefix in https://github.com/piwik/piwik/blob/master/core/Sequence.php#L46-46

The sequence uses Common::prefixTable to prefix the table and does not use the internal DB object this->db.

Cryptic error message when --db-prefix is not specified (should be a mandatory argument/question?)

Hi there!

It took me a while to figure out what I had done "wrong" here:

$ ./console migration:site 11   
Please provide the destination database host [localhost]: 
Please provide the destination database username: foobar_piwik
Please provide the destination database password: 
Please provide the destination database name: foobar_piwik-alternate
INFO [2015-09-07 02:33:59] Start transaction
INFO [2015-09-07 02:33:59] Migrating site config

  [Zend_Db_Statement_Exception]                                                                           
  SQLSTATE[42S02]: Base table or view not found: 1146 Table 'foobar_piwik-alternate.site' doesn't exist

  [PDOException]                                                                                          
  SQLSTATE[42S02]: Base table or view not found: 1146 Table 'foobar_piwik-alternate.site' doesn't exist

The error messages you see there are a bit cryptic. Eventually I figured it out:

$ ./console migration:site 11 --db-prefix="piwik_"
Please provide the destination database host [localhost]: 
Please provide the destination database username: foobar_piwik
Please provide the destination database password: 
Please provide the destination database name: foobar_piwik-alternate
INFO [2015-09-07 02:36:19] Start transaction
INFO [2015-09-07 02:36:19] Migrating site config
INFO [2015-09-07 02:36:19] Loading existing actions
INFO [2015-09-07 02:36:19] Migrating log data - visits
INFO [2015-09-07 02:36:21] Migrating log data - link visit action
INFO [2015-09-07 02:36:23] Migrating log data - conversions and conversion items
INFO [2015-09-07 02:36:23] Migrating archive data
INFO [2015-09-07 02:36:56] Commit transaction

I think the SiteMigration plugin could be a little bit smarter, by

Clarifying the error message you see further below (it says "Base table or view not found […]"; it could be clearer on the fact that the problem might be the missing db table prefix?), if possible.
In interactive mode, prompting the user for the table prefix (and prefilling it with the default "piwik_" value?) so that it's fool-proof.

how to upload files using php in lampp ubuntu

Travis CI disappeared?

It seems the project doesn't exist anymore on Travis CI pro. Is that because the project has been open sourced or was it intentional? (@mattab maybe)

If Plugins or Custom Variables count differ on both instances, got "Column not found: 1054 Unknown column 'custom_var_k6' in 'field list'"

Hi,

I have two instances of Piwik: Instance 1 and Instance 2. I am trying to merge data from Instance 1 to Instance 2. Instance 1 has one custom variable more than Instance 2. While running migration:site I get following error:

SQLSTATE[42S22]: Column not found: 1054 Unknown column 'custom_var_k6' in 'field list'

So obviously both instances must have the same number of custom vars. I think it should be at least explicitly stated in documentation, but the best option would be automatically adding additional custom variables.

Add FAQ entry

How do I migrate a particular website in Piwik to a different Piwik server?

Cannot get it to work

Hello, I have one installation for site A set to siteid 1. I have a second installation for site A set to siteid2 (because I have a Site B set to 1). I changed the sql data for the second installation so that Site A is also set to siteid 1 and Site B is set to sited 3. So that is clear and we are good to go.

I tried the command ./console migration:site 1 -v

Entered in username: getcert_piwkg
password: (xxxx)
and DB name: getcert_piwkg
(yes I have the same username and DB name)

However, I keep getting this error:

[Zend_Db_Statement_Mysqli_Exception]
Mysqli prepare error: Table 'getcert_piwkg.site' doesn't exist

I am not sure what table this is looking for but there is no table with this naming convention. This is only the DB name.

All tables in this DB are standard Piwik tables with names such as "Piwik_Archive_Blob_2014-10" and "Piwik_Report".

Is there something I am missing?

combine site 1 and site2 data?

Hi
I have one question regarding this module.
Is it possible to combine data from site 1 to site2 ?
I executed the site migration from source piwik(site1), but it has every time been created new idsite on target piwik.

It seems to that does not combine site data.

Unknown column 'location_provider' in 'field list'

Hi,
I am trying to migrate a single site from one Piwik installation to an other using your plugin.
Both Piwiks are running in version 2.16.0. The difference is the the PHP and MySQL version:
Server 1: PHP 5.4.39 and MySQL 5.1.73
Server 2: PHP 5.5.9 and MySQL 5.5.7
On both Piwiks the same addons are installed. On server 1 also SiteMigration is installed.

This is the error output:

www-data@h1980054:~/sites/piwik$ ./console migration:site -vvv 20
Please provide the destination database host [localhost]: mysite.com
Please provide the destination database username: piwik
Please provide the destination database password: 
Please provide the destination database name: piwik
INFO [2016-03-09 13:29:08] Start transaction
INFO [2016-03-09 13:29:08] Migrating site config
INFO [2016-03-09 13:29:09] Loading existing actions
INFO [2016-03-09 13:29:09] Migrating log data - visits

  [Zend_Db_Statement_Mysqli_Exception]                                      
  Mysqli prepare error: Unknown column 'location_provider' in 'field list'  

Exception trace:
 () at /var/www/sites/piwik/libs/Zend/Db/Statement/Mysqli.php:77
 Zend_Db_Statement_Mysqli->_prepare() at /var/www/sites/piwik/libs/Zend/Db/Statement.php:115
 Zend_Db_Statement->__construct() at /var/www/sites/piwik/libs/Zend/Db/Adapter/Mysqli.php:381
 Zend_Db_Adapter_Mysqli->prepare() at /var/www/sites/piwik/libs/Zend/Db/Adapter/Abstract.php:478
 Zend_Db_Adapter_Abstract->query() at /var/www/sites/piwik/libs/Zend/Db/Adapter/Abstract.php:575
 Zend_Db_Adapter_Abstract->insert() at /var/www/sites/piwik/plugins/SiteMigration/Helper/DBHelper.php:43
 Piwik\Plugins\SiteMigration\Helper\DBHelper->executeInsert() at /var/www/sites/piwik/plugins/SiteMigration/Migrator/TableMigrator.php:56
 Piwik\Plugins\SiteMigration\Migrator\TableMigrator->processRow() at /var/www/sites/piwik/plugins/SiteMigration/Migrator/TableMigrator.php:47
 Piwik\Plugins\SiteMigration\Migrator\TableMigrator->migrate() at /var/www/sites/piwik/plugins/SiteMigration/Migrator/Migrator.php:182
 Piwik\Plugins\SiteMigration\Migrator\Migrator->migrateLogVisits() at /var/www/sites/piwik/plugins/SiteMigration/Migrator/Migrator.php:112
 Piwik\Plugins\SiteMigration\Migrator\Migrator->migrate() at /var/www/sites/piwik/plugins/SiteMigration/Commands/MigrateSite.php:98
 Piwik\Plugins\SiteMigration\Commands\MigrateSite->Piwik\Plugins\SiteMigration\Commands\{closure}() at /var/www/sites/piwik/core/Piwik.php:758
 Piwik\Piwik::doAsSuperUser() at /var/www/sites/piwik/plugins/SiteMigration/Commands/MigrateSite.php:104
 Piwik\Plugins\SiteMigration\Commands\MigrateSite->execute() at /var/www/sites/piwik/vendor/symfony/console/Symfony/Component/Console/Command/Command.php:257
 Symfony\Component\Console\Command\Command->run() at /var/www/sites/piwik/vendor/symfony/console/Symfony/Component/Console/Application.php:874
 Symfony\Component\Console\Application->doRunCommand() at /var/www/sites/piwik/vendor/symfony/console/Symfony/Component/Console/Application.php:195
 Symfony\Component\Console\Application->doRun() at n/a:n/a
 call_user_func() at /var/www/sites/piwik/core/Console.php:79
 Piwik\Console->Piwik\{closure}() at /var/www/sites/piwik/core/Access.php:466
 Piwik\Access::doAsSuperUser() at /var/www/sites/piwik/core/Console.php:80
 Piwik\Console->doRun() at /var/www/sites/piwik/vendor/symfony/console/Symfony/Component/Console/Application.php:126
 Symfony\Component\Console\Application->run() at /var/www/sites/piwik/console:27


migration:site [--skip-archive-data] [--skip-log-data] [-H|--db-host="..."] [-U|--db-username="..."] [-P|--db-password="..."] [-N|--db-name="..."] [--db-prefix[="..."]] [--db-port="..."] [-F|--date-from="..."] [-T|--date-to="..."] idSite

migrate to other idSite

i want to merge the data with data on my target server, but it has another idSite. it's not mentioned here how to set the idSite of the site on the target server. does this tool just create a new site?

Very slow migration

I'm migrating a site with log data and archives from a local instance to a WAN/internet host.
For e very low traffic site (< 3 k visits over two years), it takes more than 18 hours, which seems a tad too slow.

Any suggestions on where I should look or tune the mysql or other settings?

Config:
Source: LAMP with piwik 2.14.3 on centos 6.5
Target: bitnami-piwik 2.14.3 on ubuntu 14.04. LTS

MigrateSite uses a PHP 5.4 type-hint (callable)

Reported by a forum user.

Catchable fatal error: Argument 5 passed to Piwik\Plugins\SiteMigration\Commands\MigrateSite::ensureOptionsIsProvided() must be an instance of Piwik\Plugins\SiteMigration\Commands\callable, instance of Closure given, called in /usr/home/mysite/public_html/piwik/plugins/SiteMigration/Commands/MigrateSite.php on line 156 and defined in /usr/home/mysite/public_html/piwik/plugins/SiteMigration/Commands/MigrateSite.php on line 189

(PHP 5.3)

How to find the exact number of times a keyword has been searched in Piwik

Hello,

One of our application that is integrated with Piwik is mainly used as a search engine. Hence, it is very important for us to get the exact number of times a keyword has been searched or the number of times it has produced no results. However, Piwik currently is displaying 'The number of visits that searched for a keyword in the Search engine' as 'Searches'. This means, if a keyword is searched multiple times by a single user in a single visit, the 'Searches' field will display the count as one only.

How can I get the number of searches for a keyword/the number of times a keyword has produced no results despite if it is from a single visit or not?

Regards,
Sobhika

SiteMigration without command line

Unfortunately I do not have a command line access. Is a site migration also possible from within the webinterface of Piwik?

What do I miss?

Hi, I tried the SiteMigration plugin and migrated data from "date-from" to "date-to" into another site.
This was successful, but the widget for showing the visits for each page title does not show me any data though I selected the correct site and date (which have indeed data in the database):

So I am somehow stuck now, what I need to do in order to show the data....

The connection log_visit->log_link_visit_action --> log_action seems to be correct.
The keys correspond to each other and the hash in the latter table is a crc32 of the name column in the latter table. I also deleted the respective archive tables but no change :(
What can I else do?
I am a beginner with piwik and I do not know what to do anymore...

I would be so happy if one of the developers could help me :)

Thank you in advance...

Console not executing

Hey there,
i installed the plugin on my old Piwik installation. Now I wanted to migrate one of my sites (ID is 2) to the new destination. So in my SSH client I type in the following command:

./console migration:site 2 --db-prefix piwik_

The console returns the following:
Content-type: text/html

What am I doing wrong? I tried several commands but I'm always getting the same result.

Site Migration plugin is way too slow and data isn't migrating under same userID

I have around 8 sites being tracked on my Piwik in both servers. Server 1 has new data and Server 2 has old data. I need to merge Server 2 with Server 1, I used SiteMigration plugin, some small data were transmitted well but, site data is saved on separate site ID.

Larger data files have not been transmitted at all, it was stuck for more than 7 hours.

I use Piwik new version.

SQLSTATE[42S22]: Column not found: 1054 Unknown column 'referer_visit_server_date' in 'field list'

Hi mattab,

I'm currently running 2 piwik installations both version 2.11.0 and your plugin installed via piwiki-marketplace. I tried to merge them but I get following errors:

[piwik@xxxxx analytics]# ./console migration:site -v --db-prefix="piwik_" 1
DEBUG SitesManager[2015-02-20 08:43:58] Db::fetchAll() executing SQL: SELECT idsite FROM stats_site
DEBUG SitesManager[2015-02-20 08:43:58] Db::fetchAll() executing SQL: SELECT idsite FROM stats_site
Please provide the destination database password: 
INFO SiteMigration[2015-02-20 08:44:03] Start transaction
INFO SiteMigration[2015-02-20 08:44:03] Migrating site config
INFO SiteMigration[2015-02-20 08:44:03] Loading existing actions
INFO SiteMigration[2015-02-20 08:44:03] Migrating log data - visits
INFO SiteMigration[2015-02-20 08:44:03] Migrating log data - link visit action
INFO SiteMigration[2015-02-20 08:44:04] Migrating log data - conversions and conversion items



  [Zend_Db_Statement_Exception]                                                                       
  SQLSTATE[42S22]: Column not found: 1054 Unknown column 'referer_visit_server_date' in 'field list'  


Exception trace:
 () at /var/www/vhosts/mydomain.com/application/public/analytics/libs/Zend/Db/Statement/Pdo.php:234
 Zend_Db_Statement_Pdo->_execute() at /var/www/vhosts/mydomain.com/application/public/analytics/libs/Zend/Db/Statement.php:300
 Zend_Db_Statement->execute() at /var/www/vhosts/mydomain.com/application/public/analytics/libs/Zend/Db/Adapter/Abstract.php:479
 Zend_Db_Adapter_Abstract->query() at /var/www/vhosts/mydomain.com/application/public/analytics/libs/Zend/Db/Adapter/Pdo/Abstract.php:238
 Zend_Db_Adapter_Pdo_Abstract->query() at /var/www/vhosts/mydomain.com/application/public/analytics/core/Db/Adapter/Pdo/Mysql.php:234
 Piwik\Db\Adapter\Pdo\Mysql->query() at /var/www/vhosts/mydomain.com/application/public/analytics/libs/Zend/Db/Adapter/Abstract.php:575
 Zend_Db_Adapter_Abstract->insert() at /var/www/vhosts/mydomain.com/application/public/analytics/plugins/SiteMigration/Helper/DBHelper.php:43
 Piwik\Plugins\SiteMigration\Helper\DBHelper->executeInsert() at /var/www/vhosts/mydomain.com/application/public/analytics/plugins/SiteMigration/Migrator/TableMigrator.php:56
 Piwik\Plugins\SiteMigration\Migrator\TableMigrator->processRow() at /var/www/vhosts/mydomain.com/application/public/analytics/plugins/SiteMigration/Migrator/TableMigrator.php:47
 Piwik\Plugins\SiteMigration\Migrator\TableMigrator->migrate() at /var/www/vhosts/mydomain.com/application/public/analytics/plugins/SiteMigration/Migrator/Migrator.php:204
 Piwik\Plugins\SiteMigration\Migrator\Migrator->migrateLogVisitConversions() at /var/www/vhosts/mydomain.com/application/public/analytics/plugins/SiteMigration/Migrator/Migrator.php:114
 Piwik\Plugins\SiteMigration\Migrator\Migrator->migrate() at /var/www/vhosts/mydomain.com/application/public/analytics/plugins/SiteMigration/Commands/MigrateSite.php:97
 Piwik\Plugins\SiteMigration\Commands\MigrateSite->execute() at /var/www/vhosts/mydomain.com/application/public/analytics/vendor/symfony/console/Symfony/Component/Console/Command/Command.php:253
 Symfony\Component\Console\Command\Command->run() at /var/www/vhosts/mydomain.com/application/public/analytics/vendor/symfony/console/Symfony/Component/Console/Application.php:874
 Symfony\Component\Console\Application->doRunCommand() at /var/www/vhosts/mydomain.com/application/public/analytics/vendor/symfony/console/Symfony/Component/Console/Application.php:195
 Symfony\Component\Console\Application->doRun() at n/a:n/a
 call_user_func() at /var/www/vhosts/mydomain.com/application/public/analytics/core/Console.php:58
 Piwik\Console->Piwik\{closure}() at /var/www/vhosts/mydomain.com/application/public/analytics/core/Access.php:454
 Piwik\Access::doAsSuperUser() at /var/www/vhosts/mydomain.com/application/public/analytics/core/Console.php:59
 Piwik\Console->doRun() at /var/www/vhosts/mydomain.com/application/public/analytics/vendor/symfony/console/Symfony/Component/Console/Application.php:126
 Symfony\Component\Console\Application->run() at /var/www/vhosts/mydomain.com/application/public/analytics/console:27

Error Loading MarketPlace

Hi,

I am getting an error while trying to access the MarketPlace in Piwik version 3.2.0. The error also appears while accessing the Administration tab. Attached is the error screens for the same.

The log displays below error messages:

[Fri Feb 16 12:41:52 2018] [error] [client 10.54.17.33] Error in Piwik: , referer: http://piwik-test.swissre.com/piwik/index.php?module=CoreAdminHome&action=home&idSite=1&period=day&date=yesterday&activated=

Can anybody help?