twitter-forks / mysql Goto Github PK

After the release of twitter mysql on github, I noticed https://github.com/twitter/mysql/wiki/Table-Options -- to have pre-determined initial size for per-table tablespaces;
I have some code which should complement it pretty well.

Mainly the code deals with,
1. Use fallocate in os_file_set_size for initial creation of innodb files.
2. Use #1 for extension of ibdata1 files.
3. Use #1 for ibd files.
4. Ability to set increment size for extension of ibd files through a variable.

It starts from,
http://bazaar.launchpad.net/~raghavendra-prabhu/+junk/mysql-server-fallocate/revision/3547
upto
http://bazaar.launchpad.net/~raghavendra-prabhu/+junk/mysql-server-fallocate/revision/3550

(Commits after that deal with other issues).

Also, the code is merged and up-to-date with latest 5.6 bzr pull, also I have tested it myself and results have been pretty good.

I can send these with git-format-patch + git send-email if you need it.

mysql/sql/sql_yacc.cc: In function ‘int MYSQLparse()’:
mysql/sql/sql_yacc.cc:16004:30: error: too few arguments to function ‘int MYSQLlex(void_, void_)’
yychar = yylex (&yylval);

Hi, there is a potential use-after-free bug, please check and confirm if it's real

In the file mysql/mysys/my_malloc.c and the my_realloc function, the variable oldpoint free at line 100 and return it to the caller. The path conditions of these two branches can be satisfied simultaneously.

if (my_flags & MY_FREE_ON_ERROR) //line 109
      my_free(oldpoint); // line 110

if (my_flags & MY_HOLD_ON_ERROR)//111
      DBUG_RETURN(oldpoint);/112

The function my_free is at line 130 in the same file.

Thank you
Ryan

OS: Centos6.5
MySQL 5.6.37
at first use mysql binary tar ball mysqld file size is same。but running with 3 day。mysqld size is change:
3 day ago:

find / -name mysqld | xargs ls -la
-rwxr-xr-x 1 mysql dba   87208139 Jun  3 05:30 /data/install_mysql_5.6/mysql/bin/mysqld
-rwxr-xr-x 1 mysql mysql 87208139 Jun  3 05:30 /media/database/mysql/bin/mysqld
-rwxr-xr-x 1 root  root  87208139 Oct 13 11:51 /usr/bin/mysqld
-rwxr-xr-x 1 root  root  87208139 Oct 13 11:51 /usr/local/mysql/bin/mysqld
-rwxr-xr-x 1 mysql mysql 87208139 Jun  3 05:30 /var/lib/mysql/bin/mysqld

now:

find / -name mysqld | xargs ls -la
-rwxr-xr-x 1 mysql dba   87208139 Jun  3 05:30 /data/install_mysql_5.6/mysql/bin/mysqld
-rwxr-xr-x 1 mysql mysql 87208139 Jun  3 05:30 /media/database/mysql/bin/mysqld
-rwxr-xr-x 1 root  root  87987531 Oct 13 11:51 /usr/bin/mysqld
-rwxr-xr-x 1 root  root  87208139 Oct 13 11:51 /usr/local/mysql/bin/mysqld
-rwxr-xr-x 1 mysql mysql 87208139 Jun  3 05:30 /var/lib/mysql/bin/mysqld

Is this fork still alive?

hi guys, could anyone help me developing this MDEV (mariadb, maybe usefull for twitter too, since it's a nice feature that optimize many queries using ENUM fields)

https://mariadb.atlassian.net/browse/MDEV-4419

it's a enum optimization without index (but can help index too)
example:
a field with => enum ('a','b') NOT NULL

WHERE enum = 'c' => rewrite to WHERE 0 (impossible where)
WHERE enum != 'c' => rewrite to enum IN ('','a','b') OR WHERE 1 (always true)
WHERE enum!='b' => rewrite to enum IN ('','a') (COND_OK)

check that we can use any operator (LIKE for example) the key is:
read all enum possible values, execute operator for each enum value, if we have all false return it's a impossible where, if we have all true we have a always true, if we don't have all true and all false we can rewrite to IN () operator and use very optimized index queries (using HASH or BTREE for example)

Hi Expert
Iam using simple command to change root password after MySql installation ,But getting the Above error (signal 9 kill)

script "change password" do
interpreter "bash"
user "root"
cwd "/tmp"
code <<-EOH
#MYSQL
root_temp_pass=$(grep 'A temporary password' /mysql/log/mysqld.log |tail -1 |awk '{split($0,a,": "); print a[2]}')

#Login as root change password
mysql -uroot -p"$root_temp_pass" -Be "ALTER USER 'root'@'localhost' IDENTIFIED BY 'Czt732ck#';" --connect-expired-password
EOH
end

hello, I found a potential bug of free of stack address in source code of mysql, the overview of occurrence is shown as follows.


step1:
in mysql/sql-common/client.c line 3585
defined a local variable tmp_mysql in this line

step2:
in mysql/sql-common/client.c line 3615
in a path derived from step1,there is a call of mysql_close and stack address of tmp_mysql is used as 1st parameter

step3:
in mysql/sql-common/client.c line 3823
the parameter of function mysql_close is used as 1st parameter of function my_free

step4:
in mysql/mysys/my_malloc.c line 134
free of stack address here

twitter-mysql-ca173b6 was failed to make with the newest google-breakpad
caused by two change:

1. define of callback function was changed
   in sql/minidump.cc:
   dump_callback(const char *dump_path,
   const char *minidump_id,
   void *,
   bool succeeded)

in client/linux/handler/exception_handler.h

typedef bool (MinidumpCallback)(const MinidumpDescriptor& descriptor,
void context,
bool succeeded);

1. "breakpad/src/common/linux/linux_libc_support.cc" defined a set of functions with the prefix "my_" and they were conflicted with the function of mysql.

While trying to recover a corrupted ARCHIVE table, my searching lead me here. Posting my findings in case it also helps someone in the future. (Note that archive_reader is not specific to this fork - this fork is just where I was lead.)

Patched code with:

diff --git a/storage/archive/CMakeLists.txt b/storage/archive/CMakeLists.txt
index 78fb95c0d2..799ad44a6f 100644
--- a/storage/archive/CMakeLists.txt
+++ b/storage/archive/CMakeLists.txt
@@ -16,3 +16,5 @@
 SET(ARCHIVE_SOURCES  azio.c ha_archive.cc ha_archive.h)
 MYSQL_ADD_PLUGIN(archive ${ARCHIVE_SOURCES} STORAGE_ENGINE LINK_LIBRARIES ${ZLIB_LIBRARY})
 
+MYSQL_ADD_EXECUTABLE(archive_reader archive_reader.c azio.c)
+TARGET_LINK_LIBRARIES(archive_reader mysys)
diff --git a/storage/archive/archive_reader.c b/storage/archive/archive_reader.c
index 2475418fef..7a3fd792c7 100644
--- a/storage/archive/archive_reader.c
+++ b/storage/archive/archive_reader.c
@@ -39,6 +39,8 @@ static char **default_argv;
 int opt_check, opt_force, opt_quiet, opt_backup= 0, opt_extract_frm;
 int opt_autoincrement;
 
+PSI_file_key arch_key_file_metadata, arch_key_file_data, arch_key_file_frm;
+
 int main(int argc, char *argv[])
 {
   unsigned int ret;

• Compiled with cmake && make archive_reader -j4.
• Shutdown MySQL/MariaDB server (should work with either; I do not think anyone has worked on the ARCHIVE engine for over a decade).
• Backup your mysql data dir in case something goes wrong.
• Rebuild the <table-name>.ARZ with archive_reader --backup path/to/table.ARZ new-file.ARZ.
• Replace <table-name>.ARZ with the new file.
• If <table-name>.ARN exists, delete it.
• Start up the db server.
• Before doing anything else, mysqldump the table.
  • The server thinks the table is valid, so even if corruption was not repaired, you should still be able to dump data up to the point where it had an issue. Can load this dump later, if need be.
• If there were no errors, hopefully everything is good to go.

We found ARCHIVE engine to be 6x smaller than compressed InnoDB (and 10x smaller than uncompressed), so still has value. If you too must stay with ARCHIVE, I recommend FLUSHing the table after completing any major write.