MariaDB - a MySQL replacement at South East Linux Fest 2014 - SELF2014. Learn about features that are not in MySQL 5.6, some that are only just coming in MySQL 5.7, and some that just don't exist.
1. MariaDB - a MySQL
replacement
Colin Charles, Team MariaDB, SkySQL Ab
colin@mariadb.org | http://mariadb.org/
http://bytebot.net/blog/ | @bytebot on Twitter
South East Linux Fest, Charlotte, NC, USA
20 June 2014
2. whoami
• Work on MariaDB at SkySQL Ab
• Merged with Monty Program Ab, makers of MariaDB
• Formerly MySQL AB (exit: Sun Microsystems)
• Past lives include Fedora Project (FESCO), OpenOffice.org
3. Who are you?
• Developer?
• Operator? (DBA, sysadmin)
• A bit of both?
5. 5W1H is MariaDB
• Drop-in compatible MySQL replacement
• Community developed, Foundation backed, feature enhanced,
backwards compatible, GPLv2 licensed
• Steady stream of releases in 4 years 4 months: 5.1, 5.2, 5.3, 5.5, 10.0,
MariaDB Galera Cluster 5.5, MariaDB with TokuDB 5.5
• Enterprise features open: PAM authentication plugin, threadpool, audit
plugin
• Default in Red Hat Enterprise Linux, Fedora, openSUSE, etc.
8. Microseconds & 5.6
• TIME_TO_SEC(), UNIX_TIMESTAMP() preserve microseconds of the
argument
MariaDB 10.0 MySQL 5.6
SELECT
TIME_TO_SEC('10:10:10.12345');
+-------------------------------+
| TIME_TO_SEC('10:10:10.12345') |
+-------------------------------+
| 36610.12345 |
+-------------------------------+
1 row in set (0.01 sec)
SELECT
TIME_TO_SEC('10:10:10.12345');
+-------------------------------+
| TIME_TO_SEC('10:10:10.12345') |
+-------------------------------+
| 36610 |
+-------------------------------+
1 row in set (0.00 sec)
9. Virtual Columns
• A column in a table that has its value automatically calculated either
with a pre-calculated/deterministic expression or values of other
fields in the table
• VIRTUAL - computed on the fly when data is queried (like a VIEW)
• PERSISTENT - computed when data is inserted and stored in a table
MariaDB 5.2+
10. PCRE Regular Expressions
• Powerful REGEXP/RLIKE operator
• New operators:
• REGEXP_REPLACE(sub,pattern,replace)
• REGEXP_INSTR(sub,pattern)
• REGEXP_SUBSTR(sub,pattern)
• Works with multi-byte character sets that MariaDB supports, including
East-Asian sets
MariaDB 10.0+
11. GIS
• MariaDB implements a subset of SQL with Geometry Types
• No longer just minimum bounding rectangles (MBR) - shapes
considered
CREATE TABLE geom (g GEOMETRY NOT NULL, SPATIAL
INDEX(g)) ENGINE=MyISAM;
• ST_ prefix - as per OpenGIS requirements
MariaDB 5.3+
12. Sample use cases
• Import OpenStreetMap data into MariaDB: http://www.slideshare.net/
hholzgra/fosdem-2014mariadbgis
• Use the OpenStreetMap dataset: https://mariadb.com/kb/en/
openstreetmap-dataset/
• Screencast: https://blog.mariadb.org/screencast-mariadb-gis-demo/
• node.js example use case for mapping GPX data: https://
blog.mariadb.org/node-js-mariadb-and-gis/ & jQuery usage: https://
blog.mariadb.org/jquery-and-gis-distance-in-mariadb/
13. Dynamic columns
• Allows you to create virtual columns with dynamic content for each row in
table. Store different attributes for each item (like a web store).
• Basically a BLOB with handling functions: COLUMN_CREATE,
COLUMN_ADD, COLUMN_GET, COLUMN_DELETE, COLUMN_EXISTS,
COLUMN_LIST, COLUMN_CHECK, COLUMN_JSON
• In MariaDB 10.0: name support (instead of referring to columns by numbers,
name it), convert all dynamic column content to JSON array, interface with
Cassandra
INSERT INTO tbl SET
dyncol_blob=COLUMN_CREATE("column_name", "value");
MariaDB 5.3+
15. What is SphinxSE?
• SphinxSE is just the storage engine that still depends on the Sphinx
daemon
• It doesn’t store any data itself
• Its just a built-in client to allow MariaDB to talk to Sphinx searchd,
run queries, obtain results
• Indexing, searching is performed on Sphinx
16. Sphinx search table
CREATE TABLE t1
(
id INTEGER UNSIGNED NOT NULL,
weight INTEGER NOT NULL,
query VARCHAR(3072) NOT NULL,
group_id INTEGER,
INDEX(query)
) ENGINE=SPHINX CONNECTION="sphinx://localhost:9312/test";
!
SELECT * FROM t1 WHERE query='test it;mode=any';
17. Sphinx search tables
• 1st column: INTEGER UNSIGNED or BIGINT (document ID)
• 2nd column: match weight
• 3rd column: VARCHAR or TEXT (your query)
• Query column needs indexing, no other column needs to be
18. Query Cassandra
• Data is mapped: rowkey, static columns, dynamic columns
• super columns aren’t supported
• No 1-1 direct map for data types
• Write to Cassandra from SQL (SELECT, INSERT, UPDATE, DELETE)
MariaDB 10.0+
19. Cassandra II
pk varchar(36) primary key,
data1 varchar(60),
data2 bigint
) engine=cassandra keyspace='ks1' column_family='cf1'
• Table must have a primary key
• name/type must match Cassandra’s rowkey
• Columns map to Cassandra’s static columns
• name must be same as in Cassandra, datatypes must match, can be subset of CF’s columns
20. Mapping
• Datatype mapping - complete table at KB
• Data mapping is safe - engine will refuse incorrect mappings
• Command mapping: INSERT overwrites rows, UPDATE reads then
writes, DELETE reads then writes
21. Typical use cases
• Web page hits collection, streaming data
• Sensor data
• Reads served with a lookup
• Want an auto-replicated, fault-tolerant table?
22. CONNECT
• Target: ETL for BI or analytics
• Import data from CSV, XML, ODBC, MS Access, etc.
• WHERE conditions pushed to ODBC source
• DROP TABLE just removes the stored definition, not data itself
• “Virtual” tables cannot be indexed
MariaDB 10.0+
23. SPIDER
• Horizontal partitioning, built on top of PARTITIONs
• Associates a partition with a remote server
• Transparent to user, easy to expand
• Has index condition pushdown support enabled
MariaDB 10.0+
24. TokuDB
• Opensource - separate MariaDB 5.5+TokuDB/integrated in 10.0.5
• Improved insert (10-20x faster) & query speed, compression (up to
90% space reduction), replication performance and online schema
flexibility
• Uses Fractal Tree Indexes instead of B-Tree
• Tests & builds of TokuDB on multiple platforms
25. Engines, etc
• Plan for backups - TokuDB can be cool for your uses as an example
• Galera: study your workload patterns, your application, etc.
• SPIDER (built-in sharding capabilities, partitioning & XA transaction
capable with multiple backends including Oracle)
• its not going to be straightforward to “just start” - need to know
right tables to implement, etc.
26. Threadpool
• Modified from 5.1 (libevent based), great for CPU bound
loads and short running queries
• Windows (threadpool), Linux (epoll), Solaris (event ports),
FreeBSD/OSX (kevents)
• No minimization of concurrent transactions with dynamic
pool size
• thread_handling=pool-of-threads
• https://mariadb.com/kb/en/thread-pool-in-mariadb-55/
MariaDB 5.5+
27. PAM Authentication
• Authentication using /etc/shadow
• Authentication using LDAP, SSH pass phrases, password expiration,
username mapping, logging every login attempt, etc.
• INSTALL PLUGIN pam SONAME ‘auth_pam.so’;
• CREATE USER foo@host IDENTIFIED via pam
• Remember to configure PAM (/etc/pam.d or /etc/pam.conf)
• http://www.mysqlperformanceblog.com/2013/02/24/using-two-factor-
authentication-with-percona-server/
MariaDB 5.2+
28. Non-blocking client library
• start operation, do work in thread, operation processed, result
travels back
• use cases: multiple queries against single server (utilize more
CPUs); queries against multiple servers (SHOW STATUS on many
machines)
• https://mariadb.com/kb/en/about-non-blocking-operation-in-the-
client-library/
• fast node.js driver available: mariasql
MariaDB 5.5+
29. LIMIT ROWS EXAMINED
• The purpose of this optimization is to provide the means to terminate
the execution of SELECTstatements which examine too many rows,
and thus use too many resources.
• SELECT * from t1, t2 LIMIT 10 ROWS EXAMINED 1000;
• https://mariadb.com/kb/en/limit-rows-examined/
MariaDB 5.5+
30. SQL Error Logging Plugin
• Log errors sent to clients in a log file that can be analysed later. Log
file can be rotated (recommended)
• a MYSQL_AUDIT_PLUGIN
install plugin SQL_ERROR_LOG soname 'sql_errlog.so';
MariaDB 5.5+
31. Audit Plugin
• Log server activity - who connects to the server, what queries run,
what tables touched - rotating log file or syslogd
• a MYSQL_AUDIT_PLUGIN
INSTALL PLUGIN server_audit SONAME
‘server_audit.so’;
MariaDB 10.0+
32. Replication made better
• Selective skipping of replication events (session-based or on master
or slave)
• Dynamic control of replication variables (no restarts!)
• Using row-based replication? Annotate the binary log with SQL
statements
• Slaves perform checksums on binary log events
• Slaves crash-safe (data stored inside transaction tables)
MariaDB 5.3+
33. Replication made better II
• Group commit in the binary log - finally, sync_binlog=1,
innodb_flush_log_at_trx_commit=1 performs
• START TRANSACTION WITH CONSISTENT SNAPSHOT
• mysqldump —single-transaction —master-data - full non-
blocking backup
• Parallel replication
• Multi-source replication - (real-time) analytics, shard provisioning,
backups, etc.
34. New KILL syntax
• HARD | SOFT & USER USERNAME are MariaDB-specific (5.3.2)
• KILL QUERY ID query_id (10.0.5) - kill by query id, rather than thread id
• SOFT ensures things that may leave a table in an inconsistent state
aren’t interrupted (like REPAIR or INDEX creation for MyISAM or Aria)
KILL [HARD | SOFT] [CONNECTION | QUERY] [thread_id |
USER user_name]
MariaDB 5.3+
35. Statistics
• Understand server activity better to understand database loads
• SET GLOBAL userstat=1;
• SHOW CLIENT_STATISTICS; SHOW USER_STATISTICS;
• # of connections, CPU usage, bytes received/sent, row statistics
• SHOW INDEX_STATISTICS; SHOW TABLE_STATISTICS;
• # rows read, changed, indexes
• INFORMATION_SCHEMA.PROCESSLIST has MEMORY_USAGE, EXAMINED_ROWS
(similar with SHOW STATUS output)
MariaDB 5.2+
MariaDB 10.0+
36. EXPLAIN enhanced
• Explain analyser: https://mariadb.org/explain_analyzer/analyze/
• SHOW EXPLAIN for <thread_id>
• EXPLAIN output in the slow query log
• EXPLAIN not just for SELECT but INSERT/UPDATE/DELETE
MariaDB 10.0+
37. Roles
• Bundles users together, with similar privileges - follows the SQL
standard
CREATE ROLE audit_bean_counters;
GRANT SELECT ON accounts.* to audit_bean_counters;
GRANT audit_bean_counters to ceo;
MariaDB 10.0+
38. FusionIO
• If you have nvmfs (formerly DirectFS), you can disable the
innodb_doublewrite buffer
• page level compression in background threads (reduces I/O, saves
the life of your device)
MariaDB 10.0+
39. What else is there
• Engines: Aria, OQGRAPH, FederatedX
• Progress reporting for ALTER/LOAD DATA INFILE
• Table Elimination
• HandlerSocket
• SHUTDOWN functionality
• And a lot more….
41. Connectors
• The MariaDB project provides LGPL connectors (client libraries) for:
• C
• Java
• ODBC
• Embedding a connector? Makes sense to use these LGPL licensed
ones…
43. MariaDB Galera Cluster
• MariaDB Galera Cluster is made for today’s cloud based
environments. It is fully read-write scalable, comes with synchronous
replication, allows multi-master topologies, and guarantees no lag or
lost transactions.
• Currently 5.5-based
• 10.0 is in beta (almost ready for release)
44. Trusted by many
• Google
• Wikipedia
• Tumblr
• SpamExperts
• Limelight Networks
• KakaoTalk
• Paybox Services
45. Quality matters
• security@mariadb.org is now commonly on CC when it comes to
MySQL bugs
• Selective (not blind) merging
• Tests (mysql-test/)
• MySQL 5.5: 2,466
• MySQL 5.6: 3,603
• MariaDB 10.0: 3,812
46. Going forward
• column level & block level encryption (Eperi, Google - InnoDB, Aria)
• Kerberos authentication plugin
• Query timeouts
• More Google Summer of Code features (4 students 2014; 3 students
2013)
• Full 5.6 compatibility + 5.7 features (so syntax will match for
duplicated functionality)
48. Resources
• We moved to github! https://github.com/MariaDB/server
• We’re still on launchpad for older branches: https://launchpad.net/maria
• maria-discuss@lists.launchpad.net
• maria-developers@lists.launchpad.net
• #maria on freenode
• facebook.com/MariaDB.dbms
• @mariadb / +MariaDB