MySQL Cluster

MySQL Cluster
Developer(s) Oracle
Initial release November 2004
Stable release
7.4.8 / October 16, 2015 (2015-10-16)
Operating system Cross-platform
Available in English
Type RDBMS
License GNU General Public License (version 2, with linking exception) or commercial EULA
Website [1]

MySQL Cluster is a technology providing shared-nothing clustering and auto-sharding for the MySQL database management system. It is designed to provide high availability and high throughput with low latency, while allowing for near linear scalability.[2] MySQL Cluster is implemented through the NDB or NDBCLUSTER storage engine for MySQL ("NDB" stands for Network Database).

Architecture

MySQL Cluster is designed around a distributed, multi-master ACID compliant architecture with no single point of failure. MySQL Cluster uses automatic sharding (partitioning) to scale out read and write operations on commodity hardware and can be accessed via SQL and Non-SQL (NoSQL) APIs

Replication

Internally MySQL Cluster uses synchronous replication through a two-phase commit mechanism in order to guarantee that data is written to multiple nodes upon committing the data. (This is in contrast to what is usually referred to as "MySQL Replication", which is asynchronous.) Two copies (known as replicas) of the data are required to guarantee availability. MySQL Cluster automatically creates “node groups” from the number of replicas and data nodes specified by the user. Updates are synchronously replicated between members of the node group to protect against data loss and support fast failover between nodes.

It is also possible to replicate asynchronously between clusters; this is sometimes referred to as "MySQL Cluster Replication" or "geographical replication". This is typically used to replicate clusters between data centers for disaster recovery or to reduce the effects of network latency by locating data physically closer to a set of users. Unlike standard MySQL replication, MySQL Cluster's geographic replication uses optimistic concurrency control and the concept of Epochs to provide a mechanism for conflict detection and resolution,[3] enabling active/active clustering between data centers.

Starting with MySQL Cluster 7.2, support for synchronous replication between data centers was supported with the Multi-Site Clustering feature.[4]

Horizontal data partitioning (auto-sharding)

MySQL Cluster is implemented as a fully distributed multi-master database ensuring updates made by any application or SQL node are instantly available to all of the other nodes accessing the cluster, and each data node can accept write operations.

Data within MySQL Cluster (NDB) tables is automatically partitioned across all of the data nodes in the system. This is done based on a hashing algorithm based on the primary key on the table, and is transparent to the end application. Clients can connect to any node in the cluster and have queries automatically access the correct shards needed to satisfy a query or commit a transaction. MySQL Cluster is able to support cross-shard queries and transactions.

Users can define their own partitioning schemes. This allows developers to add “distribution awareness” to applications by partitioning based on a sub-key that is common to all rows being accessed by high running transactions. This ensures that data used to complete transactions is localized on the same shard, thereby reducing network hops.

Hybrid storage

MySQL Cluster allows datasets larger than the capacity of a single machine to be stored and accessed across multiple machines.

MySQL Cluster maintains all indexed columns in distributed memory. Non-indexed columns can also be maintained in distributed memory or can be maintained on disk with an in-memory page cache. Storing non-indexed columns on disk allows MySQL Cluster to store datasets larger than the aggregate memory of the clustered machines.

MySQL Cluster writes Redo logs to disk for all data changes as well as check pointing data to disk regularly. This allows the cluster to consistently recover from disk after a full cluster outage. As the Redo logs are written asynchronously with respect to transaction commit, some small number of transactions can be lost if the full cluster fails, however this can be mitigated by using geographic replication or multi-site cluster discussed above. The current default asynchronous write delay is 2 seconds, and is configurable. Normal single point of failure scenarios do not result in any data loss due to the synchronous data replication within the cluster.

When a MySQL Cluster table is maintained in memory, the cluster will only access disk storage to write Redo records and checkpoints. As these writes are sequential and limited random access patterns are involved, MySQL Cluster can achieve higher write throughput rates with limited disk hardware compared to a traditional disk-based caching RDBMS. This checkpointing to disk of in-memory table data can be disabled (on a per-table basis) if disk-based persistence isn't needed.

Shared nothing

MySQL Cluster is designed to have no single point of failure. Provided that the cluster is set up correctly, any single node, system, or piece of hardware can fail without the entire cluster failing. Shared disk (SAN) is not required. The interconnects between nodes can be standard Ethernet, Gigabit Ethernet, InfiniBand, or SCI interconnects.

SQL and NoSQL APIs

As MySQL Cluster stores tables in data nodes, rather than in the MySQL Server, there are multiple interfaces available to access the database:

MySQL Cluster Manager

Part of the commercial MySQL Cluster CGE, MySQL Cluster Manager is a tool designed to simplify the creation and administration of the MySQL Cluster CGE database by automating common management tasks, including on-line scaling, upgrades, backup/restore and reconfiguration. MySQL Cluster Manager also monitors and automatically recovers MySQL Server application nodes and management nodes, as well as the MySQL Cluster data nodes.

Implementation

MySQL Cluster uses three different types of nodes (processes) :

Generally, it is expected that each node will run on a separate physical host, VM or cloud instance (although it is very common to co-locate Management Nodes with MySQL Servers). For best practice, it is recommended not to co-locate nodes within the same node group on a single physical host (as that would represent a single point of failure).

Versions

MySQL Cluster version numbers are no longer tied to that of MySQL Server - for example, the most recent version is MySQL Cluster 7.4 even though it is based on/contains the server component from MySQL 5.6.

Higher versions of MySQL Cluster include all of the features of lower versions, plus some new features.

Older versions (no longer in development):

First 'telco' or 'carrier grade edition' release. Supports 255 nodes, online table alter, replication latency and throughput enhancements etc.
Includes compressed backup + LCP, circular replication support, conflict detection/resolution, table optimization etc.
Includes multi-threaded data nodes (ndbmtd), Transactional DDL, Windows support.
Includes ClusterJ and ClusterJPA connectors

Currently available versions:

Includes Adaptive Query Localization (pushes JOIN operations down to the data nodes), Memcached API, simplified Active/Active Geographic replication, multi-site clustering, data node scalability enhancements, consolidated user privileges.[6]
Includes support for foreign key constraints, Node.js / JavaScript API and an auto-installer.[7]
Includes enhanced conflict detection and resolution, improved node restart times, new Event API.[8]

Requirements

For evaluation purposes, it is possible to run MySQL Cluster on a single physical server. For production deployments, the minimum system requirements are for 3 x instances / hosts:

or

Configurations as follows:

Tips and recommendations on deploying highly performant, production grade clusters can be found in the MySQL Cluster Evaluation Guide and the Guide to Optimizing Performance of the MySQL Cluster Database.

History

MySQL AB acquired the technology behind MySQL Cluster from Alzato, a small venture company started by Ericsson. NDB was originally designed for the telecom market, with its high availability and high performance requirements.[9]

MySQL Cluster based on the NDB storage engine has since been integrated into the MySQL product, with its first release being in MySQL 4.1.

Competitors

Codership provides Galera, a multi-master extension for MySQL that enables multiple MySQL nodes to handle reads and writes, synchronizing the writes on each node simultaneously. Galera is also available through Percona called variously "Percona Cluster" and "XtraDB Cluster". Galera leverages a fork of the InnoDB storage engine and custom replication plug-ins. MariaDB, a MySQL fork, now ships MariaDB Galera Cluster as its cluster solution too.

ScaleBase is a database software company located in Newton, Massachusetts, that works with next-gen apps to scale out relational databases by virtualizing (auto-sharding) a distributed database environment.

Support

MySQL Cluster is licensed under the GPLv2 license. Commercial support is available as part of MySQL Cluster CGE, which also includes non-open source addons such as MySQL Cluster Manager, MySQL Enterprise Monitor, in addition to MySQL Enterprise Security and MySQL Enterprise Audit.

References

  1. ↑ Cluster CGE. MySQL. Retrieved on 2013-09-18.
  2. ↑ Oracle Corporation. "MySQL Cluster Benchmarks: Oracle and Intel Achieve 1 Billion Writes per Minute". mysql.com. Retrieved 24 June 2013.
  3. ↑ MySQL :: MySQL 5.6 Reference Manual :: 17.6.11 MySQL Cluster Replication Conflict Resolution. Dev.mysql.com. Retrieved on 2013-09-18.
  4. ↑ Synchronously Replicating Databases Across Data Centers – Are you Insane? (Oracle's MySQL Blog). Blogs.oracle.com (2011-10-03). Retrieved on 2013-09-18.
  5. ↑ Jon Stephens, Mike Kruckenberg, Roland Bouman (2007): "MySQL 5.1 Cluster DBA Certification Study Guide", pp. 86
  6. ↑ New features of MySQL Cluster 7.2, MySQL Developer Zone
  7. ↑ MySQL Cluster 7.3 is now Generally Available – an overview, MySQL Cluster 7.3 Summary
  8. ↑ MySQL Cluster 7.4 GA: 200 Million QPS, Active-Active Geographic Replication and more, MySQL Cluster 7.4 Summary
  9. ↑ Todd R. Weiss (2003-10-14). "MySQL acquiring data management system vendor Alzato". Computerworld.com. Retrieved 2012-11-05.

MySQL

Other

This article is issued from Wikipedia - version of the 11/27/2016. The text is available under the Creative Commons Attribution/Share Alike but additional terms may apply for the media files.