I started my blog to share more information about Couchbase Server and NoSQL mobile solution in local language (Japanese), but there are things going on in Japan and I thought it’s also beneficial to share those information in English.
So today, I would like to introduce use case of one of the top Japanese telecom company, KDDI, using Couchbase Server to expand their business on cloud platform.
This is the article I’ve shared in my blog a while ago, and I hope you will enjoy why NoSQL database (Couchbase Server) was chosen over RDBMS. I also would like to thank ASAHI INTERACTIVE, Inc. who authorised me to translate the article.
Below it the original Article written by Koji Kawamura, SE, Couchbase Japan KK:
*It is not a word-to-word translation of the original article above.
A while ago, Couchbase Japan had a talk with other NoSQL companies to discuss how we can cooperate together to expand the idea how people can use NoSQL database. So we decided to write articles about how different it is from relational database from each NoSQL database’s perspective (eg. HBase, MongoDB, Riak, Couchbase, Redis, Cassandra, and Neo4j).
In this blog, I would like to introduce one of the article which explains about Couchbase Server, a document oriented NoSQL database, along with customer use case how enterprise company applied Couchbase in their mission critical enterprise service.
One of the biggest telecom company, KDDI, launched a cloud service called “KDDI Business ID (http://www.kddi.com/business/security-managed/security/business-id/)” in summer 2014, and this cloud service is powered by Couchbase Server.
KDDI Business ID is an enterprise cloud service which is secured by incoming call and one time password. KDDI succeeded to simplify the system management in order to have a unified platform to manage user information for enterprise customers who use cloud service such as Google Apps for Work and Office 365 with KDDI. It is still usual to be thought that RDBMS is more suitable for such system to secure data. So let’s get down to reasons why Couchbase Server was chosen for such enterprise service which was crucial to have rigid requirements for security, performance and scalability to provide non-stop service to their enterprise customers.
**Flexible Data Model
First of all, KDDI had already decided to use agile development style in order to implement priorities in the service. So fast and flexible development was a top priority to achieve this project, and you can easily imagine that RDBMS is not the best fit for this requirement as it is hard to make changes in the process or after system is developed.
Couchbase Server was one of the candidates among other NoSQL databases, as it is well known as a flexible database which you can store both JSON or binary data very easily.
Couchbase Server is a Key Value Store (KVS) type database, and JSON can be stored in value of KVS. You can easily search or aggregate data from multiple JSON documents by using those values.
KDDI focused on NoSQL database as a solution for their project from the beginning as they knew they could not achieve what they wanted with RDBMS.
But that is still too weak of a reason to understand, especially it is the authentication service for enterprise companies.
So let’s look at another side of story.
**High Performance and Enterprise Level Reliability
Next focus was performance. As you know, read workload can be distributed with master-slave architecture with RDBMS, but master database needs to handle all write/update workload. If you need to keep consistency for replica read, then you also need to update replica data immediately. It means that you also need to sacrifice the performance of update.
In contrast, Couchbase Server is a evenly and horizontally distributed database with its auto sharding feature. Each data is evenly distributed across the cluster for update and read, thus Couchbase Server is well known as high performance database and used for systems which requires high through put.
In addition, the same key access will be read from the same node, and a single node can handle 100k to 200k requests per second, so you don’t need to read replica data to keep high performance. You can maintain high consistency for read, write, and update operation.
Another unique feature of Couchbase Server is that you can store cache for each document inside Couchbase Server. Document from application will be stored in cache layer of Couchbase Server first, and Couchbase Server sends back the cached data before writing to disk. Disk persistence and replication will be done asynchronously. This means that data store is archived with significant high throughput without depending on disk I/O performance.
However, we also understand that some enterprise companies hesitate to asynchronously write to disk to persist data. KDDI also needed to guarantee that data is written to disk when it is updated. In addition to the default operation mentioned above, you can also configure Couchbase Server to wait until data is written to disk or finish replication from application side.
In KDDI use case, documents are distributed and stored in multiple servers, and update data are written to disk, and also it was developed to create replica data for certain.
This shows that you can easily make a balance of performance and reliability with Couchbase Sever and it’s one of the benefits as well.
**Provide Non-Stop Service with Linear Scale Capability
As previously mentioned, KDDI Business ID is a secure cloud-based central authentication service for enterprise companies.
It is beneficial to have a cloud service as users can use the service anytime they want to buy, operate, or setup without complex work. However, service provider also needs to consider the increase of users or access numbers, or disasters in order to provide the service consistently.
Thus, KDDI had done a lot of research about databases including Couchbase Server to understand architecture, performance, operation, etc to handle such considerations of cloud.
Couchbase Server was highly evaluated for its linear scalability and maintain performance with simple one-click button to add server to cluster, scale cluster and maintenance without downtime, and is capable of scaling cluster with growth of service.
Couchbase Server returns the data on its memory when there is a read operation, so it makes it possible of less than 1mil second latency for data get.
You can easily configure the size of cache on memory, and if there is not much space left, then an infrequently read doc, which is already stored in disk, will be automatically removed from cache.
If there is still a need of memory space, or disk write operation cannot catch up the write operation to cache, then adding more nodes to cluster is another solution. You can easily redistribute data by adding more nodes without downtime or stop service.
Thanks to the above reasons, you don’t have to worry about stopping the service due to software update or hardware maintenance during operation as Couchbase Server lets you remove and add nodes to cluster without downtime.
With Couchbase Server, you can easily shrink or scale cluster without considering data increase or decrease and provide always consistent high performance no matter how many users or access you have as Couchbase Server provides online scalability which is hard to do with RDBMS.
**24×7 Operation with High Availability
It doesn’t matter whether the service is for consumers or enterprise users, but it’s always a question and primary concern for service providers how they can manage to provide always-on application which increases business opportunities and decrease business loss at the same time.
In KDDI Business ID service, KDDI stores user ID data in Couchbase Server and authenticate users and manage authentication token. This service was highly required to have high availability especially for disaster recovery as they have to provide cloud service such as Google Apps for Work or Office 365 with KDDI.
In order to meet such severe requirement, KDDI chose two methods with Couchbase; one is to replicate inside cluster, and another is using cross datacenter replication (XDCR) to synchronize multiple clusters in data centers in eastern and western Japan.
Thus, KDDI has achieved to develop a system which can even bear a data centers level disasters.
XDCR makes it possible to achieve secure communication with SSL encryption even with public network, and handle problems such as recovery from network shutdown, and band reduction differently compared to cluster replication method.
It is very important to choose the right solution with high availability, but you also need to test candidate solutions in several scenarios you can think of in real world.
KDDI had done testing with many scenarios such as single node disaster recovery, data centers level disaster recovery, and any disaster recovery scenarios they could think of to compare their service SLA with recovery operation, and they have successfully made efficient recovery model for the system.
Couchbase Server passed all the tests, and now is in operation to support authenticate infrastructure.
Moreover, it is possible to recover servers with failover feature when there is server failure.
Another option to avoid server failures is to configure a group such as rack or zone so that replica data will be stored in different node which does not have master data. This feature is only available to enterprise edition but you can configure rack or zone within a cluster to ensure the level of disaster recovery you want.
**Ease of Operation
This cannot be simply compared to RDBMS, ease of operation is another benefit of Couchbase Server.
Couchbase Server can be used to install binary package in Linux or Windows. It is a distributed system, but you only need to install one package. It is also easy to configure Couchbase Server cluster to add, remove, rebalance nodes, and also configure XDCR (unidirectional or bidirectional) from GUI.
You can also monitor aggregate information of cluster which makes it easy for you to understand resource usage or data increase/decrease so that you can predict when is the right time to add another node to scale your cluster.
Operations from GUI are also available from command line or REST API.
By using these APIs, you can even configure an automatic scaling which cooperates with cluster condition, then add a new server to a cluster and rebalance data across the cluster.
From the perspective of this ease of operation, Couchbase Server might give you another benefit for you to decrease the operation cost compared to RDBMS.
KDDI is automating cluster environment setup. There might be an honest mistake if multiple servers are structured one by one.
So KDDI is taking an advantage of Couchbase Server command line to enable automatic structure for clusters located in east and west data centers.
KDDI also uses cluster monitoring to send alert when there is disaster or server failure.
Couchbase Server makes it possible to monitor cluster condition and log data, and which enables you to take an immediate action when needed.
It is important to ensure disaster recovery with Couchbase Server, but also with Couchbase monitoring system, you need to construct a framework for disasters or traffic increase in order to provide enterprise level service with secure environment.
Couchbase Server also provides rich APIs which are needed to cooperate with operation management tools.
We often hear a question about transaction from people who new to NoSQL.
When we say there is no transaction feature with Couchbase Server, then those people might think that Couchbase Server cannot be used as a database.
It might a serious problem for engineers who have been operating only with RDBMS.
However, we would like to question whether transaction really a necessary feature for all systems today. In KDDI Business ID use case for user authentication system, it was more important to have high scalability and flexibility which NoSQL is more appreciated, rather than transaction with rigid schema model.
Of course, NoSQL does not mean a denial of RDBMS, as it stands for “Not Only SQL”, and there are situations that RDBMS are best fit although NoSQL is now becoming a sensation to solve today’s difficulties such as big data, big users, or IoT.
RDBMS still makes strong showing in financial systems and a system which requires ACID property. Data validation with rigid schema is also another benefit of RDBMS.
I strongly believe that it is more important to understand the difference between NoSQL and RDBMS, and which solution is the best for which use case.
This series of NoSQL vs RDBMS articles introduces only a few NoSQL databases, and there are a lot of use case even only with Couchbase Server. KDDI use case (use ID store) is just an example how to leverage Couchbase Server. Other use cases are real time big data which is hard to handle with RDBMS, and IoT (Internet of Things) with our NoSQL mobile solution (http://www.couchbase.com/nosql-databases/couchbase-mobile).
So as you can see, Couchbase Server and mobile provide wide range of solutions.
I hope this article will help you to understand Couchbase Server more and the difference between RDBMS. Thank you very much for reading and please install Couchbase Server if you haven’t! (http://www.couchbase.com/nosql-databases/couchbase-server)
You can also find tech info from Koji’s and other engineers’ blogs below.
Koji’s blog in Japanese: http://ijokarumawak.github.io
Couchbase blogs (English): http://blog.couchbase.com
There also other resources like webinars (http://www.couchbase.com/nosql-resources/webinar) and whitepapers (http://www.couchbase.com/nosql-resources/nosql-whitepapers) you can learn about Couchbase Server and Couchbase mobile.
If you are located in U.S.A, then there will be a conference “Couchbase Connect 2015” (http://www.cvent.com/events/couchbase-connect-15/event-summary-b7744ca960364b75aba41de42cbef19e.aspx?page=overview) in Santa Clara, CA. You can learn a lot more about Couchbase Server and mobile as there are a lot of user sessions.
Please come and join us to connect with COUCHBASE from June 2-4th in Levis’ Stadium! 🙂