Monday, April 24, 2017

Deep dive on AWS vs. Azure vs. Google cloud storage options

One of the most common use cases for IaaS public cloud computing is storage and for good reason: instead of buying hardware and administering, users simply load the data into the cloud and pay what they put in their place .

It sounds simple. But in reality, the world storing in the cloud has many facets to consider. Each of the top three public cloud providers IaaS - Amazon Web Services, Microsoft Azure and Google Cloud Platform - has a variety of storage options and, in some cases, complicated diagrams for how much.

According to Brian Adler, the company's architecture director at CloudScale, cloud management provider, who recently conducted a seminar comparing storage options in the cloud, there is clearly clearly better than other providers. "Is anyone in mind? It really depends on what is being used (the cloud)," he said. Each provider has its own strengths and weaknesses as the specific use case, he said. Three cases of the most popular use cloud storage and how providers accumulate.

Block storage

The storage drive is a persistent disk storage used in conjunction with virtual machines based on the cloud. Each of the suppliers break their block storage offerings into two categories: traditional dynamic rotating magnetic hard drives or the latest static (SSD) drives, generally more expensive but with better performance. Clients can also pay a premium to get a certain amount of I / O guarantees per second (IOPS), which is essentially an indication of how fast they will back up new log information and read the information stored on it.

The product is called Amazon Elastic Block Store (EBS) and comes in three main flavors: HHD optimized performance, featuring traditional magnetic disc and spinning; General purpose SSD new generation of readers; And provisional SOPI IOPS, which offers a guaranteed rate of read and write data.

The Azure storage offer blocks called managed disks and is available in standard or premium with the latter based on the SSD.

The version of Google called Persistent Disks (PDS), which is in a standard option or SSD.



AWS and Google have 99.95% availability, while Azure offers a 99.99% service level agreement (SLA) for the bulk storage service.

One of the most important factors to consider when buying storage units is the speed with which you need to access the data stored on the SSD. For this, providers offer different rates guaranteed PIO. Google is in the lead; The company offers 40,000 IOPS to read and 30,000 to write in their records. The AWS general purpose SSD offers 10,000 IOPS, but its offering provides the IOP can offer up to 20,000 IOP example, with a maximum IOP of 65,000 by volume. Azure provides 5000 IOP.

Google not only has the highest IOP, but offers customers the widest range in the size of block storage volumes. For a more traditional hard drive based storage, Google offers volume sizes ranging from 1GB to 64TB. AWS offers volumes from 500 GB to 16 TB. Azure offers volume sizes of 1GB and 1TB. As for SSDs, Google offers the highest level of IOP on hard drives by volume in 3000 for reads and 15 000 for writing. AWS and Azure are 500 GPI max. In terms of volume. Azure maximum rates are 60 MB Google 180 for reading and 120 for writing, and AWS 500 MBps.





As for prices, it becomes a bit complicated (all prices are per GB / month), but for HHD, AWS starts at $ 0.045, Google is $ 0.04 and Azure is $ 0.03.

The SSD price starts at $ 0.10 in AWS, $ 0.17 for Google and between $ 0.12 and $ 0.14 for Azure, depending on the size of the drive.

In a price analysis conducted by RightScale, the company found that, generally, the pricing structure means that Azure has the best price / quality ratio for block storage. But, for workloads requiring higher IOPs, Google becomes the most profitable option.

There are reservations when using the provisioned IOPs, says Kim Weins, vice president of marketing at RightScale. In AWS, if you need a guaranteed amount of IOP, it costs a premium. "You pay a higher cost per GB, but you also pay the required IOPs in addition, which results in a higher cost," said Weins. "Be smart about choosing your IOP level supported because you're going to pay."

Weins adds that RightScale found that some customers paid for IOPs and then forgot to unprotect the EBS instance when they finished using it, which cost money.

Storage of objects

Do you have a file that you need to put in the cloud? Object storage is the service for you. Again, cloud providers have different types of storage, classified by the frequency at which the customer expects to access them. "Warm" storage is a data that must be almost instantly accessible. "Cool" storage is more rarely available, and cold storage is an archival material that is rarely accessed. The colder the storage, the less expensive it is.

The primary storage platform for AWS objects is Simple Storage Service (S3). It offers S3 Infrequent Access for a cool storage and Glacier for cold storage. Google has Google Cloud Storage, GCS Nearline for cool storage and GCS Coldline for archiving. Azure has only one hot and cool option with hot and fresh Azure storage drops; Clients must use the cool storage for archive data. AWS and Google each have an object size limit of 5 TB, while Azure has a limit of 500 TB per account. AWS and Google each release 99.999999999% durability for objects stored in their cloud. This means that if you store 10,000 objects in the cloud, on average, a file will be lost every 10 million years, according to AWS. The goal is that these systems are designed to be ultra-durable. Azure does not publish sustainability service level agreements.



Prices on storage of objects are slightly more complicated because customers can choose to host their data in a single region, or at a slightly increased cost, they can save it in several regions, which is an optimal practice for you Ensure access to your data If there is a breakdown in a region.

In AWS, for example, costs S3 (all prices are in GB / month) $ 0.023; To replicate data across multiple regions, costs twice as much: $ 0.046, plus a transfer fee of $ 0.01 per GB. AWS's S3 Infrequent Access (AI) storage service is $ 0.0125 and its Glacier cold storage / archiving service costs $ 0.004

Google has the most similar offers: its cost of storage at a single region costs $ 0.02, while the multi-region is $ 0.026, with a free data transfer. The company's cool storage platform, Nearline, is $ 0.01 and the Cold / Archival Coldline product is $ 0.007. Google states that Coldline's data recovery is faster (in milliseconds) than in Glacier, which according to AWS could take between minutes and hours.

Azure offers one-region storage for $ 0.0184, and what it calls "Global Redundant Storage" for $ 0.046, but it's read-only, which means you can not write changes, which means Costs more. The cool storage of Azure is called Cool Blob Storage is $ 0.01. Azure does not yet offer a cold or archival storage platform, so customers must use the Cool Blob storage for this use case.

Based on these price scenarios, Google has the cheapest storage costs of pure objects plus the free data transfer, RightScale found. However, AWS beats Google's cold storage costs.

Storage of files

An emerging use case is the use of a cloud-based file storage system. Think of it as a cloud-based version of a more traditional network file system (NFS): users can mount files on the system from any connected device or virtual machine, and then read and recover Files. This is a case of relatively nascent cloud storage use and, as a result, offers are still incomplete compared to storage of blocks and objects, according to Adler.

AWS 'offer in this category is called Elastic File Storage, a beta version of June 2016. It allows users to mount files from AWS Elastic Compute Cloud (EC2) or local Using AWS Direct Connect or a Virtual Private Connection (VPC). There is no size limit, so it varies automatically according to needs and offers a throughput of 50 MB per second per TB of storage; Customers can pay up to 100 Mbps throughput. It starts at $ 0.30 / GB / month.

Azure, on the other hand, offers Azure File Storage, of a similar nature, but has a capacity of 5 TB per file and 500 TB per account and requires manual scaling. It offers a throughput of 60MBps to play files.



Google does not have a native file storage offering, but offers the open source FUSE adapter, which allows users to mount files from Google Cloud Storage buckets and convert them to a file system. Google says it provides the highest throughput of the three vendors with 180MBps read and 120MBps on writes. But Adler said in his experience that the FUSE adapter is not as well integrated into Google's cloud platform as compared to the other two offerings, resulting in potentially frustrating experiences with users. Adler also notes that AWS EFS does not have a native backup solution, while Azure does. AWS encourages EFS users to rely on third-party backup tools at this point.

Azure and Google offer lower prices for their file storage systems compared to AWS: Azure is $ 0.80 per GB / month and Google is $ 0.20, but Adler says these costs do not account for Replication or transfer costs. While the AWS base price might seem higher, taking into account all it affects scaling, it could be a wash between the three vendors .

Monday, April 10, 2017

AWS now lets you migrate MongoDB databases to DynamoDB




The public cloud infrastructure provider Amazon Web Services (AWS) today announced an upgrade to its database migration service (DMS). Now, people can transfer their databases into the NoSQL MongoDB open source database in the NoSQL service managed by DynamoDB owning AWS with the help of DMS.

In fact, DMS now supports the migration of NoSQL databases in general, AWS said in a blog post. This suggests that more NoSQL databases could get official DMS support in the future. Currently, DMS can work with Oracle databases, Microsoft SQL Server, MySQL, Amazon Aurora, PostgreSQL and SAP ASE, said Amazon.

AWS presented the DMS and the compatible schema conversion tool in 2015. In December, AWS CEO Andy Jassy said that DMS had made 16,000 migrations in 2016. In total, More than 22,000 migrations, Jassy said in a tweet last month.

In February AWS announced that the schema conversion tool could take data from Oracle and Teradata data warehouses and prepare it for installation in the AWS Redshift data storage service.

MongoDB was once a very trendy technology among developers. The company behind it, also called MongoDB, offers a managed version of the database hosted on AWS. Now, AWS will be able to generate revenue where organizations had previously sought to use MongoDB for databases in their on-site data centers. In other words, AWS is now challenging its own client, and this is not the first time it's happening .