6 Tips To Make Your OpenStack Enterprise Ready.

 

How to Make Your Openstack Environment Enterprise Ready. 6 Tips.

At a baseline, let’s first come to an agreement of what “Enterprise Ready” means. As a storage consultant and IT generalist with a specialty in cloud architecture, I would define enterprise ready as an environment with the following characteristics:

Predictable

No surprises here: we know and understand the environment’s behaviors during any stress point.

Available

Availability, measured in uptime, indicates how many nines are supported and in general the practices that need to be in place to guarantee a highly available environment.

Fast

The performance of the environment should be dependable and we should be able to set clear expectations with our clients and know which workloads to avoid.

Well Supported

There should be a help line with somebody reliable to back you up in knowledge and expertise.

Expandable

We should know where we can grow and by how much.

Low Maintenance

The environment should so low-maintenance as to be a “set it and forget it” type of experience.

How to Get There: Artificial Intelligence

Now that we know the characteristics and their meanings, the question is, how do we make our open source environment enterprise ready? Let’s take it one at a time. Hint: artificial intelligence can help at every turn.

Predictable

To make your OpenStack environment enterprise ready, you need to perform a wide range of testing to discover functionality during issues, failures, and high workloads. At KIO Networks, we do continuous testing and internal documentation so our operations teams knows exactly what testing was done and the environment’s behavior.

Artificial Intelligence can help by documenting historical behavior and predicting potential issues down to the minute that our operations team will encounter an anomaly. It’s the fastest indication that something’s not running the way it’s supposed to.

Available

To test high availability, we perform component failures and document behavior. It is important to fail every single component including hardware, software, and supporting dependencies for the cloud environment like Internet lines, power supplies, load balancers, and physical or logical components. In our tests, there are always multiple elements that fail and are either recovered or replaced. You need to know your exposure time: how long does it take your team to both recover and replace an element.

AI-powered tools complement traditional monitoring mechanisms. Monitoring mechanisms need to know what your KPIs are. From time to time you may encounter a new problem and need to establish a new KPI for it alongside additional monitoring.  With AI, you can see that something abnormal is happening and that clarity will help your administrators hone in to the issue, fix it, and create a new KPI to monitor. The biggest difference with an AI-powered tool is that you’re able to do that without the surprise outage.

Fast

Really, this is about understanding speed and either documenting limitations or opting for a better solution. Stress testing memory, CPU, and storage IO is a great start. Doing so in a larger scale is desirable in order to learn breaking points and establish KPIs for capacity planning and, just as important, day-to-day monitoring.

Do you know of a single person who would be able to manually correlate logs to understand if performance latency is improving based on what’s happening now compared to yesterday, 3 weeks ago, and 5 months ago? It’s impossible! Now, imagine your AI-powered platform receiving all your logs from your hardware and software. This platform would be able to identify normal running conditions and notify you of an issue as soon as it sees something unusual. This would happen before it hits your established KPIs, before it slows down your parallel storage, before your software-defined storage is impacted, and before the end user’s virtual machine times out.

Well Supported

We emphasize the importance of continuously building our expertise in-house but also rely on certain vendors as the originators of code that we use and/or as huge contributors to open source projects. It’s crucial for businesses to keep growing their knowledge base and to continue conducting lab tests for ongoing learning.

I don’t expect anyone to build their own AI-powered platform. Many have done log platforms with visualization fronts, but this is still a manual process that relies heavily on someone to do the correlation and create new signatures for searching specific information as needed.  However, if you are interested in a set of signatures that’s self-adjusting, never rests, and can predict what will go wrong, alongside an outside team that’s ready to assist you, I would recommend Loom Systems. I have not found anything in the market yet that comes close to what they do.

Expandable

When testing growth, the question always is, what does theory tell you and what can you prove? Having built some of the largest clouds in LATAM, KIO knows how to manage a large-volume cloud, but smaller companies can always reach out to peers or hardware partners to borrow hardware. Of course, there’s always the good, old-fashioned way: you buy it all, build it it all, test it all, shrink it afterwards, and sell it. All of the non-utilized parts can be recycled to other projects. Loom Systems and its AI-powered platform can help you keep watch over your infrastructure as your human DevOps teams continue to streamline operations.

Low Maintenance

Every DevOps team wants a set-it-and-forget-it experience. Yes, this is achievable, but how do you get there? Unfortunately, there’s no short cut. It takes learning, documenting, and applying lessons to all of your environments. After many man hours of managing such an environment, our DevOps team has applied scripts to self-heal and correct, built templates to monitor and detect conditions, and set up monitors to alert themselves when KPIs are being hit. The process is intensive initially, but eventually dedicated DevOps teams get to a place where their environment is low maintenance.

The AI-powered platform from Loom Systems helps you by alerting you of the unknown. Your team will be shown potential fixes and be prompted to add new fixes. As time goes by, the entire team will have extensive documentation available that will help new or junior admins just joining the team. This generates a large knowledge base, a mature project, and also a lower-maintenance team.

All serious businesses should enjoy the benefits of running a predictable, highly available, fast, well supported, easily expandable and low-maintenance environment.  The AI-powered platform built by Loom Systems takes us there much faster and gives us benefits that are usually reserved for huge corporations. Just as an example, if you’re the first in the market offering a new product or service, you can feel confident with Loom Systems that they’ll detect problems early and give you actionable intelligence so you can fix them with surgical precision.

It’s been a pleasure sharing my learnings with you and I look forward to hearing your feedback. Please share your comments and points of view – they’re all welcome!

 

Best Regards,

Julio Calderon

Twitter: @JulioCUS

Skype: Storagepro

Email: juliocus@gmail.com

Advertisements

COPY DATA MANAGEMENT

IS COPY DATA MANAGEMENT A REPLACEMENT FOR BACKUP? (EASY!)

I invite you to read up the article below I included the source and all of the article for your reference.

Data has deserved a new way of management and an entire overhaul for a long time now. Copy data management as a subject is one of those new ways. However it is still lacking a few pieces.

First, what is copy data management?

You have data, that data requires copies, these copies are for multiple purposes. One of those purposes is backup, redundancy, test and Dev, QA.  Each of these types of copies of data usually have a specific requirement.

Let’s review some of these requirements as examples:

QA = quality assurance… Depending on the applications, clients require same IO performance when spinning up a copy for QA. Given that this is not a Functionality type of QA only.

              Backup ….  Clients ask that they need to be able to search the objects being protected and granularly restore that object.

             Test and Dev = Testing and Development, same as QA, same IO Performance capability is required as response time on applications can come under test.

 Redundancy = Replication, some tight RTOs and RPOs near zero are sometimes required. This is a live or replica copy for redundancy purpose and also requires same IO performance.

The graphic above was found in the actifio site a few months ago.

Let’s remember that copies also lie in the mobile devices, laptops, and desktops.

Furthermore; as the article below mentions, be aware of copy data management limitations.

There are companies that have a LONG history in the market and do copy data management already but do not market themselves in this way.  My suggestion is, get with the program CMOs!

Quick example: Netapp! And no, there is no PERFECT copy data management platform! No matter what anyone tells you, there is no perfection.

When you look at copy data management platforms ask the following questions and see if they meet your requirements.

  1. What is the shortest RTOs and RPOs?
    1. So you can address your toughest issues if you can address your toughest issue by default you will able to address any less critical issue as far as RTO and RPOs are concerned.
  2. Is data being separated from live environment onto a protection environment?
    1. Best practice to protect data is to separate platforms
  3. For this critical application, how many copies of data will you eliminate?
  4. This copy data management platform, how many copies of data can it hold for this particular critical application?
  5. Considering other less critical applications and in combination with this critical app, How many copies of data can you hold in this copy data management platform?

My personal recommendation is, know your current storage platforms first. Do they take snapshots? Can it replicate snapshots to a different tier of storage on a separate controller? Can that controller or tier of storage give you the IO you require? Do you have enhanced snapshot management tools that quiescent the DB or application so you capture a restore point in an application consistent form? How many snapshots can you take?

You might find that you are already in possession of a copy data management capable technologies you just needed a lite help in identifying it!
Actifio was the first to talk about Copy Data Management as a subject then followed HDS.

http://datatrend.com/optimize-it/what-is-copy-data-and-why-is-it-a-problem/

http://www.hds.com/products/storage-software/hitachi-data-instance-manager.html

 

– – – Start of article info – – – –

I found the following article published at: http://searchdatabackup.techtarget.com/answer/Is-copy-data-management-a-replacement-for-backup?utm_medium=EM&asrc=EM_ERU_42220875&utm_campaign=20150427_ERU%20Transmission%20for%2004/27/2015%20(UserUniverse:%201490131)_myka-reports@techtarget.com&utm_source=ERU&src=5383111

Is copy data management a replacement for backup?

Copy data management can be a backup replacement in certain situations, but organizations need to be aware of its limitations.

A number of vendors have copy data management offerings, and each has its own philosophy about how the technology should be used. That said, copy data management is more about storage efficiency than data protection.

Generally speaking, redundancy accounts for a large percentage of an organization’s storage cost. According to some estimates, the average enterprise creates eight to 10 copies of every production data source. These redundant copies are used for a wide variety of purposes, including dev/test, support, end-user training, and reporting. If these estimates are true, a relatively modest 100 GB database could ultimately account for up to a terabyte of storage consumption. When you factor in the number of production data sources that exist within the average enterprise environment, you can begin to see how quickly redundant data can consume the available storage.

Copy data management allows everyone to work from a common copy of a data source. Rather than a development team making a full-blown copy of a production database, the copy data management software might instead use snapshot technology to provide the development team with an isolated development environment that perfectly mimics the production environment. In other words, the dev team is using the production database, but in a way that protects the integrity of the production data.

Although copy data management could conceivably be used to create data recovery points, the software never creates a true backup copy of the data source. Any redundancy exists only at the storage level. With the proper level of redundancy, copy data management might be able to act as a backup replacement, but it is not a good solution for organizations that require an offline (tape) copy of their data.

This was first published in March 2015

      – – – End of article info – – – –

Hope you enjoyed my comments and recommendations. I urge you to contact KIO Networks and explore how we can help you reduce copies of data in your environment independent from any technology sometimes, new practices can help you streamline a bunch!
Have a wonderful week.

Regards,

Julio Calderon

@JuliocUS Email: juliocus@gmail.com Skype: Storagepro

Data Virtualization and Strategy

Do you have a storage strategy? Are you getting the most of all your storage platforms? Are you underspending with substandard results or are you over spending with AOK results?

There are multiple forces driving you to a choice and to help you narrow down to the basic aspects here is a picture.   In the center we have our users. Our users want it all, Reliability, Performance and Low Cost.  Well, low cost, highly available and high performance storage doesn’t exist! You will have to choose a compromise between these key aspects.  In a nutshell you have to choose how much of each aspect you get and there is no way to get all for low cost.

In this triangle we show 3 layers of technologies, Media, Access Type and Features on storage. In the media type as we change media we lower costs. In the Access layer we show some basic types you will recognize as NFS, SMB or CIFS, WebDav, S3, Block storage over FC or over iSCSI. Yes, there are many more as block over InfiniBand and more protocols as FTP, SFTP. In the storage feature layer we start to see levels of data protection and higher availability as local or remote snapshots, replication, offsite backups and storage clustering there are RAID levels and # of controllers that increase availability as well.

To dig deeper into the media type, lest look at the technologies invloved. For the highest performance we have Flash, SSD, then we lower performance with SAS, SATA and finally the lowest performance in Object storage. As we lower performance we also lower costs. These are just general examples, we all know that there are other type of media as Tape with LTFS that could lower costs further. You as a storage administrator would have to choose what to give your user given that you actually have it.

So, if you provide storage as a service then you most likely already have a compromise between all aspects, what you might call the sweet spot.  However, as this is a sweet spot of all key aspects it is not responding the wide spectrum of needs. Some workloads require super high availability, replication, clustering 100% uptime while others require low cost via Object Storage and then there is the LOCAL Flash cards for high performance needs. After all most high performance and requirements of IO come from local workloads like TMP, SCRATH and SWAP (after all, this data does not require high durability as it is recreated on the fly as the OS and applications run)

What we need is a solution that does not force you into a compromise. Much easier said than done. Example below.

As you can gather from the graphic, via a single point of access you receive a slice across 3 aspects, Reliability, Performance and Cost.  If you provide storage solutions you know this is very expensive and complex because you have to acquire, configure, manage all the technologies previously covered and you must then choose a level of performance with data protection and reliability and finally the cost is associated to each platform and configuration.

So, what is out in the world to help you with this? Well, take INFINIDAT, a centralized storage platform with higher levels of redundancy and good performance.  Look at PureStorage an all SSD solution running in deduplicated mode with good density and TCO across a longer than usual period of time, you should also look at SolidFire as they can assign # of IOPs to the storage you provide helping you align performance to workload, then look at ExtremeIO from EMC great performance, Netapp for an all-inclusive feature and function of data storage and data protection by far the most mature in the market for combining storage and protection. Look at new players as Actifio, Rubrik for copy data management and the previous block storage virtualization engines as IBMs SVC or the Virtualization engine from HDS also OEM by HPE.  There are many more technologies that I am leaving out.  The list is HUGE! And your time could be consumed just in learning from each one of these.

So, next time you look for storage on demand look at KIO Networks. In the spirit of help, you learn something new I like to introduce a new subject that promises to help businesses align the right storage to the application at the right price.

There is storage player out in the market promising to address this complex challenge. Its called Primary_Data

What will data virtualization do for you?

The simplest term, allow you to move data heterogeneously across multiple platforms.

Hope you enjoy the set of videos I know I did.

The Launch of Primary_Data

 

Introduction to Primary_Data

 

What is data virtualization: I think the video is over complicated but good video.

 

Use cases for Data Virtualization


Primary_Data User Experience demo

 

Vsphere Demo

 

Hope you enjoyed the videos, as with all cool technology, the devil in the details. From my side, I will make sure to take a deep dive of the technology and report back to you guys.

Have a great week!

Julio Calderon  Twitter: @JulioCUS  Skype: Storagepro email: juliocus@gmail.com

Replication! Anyway you want it from Rubrik. Come read about it.

This article was posted by a great profesional in the data protection arena. Rolland Miller, his info can be found here: http://www.rubrik.com/blog/author/rolland-miller/

Rolland is currently investing his time at Rubrik I am sure they will absolutely make it big! Here is the original post location:  http://www.rubrik.com/blog/unlimited-replication-with-rubrik-2-0/#.VdXN-vu9esk.linkedin

— Start of Snip Original Post —

Today we announced Rubrik 2.0, which is packed with exciting new features. I’ve been working in the storage industry for the past 16 years, the majority of time spent working on backup and DR solutions for companies. This isn’t my first rodeo–I’ve seen a lot, advised a lot of customers in how to architect their backup and disaster recovery infrastructure. Needless to say, I haven’t been this thrilled for a long time—our engineers are building something truly innovative that will simplify how recovery is done on-site or off-site for the better.

Why is Rubrik Converged Data Management 2.0 so interesting?

Our 2.0 release is anchored by Unlimited Replication. There are no limits to how many snapshot replicas you can have. There is zero impact on your production systems as replication occurs since this isn’t array-based replication. This is asynchronous, deduplicated, masterless, SLA driven replication that can be deployed any way you like, many-to-one, many-to-many, one-to-one, uni-directionally or bi-directionally. In the past, replication has always been engineered with a master-slave architecture in mind because systems have always had an active-passive view of control. Our Converged Data Management platform is fundamentally a distributed architecture that allows you to share nothing, but do everything—each node is a master of its domain. Our engineers apply the same building principles to replication. Hub and spoke? Check. Bi-directional? Check. Dual-hub, multi-spoke, and archived to the cloud. Check! Check! Check!A key property of Converged Data Management is instant data access. Data is immediately available, regardless of locality, for search and recovery. Using Rubrik for replication allows you to recover directly on the Rubrik appliance since applications can be mounted directly. Files can be found instantly with our Global Real-Time Search. There’s no need to copy files over to another storage system. We’ll give you near-zero RTO.

In this release, we extend our SLA policy engine concept into the realm of replication. You can define near-continuous data replication on a per-VM basis within the same place that backup policies are set. There’s no need to individually manage replication and backup jobs—instead, you’ve freed up your time by managing SLA policies instead of individual replication targets. Once you specify a few parameters, the engine automates schedule execution. For more on managing SLA policies, see Chris Wahl’s Part 1 and Part 2 posts.

No SLA policy is complete without measurement. In 2.0, we’re releasing beautifully simple reporting that helps you validate whether your backup snapshots are successful and whether they’re meeting the defined SLA policies. Our reporting will help you keep an eye on system capacity utilization, growth, and runway—so you’ll never be caught short-handed.

A New Addition to the Family

Finally, we’re welcoming the new r348, our smartest dense machine yet. We’re doubling the capacity within the same 2U form factor, while maintaining the fast, flash-optimized performance for all data operations, from ingest to archival.

Catch Us at VMworld

In less than two weeks, we’ll be at VMworld. Make sure to stop by our Booth 1045 to see a live demo. Arvind “Nitro” Nithrakashyap and Chris Wahl will be leading a breakout session on Wednesday, 9/2 at 10 am and giving away epic Battle of Hoth LEGO sets.

— End of Snip Original Post —

WOW!  Let me quote Rolland, “There is zero impact on your production systems as replication occurs since this isn’t array-based replication. This is asynchronous, deduplicated, masterless, SLA driven replication that can be deployed any way you like, many-to-one, many-to-many, one-to-one, uni-directionally or bi-directionally.”

WOW! KICK ASS! Love it. If you are a techno-geek like I am you will be as super excited about this as I am.

Just imagine the ramification? This could be a true base platform for a service provider. Without limits, you really could service a huge pool of clients and their needs.   Obviously, you still need size properly for your forecasted use and growth. Anyways, hopefully, I can join you all at VMworld where I am sure Rubrik will WOW all of you!

Enjoy!

Regards,

Julio Calderon / @JuliocUS /email: juliocus@gmail.com