sorted by: recent | see :
popular
Content Tagged joukovski
New version of ZRM expands platform coverage, adds backup of application files
Today we released version 3.0 of Zmanda Recovery Manger for MySQL. Here are the highlights of the release.
The new version of ZRM supports Zmanda Management Console (ZMC) running on 32- and 64-bit Ubuntu and Debian. We received quite a few requests for providing such support in the past.
In addition to Linux and Solaris, the ZRM server can now run on Windows. We supported logical and VSS-based backup for MySQL on Windows for a long time, but sometimes it was a challenge to introduce ZRM running on Linux/Solaris to all-Windows shops. Based on feedback from customers, prospects and from our partner MySQL, we have developed the version of ZRM that can protect MySQL in all-Windows environments.
We expanded the number of supported snapshot technologies by adding EMC CLARiiON SnapView to the list. This mid-range storage array is a staple for many data centers. Now the users who run MySQL on CLARiiON, will be able to take advantage of very fast and space-efficient snapshot capabilities of SnapView.
Lastly, based on requests of many users, the new version of ZRM provides the capability of doing backup of files at the same time as backup of MySQL database. For example, you can backup the directory and configuration files of application that is powered by MySQL. That simplifies recovery of a database and the application that uses MySQL.
——–
Dmitri Joukovski
Three C of Oracle backup: Cost, Complexity and Control
Backup continues to be a challenge for Oracle DBAs. That message was repeated again and again by attendees of OracleWorld in San Francisco last week.
The DBAs consitently bring up three challenges of Oracle backup: Cost, Complexity and Contol
Cost
For many organizations the Oracle modules for Veritas NetBackup and Legato NetWorker are out of reach because of high cost. According to Sun ’s online store, the list price for NetBackup Oracle module running on a Solaris server with 2 CPUs is $7,500. What if you have multiple Oracle servers to backup? This can add up very quickly.
Complexity
To avoid paying unreasonable licensing and maintenance fees for NetBackup and Legato, many DBAs write their own RMAN scripts. RMAN is a well documented tool and it seems relatively easy to write a few lines to schedule backup of a single database. However, over time the script gets more complicated, configurations and requirements change, a DBA who wrote the original script gets promoted and moves to another group … and you are stuck trying to figure out when was the last successful backup and how to get your data back.
Control
Who do you depend on to get your data back? In most organizations the roles of DBA and backup administrators are separate. You, as a DBA are paid to protect Oracle data, but often times you have to ask the backup guys to get your data back. The lack of control could be even worse. As soon as your organization starts using NetBackup (or any other backup that keeps data in proprietory formats), your data becomes hostage to Veritas software. You can never recover your own Oracle or any other backups unless you continue paying Veritas.
If you face any of the chalenges above, check out Amanda Enterprise for backup of Oracle. For a low cost you will get an easy to use solution that provides you with a full control over Oracle backups and recoveries.
——-
Dmitri Joukovski
Poll of MySQL Quickpolls
MySQL Quickpolls might be insightful for people who develop products and services for MySQL. Recently I was looking again at “How do you backup your production database” poll. To interpret the results, I wanted to know who are the people answering that and other Quickpolls. Are they the DBAs responsible for running MySQL in production or the developers writing applications that use MySQL? For a backup guy like me knowing that makes a difference.
Every Quickpoll gets a time stamp when opened and tells how many people answered the poll. It occurred to me that the normalized number of people (MySQL polls run for different periods of time) answering each poll could give me some insight. The graph below shows the daily number of people answering each poll in the last 24 months.
Of course, I understand there could be self-selection in answering the polls. For example, the DBAs could be more likely to answer “operational” questions and developers could be more likely to answer the questions relevant to them. However, I still think that the size of the horizontal blue bar is a proxy for relative interest for each poll.
Well, it seems that it is mostly the developers who are answering Quickpolls.The question that generated most interest was the one about primary programming language for MySQL applications. The question about backup was number three by relative interest, but it was only 20% of the most interesting (programming language) question.
Interestingly, the respondents were much more interested in sharing what type of hardware they consider for MySQL server vs. what type of storage they use for MySQL data. I am sure that with MySQL moving up and playing even more significant role for mission critical applications, more people will understand the importance of underlying storage such as Sun Thumper with ZFS, NetApp etc for building scalable and high-performance MySQL implementations.
———–
Dmitri Joukovski
Change the world? Our job is to keep the world the way it was.
One of the candidates for a marketing position (btw, we are looking for a very technical marketing talent, drop me a line at dj at zmanda dot com) asked me if Zmanda wants to change the world. The way I see it, our job is just opposite to changing the world. We want to keep the world the way it was. If your hardware dies or a user drops a table, we give you the ability to go back in time and to recover your file with Amanda Enterprise or your MySQL database with Zmanda Recovery Manager (ZRM).
How do we actually accomplish going back in time? Are we talking here about super-expensive and complicated Continues Data Protection (CDP) technologies? Our solution delivers the same result, but without the cost and complexity of CDP. ZRM incremental backups result in a copy of the MySQL binary log being flushed to disk. This log can be used to restore the database to its original condition in any point in time between two valid backups. Wouldn?t it be wonderful if you could go back in time and pinpoint the exact time or the transaction to which you want to restore your database? We provided this capability via the command line interface before, but the recently released ZRM 2.0 makes it very easy to identify the Recovery Point Objective, right in the Management Console, by way of the Database Events Analyzer.
The Database Event Analyzer allows you to select a binary log and view its contents. The date and time of all database activities and events are shown as individual records. You can easily scroll through the entries. Of course, there may be tens of thousands of entries or more in the log. Using Google-like search syntax you can easily query for events and actions to help you locate specific items to identify activity that impacted or even damaged your database. Your Recovery Point Objective could be very specific — down to the transaction level. You can even set the Recovery Point right from the log analyzer making going back in time and recovering the database very simple.
Here is a short (7 minutes) demo of Zmanda Management Console for MySQL backup.
Dmitri Joukovski
Managing backup of MySQL via iPhone
Given a chance, I always check www.zmanda.com on different gadgets with various browsers and form factors. As soon as iPhone became available, I was at the Apple store checking out the new device and especially its web browsing capabilities.
It took me a couple of attempts to type the URL, but then I got it right and I could see our web site, wiki and forums. Everything looked great. Surprisingly, I could see most of the page without too much zooming and moving the window. Our upcoming Management Console for backup and recovery of MySQL is built on a LAMP stack with a look and feel similar to our website. What if we could actually manage all or at least some frequent day to day backup activities of MySQL DBA via iPhone?
Here at Zmanda we are moving fast, and within a few days we presented to a couple of our customers how to manage MySQL backups via iPhone. Here is a use case. A DBA receives an e-mail with notification that MySQL backup failed. To figure out if she needs to take action, the DBA logs into Zmanda Management Console for MySQL backup. She checks first the calendar view to see when was the last successful backup (see the actual photo). Green square means that backup was successful and red square indicates that a backup failed. She can tap on any day in a calendar when backups were done, and see details for a particular backup. For example, on July 11th at 02:57:22 there was a successful backup of Netflix database with retention policy of 10 days. That particular backup was not compressed and not encrypted. Knowing that she had a recent good backup, she can drill then into an event log, identify the source of failure and decide if any further action should be taken.
So far reaction of customers who have seen our Management Console in action on an iPhone has been divided. One DBA expressed an opinion that no serious administrative task such as backup of MySQL should be done via “a toy that lacks enterprise security features”. Another customer got very excited and is open to the possibility of opening a port in a firewall to connect to the Zmanda Management Console server via iPhone and manage many of his backup tasks remotely.
To get more feedback we will show how to manage backups of MySQL using Zmanda Recovery Manager and backups of file systems using Amanda Enterprise via iPhone at the upcoming LinuxWorld in San Francisco on August 6-9.
———-
Dmitri Joukovski
New ideas and users? requirements for backup of MySQL
The MySQL User’s Conference provides very fertile environment for new ideas. For example, last year we came to the conference with a conviction to build a MySQL agent for our Amanda Enterprise backup and recovery software. However, after talking to many MySQL users about their backup needs we got convinced that stand-alone solution optimized for MySQL DBAs was a better way to go, and that is how Zmanda Recovery Manager for MySQL was conceived.
This year we received a lot of insightful feedback and suggestions from MySQL users and our customers. Special thanks to MySQL engineering and product management for providing their perspective. Here are the major takeaways for us, the MySQL backup guys:
- Back up the MySQL server configuration files when you do backup of the database — that will simplify recovery to a different server or to the same server back in time.
- Provide verification of backups ? users have backups, but not so many DBAs test their backups.
- Predictive reporting will help. For example, based on historical data, alert the user that in 2 months he will start missing his backup window.
- MySQL log analyzer to identify points of interest from backup & recovery perspective, e.g. pinpoint when a user dropped a table.
- Auditing of MySQL backup operations, for example, alert the DBA that 2 of his recently installed MySQL servers don’t have any backups.
- Bare metal recovery of MySQL server and databases.
Our engineering is already looking how to implement these and many other ideas. Tell us about your backup requirements for MySQL by posting on our forums
——-
Dmitri Joukovski
Backup and recovery benchmarks with Zmanda Recovery Manager for MySQL
The MySQL database has become the world?s most popular open source database because of its fast performance, high reliability and ease of use. The DBAs expect the same virtues from a backup solution for MySQL. Often times we are asked what performance to expect from Zmanda Recovery Manager (ZRM) for MySQL. That is why recently we measured backup and recovery performance for MyISAM and InnoDB storage engines using various backup methods available with ZRM.
In our testing we used MySQL 5.0 running Netflix databases available as part of Netflix Prize project. Our 2-way Red Hat Enterprise 4 server with 4 GB of memory was able to copy large files with an average transfer rate of 37 Mb/s.
All details and results are described in a white paper available with a free registration to the Zmanda Network. Here are the important takeaways:
- Logical backups and especially recoveries are slo-o-o-o-w. For MyISAM storage engine the transfer rate for backup was 5 MB/s and only 2 MB/s for recovery. Backup of InnoDB was only slightly faster.
- Raw backups and recoveries are much faster than logical ones. For example, for MyISAM the transfer rate was 25 MB/s for raw backup and 40 MB/s for recovery.
- In our testing we did not find any significant difference in backup and recovery transfer rates for different sizes of the database.
- The READLOCK time is the shortest with raw backups done via LVM snapshots managed by ZRM. The shorter the READLOCK time the better for an application and users.
That testing was our first step in measuring the performance of ZRM for MySQL when there was no user activity. We do understand that DBAs are concerned with backup performance when MySQL is used. That is why we will continue to measure ZRM performance including conditions with heavy user activity.
——
Dmitri Joukovski
Personal comments about recently published results of Amanda usage survey
We just published the results of Amanda usage survey and I want to share my personal observations.One of the most inspiring findings is that 95% of respondents would recommend Amanda to a friend. The Loyalty expert Fred Reichheld would be very impressed with such a result. The overall satisfaction with Amanda is also highlighted by the fact that 70% of respondents have been using Amanda for more than 2 years. People use software for such a long time only if it does the job well. Many users reported that Amanda was up to the task recovering files in most critical situations.
I think it is just great that more than 60% of users provided suggestions for new functionality.
Here are the surprises:
- I expected that ‘old’ Amanda users (with more that 2 years of usage) might have a higher ratio of old, pre-2.4.5 version of Amanda. However, there was no significant difference between ‘all’ and ‘old’ users, who are up to speed with Amanda progress.
- I knew that we would have many users running MySQL, but I did not expect the number to be so high. 64% of respondents reported they have MySQL in production.
- I did not expect that so many people (62%) use Amanda to backup Linux e-mail.
- My guess was that users with more clients and more data would recover files relatively more often. However, the observed positive correlation of 0.32 between the amount of data and number of restores per month is actually not that strong.
- 10% reported using VMware or Xen. I expected more people taking advantage of virtualization. Maybe virtualization is more common in the Windows world . . .
Overall, the results of the survey prove that Amanda does the job very well. For example, one user reported 6 TB of data per backup client, which illustrates how Amanda scales up. Another user reported more than 120 clients per Amanda server and that shows how Amanda scales out. One University reported having 13 different types of architecture protected by Amanda including the relatively new Mac OS-X on Intel and Solaris 10 on AMD workstations.
In the future we will continue to conduct in-depth usage surveys every 12 to 18 months and we plan to supplement full surveys with occasional quick polls.
——–
Dmitri Joukovski
Backup and recovery of the remote MySQL server with ZRM
We just published a technical document about what to consider for backup and recovery of the remote MySQL server, for example, when your online forums and wiki are located at managed hosting service provider.
For any database backup the primary considerations are the consistency of the backup and impact on users and applications. A backup of the remote MySQL, however, has extra challenges related to:
- network usage
- security, and
- flexibility of recoverability of MySQL data to a different host.
We discuss the trade-offs to consider, and describe how to use Zmanda Recovery Manager (ZRM) for several common scenarios. For example, we provide HOWTO for a use case when you want to ensure security of backup data on the wire and at rest.
