sorted by: recent | see :
popular
Content Tagged with and + recovery
Open Storage and Open Source Backup - A Perfect Combination
Today Sun announced release of X4540 Open Storage server (a.k.a Thor). We were fortunate to get early access to Thor to certify Amanda Enterprise and Zmanda Recovery Manager (ZRM) for MySQL. Both solutions are optimized for backing up to disk, are already certified with Solaris 10, and leverage capabilities of ZFS. So it made perfect sense to certify them on Thor, effectively creating a high-performance and yet relatively inexpensive backup appliance.
The unit we were using is powered by 8 CPUs operating at 2300 MHz and provides 48 SATA drives with total capacity close to 50TB in 4U enclosure. That is good packing; a typical EMC unit with so much capacity will take the whole rack.
The two boot drives configured as a mirror were running UFS. To get capacity for backups, we easily created Zpool with 12 drives and 6 spares in RAIDZ2 configuration. That gave us 7 TB of capacity for backups with very high level of protection against drive failure:
bash-3.00# zfs list
NAME USED AVAIL REFER MOUNTPOINT
zmanda 203K 7.13T 49.0K /zmanda
zmanda/vtapes 49.0K 7.13T 49.0K /var/lib/amanda/vtapes
While installing ZRM and Amanda Enterprise on Thor, we found that a couple of Solaris packages that we require were missing, and our coverage of dependencies in documentation was not perfect. Luckily, that was the only hiccup. After fixing the dependencies, the rest of installation and configuration went smoothly.
This screenshot shows part of a summary report about backup of five Solaris clients to Thor over 1 Gb network. Each client had about 100 MB of data. In that test we forced full backup on each run, with each client pushing data at 60 MB/s. That is not bad at all considering that you can get only 20-25 MB/s for a typical LTO-2 tape drive.
The X4540 Open Storage server is an excellent choice for creating your own backup to disk appliance. It provides high performance and very large capacity. One key feature is how easy it is to manage all that capacity with ZFS.
Special thanks to Sun?s Menlo Park and Colorado interoperability teams who helped us to get started and advised on considerations in configuring ZFS for backup to disk.
———
Dmitri Joukovski
Open Storage and Open Source Backup ? A Perfect Combination
Today Sun announced release of X4540 Open Storage server (a.k.a Thor). We were fortunate to get early access to Thor to certify Amanda Enterprise and Zmanda Recovery Manager (ZRM) for MySQL. Both solutions are optimized for backing up to disk, are already certified with Solaris 10, and leverage capabilities of ZFS. So it made perfect sense to certify them on Thor, effectively creating a high-performance and yet relatively inexpensive backup appliance.
The unit we were using is powered by 8 CPUs operating at 2300 MHz and provides 48 SATA drives with total capacity close to 50TB in 4U enclosure. That is good packing; a typical EMC unit with so much capacity will take the whole rack.
The two boot drives configured as a mirror were running UFS. To get capacity for backups, we easily created Zpool with 12 drives and 6 spares in RAIDZ2 configuration. That gave us 7 TB of capacity for backups with very high level of protection against drive failure:
bash-3.00# zfs list
NAME USED AVAIL REFER MOUNTPOINT
zmanda 203K 7.13T 49.0K /zmanda
zmanda/vtapes 49.0K 7.13T 49.0K /var/lib/amanda/vtapes
While installing ZRM and Amanda Enterprise on Thor, we found that a couple of Solaris packages that we require were missing, and our coverage of dependencies in documentation was not perfect. Luckily, that was the only hiccup. After fixing the dependencies, the rest of installation and configuration went smoothly.
This screenshot shows part of a summary report about backup of five Solaris clients to Thor over 1 Gb network. Each client had about 100 MB of data. In that test we forced full backup on each run, with each client pushing data at 60 MB/s. That is not bad at all considering that you can get only 20-25 MB/s for a typical LTO-2 tape drive.
The X4540 Open Storage server is an excellent choice for creating your own backup to disk appliance. It provides high performance and very large capacity. One key feature is how easy it is to manage all that capacity with ZFS.
Special thanks to Sun?s Menlo Park and Colorado interoperability teams who helped us to get started and advised on considerations in configuring ZFS for backup to disk.
———
Dmitri Joukovski
Poll of MySQL Quickpolls
MySQL Quickpolls might be insightful for people who develop products and services for MySQL. Recently I was looking again at “How do you backup your production database” poll. To interpret the results, I wanted to know who are the people answering that and other Quickpolls. Are they the DBAs responsible for running MySQL in production or the developers writing applications that use MySQL? For a backup guy like me knowing that makes a difference.
Every Quickpoll gets a time stamp when opened and tells how many people answered the poll. It occurred to me that the normalized number of people (MySQL polls run for different periods of time) answering each poll could give me some insight. The graph below shows the daily number of people answering each poll in the last 24 months.
Of course, I understand there could be self-selection in answering the polls. For example, the DBAs could be more likely to answer “operational” questions and developers could be more likely to answer the questions relevant to them. However, I still think that the size of the horizontal blue bar is a proxy for relative interest for each poll.
Well, it seems that it is mostly the developers who are answering Quickpolls.The question that generated most interest was the one about primary programming language for MySQL applications. The question about backup was number three by relative interest, but it was only 20% of the most interesting (programming language) question.
Interestingly, the respondents were much more interested in sharing what type of hardware they consider for MySQL server vs. what type of storage they use for MySQL data. I am sure that with MySQL moving up and playing even more significant role for mission critical applications, more people will understand the importance of underlying storage such as Sun Thumper with ZFS, NetApp etc for building scalable and high-performance MySQL implementations.
———–
Dmitri Joukovski
Scaling Up: MySQL on NetApp
For many MySQL implementations scalability is a crucial requirement. Scaling on various dimensions: Size, Performance, Cost and DBA-Stress-Level.
While several factors impact the scalability of the database, underlying storage probably has the highest impact. You need to make sure that the storage will fill up the buffers fast enough to keep queries happy, and acknowledge the writes fast enough to keep transactions happy. You need to make sure that you can keep growing the available storage as your database grows - MySQL databases are particularly prone to collect more and more data. You also need to make sure that maintenance tasks such as backup, cloning, and application testing will scale gracefully with the size of your database.
Storage systems from NetApp are a great choice when designing for MySQL scalability. Administrators can expand storage on the go (using FlexVol) eliminating the need for over-provisioning. NetApp’s ability to create snapshots in a highly storage efficient manner is particularly important for scaling the maintenance tasks. You can quickly clone your MySQL database and use it to test and deploy new applications. Most importantly, this snapshot capability enables fast and scalable backups.
As I had indicated before, we have been working with the engineering team at NetApp to create highly scalable backup and archiving solutions for MySQL. Today we formally announced this joint work. Our MySQL backup product, Zmanda Recovery Manager (ZRM) for MySQL with NetApp SnapShot Manager, meets two crucial design goals for scaling MySQL backups: least impact on the live production applications and quick point-in-time recovery. Database snapshots created by ZRM can also act as source for NetApp SnapMirror software, creating a highly scalable disaster recovery solution.
With your MySQL databases stored on a NetApp filer and protected by Zmanda, you can scale up while reducing your stress level.
A Snapshot of Snapshots
Storage Snapshots are excellent tools in arsenal of a system administrator to create quick and consistent backups of their databases and applications. Snapshot is a “picture” of a filesystem at a point-in-time. In most modern snapshot implementations, this “picture” is not a full copy of the data, but rather a set of pointers to the data.
Here is a list of current industry leading snapshot technologies:
- LVM Snapshots: Snapshot capability built into the Linux Volume Manager (LVM) - the default option on a Linux system.
- ZFS Snapshots: Snapshot capability built into the ZFS filesystem on Solaris and OpenSolaris
- NetApp SnapShots: A feature of the WAFL® (Write Anywhere File Layout) filesystem
- Windows VSS: Volume Shadow Copy Service from Microsoft introduced in Windows Server 2003. VSS is also available with Windows XP, Vista, and 2008 Server.
- VxFS Snapshots: Snapshot capability built into the Veritas File System
- EMC SnapView: Snapshot capability on the EMC CLARiiON storage systems
- IBM FlashCopy: Available in the IBM Storage Subsystems (DS8000, DS6000, ESS 800, SAN VC)
- EqualLogic Smart Copy: Feature in the Dell EqualLogic PS Series
Our Zmanda Recovery Manager for MySQL product uses storage snapshots as one of the technologies to do a quick full backup of the MySQL database. We currently support LVM, ZFS, NetApp, VSS and VxFS Snapshots. Rest are coming soon…
Webinar:The Essential MySQL Backup & Recovery Solution ? ZRM 2.2 Overview
MySQL databases are increasingly used by high volume, high transaction applications
that support businesses running full throttle 24-hours a day, seven days a week.
Backup and recovery operations need to be conducted in such as way that is non-disruptive
to users and applications. The latest release of Zmanda Recovery Manager (ZRM) 2.2 provides
Continuous Data Protection (CDP) as well as instant point-in-time recovery for MySQL.
In this webinar, we will demonstrate the latest ZRM enhancements so that you can use ZRM
to build a robust, flexible, and easy to use backup and recovery solution.
When: Thursday May 29, 2008, 10:00am Pacific / 1:00pm Eastern
Efficient CDP solution with instantaneous point-in-time recovery for MySQL
Ask any DBA what are his top priorities for backup of MySQL. Chances are that hot backup and recovery to a specific point-in-time (or transaction) will be on top of everybody?s list.
The recovery to any point-in-time has been always an ultimate goal for data protection. Traditional backup solutions allow recovering only to a point in time when the last backup took place, e.g. last night. A lot of new data could be created and lost since that last backup. That is why in the last several years many vendors have been working on Continuous Data Protection (CDP).
Storage Networking Industry Association (SNIA) defines Continuous Data Protection (CDP) as “a class of mechanisms that continuously capture or track data modifications enabling recovery to previous points in time”.
ZRM 2.2 enables CDP by using LVM, VxFS, VSS, ZFS and NetApp snapshots in conjunction with MySQL transaction logs. When doing a recovery to a particular point in time, ZRM reads data from the snapshot and then replays MySQL transaction log from that point forward. This enables almost instantaneous point-in-time recovery.
For each point in time T, ZRM creates a snapshot that includes a MySQL binary log. To recover MySQL database to a specific Recovery Point Objective (RPO) between T2 and T3, ZRM reads data from snapshot T2 and replays transactions from binlog T3 up to RPO providing the user with point-in-time recovery. To enable instantaneous recovery snapshots are stored where the database files are, for example, on the same NetApp filer.
Since real life recoveries are performed under stressful conditions, we designed our product to simplify your life and ensure successful recovery:
- With a single click you can initiate point-in-time recovery that reads data from the snapshot and replays transactions so you get your database back in time.
- Our unified recovery mechanism allows the very same recovery workflow not just for snapshots, but for all other backup methods such as logical and raw backup. The unified recovery works even with third party data extraction utilities for MySQL such as InnoDB HotBackup.
- ZRM allows recovery to any point in time for multiple MySQL servers in the organization. MySQL servers could have different storage engines and run on different operating systems and all backup and recovery operations are centrally managed by ZRM.
Some CDP solutions capture all writes to the database effectively duplicating what a binary log is designed to do. Why re-invent the wheel and waste computing resources? ZRM is integrated with MySQL on a deeper level than any other backup solution. That allows us to provide very efficient CDP solution with almost instantaneous point-in-time recovery.
——
Dmitri Joukovski
Wow! What a Great First Day at MySQL UC
You have to have been living under a rock, if you did not know that today is the first day of MySQL Users’ Conference and Expo. We at Zmanda are so proud and privileged to have been awarded the “Partner of The Year”. The award was more meaningful to our vision of Simplified, Easy to use, commercial Open Source Backup and Recovery. We appreciate the award and are committed to making the life of the MySQL DBA hassle free. We have had a ton of visitors from all walks of MySQL user community talk to us today. Its fascinating to talk to customer and prospects on how they leverage the power of MySQL. To learn more about how Zmanda provides the Best in Class Backup and Recovery solution for MySQL, you can attend Zmanda’s sessions at the MySQL Conference & Expo include:
What: “Radically Simple Backup & Recovery for Live MySQL”
Who: Chander Kant, Zmanda CEO and founder
When: Thursday, April 17, 2008, 10:00 a.m. PT
Where: Santa Clara Convention Center, Ballroom E
In this keynote presentation Zmanda CEO and founder Chander Kant will discuss the most critical task for database administrators - protecting corporate data using online backup and recovery solutions. He will explain how Zmanda enables MySQL DBAs to take advantage of the latest advancements in snapshot technologies and storage engines to take on mission-critical online transaction processing challenges with confidence.
What: “Zmanda Recovery Manager for MySQL”
Who: Ann Ruckstuhl, Zmanda vice president, sales and marketing
When: Wednesday, April 16 at 5:15 p.m. PT
Where: Santa Clara Convention Center, Ballroom H
Ann Ruckstuhl will present a session on Zmanda Recovery Manager (ZRM) for MySQL and how DBAs can deploy ZRM to protect their MySQL databases. In this session, Ann will discuss in detail how DBAs can use ZRM to select the right types of backup (e.g. logical, raw, snapshot, full or incremental), optimize backup performance, and activate point-in-time granular recovery right from the MySQL Visual Log Analyzer. She will also outline how ZRM can dramatically simplify and streamline day-to-day backup management tasks via its built-in administration tools, reports, monitoring and alerts.
What: “Top 5 Considerations While Setting up Your MySQL Backup”
Who: Dmitri Joukovski, Zmanda vice president, product management
When: Wednesday, April 16 at 4:25 p.m. PT
Where: Santa Clara Convention Center, Ballroom G
MySQL Backup and Recovery Training from Zmanda
Worried about Backup and Recovery of your MySQL Databases? MySQL Backup school from Zmanda provides hands on and in depth training on Backup and Recovery of MySQL. Just sign up and show up with your laptop. More information available here.
Sun/MySQL to resell Zmanda Recovery Manager for MySQL
Today Sun and Zmanda announced our agreement to deliver a comprehensive, global data backup and recovery solution for MySQL Enterprise subscribers. Starting April 1st, MySQL Enterprise customers will be able to purchase ZRM for MySQL directly from Sun worldwide.
I think Zack’s comment in the press release captures the rationale for the deal:
“Protecting corporate data through effective backup and recovery is one of the most crucial tasks for a database administrator, and it can be a complex undertaking — especially for today’s large Web-scale applications,” said Zack Urlocker, VP of products, Sun Microsystems database group. “MySQL users have told us that global backup and recovery is very important to them, and we are thrilled that we can now offer ZRM for MySQL as an easy-to-use solution for protecting all of their MySQL data.”
Of course, at Zmanda we are thrilled as well. Sun’s sales channels will give us the opportunity to make MySQL Backups radically simple for ever increasing MySQL installations around the world.
Webinar: New Levels of Backup and Recovery for MySQL
Zmanda has introduced a new version of Zmanda Recovery Manager (ZRM) which adds quite a few capabilities for MySQL users. Version 2.1 includes enhanced snapshot support that enables backup without application downtime or interruption to online data access. Version 2.1 also has global management of backups so that you can manage all your MySQL backup jobs from a single graphical console. You can now run the entire ZRM solution on Solaris as well as on Linux.
Join us for an overview and a live demo of ZRM 2.1 on Thursday 3/20 at 10am PST. Click here to register.
Zmanda Recovery Manager 2.1 - Keep those MySQL Databases Zipping
Today we announced a significant enhancement to our Zmanda Recovery Manager (ZRM) for MySQL product. Here are major highlights of this new version (2.1):
End-to-end support for Solaris: We have had increasing number of requests from customers for supporting Solaris. While we always protected MySQL databases running on Solaris, we needed customers to run the ZRM core engine on a Linux box. Now we are able to run the ZRM core engine on Solaris itself. So a pure Solaris shop can use ZRM without getting a Linux server. The coinciding of this support with MySQL’s acquisition by Sun was not a planned thing 
Enhanced Snapshot support: One of the coolest features of ZRM is to be able to take advantage of underlying storage infrastructure whenever possible. Since version 1.0 of ZRM we have provided support for backing up MySQL using LVM on Linux. Now we are adding support for four major snapshot options: Windows Volume Shadow Services (VSS) snapshot, Network Appliance SnapManager, Veritas VxFS and Solaris ZFS. If your storage infrastructure supports any of these snapshot options, this will be the fastest way to backup your MySQL database - with zero impact on the application using the database. In fact, on Windows platform, ZRM 2.1 enables for the first time the capability of taking a fast raw backup of MySQL (using VSS). BTW, a crucial feature here is how ZRM uses snapshots for Restore. When a DBA requests a point-in-time restore, ZRM is intelligent enough to collate snapshot-based full backups with log-based incremental backups, to deliver the MySQL database in the precise state at requested point-in-time.
Global Management of MySQL databases: One key behavior we observe about MySQL databases is that they tend to propagate rapidly within organizations. Here at Zmanda, I can easily count close to 10 production MySQL databases, both inside and outside our firewall. Maintaining backups of multiple MySQL databases at a single repository tremendously reduces the complexity for DBAs. While we always had some form of remote backup capability, with ZRM 2.1 we now support centralized backup across all storage engines and across all operating systems. Our web based Zmanda Management Console allows this centralized management from anywhere, including from an iPhone!
Our goal is to make backup of live MySQL databases radically simple - enabling you to focus on your business applications.
Webcast - Enterprise Backup to Amazon S3
We are hosting a webcast tomorrow to talk about backing up corporate data to Amazon?s Simple Storage Service (S3) using Amanda Enterprise. We will demonstrate live, how enterprise users can now use Amanda Enterprise to harness Amazon S3 for a complete data backup, archiving and disaster recovery solution. The webcast will take place on Wednesday February 13 at 10:00AM PST and will last about an hour, including a Q&A session at the end. We hope you can join us to discover a new & cost effective way to leverage online storage services such as Amazon S3 as a part of your corporate data protection strategy. Click here to register.
Congratulations to Sun and MySQL
Congratulations to Marten and team on Sun’s acquisition of MySQL today! This is a billion dollar stamp of approval on the importance of the LAMP stack. This also gives another strong backing to the business model which involves giving one’s software away for free to up to 99+% of users!
This announcement comes at an interesting time for Zmanda. Just last week we announced support for the Solaris platform for our Amanda Enterprise product line. With our industry leading MySQL backup solution, we now have full coverage of operating systems and database software from the new Sun!
A great day for open source software and business of open source software!
Webinar Q & A: MySQL Backup Techniques for the Oracle DBA
We presented a webinar along with MySQL last week. The topic was MySQL Backup from the perspective of an Oracle DBA. We got some really good questions at the end of the webinar. Here is a transcript of Q&A:
Q from Trey: Will this Webinar be available online for viewing later?
A from MySQL: Yes. It will be on our site in a few days at http://www.mysql.com/news-and-events/on-demand-webinars/
Q from Randy: Is a copy of presentation available on web? Where?
A from Zmanda : slides are available on Zmanda Network: http://network.zmanda.com/
Q from Jing: Converting database from oracle to MySQL, any limitation regards the version compatibility?
A from MySQL: We provide a free GUI migration tool that helps migrate Oracle data objects to MySQL. For a complete and very detailed discussion on the subject, download our white paper on the subject at http://www.mysql.com/why-mysql/white-papers/mysql_wp_oracle2mysql.php
Q from Julien: Is MySQL supported on NetApp?
A from MySQL: Yes
Q from Julien: For mysql on NetApp, which protocols are supported? nfs? iscsi? cifs?
A from MySQL: I believe the key protocols are supported.
Q from Haris: How does ZRM integrate with NetApp
A from Zmanda: ZRM takes advantage of snapshots provided by Data ONTAP OS in NetApp filers.
Q from Jim: Will the “new non-blocking backup utility” for MySQL for hot backup be free? or an extra cost option? Thank you.
A from MySQL: At this time, the core backup utility is planned to be GPL and in the community server
Q from Neil: How much does ZRM cost?
A from Zmanda: Annual subscription starts from $200 per MySQL server. We provide subscriptions with three levels of support — Basic (email only), Standard (business hrs phone and email) and Premium (7 X 24). Pricing is available at www.zmanda.com/pricing.html
Q from Franck: so the licensing is per MySQL Server
A from Zmanda: Yes, subscription is per MySQL server and you can backup as many databases within that server as you have.
Q from Isaac: Is the community edition of Zmanda fully functional or somehow restricted?
A from Zmanda: The Community Edition is fully functional. However, only the Enterprise version provides web-based Management Console that simplifies all operations. It also provides log analyzer that simplifies point-in-time recovery. To see additional details, please check http://www.zmanda.com/zrm-mysql-enterprise.html
Q from Pavel: Does Zmanda GUI manager comes for free???
A from Zmanda: No. Only the Enterprise (which requires annual subscription) version provides a web based console.
Q from Tim: Could you provide onsite support if we need that?
A from Zmanda: Yes. We can provide onsite support and training if needed.
Q from Haijun: Does ZRM include InnoDB online backup?
A from Zmanda: Not in the current release. It is planned for a future release
Q from Alicia: Any idea how long before the release of a ZRM version that supports hot backup of InnoDB databases?
A from Zmanda: 3-6 months (likely next release)
Q from Mike: mysqldump cmdline for non-blocking with innodb?
A from Zmanda: mysqldump –single-transaction is non-blocking for transaction storage engines such as InnoDB
Q from Kevin: Now that Oracle owns the rights for InnoDB, what is it’s future. Will Oracle charge for InnoDB? Is there a replacement for Innodb from MySQL
A from MySQL: For MySQL, it’s business as usual so no changes from a support standpoint. We do have other transactional engines available - NDB (cluster) and our new Falcon transaction engine (about to enter beta). Also we have SolidDB and PBXT, - external engines
Q from Srinivas: what is the dependency of InnoDb during backup process?
A from Zmanda: There is no dependency on storage engines, but we can take advantage on functionality provided with some engines, e.g. we can use mysqlhotcopy as a data extraction utility for MyISAM. Another example — using snapshots with InnoDB can provide truly hot backup of MySQL.
Q from Madhavi: Having a combination of MyISAM and innodb, do we need separate jobs for backups?
A from Zmanda: No, you don’t. ZRM is designed to handle variety of MySQL configurations and the same backup job can handle different engines.
Q from Alicia: Which backup modes of ZRM support Innodb backups?
A from Zmanda: All backup methods are supported with Innodb storage engine (logical, raw, snapshot, replication)
Q from Rich: For datacenters with centralized backup to tape, will ZRM work in this environment? If so, am I able to have each customer have their own tape and can it be incremental and/or full?
A from Zmanda: Absolutely. ZRM can be easily integrated with any enterprise media manager such as Amanda, Netbackup by Veritas, NetWorker by Legato and TSM by IBM. Think of ZRM as “RMAN” for MySQL.
Q from Kevin: With Oracle we use RMAN and Netbackup using scripts for level 0,1,2. Can ZRM do the same?
A from Zmanda: Yes you can. You can use ZRM in conjunction with Amanda Enterprise (network backup software similar to NetBackup). If you are using NetBackup pre-backup and post-backup scripts to integrate with Oracle/RMAN, you can write similar scripts with ZRM.
Q from Charlie: If my 20 backups are successful and usually they are, I don’t want 20 emails. I only want an email if it “fails”.
A from Zmanda: You may have to tailor the email being sent. This can be done with some scripting. We plan to improve this functionality in the next release of Zmanda Recovery Manager for MySQL
Q from Jos: When in a replicated environment, can you restore all replicated databases at the same time from one backup?
A from Zmanda: Yes. You should configure all databases in the same backup set.
Q from Srinivas: Is selective recovery feature allows deselecting the corrupted records ?
A from Zmanda: If you have incremental backups covering the time frame when the records were not corrupted — then yes
Q from Isaac: How is recovery handled with the community edition of ZRM?
A from Zmanda: Recovery in community and enterprise editions is the same. Recovery depends on the backup method used. The DBA does not need to remember the backup method used. Enterprise edition provides easy to use recovery manager in Zmanda Management Console.
Q from Bob: Can Zmanda use multiple streams in parallel to backup a database?
A from Zmanda: You can configure ZRM to backup in multiple streams by configuring tables in the database in different backup sets.
Q from Raju: Can I resume my backup after a failure instead of starting over?
A from Zmanda: If there is backup failure, ZRM uses a different backup method. ZRM uses a hierarchy of backup methods depending on configuration and storage engine in use. Currently we don’t have a built-in checkpoint mechanism to resume a particular failed backup run.
Q from Isaac: Can ZRM be used to take backups from a slave, or is there some writing that is performed during the backup?
A from Zmanda: You can take backup from slaves in a replication setup.
Q from Rich: How does the recovery time compare between ZRM and RMAN for similar sized (say 800GB) DBs?
A from Zmanda: ZRM supports multiple backup methods and multiple storage engines. If you are using InnoDB storage engine (size 800GB) and snapshot backup method, the recovery time will be comparable.
Q from Eric: Does ZRM perform any compression? or encryption options?
A from Zmanda: Yes. ZRM uses plug-in architecture. You can create plug-ins for compression and encryption. The default configuration has lz and bzip2 compression similar to Oracle/RMAN.
Q from Steve: Can ZRM restore a single mysql database if you have it configured to “backup all databases” on a server?
A from Zmanda: Yes.
Q from Adam: So, when I enable the Zmanda backup, should I disable the collection of the binary logs, or can Zmanda use the logs collected by MySQL?
A from Zmanda: Zmanda can use logs collected by MySQL. Binary logs are used for incremental backups.
Q from Rich: what gets installed on the DB - an agent? If so, how lightweight is it?
A from Zmanda: On MySQL server, you need to install small footprint software that does data transfer to the ZRM server. This software runs only when needed (during backup run).
Q from Nitish: how to you integrate third part snapshots methods within zmanda?
A from Zmanda: There is a snapshot plugin interface which is documented in ZRM for MySQL wiki http://mysqlbackup.zmanda.com/index.php/Snapshot_Plugin
Q from Mike: Can you speak more about the feature that will delay a backup during times of heavy usage?
A from Zmanda: This is done using scheduling plugin. Please see ZRM for MySQL wiki http://mysqlbackup.zmanda.com/index.php/How_to_create_custom_plugins_for_MySQL_ZRM%3F#Pre_Scheduler_plugin
Q from Sarah: are zrm and zmc integrated or can they be? or are they completely free standing applications such as rman and netbackup.
A from Zmanda: ZRM can be used without ZMC, but then you loose ease of use and extensive monitoring and reporting available only via ZMC.
Q from Srinivas: does ZRM supports Community Version of MySQL?
A from Zmanda: ZRM enterprise version supports both the community and enterprise versions of MySQL.
Q from Rich: Will ZRM Community work with MySQL Enterprise or is that a licensing restriction?
A from Zmanda: Yes, it will, but we highly recommend to use the Enterprise version of ZRM for Enterprise version of MySQL.
Q from Eric: when is support for Falcon storage engine expected from ZRM
A from Zmanda: Zmanda works very closely with MySQL engineering. We will support Falcon when released and we are already working with alpha version of Falcon.
Q from Greg: What is ZRM’s backward compatibility to previous versions of MySQL? I notice your examples all show MySQL v5.
A from Zmanda: We support all versions from MySQL 3.X. There are restrictions on the backup methods supported in older releases. Please contact zsales (at) zmanda.com if you are interested in compatibility with a particular MySQL release.
Q from Rick: Is ZRM available on Suse?
A from Zmanda:: Yes. It works on SLES as well as Open SuSE
Q from Franck: Do you also support Mac OS X?
A from Zmanda: Yes, remote backup of MySQL on Mac OS X is supported.
Q from Neil: No ZRM for Windows server? Is that correct?
A from Zmanda: ZRM can be used to protect MySQL database on ANY operating system. Some OS are supported in remote configurations only. Windows is supported remotely and we are working on a new version that runs either locally remotely on Windows and uses VSS snapshots for backup of MySQL.
Q from Franck: I’m confused. Is Zmanda a backup solution for Mysql only? Of can it be used to BackUp Oracle too?
A from Zmanda: Zmanda Recovery Manager (ZRM) has been designed specifically for backup and recovery of MySQL. It is closely integrated with MySQL. Think of ZRM as “RMAN” for MySQL.
Q from Alexandre: Our organization has both Oracle and MySQL databases. Can ZRM provide some uniformity in managing both backups, e.g. backup MySQL to the same physical backend tape storage as Oracle?
A from Zmanda: You can backup both to same physical backend by using a third party media manager, such as Amanda Enterprise or NetBackup.
Q from Bob: From a ZRM standpoint is there any difference backing up a MySQL DB on Linux vs. Windows?
A from Zmanda: In current release, we do not support snapshots for MySQL on Windows. The next release will support VSS snapshot method for MySQL on Windows.
Q from Bob: What media managers does Zmanda integrate with?
A from Zmanda: Any media manager, for example Amanda Enterprise, NetBackup, Legato and others.
Q from Alexandre: Are there any competing products on the market to manage MySQL backups?
A from Zmanda: Our competition is adhoc scripts written by MySQL DBA and our own community edition
Q from Adam: I’m a little confused… What is the ZRM DB Event log exactly? Is it like the InnoDB binary logs? What is the format of it?
A from Zmanda: ZRM DB event log is same as MySQL binary log (that is used for MySQL replication). It contains SQL statements and results of SQL statements.
Q from Nitish: who are your major customers?
A from Zmanda: We have organizations of all sizes as our customers ? from large online retailers and airline reservation systems to small manufacturers and research labs. We are where MySQL is run in production and where organizations value their MySQL data. Please send an email to zsales (at) zmanda.com to get a reference list of our ZRM customers.
Q from Neil: Where is Zmanda located?
A from Zmanda: We are in Sunnyvale, CA and Pune, India. Many employees work remotely in different countries.
Backup and Recovery Techniques for MySQL Database
If you are migrating your production environment to MySQL database or even thinking about it, you do not want to miss this webinar Experts from Zmanda and MySQL will present various methods to backup MySQL database and correlate them with tools and terminology you may be familiar with in other databases, including Oracle.
A good pre-read for this webinar would be Paddy’s blog on online MySQL Backup
WHO:
Chander Kant, CEO, Zmanda
Robin Schumacher, Director Product Management, MySQL
WHAT:
MySQL Backup Techniques for the Oracle DBA (web presentation)
WHEN:
Thursday, December 13, 2007, 10:00 am PDT, 1:00 pm EDT, 18:00 GMT
The presentation will be approximately 45 minutes long followed by Q&A.
 |
MySQL Backup and Recovery - Today and Tomorrow
We have had lots of questions and very interesting feedback and suggestions.
You can find the slides and the Webex presentation here.
I would like to post more on this subject, specifically on snapshot backup, comparing the different options with NetApp, DRBD and LVM.
Questions and Answers from the Backup Webinar
mysqlhotcopy is available in all releases. ibbackup is a separate offering that has been available for a long time
Q from Adam: Is the SQL based output portable to another database server engine e.g. MSSQL?
There are options of mysqldump that make the output as much generic as possible, and that is portable, but we can't guarantee it works with every RDBMS. The DDL is the less portable section
Q from Andrea: Is a hot mysqldump generally a good idea?
Unfortunately the answer is "it depends". With MyISAM, for example, this is not generally a good idea, since you have to lock the tables to make the dump consistent - i.e. it is not a hot dump anymore, you may call it "warm". With InnoDB the dump is consistent, provided we execute it with the Repeatable read isolation level.
Q from Simon: What is your suggestion to backup databases with tables with mixed engines?
A LVM snapshot will work. In MySQL 5.2, this will be available with the built-in backup facility. mysqldump can still do a good job, but it will lock the MyISAM tables.
Q from Francisco: Can I execute a mysqldump from a remote machine, using ftp or sftp for example?
It is possible to execute mysqldump from a remote machine, but you may experience poor performance due to the network. mysqldump will not use ftp or sftp, it will use the usual tcp connection to connect to the db server and retrieve the data.
Q from Roalt: If I apply a the lock-all-tables in the mysqldump, would that affect replication too?
If you are dumping the slave, the sql thread of replication (the thread responsible to update the slave) will stop, but in the meantime the master and the I/O thread can still work.
Q from Sven: How long will mysqlhotcopy take to backup a 50 GB database?
The time depends on several factors and it's hard to say. One thing to consider though is that mysqlhotcopy will be in any case faster than mysqldump
Q from Simon: Can I use replication and then a binary copy of the data from a slave?
You can, but you have to consider several aspects. First of all, it will always be a binary copy of that server (slave) and it may not work on the master. MyISAM tables can be recovered on the master as well, but InnoDB data files will be different.
Q from Michael: Are indexes re-generated when we apply a restore from a logical backup?
Yes, the whole table, with data and indexes, will be recovered from scratch.
Q from Martjin: Can you restore a db to a new copy, still keeping the old one (with mysqlhotcopy)?
Yes, this is definitely possible. Simply use different names for the db schema
Q from Kristiaan: Is there any way to automate the backups to run on a regular basis?
Yes, You can use any scheduling system, crontab etc, depending of your operating system.
Q from Sven-Ake: What tool do you recommend to backup a MyISAM database that is about 50 GB? We want to do a hot backup, or at least a warm backup.
A LVM snapshot would be my first choice. I would also consider replication or the use DRBD, if you need HA as well. If you use InnoDB, hotbackup may be a solution.
Q from Roalt: Is DRBD stable enough for production of large databases?
I would say that DRBD is stable and definitely good for production. The size of the DB is not important, it is more important to consider the throughput required by the write operations.
Q from Ludovico: Will the new backup API allow integration with other backup solutions? (Legato, Omniback, Data Protector, Veritas etc.)
Yes, but you should not expect this integration available when we will release the API
Q from Kelly: Can binary logs be created for a specific database or will binaries always be for all databases within your instance?
The logs are created per server, therefore there is no way to distinguish logs for different schemas
Q from Kristiaan: If we want to transfer a copy of a MySQL database each night from the production system to the database, is there a way to get it to merged?
I would not call it a typical backup procedure. Anyway, it depends on what you mean with "merge", how you want to handle conflicts and so on.
Q from Phil: Does mysql have anything that combines the core DB backup with ftp mechanism?
Not today, but mysqldump can be run remotely.
Q from Jon: In a mixed engine schema, if LVM snapshots are not an option, what would you recommend? mysqldump is quite slow for us in practice.
Backup on a slave server with replication, for example. If you use InnoDB , Innobackup is also an option.
Q from Roalt: Are there any scripts to provide incremental backups from a replicating slave database server?
Incremental backups rely on the use of binary logs or, in case of use within the server, using relay logs. The procedure is fairly simple and it can be found in the Ref Manual.
Q from Nadav: Is there a flag when using mysqldump not to insert the schema name into the dump file?
Yes, you may use the --no-create-info and --no-create-db params
Q from Krzysztof: What about backup/restore function available in MySQL Administrator?
MySQL Administrator executes a mysqldump, but it handles it through a GUI and keeps track of the backups
Q from Adam: We have a 200GB customer file, what is the best option to store?
for a db of this size, I would definitely recommend a physical/raw backup. Logical backups may be too slow.
Q from Phil: How much additional performance load does running replication create?
Very little, since it's asynchronous and transactions are acknowledge immediately on the master server. The main impact on performance is related to the use of the binlog, which is actually another write on disk.
Q from Michael: Are you aware of any problems using mysqldump with non-English characters. Do I need to have a supported locale installed on my linux to have these statement "write" correctly?
There should not be any problem, if you have encountered them, I would suggest to check for bugs
Q from Roalt: Are LVM snapshots using LVM2 on Linux any good with MyISAM?
Yes, there should not be any problem, assuming you do a FLUSH TABLES first
Q from Alessandro: How are managed calls to functions like "NOW()" in Replication?
They are converted to the current time in the binary log
Q from Ewerton: Is there any tool to verify physical or logical corruptions in tables and indexes?
Unfortunately no, not at the moment.
Q from Jonathan: NAS backup: what's the recommended method for ensuring that NAS snapshots are restorable with minimal work after the restore?
There are different methods and they all depend on your environment. The main difference is related to the use of InnoDB or MyISAM. With MyISAM, things are pretty simple whilst with InnoDB it's a little bit trickier. If you can afford to make the database read only for few seconds, that is the best and safest way to apply the snapshot.
Q from Nadav: is ZMANDA for Unix only, and if yes, is it GUI only or command line also?
ZManda Recovery Manager (ZRM) runs on Linux and it can execute backups on Linux, Solari, OS X and Windows. ZRM can be executed from command line.
Q from Cristian: If I backup a slave server, can I restore the backup on the master?
If you have not applied any change to the data model, you can restore your logical backup on the master as well on the slave. If you have a physical backup, you master and server must be configured in the same way (DB version, location, parameters etc.).
Q from Arthur: Could you explain the --tab param in mysqldump?
With --tab you define that the output will be a tab delimited set of files with the same name of the tables and sql files with the same name to create the tables. The parameter is used to pass the path where these files must be created.
Q from Jonathan: How does the NAS-based backup ensure DB consistency?
It doesn't, that is up to the database. You may ensure consistency with MyISAM flushing the tables and switching to read-only mode. For InnoDB, this approach will leave the database in a recoverable state that must be applied when the backup will be restored.
Q from Arthur: Can you point my to some more text on using the NetApp with mysql
You may have a look at this document: http://www.netapp.com/library/tr/3601.pdf
Do I really need backups?
I was reading, this article (subscription required) by Ben Worthen on WSJ about a study on how well companies are prepared for a disaster. I was not at all surprised by (one of) the findings (paraphrased):
…While 70% of 189 tech leaders surveyed by Forrester Research Inc. said that their companies are prepared for a disaster, the reality was quite different…
We run into this all the time, in talking to (prospective) customers. People “think” they are prepared for a disaster. The truth hits them when you start peeling the layers of the supposed plan. Some things to keep an eye on are
- Is it robust and reliable?
- Do you have the right resources engaged in the plan - both design and implementation?
- How often are the plans tested?
- How well is the plan documented?
- How often is it updated?
- How far away from the primary site is the disaster recovery site ?
- Is there a plan B?
Just because someone in the company is keeping a copy of the files or database on a USB stick, does not mean you have backup. You need a robust solution which is flexible and simple to use. A good Backup and recovery plan is not a luxury, it is a necessity. The article concludes with:
Business people dont want to spend money on an IT project unless it’s likely to cut costs or add revenue. A backup plan does neither…..until disaster strikes
Enough said!
ZRM for MySQL 1.2.1 released
Community version 1.2.1 of Zmanda Recovery Manager (ZRM) for MySQL, a robust and intelligent solution for backup and recovery of MySQL databases is available for download at Zmanda downloads page.
Changes in this release:
- Bug fixes
- Source tar ball (including debian package files)
- Help from ZRM online sites
For documentation see, ZRM for MySQL users manual. Please use bugzilla to report bugs and feature requests. If you have questions or feedback, please post them in forums.
The a,b,c?s of MySQL Backup Planning
The role of data stored in MySQL database in today’s world is already known - it powers business and mission critical apps. Protecting this data is of utmost important to any enterprise. Before you actually have a backup in place, that are some very important things to think about the backup implementation. Like for instance, how soon would you want your data recovered after a user or system error? This article provides an indepth analysis about the basics of what needs to be considered while planning a Backup and Recovery solution for your MySQL database.
Webinar: Zmanda Management Console for MySQL Backup
Zmanda’s MySQL Backup and Recovery Solution is the only comprehensive backup solution that addresses the unique needs of MySQL databases — such as multiple database storage engines. Zmanda Recovery Manager (ZRM) for MySQL now features graphical tools to simplify the process of protecting MySQL data.
This free presentation from experts in open-source backup and recovery of MySQL will demonstrate Zmanda Management Console for ZRM , which is a web based service integrated with Zmanda Network. It provides management and visualization tools for defining all backup activities of a MySQL DBA:
- what to backup, e.g. all databases or just the selected tables
- where to store the backup images
- when to perform backups, e.g. schedule daily, weekly or monthly full and incremental backups
WHO:
Chander Kant, CEO and Co-Founder of Zmanda
Paddy Sreenivasan,VP of Engineering and Co-Founder of Zmanda
WHAT:
Zmanda Management Console for MySQL Backup web presentation.
WHEN:
Wednesday, August 22, 2007, 10:00 am PDT, 1:00 pm EDT, 17:00 GMT
The presentation will be approximately 45 minutes long followed by Q&A.
Change the world? Our job is to keep the world the way it was.
One of the candidates for a marketing position (btw, we are looking for a very technical marketing talent, drop me a line at dj at zmanda dot com) asked me if Zmanda wants to change the world. The way I see it, our job is just opposite to changing the world. We want to keep the world the way it was. If your hardware dies or a user drops a table, we give you the ability to go back in time and to recover your file with Amanda Enterprise or your MySQL database with Zmanda Recovery Manager (ZRM).
How do we actually accomplish going back in time? Are we talking here about super-expensive and complicated Continues Data Protection (CDP) technologies? Our solution delivers the same result, but without the cost and complexity of CDP. ZRM incremental backups result in a copy of the MySQL binary log being flushed to disk. This log can be used to restore the database to its original condition in any point in time between two valid backups. Wouldn?t it be wonderful if you could go back in time and pinpoint the exact time or the transaction to which you want to restore your database? We provided this capability via the command line interface before, but the recently released ZRM 2.0 makes it very easy to identify the Recovery Point Objective, right in the Management Console, by way of the Database Events Analyzer.
The Database Event Analyzer allows you to select a binary log and view its contents. The date and time of all database activities and events are shown as individual records. You can easily scroll through the entries. Of course, there may be tens of thousands of entries or more in the log. Using Google-like search syntax you can easily query for events and actions to help you locate specific items to identify activity that impacted or even damaged your database. Your Recovery Point Objective could be very specific — down to the transaction level. You can even set the Recovery Point right from the log analyzer making going back in time and recovering the database very simple.
Here is a short (7 minutes) demo of Zmanda Management Console for MySQL backup.
Dmitri Joukovski
New face of MySQL backups
Check out this short demo of Management Console of MySQL backup
tk
Sexy back(up)
Could not resist the title after reading what Matt wrote. Sexy and exciting, indeed. Most of the crew here have their heads buried with the upcoming releases of Amanda Enterprise and Zmanda Recovery Manager (ZRM) for MySQL. We are continuing to enhance the Management console for Network backup. And with the next release of ZRM for MySQL, backup of MySQL database will never be the same. Dmitri already wrote about how easy it will be to manage MySQL backups from Iphone. Additionally we are fixing stuff in the Management console for ease of use. You can see, touch and feel them at LinuxWorld. Come check us out. And watch this space for more excitement !
Managing backup of MySQL via iPhone
Given a chance, I always check www.zmanda.com on different gadgets with various browsers and form factors. As soon as iPhone became available, I was at the Apple store checking out the new device and especially its web browsing capabilities.
It took me a couple of attempts to type the URL, but then I got it right and I could see our web site, wiki and forums. Everything looked great. Surprisingly, I could see most of the page without too much zooming and moving the window. Our upcoming Management Console for backup and recovery of MySQL is built on a LAMP stack with a look and feel similar to our website. What if we could actually manage all or at least some frequent day to day backup activities of MySQL DBA via iPhone?
Here at Zmanda we are moving fast, and within a few days we presented to a couple of our customers how to manage MySQL backups via iPhone. Here is a use case. A DBA receives an e-mail with notification that MySQL backup failed. To figure out if she needs to take action, the DBA logs into Zmanda Management Console for MySQL backup. She checks first the calendar view to see when was the last successful backup (see the actual photo). Green square means that backup was successful and red square indicates that a backup failed. She can tap on any day in a calendar when backups were done, and see details for a particular backup. For example, on July 11th at 02:57:22 there was a successful backup of Netflix database with retention policy of 10 days. That particular backup was not compressed and not encrypted. Knowing that she had a recent good backup, she can drill then into an event log, identify the source of failure and decide if any further action should be taken.
So far reaction of customers who have seen our Management Console in action on an iPhone has been divided. One DBA expressed an opinion that no serious administrative task such as backup of MySQL should be done via “a toy that lacks enterprise security features”. Another customer got very excited and is open to the possibility of opening a port in a firewall to connect to the Zmanda Management Console server via iPhone and manage many of his backup tasks remotely.
To get more feedback we will show how to manage backups of MySQL using Zmanda Recovery Manager and backups of file systems using Amanda Enterprise via iPhone at the upcoming LinuxWorld in San Francisco on August 6-9.
———-
Dmitri Joukovski
ZRM for MySQL backup version 1.2 released
Version 1.2 of Zmanda Recovery Manager (ZRM) for MySQL, a robust and intelligent solution for backup and recovery of MySQL databases is available at Zmanda downloads page.
Changes in this release:
- Code restructured into modules
- Snapshot plugin interface. LVM snapshots implemented as a plugin.
- ZRM should be run as user belonging to mysql group.
- Numerous bug fixes
Documentation is available at ZRM for MySQL wiki. Ask questions and provide feedback in Zmanda forums.
Provide feedback on MySQL backup console
We are working on Zmanda Management Console for our MySQL backup product line: Zmanda Recovery Manager (ZRM) for MySQL. ZRM for MySQL is an enterprise backup and recovery solution for MySQL.
We are looking for MySQL administrators in San Francisco bay area who would be interested in providing functionality and usability feedback for the user interface. We are particularly interested in MySQL administrators responsible for implementing backup solution for their MySQL databases. Please send an email to me (paddy-at-zmanda-dot-com) with your availability and why you would meet our requirement. We will pay a small stipend for your time.
ZRM for MySQL Cluster Edition
ZRM for MySQL Cluster Edition, released during MySQL users conference last month, provides non-blocking backup and recovery for MySQL cluster databases. In addition to backup and recovery for MySQL clusters, the cluster edition also supports online backup and recovery of other MySQL storage engines such as MyISAM, InnoDB. Most applications using MySQL Clusters also store data in other storage engines. The Cluster edition provides a common user interface for backing and recovering applications that use multiple MySQL storage engines.
ZRM for MySQL supports both MySQL Cluster as well as the MySQL Cluster Carrier Grade edition.
A demo of ZRM for MySQL Cluster Edition is available here. If you want to buy or try ZRM for MySQL cluster edition, please contact us.