sorted by: recent | see : popular

Content Tagged with solaris + MySQL

We announced support for Oracle today. Specifically, Amanda Enterprise now protects:

• Oracle 11g on Windows
• Oracle 10g and 11g on Linux and Solaris
• Oracle Enterprise Linux

On Linux and Solaris, we took the approach of integrating Amanda with RMAN. Keeping with Amanda’s philosophy of using native tools as closely as possible, we allow Oracle DBAs to perform recovery right from within Oracle Enterprise Manager. Amanda takes care of restoring all files needed for a DBA to perform recovery specific to their need (e.g. table level recovery or point-in-time recovery).

For Windows, on recommendation from backup experts at Oracle, we integrated with their new VSS writer. This enables us to take quick snapshot of Oracle and basically have no processing overhead on Oracle for bulk of the backup process (i.e. actual movement of data).

We also added Oracle Enterprise Linux as one of the Linux distributions that we support. Now our Linux backup list reads: Red Hat, Fedora, CentOS, Ubuntu, Debian, SUSE, OpenSUSE and Oracle Enterprise Linux.

Oracle DBAs now benefit from open formats (relieving them of costly vendor lock-in) and ease of use of Amanda. Of course, they benefit from huge cost savings offered by Amanda Enterprise - in many cases one-fifth the cost of other solutions. In addition, for the first time, they can backup their databases directly to the Amazon S3 storage, combining backup and disaster recovery in one step.

At Zmanda our strategy is to continue to stretch our umbrella to protect more and more platforms and applications. Supporting Oracle is a major step in this direction.

..aka how to spend a magnificent Sunday-Monday in SUN’s company :)

If your applications run on Windows? and you need a MySQL? Cluster for testing purposes you can use Virtual Box to install SUN Solaris?  10 and set up the Database Cluster creating 4 Solaris? Zones (4 IP are required for a minimal Cluster setup).

It’s an interesting scenario.

You will need:

• Virtual Box
• Solaris? 10
• the binary distribution of MySQL? Cluster for Solaris?

You can create the Solaris? Zones manually or you can create them at once using a script file.

Note that you don’t need to install 4 Operating Systems. You’ll install 1 Solaris? 10 and then 4 Zones, each with a different IP: one for the MGT Node, one for the SQL Node, one for the Data Node 1 and one for the Data Node 2.

To get started with Solaris? Zones, I followed a very useful article by a Sun blogger - “Setting up MySQL Cluster using Solaris Zone”.

Unfortunately, after setting up Solaris? 10 and the Database Cluster I was not able to connect through SSH to the SQL Node from my Vista? machine, thus my Cluster was up, running but isolated and unreachable as well :(

After a long pause I got the idea for the solution of the issue (it’s typical): I forgot the configuration of port forwarding with NAT in the Solaris? 10 Guest System.

…The disadvantage of NAT mode is that, much like a private network behind a router, the virtual machine is invisible and unreachable from the outside internet; you cannot run a server this way unless you set up port forwarding..

ummh.. always RTFM (and check it again, and again…)!

I set up the port forwarding with the commands:

C:\Program Files\Sun\xVM VirtualBox>VBoxManage.exe setextradata “Solaris-10″ “VB
oxInternal/Devices/pcnet/0/LUN#0/Config/guestssh/Protocol” TCP
VirtualBox Command Line Management Interface Version 1.6.2
(C) 2005-2008 Sun Microsystems, Inc.
All rights reserved.

C:\Program Files\Sun\xVM VirtualBox>VBoxManage.exe setextradata “Solaris-10″ “VB
oxInternal/Devices/pcnet/0/LUN#0/Config/guestssh/GuestPort” 22
VirtualBox Command Line Management Interface Version 1.6.2
(C) 2005-2008 Sun Microsystems, Inc.
All rights reserved.

C:\Program Files\Sun\xVM VirtualBox>VBoxManage.exe setextradata “Solaris-10″ “VB
oxInternal/Devices/pcnet/0/LUN#0/Config/guestssh/HostPort” 2222
VirtualBox Command Line Management Interface Version 1.6.2
(C) 2005-2008 Sun Microsystems, Inc.
All rights reserved.

However since making this change the Solaris? Guest did not start anymore and I got the following error:

Configuration error: Failed to get the “MAC” value.
VBox status code: -2103 (VERR_CFGM_VALUE_NOT_FOUND).

I found some posts - like this  - but they didn’t help me.

Conclusion: it’s 4:20 am and I am still looking for a solution to get my Cluster reachable, wondering why I want to use Solaris? before attending an intensive SUN Training ;-)

I really want to figure out why if I set up the port forwarding for SSH Connections the Guest does not start.. but.. I’d go to bed ;-)

I’ll try to comment in the next days after getting help from VB guru :)

So… some colleagues have been experimenting with DTrace a bit, and I’ve been (for a while now) wanting to experiment with it.

The challenge now, instead of in the past, is that I’m setting up a Solaris based system - not getting one premade.

I chose OpenSolaris as I’d previously tried Solaris 10 and just sunk too much time trying to get updates and a development environment installed (another colleague could get the opposite to me going: he got devtools but no updates. at least mine was up to date and secure… but without a compiler).

So… OpenSolaris. It isn’t 100% open, there’s binary only drivers and such… but compared to previous Solaris, a whole lot better. Now, if only it was GPL licensed so we could have cross-pollination with Linux.

I grabbed the 2008.05 ISO as soon (in fact, slightly before) it was released and installed it in VirtualBox.

The installation was shiny - one of the best OS installs I’ve seen in a while. It set up nice things (zfs, X) and (an improvement on the previous release) even managed to get all the hardware going (not sound though).

However, on first reboot, nasty surprise. DNS isn’t enabled by default.

I found out why DNS isn’t enabled by default - and (as usual) this comes down to hysterical raisins. Back in what we laughingly call the past, during install Solaris would ask you what services you wanted to use for name resolution (which I guess made sense when people used yp/NIS more often than DNS). The default didn’t include DNS.

In the graphical installer, it just chose the default without asking… which is no DNS. So my mother would be able to install OpenSolaris, but once done, she’d have to know to type in 150.101.98.214 instead of www.google.com.au into Firefox. However, I swallowed my pride, edited /etc/nsswitch.conf and went along my business (I wonder the percentage of users who would actually go from “hrrm, internet not working” to editing /etc/nsswitch.conf without intense googling).

The UI did look nice though. Nice looking GDM, GNOME desktop looked nice. You could tell that whoever did the theme had spent too much time near MacOS X, but I’ll forgive them for that. The default shell is remotely sane and even though the bash completions aren’t as funky as on Ubuntu, I managed (unlike sitting at cmd.exe, where somebody is likely to die each time my keystrokes end up there).

I even had a look at the graphical package management tool - which looked quite nice. I even tried to do an update via it… which ended in what seemed to be a locked package manager and general amounts of fail. To see if it had just stopped or was chewing up my CPU or memory, I opened a terminal and ran ‘top’.

I then found out that top isn’t installed by default. It’s 57kb on my Ubuntu 8.04 laptop so disk space couldn’t be the reason why it’s not installed. It’s certainly not a “it’s a minimal install” argument, there’s lots of other things there by default.

Next step, let’s get updates (some time had elapsed between first install and now).

Seeing as I hadn’t met too much success with the graphical utility (it was at version 0.0000001 or something, so I don’t lay blame there). I find out that ‘pkg image-update’ is what you want to run. So I do.

It chugs for a while and says there’s 1GB of updates. That’s okay, I (where I=Sun) pay for what here on the arse end of the Internet is considered a decent link to my home office. About 20-30minutes later, having downloaded about 600MB, it goes “url timedout error” and aborts. Oh well I think, that’s easy - i’ll run it again and it’ll just resume downloading (remember the revolution when that started working, you know, in 1997).

I then discovered that pkg doesn’t resume downloads. It creates a snapshot using ZFS and puts the updates in it. If anything goes wrong, it just deletes the snapshot. This is a huge benefit over (say) dpkg, which if you press the reset button at the right time will leave your system very, very fucked (magic incantations can revive it, but it’s not fun - and the dpkg developers don’t think it’s a problem - come to my “Eat My Data” talk at OSCON to find out the full story). So OpenSolaris pkg wins on the “don’t ruin my working OS install already” front, but fails on resuming downloads.

I try again. Same story.

It’s now wasted a bit over 1GB of downloads… which equates to a couple of dollars.

I wait a few days, a week, and try again. Same story. I even try with a few hints found online that should fix things (well.. they did let another 100MB on average download before dying with the same story).

I then decided to just try and do the minimal - I wanted a development environment so I could build a MySQL Server with NDB and then play with DTrace to help nut out a performance problem or two.

So i tell pkg to install SunStudio Express. I’m even using instructions off sun.com, so it has to work.

It’s only ~500MB now (IIRC). Fails with exactly the same error as before (url timedout). Gah!

So, this brings us to today. I head into the Sun office.

I figure “this just has to work from a Sun office… ” and I was right!

It got through the (now) 1500MB download of updates!

It even applied them!

Success!

Win!

Well, no, - FAIL.

It now refused to boot with the updates. Or rather, it just rebooted soon after having started booting. No panic, no error screen, no “will reboot in 120 seconds” or anything useful. Instead, you just saw a flicker of the error message before it rebooted.

So… with some very careful pause/unpause of the VM (thanks VirtualBox… I also have a feature request now - pause before reboot :) I got this:

Aparrently the successful update, not so much.

Hrrm… perhaps select the known good one from the GRUB menu? It did actually boot! But this wasn’t just the old kernel, it was the whole older system. I guess that’s a possible upside of ZFS snapshots…. but oh my, that could be sooooo subtle and lead to data loss that it’s really quite dangerous.

I was still no closer to getting an up to date opensolaris system with enough developer tools to build a MySQL Server and use dtrace.

And this was enough. It’s now gone and I get my 10GB of disk back.

Maybe I’ll try again later… but I’m finding the google-perftools to be rather exciting and they’re really satisfying shiny thing urges at the moment.

Today Sun announced release of X4540 Open Storage server (a.k.a Thor). We were fortunate to get early access to Thor to certify Amanda Enterprise and Zmanda Recovery Manager (ZRM) for MySQL. Both solutions are optimized for backing up to disk, are already certified with Solaris 10, and leverage capabilities of ZFS. So it made perfect sense to certify them on Thor, effectively creating a high-performance and yet relatively inexpensive backup appliance.

The unit we were using is powered by 8 CPUs operating at 2300 MHz and provides 48 SATA drives with total capacity close to 50TB in 4U enclosure. That is good packing; a typical EMC unit with so much capacity will take the whole rack.

The two boot drives configured as a mirror were running UFS. To get capacity for backups, we easily created Zpool with 12 drives and 6 spares in RAIDZ2 configuration. That gave us 7 TB of capacity for backups with very high level of protection against drive failure:

bash-3.00# zfs list
NAME USED AVAIL REFER MOUNTPOINT
zmanda 203K 7.13T 49.0K /zmanda
zmanda/vtapes 49.0K 7.13T 49.0K /var/lib/amanda/vtapes

While installing ZRM and Amanda Enterprise on Thor, we found that a couple of Solaris packages that we require were missing, and our coverage of dependencies in documentation was not perfect. Luckily, that was the only hiccup. After fixing the dependencies, the rest of installation and configuration went smoothly.

This screenshot shows part of a summary report about backup of five Solaris clients to Thor over 1 Gb network. Each client had about 100 MB of data. In that test we forced full backup on each run, with each client pushing data at 60 MB/s. That is not bad at all considering that you can get only 20-25 MB/s for a typical LTO-2 tape drive.

The X4540 Open Storage server is an excellent choice for creating your own backup to disk appliance. It provides high performance and very large capacity. One key feature is how easy it is to manage all that capacity with ZFS.

Special thanks to Sun?s Menlo Park and Colorado interoperability teams who helped us to get started and advised on considerations in configuring ZFS for backup to disk.
———
Dmitri Joukovski

No, this is not a blog about Computational Fluid Dynamics - my least favorite subject in college. This is about a more exciting (sorry mechanical engineers!) CFD: Customer Facing Data. This is the data that is typically available on the website of an organization that their customers interact with. CFD can range all the way from profiles of users on a social networking site such as Facebook to the customer information database of an e-commerce company such as Travelocity.

CFD represents today’s data protection challenge. Probably the biggest challenge while planning a backup solution for CFD is that it is very hard to figure out what to plan for. You might be starting with a very small database which might grow much more rapidly than what you think. If the data can be segmented based on users or some other characteristic, then you will find that your databases may scale-out instead of scale-up. Also, rate of change can be very rapid, sometimes with lots of small changes (e.g. tags) or sometimes with big data changes (e.g. addition of user generated media content), very similar to the rather unpredictable viral growth pattern of the Internet.

Many organizations are keen to save and analyze behavior of users as they interact with CFD. This metadata can itself pose a data protection challenge of its own, since it may change very rapidly, while the actual data is not changing (e.g. tracking of customer intelligence metadata about which prospects downloaded which whitepapers from your corporate website, so that you can determine the best marketing & sales approach for these prospects).

MySQL is the DBMS of choice for CFD. Some skeptical press and analysts have said a few times that MySQL is not being deployed at the back-end of the enterprise (financials, billing etc.), but only on the web-tier. The implicit (sometimes explicit) implication being that web-tier applications are less important for the organization. I think this analysis fails to realize the actual importance of the data being stored in web-tier applications. Most of this is CFD, loss of which will cause huge pain and costs in the form of lost revenues, customers and reputation.

From its very inception, Zmanda has been focused on technologies powering CFD (MySQL, LAMP stack, and now increasingly Solaris). We have dug deeper than anyone else in understanding the needs of protecting this crucial part of any organization and very rapidly delivered products to address these needs. We are the data protection company for CFD. While we do protect back-end applications and platforms, our technology and business focus remains CFD.

I will be talking about deploying radically simple backup solutions for CFD on Thursday (April 17th, 10AM) at the MySQL user conference. I am in the unenviable position of making a Backup presentation sound interesting after the Facebook keynote (”A Match Made in Heaven? The Social Graph and the Database”). Well, I guess the point that conference organizers are trying to make is that if you have a radically simple MySQL backup and recovery solution, you will have more time to spend on Facebook!

Speaking of analysts, I recently read an analyst report which indicates that, on average, salaries offered to MySQL DBAs can be up to 40% lower than those offered to Oracle or DB2 DBAs. This ironically is considered one of the barriers for entry of MySQL in some environments. Per this report: higher paid database personnel have vested interest to keep MySQL out. Another claim is that somehow the lower salary to MySQL DBAs reflects the relative importance of the MySQL powered application for the business. Well if you are a MySQL DBA, you will do well to make your management realize that you are the keeper of their Customer Facing Data. Any business deploys its best resources for customer facing activities. Hopefully this will start reflecting on your paycheck soon.

If you would like to talk about your data protection challenges, or your salary as a MySQL DBA, stop by our booth #307 at the MySQL user conference.

We’ve been running into a problem with one client:

SELECT COUNT(*) FROM tbl;

takes 0.25 seconds on one db, and 0.06 seconds on another.

Consistently. That’s a fourfold difference.

There aren’t any significant configuration differences (like query cache, etc.), the software versions are the same, and the table fits into memory. This has been looked at by at least 3 in-house MySQL experts, and the only thing we can determine is that it’s a hardware difference.

The table fits into memory so it’s not a disk issue, and the only other difference among the hardware is that the slower machine has Solaris virtualization in place in the form of “containers” (cpu is the same, etc). Is this something that’s known to cause issues with speed? The “tbl” in question is an InnoDB table, if that means anything. Is there something like the “speed” of RAM?

Note that Sun has already been called in, and scripts have been run and analyzed on their side and they say “must be a db config issue” even though the configs have no significant differences (and neither does the hardware and OS spec).