Does open source need to be ?organic??
Sparse notes from the talk, I noticed Sheeri recording some video, so sitting through that at some stage might make sense. There were no slides, this was a panel discussion. Suggested reading: Organic vs. Non-organic Open Source.
Does Open Source need to be “Organic”?
Brian Aker, Rob Lanphier, Stephen O’Grady, Theodore Ts’o
Taking code, and slapping a certain license on it, doesn’t a successful software project make.
Blurring the distinction, by marketing. Not doing any work to get external contributions.
Open sourcing a product one plans on “genociding”, its really bad.
“Corporate sociopathic Druckerism” — Brian Aker
“As long as the source code is open, let the market decide. MySQL is largely inorganic, and its a success. Much of it comes down to choice.” — Stephen O’Grady
Mark Shuttleworth has pushed the idea that forking is OK. Look at Launchpad: take a project, fork the project, make your change, and you can publish your tree that people can use. The wonders of distributed version control.
Its up to a company to decide if they want an organic or an inorganic project. Its your code, do what you want with it. In the future, an organic project may outstrip your inorganic project.
Netscape: inorganic piece of open source (with Mozilla). Firefox: forked the code, turned it into an organic model, then there was success.
Is Firefox really the best example? Look at what it did for Netscape Corporation or AOL? This won’t work well with the Pointy Haired Boss.
What was your goal of releasing the product under an open source license? If marketing buzz, then you make lots of PR, etc… then go home. If your goal is wanting to cut your development cost, you’re going to be disappointed with an organic model. If your goal is ubiquity, you aim for an organic model.
Commit access actually means you’re a worker bee. It doesn’t mean a free wheel to push features, it means you’re the garbage man - you collect everything, you sort everything, and so on. Let’s rethink what it means to have commit access.
OSCON day 3: Reflections on OSCON 2008
Today was the last day of OSCON and I'm in the mood to think about the conference and share some of my random observations that didn't make it into any of my other blog posts.
First up is a comment that Brian Aker of MySQL fame made during the "Tim O'Reilly Interviews Monty Widenius & Brian Aker" interview:
Microsoft is irrelevant. ... We're more worried about Apple.
Woah. That's a tall statement! One that resonates with me since just a few weeks ago I realized that my life is now fully free of Microsoft. And I used to be a full time Win32 programmer 10 years ago. While Microsoft may not be fully irrelevant in all scopes, a comment like this shows that the open source movement has made an amazing amount of progress in the last 10 years. Consumers have a lot more operating choices today than they did 10 years ago. And to think that Apple had been written off for dead -- now they're back and they're feared!
And while we're on the topic of Microsoft's relevance, Thorsten von Eicken in his "Scale into the Cloud with Open Source" presentation said:
Inflexible licensing on behalf of vendors has caused most of the software running in the cloud to be open source software.
(The "cloud" in this context is Amazon's EC2 for which his company provides services). This is a huge deal! We're talking about closed source software vendors shutting themselves out by not quickly providing flexible licenses that can deal with the rapid deployment of servers to a cloud. And even when they do provide flexible licenses, what will the costs be of handling a massive load spike of your services running in the cloud? You may have to watch out for more than just your bandwidth costs during a load spike -- what could the operating system licensing costs do to your company? Go open source software! This keeps getting better and better!
At the Google dinner I sat next to Greg Kroah-Hartman, the Linux kernel hacker and talked about all sorts of things. I asked him what the strangest place was that he found Linux running. His response:
- VW Production plants: Assembly robots are running Linux
- Motorola cell phones: Apparently all motorola phones run Linux. I didn't know that!
- Sail boats: Some fancy sailboats are controlled with embedded Linux systems.
- Cars: Many cars run Linux now and when customers demanded iPod support in their car, Linux made a huge surge forward as manufacturers raced to provide this new feature.
I didn't realize how many places used embedded Linux to control various contraptions. I personally have run an embedded Linux computer in my Burning Man project last year, so I shouldn't be surprised at the various places where Linux crops up.
Finally some comments about OSCON itself. Besides having a blast once again and meeting many interesting new people, the sessions all engaged me fully. Not even once did a speaker fail to capture my attention throughout the whole session. The selection of topics was also spot on -- many of the things that are on my personal radar were covered. And the gender balance had never been better at an OSCON! I was pleased to see many more women present at OSCON -- I suspect that Danese Cooper and Allison Randal may have had something to do with that. Regardless of who deserves the credit, please continue to encourage more women to attend!
That's it for me. I have to run off to catch my plane back to California. Thanks for the excellent conference and thanks for having my on the Radar!
MySQL Query Analyzer: Finds good code, gone bad
As a Product Manager with MySQL I have learned from meeting with friends/customers that this experience hasn't really changed much since I left the field. I hear things like:
- MySQL is not well instrumented for tracking code level performance metrics
- Logs are OK, but not centralized and too low-level for easy navigation
- We need help identifying "good code gone bad" and "bad code gone worse"
To these friends I say we have listened (and will continue to) and help is on the way! My team has been busy working on the new MySQL Query Analyzer that will ship as part of the Enterprise Monitor a little later this year. Just a few bullets on what it does and what you can do with it.
Cool things it does:
- Proactive collection and identification of problematic SQL code.
- Historical browsing/analysis of queries across all servers.
- Aggregated roll ups of all queries in canonical form (no variables).
- Fully qualified views of worst performing statements.
- Drill downs into query details, number of executions, execution stats
These things are baked into the Monitor to help:
- Tune your SQL code before it is promoted to production
- Quickly identify queries that are sapping performance
-- by query type, content, server or application user
- Once problematic code is surfaced you can:
-- view an EXPLAIN plan to see where you need indexes
Here's a screenshot of the Monitor with Quan enabled:
We are currently working with a few "friendlies" on the first beta, but plan to open things for general beta soon. Stay tuned (no pun intended...), more details to follow on how you can help with this.
The #1 mistake hosting providers make for MySQL servers
This article is not meant to malign hosting providers, but I want to point out something you should be aware of if you’re getting someone else to build and host your servers for you.
Most hosting providers — even the big names — continue to install 32-bit GNU/Linux operating systems on 64-bit hardware. This is a serious mistake.
You have to tell them to install a 64-bit operating system. If you don’t then you will come to a point where your needs grow and you want to use more memory — and they will gladly install 8 or 16GB of memory for you, but MySQL can’t use it because it runs in a single process, which is limited to about 2.5GB of memory. And then you have to rebuild the whole operating system from scratch. But you don’t want any downtime, so you have to buy another server, set it up as a slave, switch your site to use it, and then rebuild the old server. That 32-bit OS turned into a pretty expensive mistake.
I do not know why the hosting providers keep doing this. Just yesterday I got a quote from a hosting provider for a medium-high-end system with 8GB of RAM, and forgot to tell them 64-bit OS, and they actually listed 32-bit explicitly on the quote — useless! I would estimate about half of all the hosted systems I’ve seen so far have this mismatch. I don’t know why they do this — maybe there is a reason, but I don’t know it and it looks pretty silly to me. 64-bit hardware and operating systems aren’t new anymore. In fact, 32-bit is hard to find in server-class hardware these days. So it certainly looks like the hosting companies need to change what they’re doing, but maybe there’s a different reason.
Entry posted by Baron Schwartz | 6 comments
Add to: 
| 
| 
| 
| 
MySQL Certified Professionals LinkedIn Group
If you've requested to join the MySQL Certified Professionals LinkedIn Group and haven't heard back, I do apologize! I've been swamped, and haven't had the free time to manage the group lately. I've approved all the remaining requests, so welcome! We've reached 140 members in the group.
For future requests to join, please include a link to your corresponding certification starting from here: http://www.mysql.com/certification/candidates.php. This will greatly help speed up the approval process.
This MySQL Certified Professionals group is for certified MySQL professionals, technical recruiters, hiring managers or human resource managers to help locate MySQL Certified Professionals in their area.
MySQL Back to Basics: Analyze, Check, Optimize, and Repair
It felt like the right time for us to look back at some useful commands for table maintenance that some of us may not have mastered as much as we might like to think.
In my post about gathering index statistics, I referred to OPTIMIZE TABLE, ANALYZE TABLE, and REPAIR TABLE — but I never explained in depth what the different commands do, and what the differences between them are. That is what I thought I would do with this post, focusing on InnoDB and MyISAM, and the differences in how they treat those commands. I will also look at different cases and see which one is right for in each case.
Hey Drizzle, share the love
And speaking of 5.1, what is the plan for releasing 5.1 plugins for Maria and Falcon? Given that the storage engine API supports plugins, the release of a storage engine doesn't require a new server release. InnoDB has already proven that by giving us a plugin that provides compression and fast DDL.
Hey Drizzle, share the love
And speaking of 5.1, what is the plan for releasing 5.1 plugins for Maria and Falcon? Given that the storage engine API supports plugins, the release of a storage engine doesn't require a new server release. InnoDB has already proven that by giving us a plugin that provides compression and fast DDL.
Log Buffer #107: A Carnival of the Vanities for DBAs
Welcome to the 107th edition of the Log Buffer. My name is Keith Murphy and I am a MySQL database administrator for the Pythian Group. In addition, I am the editor of MySQL Magazine. This is my second go for the Log Buffer, so I must be doing something right!
This week for the open source world brings OSCON in Portland Oregon. There are plenty of MySQL people present and there have been more posting this week from these realms than normal. Also, Lewis Cunningham, among others, posted news that EnterpriseDB released the results of their open source survey at OSCON. The 451 CAOS Theory published their thoughts on the survey.
Of interest to all DBAs is John Duncan’s post about what is called the “five minute rule”. This was introduced in 1987 by Jim Grey. And finally, before we dive into the specific server news, here is a post on Facebook’s project to build a distributed database similar to Google’s BigTable.
MySQL News:
Friday saw the release of the summer issue of MySQL Magazine. The highlights of the issue include the first annual MySQL Usage Survey. The magazine is available here. Peter, over at Percona, shows some initial benchmarks for the latest version (0.9.8) of Sphinx. If you aren’t familier with Sphinx, it is a full text search engine that easily integrates with MySQL.
Probably the biggest news this week was the announcement by Brian Aker of “Drizzle”. It is what amounts to a slimmed down version of MySQL server. These comments are from his initial post “Stored Procedures, Views, Triggers, Query Cache, and Prepared Statements are gone for now.” Interesting. My co-worker, Sheeri K. Cabral, posted a video of Brian Aker talking about the Drizzle project at this week’s OSCON. Monty Widenius writes a good summary how Drizzle can/might integrate with the MySQL “ecosphere” at large. It was very good to hear Monty say that Sun management is encouraging this project. There has been some other interesting posts about this as well including Arjen’s thoughts, and Paul McCullagh’s. Brian Moon gives a very thoughtful view on how Drizzle could potentially fit in at dealnews. While I could probably fill up the entire Log Buffer with links to post about Drizzle, I better leave it at that.
To be completely open and honest...
Robin left Embarcadero to start the Product Management group at MySQL in June, 2005. He recruited me again and I joined him in December of 2005 to manage what was then call MySQL Network. Having worked primarily for corporate IT shops and commercial software companies, it was quite an adjustment coming to work for MySQL. Front and center, the primary product is open source and free to use; hard to swallow for the capitalist in me, but hey I quickly adapted. Actually, I prefer the MySQL model to the old school, licensed, proprietary model. Basically, you use our product to build great apps and when those apps become critical to your business, you pay us to help you make sure they are up and running great. Win-win!
At MySQL I work with some of the best and brightest Engineering and business minds in the industry. My primary partnership is with the Enterprise Tools Engineering team led by Gary Whizin and Mark Matthews. Our primary work is focused on the MySQL Enterprise Monitor which comes as part of a MySQL Enterprise subscription. MySQL Enterprise is our subscription based solution that includes the Enterprise Server and regular bug fixes, the Enterprise Monitor and Expert Advisors and our stellar Production Support Services. You can learn more here. We (PM and the Enterprise Tools team) release new versions of the Monitor 2-3 times a year to ensure that the product continues to evolve. Along these lines we are currently working on Enterprise Monitor 2.0 that will ship with advanced Query collection and analysis capabilities. Of all the product releases I have been involved with since jumping from development to Product Management, this one has me the most excited. It might be because we are addressing a common DBA/Developer pain point with a very real, easy to use solution. It might be because every single customer we have spoken to echoes this pain. It might be those things, but I think I am most excited because some of the proprietary tool vendors in the MySQL monitoring space have creatively adopted our very public facing roadmap and announced similar features in their products. I think this is great! We want them to be successful and to enable more people to adopt and deploy MySQL as part of their move to innovative Web 2.0 technologies. IMHO this is a win all the way around!
I will post more details on the new beta release of the Enterprise Monitor with the new Query Analyzer in my upcoming posts. It is mighty cool!
On open source and piracy
Dana Blankenhorn asks whether open source is hurt by piracy, prompted by comments made by Louis Suarez-Potts, Sun’s community manager for OpenOffice.org at OSCON.
Dana is unconvinced that open source supporters should necessarily be doing anything about piracy, noting that “There is no direct financial loss to Open Office when someone has a pirated copy of Microsoft Office. To the extent that BSA enforcement actions cause fear in the market, that just benefits open source, so why join it?”
He also notes that “On the other hand if we helped Oracle enforce its license terms we might accelerate the move to MySQL and Ingres.”
However, one need only remember these comments from last year made by the president of Microsoft’s business division, Jeff Raikes, to understand why piracy is bad for open source:
“Our number one goal is that we want people to use our product. If they’re going to pirate somebody, we want it to be us rather than somebody else. And that’s because we understand that in the long run the fundamental asset is the install base of people who are using our products.
What you hope to do is over time you hope to convert them to licensing the software, legally licensing it, so on, and so forth,” he added, neatly - and presumably accidentally - describing the method by which commercial open source vendors benefit by making their core code available free of charge.
So it’s always a delicate balance, because what you want to do is you want to push towards getting legal licensing, but you don’t want to push so hard that you lose the asset that’s most fundamental in the business.”
Additionally on our recent virtual tour of Europe we saw how piracy was seen as a barrier to further adoption of open source in countries like Greece and Romania.
Supported by government, open source can be used as a tool defeat piracy. Louis explained, ComputerWorld reports that: “By cracking down on software piracy, nations around the globe are starting to see that they can help themselves dramatically by encouraging innovation and creativity — as well as job growth and richer economies — through open-source development.”
An example of this is Russia, where Microsoft’s bungled attempt to crack down on software piracy resulted in a decision by the government to reduce piracy and encourage local business by encouraging the use of open source software.
Daniel Nichter and Percona join forces on Maatkit
Daniel Nichter, the author of several very useful MySQL tools, has joined with Percona to continue to improve Maatkit — the “other” toolkit we rely on daily. Daniel is a skilled Perl programmer who understands MySQL. He is in a perfect position to move the project forward on a more regular basis than I’ve been able to commit time to since I’ve taken on a leadership role in Percona.
Right now he’s working on an unreleased tool we will use to make our performance audits more efficient. As a result, we’ll be able to deliver more value to our customers and assess servers more quickly in the event of an emergency. (By the way, if you’d like to sponsor features you need for a Maatkit tool, contact us and we’ll be glad to discuss it with you.)
Welcome, Daniel! And a toast to Maatkit’s continued improvement and usefulness!
Entry posted by Baron Schwartz | One comment
Add to: | | | |
OSCON 2008 in photos
Paris, City of Love and MySQL ? 19 September 2008
In an internal mail thread, I was asked whether there would be any “objections from an integration perspective” to some Sun initiated plans for a more organised French MySQL community.
My reply was that it’s great, if it’s something related to the self-organisation of the already very active French MySQL community (as witnessed for instance by the huge numbers that Véronique Loquet of Al’x Communication attracted to our Paris meetup in April). But if it’s about a centrally-imposed structure of “marketing towards the user base”, then I want to understand more and we need to discuss a bit further.
Based on the video link that Véronique sent me, it seems to be more of the former — or if it’s the latter, then they seem to have got it just right, for an event planned for 19 September 2008.
This is one of the best MySQL videos — at least the funniest — that I’ve seen so far!
As for the 19 Sep 2008 event, I hope I can make it, but it might be tough. Take a look at the video, and you know why (specifically why I hope to make it, not why it might be tough).
Text:
START TRANSACTIONS;
INSERT INTO plage (fille) VALUE ('mignonne');
DELETE FROM beaugosse WHERE cheveux LIKE 'Blond';
UPDATE geek SET serial = 'Lover';
REPLACE HIGH_PRIORITY INTO Love (FILLE, MEC) VALUES ('MOI', 'TOI');
COMMIT;
Glossary:
- plage = beach
- fille = girl
- mec = guy
- mignonne = beautiful
- beaugosse = handsome
- cheveux = hair
- moi = me
- toi = you
La communauté MySQL se donne rdv;
le 19/09/2008;
à la cantine;
à partir de 18H;
Plus d’informations bientôt sur lacantine.org
Geek: Victor Rieunier;
fille: Mathilde Mallen;
Production: QNTV;
Réalisation: Morgan;
Musique: amsteroller;
Links:
- Movie-teaser: http://vpod.tv/QuartierNumeriqueTV/526842
Drizzle? seeing sun through the clouds
Brian Aker was at the Sun booth today, in a premier slot, where there must have been about 50-60 people, huddled around, to listen to him talk about Drizzle. The project motivations, what’s behind it, what its not aimed to be, and so much more. Check the video out (21 minutes long)!
The Birds of a Feather (BoF) session in the night, was well attended, and there was lots of large discussion on what’s next. I think the important message to take away is that Drizzle doesn’t aim to be MySQL, and there are no plans to “merge” things back (fixes where the code-base is shared though, might make sense). Its also important that the design is for the future, i.e. multi-core machines. It was great to see Brian say that this really leverages Sun in many ways.
Its worth nothing that Sheeri was at the lightning talk, and has a shorter, 8 minute video recording too.
MySQL Binaries with patches
As there were many requests for Debian binary packages in our announcement of MySQL releases with custom patchsets, we decided to play with it and built .deb, which you can find there:
http://www.mysqlperformanceblog.com/mysql/debian/
Also we have generic RPMs for x86-64 architecture:
http://www.mysqlperformanceblog.com/mysql/RPM/. They are based on .spec file provided in MySQL source tree and should be compatible with SuSe, CentOS and RedHat.
Your comments are welcome!
Entry posted by Vadim | 4 comments
Add to: | | | |
SourceForge Community Choice Awards winners
InnoDB Performance Tuning (training course day for MySQL)
Just like with other topics, extending this to a full day allows the issue to be explored much better rather than getting skimmed over as a small part of a much bigger course. After all, if you understand the tool, how it does things and why, you can utilise it more optimally. The skimming wouldn't help you in this sense.
I still have some customers use MyISAM (out of habit mainly) but the default storage engine really is InnoDB, effectively, even if the server build does not see it that way by default ;-) But there is more to tune, things to know. Just converting to InnoDB does not make the application run any better necessarily. It might reduce lock contention in some cases (like update heavy apps), but the overall running might be slower without the right setup. While maintaining ACID, there's much that can be done.
The course is part of the DBA course days that Open Query has currently scheduled in cities around Australia, and in Auckland New Zealand. For dates that are still a few months away you can get up to $50 off (per person/per day) for early bookings.
MindTouch Deki's new release integrates...just about everything
Disclosure: I am on the advisory board for MindTouch.
Double disclosure: I really, really like the latest release of MindTouch Deki (formerly "Deki Wiki").
MindTouch has always thought that a wiki should be about more than simply creating basic web pages. With its new "Kilen Woods" release, the company has significantly bent the rules as to what constitutes a wiki, and just which data sources can feed into a wiki.
LinkedIn? Yep. Salesforce.com? Sure. SugarCRM. Uh-huh.
MindTouch Deki enables businesses to connect and mashup the growing number of application and data silos that exist across an enterprise - including legacy systems, CRM and ERP applications, databases, and Web 2.0 applications....For example, MindTouch Deki can visualize content from a Microsoft SQL Server or Microsoft Access databases and mash it up with other services, such as Microsoft Live Earth or Google maps, LinkedIn and a CRM system - offering a common wiki and web-service interface for content and behavior from multiple sources.
eWeek and others have some good reviews. Me? I just like that I don't have to learn any wiki language to use it. Take a look for yourself:
...OSCON 2008 - MySQL Proxy - from architecture to implementation
The presentation about MySQL Proxy at OSCON 2008 is over.
Here are the slides.
Thanks to my co-presenter Ronald Bradford and to all the participants. If you have more questions about the session, please use this blog's comments.