sorted by: recent | see :
popular
Content Tagged with Drupal + MySQL
Linux Symposium
As Tom and I will be heading to Ottawa for OLS Tomorrow you can expect some active blogging here this week..
That is if we can manage to find quality Wifi and our batteries last long enough..
before we find power :)
Anyway .. I`ll be heading to the Virtualization Mini Summit on tuesday, and then of to the big conference.
I`ll be presenting twice, once on the miniconf about openQRM4 and Tom and I will be presenting our findings comparing different monitoring tools such as Nagios, Hyperic, Zabbix , Zenoss and others at OLS itselve.
But don't hesitate to talk to me about other interresting topics such as MySQL or Drupal :)
Now first we have to cross a couple of borders, and an ocean :)
Tuning Search In Drupal 5
In previous search benchmarks, I utilized random content generated with Drupal's devel module. In these latest benchmarks, I used an actual sanitized copy of the Drupal.org community website database, with email addresses and passwords removed. The first tests were intended to confirm that Xapian continues to perform well with large amounts of actual data. Additional tests were performed to measure the effect of various MySQL tunings and configurations. The following data was derived from several hundred benchmarks run on an Amazon AWS instance over the past week using the SearchBench module.
These tests confirm that Xapian continues to offer better search performance than Drupal's core search module. Contrary to popular belief, the data also shows that using the InnoDB storage engine for search tables significantly outperforms using the MyISAM storage engine for search tables, especially when your database server has sufficient RAM. The data also confirms that allocating additional RAM for MySQL's temporary tables can also improve search performance.
Online Performance and Scalability Book
Tag1 Consulting is focused on improving Drupal's performance and scalability. We also believe that when information is freely shared, everyone wins. Toward these ends, we are working on an online book titled, "Drupal Performance and Scalability". The book is divided into five main sections, Drupal Performance, Front End Performance, Improved Caching and Searching, Optimizing the Database Layer, and Drupal In The Cloud. The book is primarily aimed toward users running Drupal on the LAMP stack, with chapters applicable to everything from low-end shared hosts to large-scale multi-server installations.
By publishing on-line, we aim to encourage you to participate in the book writing process as an editor and a technical reviewer. You will currently find the book's complete outline online, along with descriptions of each planned section and chapter. As the book evolves, it will continue to be updated online in real time. We encourage you to post comments with suggestions, critical feedback, grammatical corrections, or anything else relevant to our ongoing effort.
Web Hosting Services, SSL Certificates & Domain Registration
MySQL Trigger Woes
After a period of inactivity I was hacking back on a Drupal project, I had taken a mysql dump from a production platform and imported into my local dev setup, just to have some realistic data.
All of a sudden some forms started failing with the following error:
user warning: There is no 'user'@'nonlocalhost registered query: insert into blah (stuff,morestuff) values (x,y) in /var/vhost/drupal-tree/includes/database.mysql.inc on line 172.
My Drupal data connection was correct and working for selects etc.. only a limited set of inserts failed.
After some debugging I realised that the error was not Drupal related, running the same query on my MySQL console gave the same error.
ERROR 1449 (HY000): There is no 'user'@'nonlocalhost' registered
The error came from a trigger on the table I was inserting data into that had been created on the production machine by a user@'host' that didn't exist on my development machine. the user was identical but the host wasn't.
MySQLdump includes that information in a dump and uses it to restore the same values.
So recreating the trigger on the development machine solved the problem.
I should probably look a bit closer into the MySQL bugs to figure out if this is a bug or just expected behaviour.
There might even be a parameter to disable this feature , but I didn't find it yet.
T-Dose 2008 CFP
At this weeks' geekdinner some people wondered what was up with T-Dose, and guess what .. their CFP has been out for ages.
Last year I just catched the end of Bert's talk and Some Abstract Type has also been spotted there before.
No reason to miss this year's edition.
drupal.org | Community plumbing
The Consequences of Being an Open Source Company
No Matt, my brain definitely wasn't idle.. I've been thinking about these problems for the better part of the last decade. And it seems like I`m not the only one who wants this discussion.
Dries told me that as a follow up to my previous post I should write a post with solutions to the problem. Difficult as I don't have the solutions yet.. If I had them .. well :)
Fact is that different types of opensource products might require different approaches Alfresco to my knowledge has little to no contributing community , Linux distributions tend to have a big one, if not just in the form of the different open source projects they pacakge. The MySQL community is more one of documentation, helping out and bugsquashing. So my ideas aren't valuable for everybody, which is maybe why Matt Asay can't understand me, he might be looking at only one side of the picture.
There are some little things that I can suggest however.
Open Source works because of people contributing to projects, Open Source companies should recognize that and figure out a way to return more business the partners that also contribute to their code , this way they can contribute both on commercial and financial level. If you keep sending business to non contributing partners at the loss of the ones that actually commit code, some people will be unhappy. Those contributing shops might not be bringing the big revenue for the vendors, but they sure are contributing.
The other part is in the support model, Matt somehow thinks I`m in the "everything must be free" camp. Wrong, I`m in the right price for the right product/service camp.
Which means that if I`m escalating a support issue of a customer of mine to a vendor, my time must also be paid for. However that's a difficult sale, my client already paid for his support contract , to the software vendor.
So my suggestion, back when RedHat came to the Bemelux, was to have different types of support contracts, a customer could get a direct contract with a vendor where no integrator could log the calls. Then with another contract type if the a partner actually logs a call for his customer he must get some kind of kickback for that...
One of the advantages there are that more first line calls can be tackled by local partners, partners that might know their customers better.. but they still have a backup if they can't solve the problem. Therefore less investments are to be made in a support organization by the vendor.
And last but not least , don't tell your partners what they can't do. They should be listening to their customers, if their customers choose for the open source version it's the customers choice, and the partner should be able to help his customer, the last thing you need to do is punish them for listening to their customers needs rather than the vendors. This is how the proprietary world works.
Oh and Matt, next time you are in Belgium, let's do another round of Buytaert vs Asay :)
Maybe we come up with some better ideas than the above ones.
Doesn't Matt Asay want Open Source integrators to earn a living ?
Or, why the Inuits won't partner on selling Ice from Alfresco unless they change their strategy.
I usually agree with lot of the things Matt Asay writes but today in Closing an open-source deal trough your systems integrator , he thinks the way to work with partners in an opensource environment is to force them to sell the commercial solutions of your products.
He also thinks you should block them from starting an implementation before the end customer has signed a purchase order.
Whew.. this must be the most stupid idea he had since he started his opensource career. The sad part is that I haven't seen a commercially backer of an opensource project dealing correctly with its contributing partners. He isn't solving the problem , he is creating a bigger one :(
Integrators and consultants are often the bigger contributors to a project because they are integrating new features for their customers, You know, their local , we speak your language , customers. So now Matt wants to force them not to sell services around GPL software anymore but sell the commercial versions ?
As lots of commercial opensource versions do not allow you to make changes to the code if you don't want to loose support your hands are tied again. And yes I have been in this situation before multiple times, a situation where , a commercially backed opensource project, required a couple of small changes to fit with a customer, because of these changes the commercial vendor would drop support , so the customer decided not to buy the license. Should a local integrator capable of helping such customer loose that deal because of a partnership ? Off course not .. It's perfectly understandable that a software vendor can't support every different patch. Shouldn't an integrator have the freedom to assist a customer in making these choices, and give him valuable advise ?
Forcing the integrator to sell the commercial version brings them back to the proprietary software vendor situation , where they couldn't solve issues either.
Mind this is a Category "C" user ,(an organization that has more money than time), which should be an easy win for the commercial opensource vendors.
Then there is the issue of Paying twice where a customer both pays for the time the integrator spends on solving his issue and the support contract. I`m stil looking for a solution for this one.
In the past we invested in different partnerships , some requiring certification, with different Open source vendors before, never got a dime back from these investments.
While our shop was a small but specialist expert knowledge center most deals that those other vendors had in our area went to the incompetent boxmovers that did volume, often totally screwing up the actual implementation. Whether we had contributed to the project, or in the case of Linux distributions were probably equally skilled to support the environment as the vendor itself didn't matter.
We didn't sell enough boxes , so we never got any deals back. Our business is advising people on how to implement open source , implement it for them and support them. We are working with both type A,B and C customers. But the commercial opensource vendors want to force us to go back to the old proprietary boxmoving model, sell licenses, don't sell solutions, Oh and No you can't fix that .. you'll have to wait for the next commercial release or lose support.
So how many of the opensource benefits should the customer give up ?
No Matt, this time your idea stinks,
This way skilled consultants that care about open source and contribute to the community are being punished for doing so, whereas they should actually be getting business back from the vendors, so they can earn money and contribute more on your product you force them to waste more time on the sales side. While the people that just move boxes, don't care if its an open source application or a proprietary package gain more. For them its just business as usual .. selling boxen.
It just doesn't make sense
This concept is just bad for opensource in general, motivated people will stop contributing to products they implement, as they see that their efforts aren't appreciated by the vendors.
What does an opensource project need ?
besides a community ?
Some people think that apart from a community and users you also need an infrastructure to support these users.
Murray ignited the discussion by pointing us to the fact that PostgreSQL doesn't have a bugzilla to report an track issues.
Different projects have different approaches. Both the kernel, Drupal and MySQL have build their own infrastructure. Others choose Sourceforge for their projects.
What's your approach ?
MySQL Engines: MyISAM vs. InnoDB
Why use InnoDB?
InnoDB is commonly viewed as anything but performant, especially when compared to MyISAM. Many actually call it slow. This view is mostly supported by old facts and mis-information. In reality, you would be very hard-pressed to find a current, production-quality MySQL Database Engine with the CPU efficiency of InnoDB. It has its performance "quirks" and there are definitely workloads for which it is not optimal, but for standard OLTP (Online Transaction Processing) loads, it is tough to find a better, safer fit.
Yummie MySQL Repository
It seems like Jeremy wants to be MySQL community president this week :)
The announcement of a MySQL yum repository is a good one but it's slightly confusing me .. didn't Jeremy already have this with
Dorsal, where there are also 5.1 builds. So what's the difference between Dorsal and the new yum repo anyway .
But he asks for Adittionals packages , well 5.1 to start with, apart from that the CentosPlus repo also has builds for Cluster , having a uniform place go get those to would be good.
And what about builds for CGE ?
Oh and while you are at it .. can you run genbasedir also .. that way we can also use apt4rpm :)
Now I all need is a repository with all drupal modules packaged separatly :)
Drupal and MySQL
For a company that wants to become the RedHat of Drupal, Kieran is pointing a lot to RedHat's competition :)
But indeed the Brainstorm idea is a good one... if Sun wants to keep up the "big user community, no contributor community" model for it's products this is the least they can do.
Kieran also calls for more crossposting between the mysql and drupal planets :)
Drupal in the cloud
It is not always easy to scale Drupal -- not because Drupal sucks, but simply because scaling the LAMP stack (including Drupal) takes no small amount of skill. You need to buy the right hardware, install load balancers, setup MySQL servers in master-slave mode, setup static file servers, setup web servers, get PHP working with an opcode cacher, tie in a distributed memory object caching system like memcached, integrate with a content delivery network, watch security advisories for every component in your system and configure and tune the hell out of everything.
Either you can do all of the above yourself, or you outsource it to a company that knows how to do this for you. Both are non-trivial and I can count the number of truly qualified companies on one hand. Tag1 Consulting is one of the few Drupal companies that excel at this, in case you're wondering.
My experience is that MySQL takes the most skill and effort to scale. While proxy-based solutions like MySQL Proxy look promising, I don't see strong signals about it becoming fundamentally easier for mere mortals to scale MySQL.
It is not unlikely that in the future, scaling a Drupal site is done using a radically different model. Amazon EC2, Google App Engine and even Sun Caroline are examples of the hosting revolution that is ahead of us. What is interesting is how these systems already seem to evolve: Amazon EC2 allows you to launch any number of servers but you are pretty much on your own to take advantage of them. Like, you still have to pick the operating system, install and configure MySQL, Apache, PHP and Drupal. Not to mention the fact that you don't have access to a good persistent storage mechanism. No, Amazon S3 doesn't qualify, and yes, they are working to fix this by adding Elastic IP addresses and Availability Zones. Either way, Amazon doesn't make it easier to scale Drupal. Frankly, all it does is making capacity planning a bit easier ...
Then comes along Amazon SimpleDB, Google App Engine and Sun Caroline. Just like Amazon EC2/S3 they provide instant scalability, only they moved things up the stack a level. They provide a managed application environment on top of a managed hosting environment. Google App Engine provides APIs that allow you to do user management, e-mail communication, persistent storage, etc. You no longer have to worry about server management or all of the scale-out configuration. Sun Caroline seems to be positioned somewhere in the middle -- they provide APIs to provision lower level concepts such as processes, disk, network, etc.
Unfortunately for Drupal, Google App Engine is Python-only, but more importantly, a lot of the concepts and APIs don't map onto Drupal. Also, the more I dabble with tools like Hadoop (MapReduce) and CouchDB, the more excited I get, but the more it feels like everything that we do to scale the LAMP stack is suddenly wrong. I'm trying hard to think beyond the relational database model, but I can't figure out how to map Drupal onto this completely different paradigm.
So while the center of gravity may be shifting, I've decided to keep an eye on Amazon's EC2/S3 and Sun's Caroline as they are "relational database friendly". Tools like Elastra are showing a lot of promise. Elastra claims to be the world's first infinitely scalable solution for running standard relational databases in an on-demand computing cloud. If they deliver what they promise, we can instantly scale Drupal without having to embrace a different computing model and without having to do all of the heavy lifting. Specifically exciting is the fact that Elastra teamed up with EnterpriseDB to make their version of PostgreSQL virtually expand across multiple Amazon EC2 nodes. I've already reached out to Elastra, EnterpriseDB and Sun to keep tabs on what is happening.
Hopefully, companies like Elastra, EnterpriseDB, Amazon and Sun will move fast because I can't wait to see relational databases live in the cloud ...
3 strikes
I'd call this the 3rd strike and everybody knows what happens next
Marc Fleury has some good answers to the most clueless industry reporter around, starting with:
Spring is touting itself as a JBoss replacement. Smart PR, but false. Spring is a development framework comprising wrappers and dependency injection on top of Hibernate and Tomcat runtimes, both developed, and monetized by JBoss.
You can drop some balls, no one can keep track of what's going on in Open Source land, it's difficult enough to track what's going on in MySQL, Drupal, Virtualization and Distribution land but if you realize you don't have the whole picture (like not really knowing how the different tools compare to each other) please keep quiet.
Acquia Dresses Up Drupal for Corporate Users
Acquia, a North Andover, Mass.-based startup, is announcing a supported product using Drupal, the open-source content managment system that underlies many of the community aspects on the web, from sites such as Fast Company to The Onion. It’s a rite of passage for an open-source project to get its very own shepherd that provides a measure of support beyond the forums and masses of independent programmers who churn out the code. For Drupal, the move is akin to Red Hat offering support for Linux and Sun taking MySQL under its wing for $1 billion.
Acquia has hired Dries Buytaert, the founder of Drupal, as its CTO; it scored $7 million back in December to build its 12-person team to this point. Jeff Whatcott, the VP of marketing for Acquia, says the company wants to continue to contribute to the Drupal code base and has no plans to create a proprietary form of code for enterprise use.
Acquia has readied some modules targeted at specific enterprise markets and will sell subscriptions to Acquia’s Drupal add-ons as well as services that support new and existing Drupal deployments. The Acquia-supported Drupal product is called Carbon, and will be ready in the second half of this year. An automatic update service for Carbon called Spokes will be available then as well.
Drupal and MySQL hIgh-availability
While I invited Kris Buytaert to present at the Drupal developer room at FOSDEM, I couldn't make it to his presentation as it was scheduled for the last presentation of the day. Oops! Fortunately, Luc Van Braekel recorded almost all Drupal presentations at FOSDEM so I'm sharing the video of Kris's presentation to make up for it. It's is well worth a look, and no, we are not family.
Drupal Devroom Presentation Online
So yesterday I presented my findings on MySQL Cluster (ndbd) and Drupal.
I`ll be writing them out in a more detailed document some day soon ..
But the short summary is that MySQL-Cluster and Drupal are not a good fit.
One has to realise that MySQL cluster isn't always a good solution and that every problem has it's own solutions. The nature of the tables of Drupal means that you need enormous amounts of memory.
I was a bit afraid that the present MySQL crowd woudln't really appreciate my findings, however I was wrong I got some good feedback from the present MySQL crowd , they too seem to want to promote the idea that Cluster is not a fit for every problem, and they were great help in answering some of the audience questions.
The presentation is here here
Honest Open Source
While posting about my talk on MySQL Cluster and Drupal I realized something very important about Open Source and how it differs from proprietary software.
Looking at a traditional software and infrastructure consultancy shop you will see that they carry a broad range of products which they care about and which they are actively offerering to customers. For this products they have formal arrangements with suppliers that will both give them licenses, support but also incentives to sell those licenses.
So when a sales guy or a consultant arrives at a customer they are already biased to plug a certain product, and they will try to convince the customer that the product they are selling is the right fit. Even tough when it's a total Misfit.
With the Open Source consultant he won't be plugging a product, he will be listening to his customer and will use the right tool for the right project .. afterall there is no vendor pushing him to plugg a certain solution, there is no higher margin if he chooses the red box over the green box (as he isn't choosing boxes) The opensource guy will be open and explain to the customer why product X does not fit his needs and product Y does, he will give his customer some honest advise. He will help the customer to solve his problem, not create more.
I was pretty surprised yesterday that even the guys from MySQL were supportive when I stated that MySQL Cluster was not a fit for Drupal. I can't imagine people from Oracle or IBM agreeing on one of their product being a misfit for their customer ..
Fosdem 2008, looking back
Fosdem is over .. and it was ... overcrowded :(
Honestly trying to squeeze into an overcrowded bar, then on saturday overcrowded rooms, or even not being able to enter that room (Mozilla and Embedded) , Fosdem is starting to become the victim of it's own success.
Some people are suggesting Fosdem to move to the Arenberg campus in "Brussels-East" dunnow if Leuven can actually host enough beds for Fosdem :)
However the Beer event problem would be solved but Philip will have to make arrangements with 'The greatest bar of Western Europe"
The talk about Xen on ARM was interresting however the grande finale missed, the MiniOS just didn't boot :( Kettle was interresting and I should start spending time with it :) But then again .. so are a zillion other things Too bad the SWOT analysis between Postgress and MySQL got cancelled.
but it left me some breathing space :)
The evening ended with a mixed crowd of local Linux geeks and Drupal folks in the restaurant on walking distance.
On sunday morning I realized it must be the Fosdem weekend when you are on the E19 direction Brussels around 0900 and there is no traffic :)
I was right on time for the Drupal 6 and 7 talks from Gabor and Dries , which off course meant I was going to be too late for Ian's talk. Luckily I catched the important parts. The virtualbox talk disappointed me .. this was a marketing talk for endusers, not a talk suitable for Fosdem :(
Pascal learned that integration Amazon basically is a fine dns problem :) Then after some chatter with the MySQL crowd I headed into the Conary talk.. I was expecting a bit more information around their rbuilder system
I wondered into too much talks on sunday afternoon . the MySQL Proxy talk , the end of atogs talk which he didn't want to repeat :) Karan's talk .. etc.
I had to miss Simon's talk about Posgtress HA on sunday for the obvious reason , but luckily I could catch him on saturday to get a short sneak preview..
But more about those obvious reasons in a separate post :)
My Pics are over here. You'll be recognising Jan Kneschke, LVB , Thomas Bonte , Geert Vanderkelen, Dries, Gabor,Matt Casters and others.
Drupal Devroom Videos Online
Luc just pointed me with to his recording of my talk about Drupal and MySQL HA yesterday at the Drupal Room at Fosdem
He also uploaded some other video's from the Drupal room.
More about my Fosdem 2008 experience later :)
Props to drupal
Congrats to Drupal for releasing version 6.0 yesterday. This site has been running 6.0 since about rc2 (upgrading every release along the way) and I've been very happy with it. Also, thanks to the developer(s) of the 'Channel Nine' theme, which I've adopted for my own (with some modifications).
Congratulations to the Drupal 6.0 release!
Yesterday, Drupal 6.0 was officially released - check out this screencast to get a 29-minute tour on the new features in this release.
We'd like to congratulate the Drupal Developer Team and Community for reaching this milestone and are happy that the MySQL Server continues to serve well as the database backend for this awesome content management platform!
I had the pleasure of evaluating and reviewing a previous release of Drupal for the Open Source Content Management System Award from Packt Publishing and it has been one of my favourites.
Keep up the good work!
Fosdem 2008
This February I`ll be heading to my 8th Fosdem in row.
I went to every single Fosdem so far, some years only one day because of other obligations but I was a round most of the time
During the first couple of years I was pretty active in the FIT team, helping out people to find the right rooms , keeping the fosdem contributions safe with Sven, funding different social events and devroom dinners ,
Later I was in charge of the HPC and Cluster devroom in which we also held the openMosix summits.
And back in 2004 I replaced Moshe at the openMosix Summit standing in front of a great audience at the Janson room.
As Wim just pointed out the Drupal Devroom schedule is out
and it seems like 2008 will be the year that I have to rush my talk in order for the Drupal Devroom to close its doors on Sunday evening. I'll be sharing my knowledge on MySQL cluster with the Drupal crowd for them to learn and benefit from.
I got Interviewed
Some people obviously think I`m still interresting enough to interview
Reminds me of my first interviews over a decade ago .. I`m still being remembered by one of my famous quotes back then rougly translated to "those toddlers steal my bandwith" , when being asked about how the growing userbase of the KULeuven PC rooms in the Dekenstraat influenced my network usage :) (Think 1994 ish :))
But the acutal news is that Profoss is coming closer and closer .. and you should now Register!
So all you Drupal and MySQL folks out there.. this is your chance to learn about virtualization !
