sorted by: recent | see :
popular
Content Tagged with User:marc + Web
Website Design Gone Horribly, Horribly Wrong
There are lots of ways in which a website can be annoying. Favourite methods include: rotating and blinking animated GIFs (or worse, Flash), popup advertising windows, unexpected background music files, or just plain all around atrociously ugly page design. (I’ve been quite guilty of this in the past!)
But until you’ve lived in China, or at least spent some time browsing around websites here on the mainland, there’s probably one way to annoy the living bejeezus out of people that you’ve never thought of.
To demonstrate, simply visit any Chinese website, such as the Bank of China or something else such as Chinaren. Don’t worry if you can’t see the characters, they’re not important for this experiment. (Windows XP users can add them by going to Control Panel /International and installing the Asian Font Pack, while Vista and Mac users will have all these fonts installed already).
Once you have one of these pages up in your browser window, click on a link or two. Click on some more links on those pages. Try to get back to where you came from. Within minutes, you’ll have at least a dozen browser windows littering your desktop, or at best, for those Firefox users with the correct settings, dozens of tabs.
You could be forgiven for thinking that this was specific to a few sites with particularly bad design. And you’d be totally wrong. This is completely endemic here in local website design, and is how the locals think that the “Internets” should work. Indeed, there is almost no concept of forward or back button usage any more, and it is not uncommon to see users with well over twenty browser windows littering their desktop at any given time. While Windows users can at least expect the Task Bar to group similar windows, Mac users just end up using the mouse to move the windows out of the way until needed later, or until they just close the browser application completely.
Ultimately, the problem becomes such that, if you want to fix the site design to not do things this way, you will confuse your user. When they click to go to a new page, and they then subsequently finish visiting it, they will close the browser window and proceed to go looking through their other browser windows until they find the one from whence (they hope) they came.
The only thing I can say? At least blatent ripoffs of other sites on the internet don’t seem to have felt compelled to introduce this behaviour into their clones. For everybody else, it’s going to take a while to change this design.
When MySQL Attacks!!!
The Setup
Imagine, if you will, the following scenario:
- You design a whole new database schema for your cool new scalable web-application. You’re using MySQL and the InnoDB datbase engine for everything, because your schema is so cool it uses all sorts of foreign keys and transactions and the like.
- You quickly set up MySQL and get your application going with your new schema on your development staging machine.
- You get MySQL up and running on your live server, play around with it for a bit to make sure it’s working, and then set up a my.cnf file with all sorts of caching and security goodies in it.
- You do a backup from your dev machine, restore it to the live server, and ta-daa!!! Your web application is up and running on your live server.
What you might not have noticed, especially if you – like me – have a few thousands rows of data, is that MySQL might have screwed you along the way and not really told you all that clearly.
The Problem
After a few days of operation on my live server, I started to notice a few weird things—foreign keys weren’t being enforced properly, and there were some values in the database that probably shouldn’t have been possible. I furrowed my brows and put it on my list of stuff to investigate.
Well, yesterday, I added a new table to the database, and it went something like this:
mysql> CREATE TABLE Fudgecicles
>(
> id INTEGER AUTO_INCREMENT PRIMARY KEY,
> value VARCHAR(255) NOT NULL,
>)
>ENGINE = InnoDB;
Query OK, 0 rows affected, 1 warning (0.10 sec)
Where did that warning come from?
mysql> SHOW WARNINGS;
It is here that MySQL tells me:
+---------+------+-----------------------------------------------------+
| Level | Code | Message |
+---------+------+-----------------------------------------------------+
| Warning | 1266 | Using storage engine MyISAM for table 'Fudgecicles' |
+---------+------+-----------------------------------------------------+
Augh! No! Bad! Bad, MySQL, Bad! Why on earth would it do that? I didn’t misspell InnoDB or even use “incorrect” casing in the name. There’s nothing wrong with the schema I specified and I’ve done this hundreds of times before.
Well, after some research, I then tried the following:
mysql> SHOW ENGINES;
And MySQL helpfully gave me the following:
+------------+---------+----------------------------------------------------------------+
| Engine | Support | Comment |
+------------+---------+----------------------------------------------------------------+
| MyISAM | DEFAULT | Default engine as of MySQL 3.23 with great performance |
| MEMORY | YES | Hash based, stored in memory, useful for temporary tables |
| InnoDB | DISABLED| Supports transactions, row-level locking, and foreign keys |
| BerkeleyDB | NO | Supports transactions and page-level locking |
| BLACKHOLE | NO | /dev/null storage engine (anything you write to it disappears) |
| EXAMPLE | NO | Example storage engine |
| ARCHIVE | YES | Archive storage engine |
| CSV | NO | CSV storage engine |
| ndbcluster | NO | Clustered, fault-tolerant, memory-based tables |
| FEDERATED | NO | Federated MySQL storage engine |
| MRG_MYISAM | YES | Collection of identical MyISAM tables |
| ISAM | NO | Obsolete storage engine |
+------------+---------+----------------------------------------------------------------+
12 rows in set (0.00 sec)
The InnoDB database engine had been disabled somewhere along the way and I hadn’t even noticed. It is enabled, by default, on the standard Linux and Mac OS X MySQL binaries that I’ve been downloading. So something changed along the way that made this all stop.
Far worse, I then began to worry about my existing tables, all of which we supposed to be InnoDB. Upon executing the following for each of them:
mysql> SHOW CREATE TABLE Fudgecicles;
I would see something like:
| Table | Create Table
| Fudgecicles | CREATE TABLE `Fudgecicles` (
`id` int(11) NOT NULL auto_increment,
`value` varchar(255) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8
Sure enough, every single one of my tables was MyISAM, and not the InnoDB it was supposed to be.
The Reason
The problem turns out to be that InnoDB is somewhat finicky about my.cnf settings, and will frequently refuse to operate if settings change in this file in such a way that makes any existing data incompatible with the way it would write new data.
In my case, it turns out that changing the settings for the InnoDB binary data and log file sizes were somehow incompatible with the existing binary data and log files (ibdata1 and ib_logfileX). Upon starting the server, InnoDB finds this inconsistent state and simply refuses to start up. This is the first problem.
The second, and far more serious problem, is that MySQL just switches the database persistence engine on you and only provides a little warning. If you’re loading in thousands – if not tens of thousands – of rows, those warnings are easily lost in the scroll-a-thon that ensues. This is bad behaviour. MySQL should simply refuse to create your table if your selected persistence engine is not available.
The Solution
Fixing this problem involved three parts. The first, and easiest, is to get the InnoDB engine back. You have two choices:
- You can either revert to the original settings you had for the my.cnf file to remove the inconsistent state that is freaking out poor ole’ InnoDB.
- You can delete the idbdata1 and ib_logfileX files with the inconsistent sizes, thus removing in a different way the same inconsistent state, and allowing InnoDB to startup with your new (and presumably better) configuration values in my.cnf.
I chose the second method.
| WARNING: Using this second method can result in data-loss if you’re not careful. Some database engines store things in the binary data and log files before writing them to the actual table files, and if they’re deleted, you might lose those changes. I only selected this path because all my tables were MyISAM, I had a full backup, and spent a good 10 minutes after deleting the files verifying that all data were correctly restored. |
You then stop and restart the MySQL database engine, and InnoDB will be back. You can verify that all is well in InnoDB land by executing:
mysql> SHOW InnoDB STATUS;
And you will receive a gloriously long and detailed set of information on how things are going.
Part two of the process involved converting my tables back to InnoDB.
mysql> ALTER TABLE FishSticks ENGINE = InnoDB;
This proves to be tedious and time consuming, as you cannot simply go from table A to table Z doing the conversion – because of the foreign keys and references, they have to be done in the right order. When one of the ALTER TABLE statements fails, you can find out what happened by re-executing the SHOW InnoDB STATUS command – it will tell you why it wouldn’t convert the table to InnoDB.
But, eventually, I got them all done. It was then that I noticed that none of the foreign keys were set up properly any more.
So, the last step of the process is to re-establish the FOREIGN KEYs. I did this by, for each foreign key in each table I had, executing the following commands:
mysql> ALTER TABLE FishSticks DROP KEY [dead foreign key name];
mysql> ALTER TABLE FishSticks ADD FOREIGN KEY (keyname) REFERENCES Table (fieldname);
The good news is that saving my database and getting back to all sorts of InnoDB goodness only took about an hour in total, for about 30 tables or so.
However, if MySQL had simply reported an error a few days earlier instead of blithely just switching tables types behind the scenes, I might have avoided this whole mess in the first place. Oh well, at least I learned a few neat little commands I can play around with now! Lesson learned!
Here’s to hoping that this article helps some other folks solve the same problem a bit quicker!
URLs as APIs
On a recent contract, I was the lead developer on a web application to help gather information about open source projects called Swik (If you haven’t checked it out yet, do so and send feedback on it). The project is written using PHP and MySQL, and one of the key things about which the designer of the site, Alex, was adamant was in how the URLs looked.
The original URL scheme in the prototype had URLs along the lines of:
http://swik.net?project=554323&list=325244&edit=true
While this worked just fine and in no way affected the basic usability of the application, the argument was made that there is a class of web applications that actually benefit from more structured URL design. I found this the most ludicrous thing I had ever heard and implemented the new scheme largely because I was paid to. In the new scheme, the above link would translate into something roughly as follows:
http://swik.net/edit/project/FreeBSD/About
We would continue to trade jibes over this for some time, with me calling him a “URL Freak”, and he calling me a dim-witted clod (well, in all fairness, he’s much nicer than that, and just calmly argued that I was wrong).
So it is with some mild amount of chagrin that I must admit that he’s probably more right on this one than wrong. As I look around at sites that I use on a regular basis, I realise that, as a more technical person, I do use well designed URL systems whenever possible. As a fast typer, the ability to enter more robust URLs ultimately helps me navigate to my final destination much quicker than I could otherwise with the mouse and multiple page reloads.
The example of a website that uses URLs to do its work that most quickly comes to mind is del.icio.us. Many users do not even realise it, but this site can be entirely manipulated from the address bar.
For those not familiar with the site, it is often described as a “social bookmarking” application: instead of using your web browser’s bookmark feature, you store bookmarks in del.icio.us, tagging them with keywords or other descriptive comments when you add them. This has a couple of key advantages:
First, if you are a multi-computer/mulit-browser user, then all of your bookmarks are stored in one place for you to come back and look at later. Since this place is public, however, this probably means that you should keep your fetish dungeon pr0n links stored somewhere else, unless you do not mind others knowing that you are a big fan.
The far more useful and powerful feature of del.icio.us is that you can use the tags that you (and others) associate with links to see what others are associating or storing with the same “subject”.
For example, to see all the links that people have associated with the tag ajax, you can enter:
http://del.icio.us/tag/ajax
Similarly, to see all those links associated with the word Lamborghini, you type:
http://del.icio.us/tag/lamborghini
And then things get crazy. You can actually search for cross-sections of multiple tags. To see all those links people have added with all three of PHP, MySQL, and Apache, you can type in the address bar:
http://del.icio.us/tag/php+mysql+apache
Want to learn about everything that only I have tagged with php ?
http://del.icio.us/marcwan/php
To get an RSS feed for any of these, you just insert “rss” as the first parameter to the URL string:
http://del.icio.us/rss/marcwan/php
This last innovation lets us see the use of URL schemes as an API (Application Programming Interface) for websites. If I am building an application, and want to use information from sites such as del.icio.us (within polite limits or with the permission of the site author(s)), I can use more structured output such as RSS feeds to get these data:
<?php
$tag = rawurlencode($tagName);
$url = "http://del.icio.us/rss/tag/$tag";
$feed = file_get_contents($url);
// pass this to an RSS reader, the XML DOM, or otherwise
// parse through it by hand.
?>
Look more closely, and you will see other web applications that do similar things. Flickr is built in a very similar way. You can obviously use the well designed user interface to browse your way through the application, but the URLs are also sufficiently well designed that you can quickly find all photos of Sendai, Japan by entering the following:
http://www.flickr.com/photos/tags/sendai
To see all the photos that the user samantha has tagged with girl, you can type:
http://www.flickr.com/photos/samantha/girl
Even chipmunkninja.com is designed with simple to use URLs in mind. All articles are given a short name when stored, and can easily be viewed with something similar to the following:
http://www.chipmunkninja.com/article/osxamp
There is indeed some extra work required to make such a URL system work, but for some reason, it is significantly more pleasant to look at when browsing to a site.
Even less “geeky” sites, such as BBC News often let you narrow your search down somewhat. Changing the 2 to a 1 in the URL news.bbc.co.uk/2/hi.html takes you from the worldwide home page to the page more tailored for local UK readers. Similarly, to see those articles about Africa or the weather, you can enter:
http://news.bbc.co.uk/2/weather
Of course, beyond that, the URLs become less workable, and you are back to more traditional navigation schemes.
Am I arguing that all sites should be designed in such a way? Not at all. For most sites, most users do not care and often do not even look in the address bar (which is why so many phishing attacks appear to work so well). Whether their URLs are neat and compact or hideous monstrosities such as any URL you will see on Amazon.com completely uninteresting.
It must also be noted that URL “prettiness” should probably have its limits. If you are actively dropping features or contorting your web application to silly extremes just to keep URLs looking nice, you are probably doing yourself and your users a disservice. By looking at the most common user scenarios or “high value” URLs, you can probably find the appropriate balance between usability and application complexity.
While not necessarily appropriate for all users and all web applications, implementing a robust URL scheme in our web applications can be one of many ways in which we help people them and interact more dynamically with them.
PHP Book Addenda I
As I sat down to edit “Core Web Application Programming with PHP and MySQL”, I would sometimes find errors in the text so blindingly obvious and stupid that I would question whether or not I was truly qualified to write such a book. And yet, after talking with some other people who write books (and recalling days when I wrote huge amounts of code), it seems that this is all common and with much proof-reading and the hard work of some friendly reviewers, I was able to write a book of extremely high quality.
Of course, that just meant I would be even more devastated when the first technical errors WERE found in the book.
There have been a couple, but they’re not that killer serious.
Chapter 21 Error
In Chapter 21, where the book discusses writing your own output handler, the constant in the $_SESSION array to check is HTTP_ACCEPT_ENCODING, without the letter ‘S’ on the word accept.
The Accompanying Source Code
There are a couple of errors in the source code, the most glaring of which DID get fixed, but never made it on to the shipping CD ROM (d’oh!). In the SimpleBlog sample, in the file lib/entrymanager.inc, the class DBManager is accidentally misspelled DBMananager. Just fix it and change it back to the correct spelling and the sample will compile fine under PHP 5.0.x (x <= 4).
The other problem with the samples is that some new things have appeared in newer versions of PHP 5.0.y (y >= 5). PHP now defines a class called InvalidArgumentException, which conflicts directly with the class I have defined using the same name. The easy fix for this problem is to simply change the name of the class slightly, to something like MyInvalidArgumentException or some such thing.
To save you the hassle of tracking down and fixing all of these problems, I have put a new copy of the book source code up on the chipmunkninja.com servers. You can download these from here: phpwasrcupdate_2005-12-01.zip.
As always, if you see any other problems or errors in the book, or just want to comment on it, please feel free to drop me some mail.
I remain chagrined, but I’ll get over it.
Announcing Payjacks, an Object-Oriented PHP/Ajax Web Application Framework
I am happy to announce the immediate availability of Payjacks, currently at version 0.2.0. Payjacks is a PHP/Ajax web application framework I’ve written using the object-oriented features in PHP5+.
Payjacks can be downloaded here:
http://chipmunkninja.com/download/payjacks-0.2.0.tar.gz
What is Payjacks?
Payjacks is an object oriented PHP-Ajax web application framework I’ve written to help write robust and organised web applications. It was designed to require a minimal amount of effort to get your own web application up and running, while helping with such tasks as accessing a (MySQL, currently) database or providing a framework for sending asynchronous Ajax requests back to the server.
Payjacks uses many of the new object-oriented features in PHP 5 to do its work, and handles most of the details required to run a robust web application.
Some of the main features are:- Fully object-oriented.
- Support for various types of Asynchronous Ajax requests, including submitting forms asynchronous or simply sending your own data strings asynchronously
- Robust error handling and exception handling, allowing for customisable error pages and otherwise helping you help your users when things go wrong.
- Support for URL processing. Thus, instead of having URLs such as http://mysite.blabhalblahblah.com/SomePageName.php?showarticle=getname you can have your site do things like: http://mysite.blhablahblah.com/showarticle/name. This helps make your application easier to use for power users and helps with search engine indexing.
- The ability to create simple pages if you just want to test things out and don’t want the hassle of setting up a web application.
- A full set of samples to show you how to set things up.
- The framework has been designed from day one with web application security in mind.
- Full support for UTF-8 and localisable web sites.
When should I use Payjacks?
I use Payjacks for almost all of my “web page writing” these days. You can use the SimplePage class to just throw out a single PHP/HTML page, or you can use the WebApplication class and template off one of the samples to create more robust web applications.
Since Payjacks takes care of so many of the details that many web application authors simply neglect (error handling, URL processing, security details), using it gives you nearly everything you want except for the actual HTML of your pages.
If you want to use asynchronous Ajax requests, Payjacks helps to make this very easy too.
What does Payjacks not do?
Payjacks does not help with generation of well-styled HTML or page content. It merely gives you the framework in which to place your content. There is nothing preventing you from designing and developing perfectly secure and robust yet hideously ugly web applications using Payjacks. It is for this reason that graphic designers earn their paycheques. Even my blog site, http://chipmunkninja.com is very rudimentary and clearly shows the limits of my design skills.
Payjacks Requirements
In order to run Payjacks, you need:
- PHP 5.0 or later. I’ve tested on 5.0.[45] and 5.1.4. PHP needs to have mbstring and mbregex built-in for any sites not using strict Latin1 pages.
- A web server. If you want this web server to support the web application
system that Payjacks is built around, it must have some sort of URL
re-writing capability.
On Apache, this is mod_rewrite, and can easily be compiled into your Apache build. I have tested and run this on Apache 1.3.xx and 2.0.yy.
For IIS, there are a number of mod_rewrite clones, ranging from very free and open source to very expensive and not open source.
More
Payjacks is currently version 0.2.0 and should be considered development quality only right now. I plan on making some minor changes to function signatures in the next few versions (mostly to get rid of some parameters that I thought I would need for localisation, but have since realised were the wrong way to go about the problem).
If you have any bugs, comments, or questions, please do not hesitate to contact me at marcwan@chipmunkninja.com.
Payjacks, an Object Oriented PHP/Ajax Web Application Framework
Payjacks is an object oriented PHP – Ajax web application framework I’ve written to help write robust and organised web applications. It was designed to require a minimal amount of effort to get your own web application up and running, while helping with such tasks as accessing a (MySQL, currently) database or providing a framework for sending asynchronous Ajax requests back to the server.
Payjacks uses many of the new object-oriented features in PHP 5 to do its work, and handles most of the details required to run a robust web application.
Web
object-oriented
user:marc
Requests
asynchronous
PHP
Ajax
Framework
Application
License:BSD