sorted by: recent | see :
popular
Content Tagged with InnoDB + falcon
Falcon Transactional Characteristics
It’s time to continue our series on the transactional storage engines for MySQL. Some might question why I even include Falcon because it is very much beta at this time. MySQL, however, has made quite an investment into Falcon, and while it is currently beta, the code is improving and it looks like that it will be production-worthy when MySQL server 6.0 hits GA.
If this is the case, it is important to begin to understand what Falcon was designed for and how it differs from other transactional engines such as InnoDB. I am going to concentrate quite a bit on the Falcon/InnoDB comparison as that is what everyone wants to talk about. This is despite my having heard MySQL employees repeatedly make statements to the effect of, “Falcon is not going to replace InnoDB,” or “Falcon is not competing with InnoDB.” Well, take that with a grain of salt. It certainly seems to me that they are competing for the same spot.
Warning
As I said, Falcon is beta. First off, don’t even try to use it in production. Using it in production means you will also be using MySQL Server 6.0, which itself is considered alpha. Your data will explode, be corrupted, or eaten by jackals. It won’t be pretty. It will cause great pain.
In addition, the features of Falcon are still changing. What I say here might or might not be accurate in the future.
End of Warning
So, why was Falcon even created?
MySQL
replication
InnoDB
falcon
mvcc
row-based
statement-based
Notes from Falcon from the beginning
Here is the quick notes from the session Falcon from the beginning by Jim Starkey and Ann Harrison
- Why Falcon
- Hardware is evolving rapidly, world is changing, so taking advantage
- Customers need ACID transactions
- Where hardware is going
- CPUS breed like rabbits (more sockets, cores, threads/core)
- Memory is bigger, faster and cheaper
- Disks are bigger and cheaper but not much faster
- In general boxes are getting cheaper
- Where applications are going
- batch - dead
- timesharing - dead
- departmental computing - dead
- client server - fading fast
- application servers for most of us
- web services for the really big buys
- Database Challenges
- Traditional challenge
- exhaust CPU, memory and disk simultaneously
- Tradeoffs
- use memory to page cache to avoid disk reads
- record cache to avoid page cache manipulation
- use CPU to find the fastest path to record
- use CPU to minimize record size
- Synchronize most data structures with user mode read/write locks
- Synchronize high contention data structures with interlocked instructions
- Architecture
- Incomplete in-memory db with disk backfill
- Multi-version concurrency control in memory
- Updates in memory until commit
- Group commits to a single serial log write
- post-commit multi-threaded pipe line to move updates to disk
- Incomplete in-memory database
- records cached in memory
- separate cache for disk pages
- record cache hits 15% the cost of a page cache hit
- record cache is more memory efficient than page cache
- Record Encoding - cache efficiency
- records encoded by value, not declaration
- string “abc” occupies the same space in varchar(3) or varchar(4096)
- the number 7 is the same where small, medium, int, bigint, decimal or numeric
- MVCC
- update ops create new record versions
- new one is tagged with id, points to old version
- keep tracks which
- Updates are in memory
- held in memory pending commit
- index changes held in memory
- verb rollback is dirt cheap
- trxs rollback is dirt cheap
- At commit
- pending record updates flushed to serial log
- pending index updates flushed to serial log
- commit record written to serial log
- serial log flushed to the oxide
- and trx is also committed
- Memory is infinite, so
- large txns chills uncommitted data (flushes it to the log early)
- chilled records can be thawed
- scavenger garbage collects unloved records periodically
- when things get really had, entire record chains flushed to backlog
- Weakness
- transactions are ACID but not serializable
- latency advantage disappears at saturation
- very large transactions degrade performance
- optimized for web, not batch
- Strengths
- runs like a memory db when data fits
- scales like disk-based db when db doesn’t fit in cache
- lowest possible latency for web apps
- absorbs huge spiky loads
- Performance
- benchmark against InnoDB vs Falcon only
- DBT2 benchmark (what about sysbench?)
- High contention
- Writes intensive - 40% records touched are updated
- measures only performance at saturation
- DBT2 is InnoDB’s best spot and Falcon’s worst, so do not take benchmark results, decide on what you want
- When should you use what ?
- don’t need ACID ? then MyISAM is good
- single processor, small memory - InnoDB is good
- large transactions, batch inserts/updates, InnoDB is good
- multi cores, more memory, more threads , use Falcon
- For web, Falcon is hard to beat
MySQL?s storage engine program picks up steam
The solidDB for MySQL database engine for MySQL may have lost its sponsor following IBM’s acquisition of Solid Info Tech but events at this week’s MySQL Conference and Expo prove the certified engines program is alive and well.
Not only has Oracle announced that its Innobase subsidiary has updated InnoDB transactional storage engine, but there is also a new member of the certified enginesprogram.
Kickfire has recently emerged from stealth mode with its data warehousing appliance based on MySQL, column-store software, data compression, and a proprietary SQL processor.
Additionally, another potential new storage engine partner emerged in the form of ScaleDB, which is promising to deliver its new scalable storage engine for MySQL in the fourth quarter of this year.
Then of course there will be MySQL’s own Falcon engine, which entered beta testing in March and will be available with MySQL 6.0.
Storage Engines at the MySQL Conference
I’ll be following closely the progression of Storage Engines available in the MySQL Database server, well soon to be available when 5.1 gets to GA (hopefully by end of Q2 which is what we have been told). Tick, Tick, time is running out.
PrimeBase XT (PBXT) and Blob Streaming is obviously my clear focus, actually now working for PrimeBase Technologies, the company which I want to note for people is an Open Source company, committed at providing an open source alternative to the other commercial players. You also have at the MySQL Conference talks on the the existing InnoDB from Innobase (a subsidiary of market RDBMS leader Oracle). There is a Nitro presentation, an Infobright presentation, no Solid presentation surprisingly (the IBM news happening after submissions closed). We also have from MySQL, presentations on the internally developed storage engines Falcon and Maria, both products that won’t even be in 5.1 but 6.0, however Maria is presently a different branch of 5.1 so I don’t know how that works. Will it be in 5.1?
But what I want to seek is more news of KickFire, a Diamond Sponsor, an engine with embedded H/W, something that’s been obviously worked on in reasonable stealth. For me it’s not just interesting, it’s a competitor in our technology space, so I’ve been researching Joseph Chamdani and some of his patents.
Plenty of news in the past few weeks on Kickfire including Kickfire Update by Keith Murphy on April 3, Kickfire: stream-processing SQL queries by Baron Schwartz on April 4, Kickfire looking to push MySQL limits by Farhan Mashraqi on April 4, and Kickfire Kickfire Kickfire by Peter Zaitsev on April 4, and myself back on March 23.
So what can I make from the lack of company information and posted information to date.
- Hardware based acceleration.
- No Solid State Drive (SSD) Technology, at least not yet but C2App mentions SSD.
- Data Warehousing, lending to thinking it’s not a transactional storage engine
- A new storage engine and a new approach to data storage. I find this surprising, as it takes years to develop a feature complete storage engine, and most new 5.1 storage engines are indeed existing products, take Nitro, Solid, Infobright and Falcon. Only PBXT has been written from the ground up for MySQL 5.1, so looking to know more about it’s development
- Expensive, it’s dedicated H/W + (assuming) MySQL Enterprise + Storage Engine