sorted by: recent | see :
popular
Content Tagged with algebra + soa
Information Algebra
I get more information from two newspapers than from one - but not twice as much information. So how much more, exactly? That depends how much difference there is between the two newspapers.
Even if two newspapers report the same general facts, they typically report different details, and they may have different sources. To the extent that there are differences in style and detail between the two newspapers, this typically reinforces my confidence in the overall story because it indicates that the journalists are not merely reusing a common source (such as a company press release).
In the real world, we are accustomed to the fact that information and intelligence needs double-checking and corroboration. And yet in the computer world, there is a widespread belief that it is always a good thing to have a single source of information - that repeated messages are not only unnecessary but wasteful. Data cleansing wipes out difference in the name of consistency and standardization, leaving the resulting information flat and attenuated. A single source of information ("single source of truth") sometimes means a single source of failure - never a good idea in an open distributed system.
Writing about this in an SOA context - when three heads are better than one - Steve Jones describes this as redundancy, and points out the potential value of redundancy to increase reliability. He quotes Lewis Carroll (as Andrew Clarke points out, it was actually the Bellman): "What I tell you three times is true."
The same quote can be found at the head of Chapter 3 of Gregory Bateson's Mind and Nature, available online as Multiple Versions of the World. This expands on Bateson's earlier slogan "Two descriptions are better than one".
Bateson himself used the word "redundancy", but it is not a simple redundancy that can be plucked out without a second thought. Thinking about the consequences of adding and subtracting redundancy is a hard problem - Paulo Rocchi calls it calculus, but I prefer to call it algebra.
Even if two newspapers report the same general facts, they typically report different details, and they may have different sources. To the extent that there are differences in style and detail between the two newspapers, this typically reinforces my confidence in the overall story because it indicates that the journalists are not merely reusing a common source (such as a company press release).
In the real world, we are accustomed to the fact that information and intelligence needs double-checking and corroboration. And yet in the computer world, there is a widespread belief that it is always a good thing to have a single source of information - that repeated messages are not only unnecessary but wasteful. Data cleansing wipes out difference in the name of consistency and standardization, leaving the resulting information flat and attenuated. A single source of information ("single source of truth") sometimes means a single source of failure - never a good idea in an open distributed system.
Writing about this in an SOA context - when three heads are better than one - Steve Jones describes this as redundancy, and points out the potential value of redundancy to increase reliability. He quotes Lewis Carroll (as Andrew Clarke points out, it was actually the Bellman): "What I tell you three times is true."
The same quote can be found at the head of Chapter 3 of Gregory Bateson's Mind and Nature, available online as Multiple Versions of the World. This expands on Bateson's earlier slogan "Two descriptions are better than one".
Bateson himself used the word "redundancy", but it is not a simple redundancy that can be plucked out without a second thought. Thinking about the consequences of adding and subtracting redundancy is a hard problem - Paulo Rocchi calls it calculus, but I prefer to call it algebra.
Doubting Events
In my previous post on Cancelling Events, I pointed out that new information sometimes causes us to revise our opinion as to whether a given event had occurred. Marco Seiriö wants to handle my example by adding probability into an event model. According to Marco, you would require that each part of the system can deal with these probabilities.
I can see that probabilistic events could be useful sometimes. However, I am not convinced we need to propagate these probabilities (and the accompanying complexity) throughout the system. I'd prefer to find a way of containing the complexity, so that some parts of the system are presented with a simple binary event statement (either it happened or it didn't) while other parts of the system may be presented with a more complex probabilistic event statement (it might have happened, with probability X%). This is a form of attenuation. - it can be regarded as an application of the need-to-know principle.
This attenuation could be managed architecturally by layering - for example we might separate a process coordination layer (which knows about the probabilities) from a process execution layer (which doesn't).
There is another problem with probabilities - which is that they may change continuously. If a promised action does not appear, the probability that the other person has forgotten increases with time. In the car accident example, if the driver fails to respond under certain conditions, it becomes increasingly likely (but still not certain) that the driver is dead or unconscious. The emergency response parts of the system may need to respond in real-time or near-real-time to this continously shifting probability, but we want to decouple this from other parts of the system that do not have this requirement.
More fundamentally, probability introduces some algebraic challenges that can be solved for relatively simple examples (and perhaps the car accident example is relatively simple) but don't scale for more complex examples (I guess I'll have to construct one).
I can see that probabilistic events could be useful sometimes. However, I am not convinced we need to propagate these probabilities (and the accompanying complexity) throughout the system. I'd prefer to find a way of containing the complexity, so that some parts of the system are presented with a simple binary event statement (either it happened or it didn't) while other parts of the system may be presented with a more complex probabilistic event statement (it might have happened, with probability X%). This is a form of attenuation. - it can be regarded as an application of the need-to-know principle.
This attenuation could be managed architecturally by layering - for example we might separate a process coordination layer (which knows about the probabilities) from a process execution layer (which doesn't).
There is another problem with probabilities - which is that they may change continuously. If a promised action does not appear, the probability that the other person has forgotten increases with time. In the car accident example, if the driver fails to respond under certain conditions, it becomes increasingly likely (but still not certain) that the driver is dead or unconscious. The emergency response parts of the system may need to respond in real-time or near-real-time to this continously shifting probability, but we want to decouple this from other parts of the system that do not have this requirement.
More fundamentally, probability introduces some algebraic challenges that can be solved for relatively simple examples (and perhaps the car accident example is relatively simple) but don't scale for more complex examples (I guess I'll have to construct one).
Economic Rent
Can the business case for SOA be based on the concept of economic rent? I'd be interested to get some reaction to the following preliminary discussion ...
Economists use the term "rent" to mean something different to the everyday meaning of the term. It is not the rental income generated by renting out an asset (such as land) but the market power associated with a given market position. For example, let's say that an executive would work for $250,000 per year, but manages to negotiate a salary of $1,250,000 per year. The difference between these two figures ($1m) is a measure of his market power, and this is what economists refer to as economic rent. Economic rent is also earned by professionals with some barrier to entry (such as doctors and lawyers) and by celebrity performers (such as musicians and sportsmen).
There are two alternative definitions of economic rent, yielding somewhat different calculations. In classical economics, it is defined as the difference between the actual income and the necessary income. For example, imagine a popular musician who currently earns around £50,000 per concert. If his popularity waned, his earnings might drop to around £10,000 per concert, but he would continue to play. The difference between these two figures is pure excess - economic rent.
Followers of Pareto use a slightly different definition - the difference between the actual income and the next best opportunity. For example, if our musician wasn't playing concerts, his next best activity might be as a session musician at £4000 per week.
John Kay writes: "economic rent arises from differentiation and migrates to wherever in the value chain scarcity is found". Kay illustrates this with The Story of Champagne. Under perfect market competition, there are no clusters of market power, and no economic rent - prices are adjusted until the level of supply exactly equals the level of demand.
With software and services, of course, the potential income sometimes bears no relationship whatsoever to the costs of production. Most of the profits of Google or Microsoft can be regarded as economic rent in this sense. Some people regard these profits as excessive, while others would justify such profits as reward for past risks and initiatives taken; in any case, the continuation of these profits depends on maintaining some exclusive advantage, something that is difficult for others to replicate.
If a service provider is to invest in developing a new service, then this may be done with the expectation of receiving sufficient reward to cover the (mostly front-loaded) costs and risk. The level of value generated by a service depends on its exclusivity and differentiation: if the service would be easy to copy then there is little prospect of economic rent; but if the service gains a strong foothold, then it may be difficult to dislodge, at least in the short term.
It is for this reason that network effects are so important to the economics of SOA. A service with ten million consumers delivers greater value to each consumer than a similar service with only ten thousand consumers. This advantage enables the provider of the more popular service to extract an economic rent.
Another important consideration is the relationship between fixed development costs (sunk costs?) and ongoing operational costs (opportunity costs?). (If I understand the Wikipedia article correctly, this relationship is one of the areas where classical economics and paretian economics diverge.) SOA and other forms of virtualization (including grid) make it easier to shift costs from fixed to variable - but of course this may not deliver the best economic advantage to the service provider.
If you have some market power, how do you calculate the best possible price? There is some tricky mathematics involved here: let's assume that the more you charge, the fewer users you have, and let's assume that there is a non-zero cost of servicing each user. Under certain conditions, the maximum rent is given by a formula known as Hotelling's Rule.
In case you were wondering, Hotelling has nothing to do with hotels. Harold Hotelling was a mathematical statistician, also known for Hotelling's Law ("in many markets it is rational for producers to make their products as similar as possible") and Hotelling's Lemma (relates the supply of a good to the profit of the good's producer).
Wikipedia: Comparative Advantage Economic Rent, Hotelling's Law, Hotelling's Rule
Economists use the term "rent" to mean something different to the everyday meaning of the term. It is not the rental income generated by renting out an asset (such as land) but the market power associated with a given market position. For example, let's say that an executive would work for $250,000 per year, but manages to negotiate a salary of $1,250,000 per year. The difference between these two figures ($1m) is a measure of his market power, and this is what economists refer to as economic rent. Economic rent is also earned by professionals with some barrier to entry (such as doctors and lawyers) and by celebrity performers (such as musicians and sportsmen).
There are two alternative definitions of economic rent, yielding somewhat different calculations. In classical economics, it is defined as the difference between the actual income and the necessary income. For example, imagine a popular musician who currently earns around £50,000 per concert. If his popularity waned, his earnings might drop to around £10,000 per concert, but he would continue to play. The difference between these two figures is pure excess - economic rent.
Followers of Pareto use a slightly different definition - the difference between the actual income and the next best opportunity. For example, if our musician wasn't playing concerts, his next best activity might be as a session musician at £4000 per week.
John Kay writes: "economic rent arises from differentiation and migrates to wherever in the value chain scarcity is found". Kay illustrates this with The Story of Champagne. Under perfect market competition, there are no clusters of market power, and no economic rent - prices are adjusted until the level of supply exactly equals the level of demand.
With software and services, of course, the potential income sometimes bears no relationship whatsoever to the costs of production. Most of the profits of Google or Microsoft can be regarded as economic rent in this sense. Some people regard these profits as excessive, while others would justify such profits as reward for past risks and initiatives taken; in any case, the continuation of these profits depends on maintaining some exclusive advantage, something that is difficult for others to replicate.
If a service provider is to invest in developing a new service, then this may be done with the expectation of receiving sufficient reward to cover the (mostly front-loaded) costs and risk. The level of value generated by a service depends on its exclusivity and differentiation: if the service would be easy to copy then there is little prospect of economic rent; but if the service gains a strong foothold, then it may be difficult to dislodge, at least in the short term.
It is for this reason that network effects are so important to the economics of SOA. A service with ten million consumers delivers greater value to each consumer than a similar service with only ten thousand consumers. This advantage enables the provider of the more popular service to extract an economic rent.
Another important consideration is the relationship between fixed development costs (sunk costs?) and ongoing operational costs (opportunity costs?). (If I understand the Wikipedia article correctly, this relationship is one of the areas where classical economics and paretian economics diverge.) SOA and other forms of virtualization (including grid) make it easier to shift costs from fixed to variable - but of course this may not deliver the best economic advantage to the service provider.
If you have some market power, how do you calculate the best possible price? There is some tricky mathematics involved here: let's assume that the more you charge, the fewer users you have, and let's assume that there is a non-zero cost of servicing each user. Under certain conditions, the maximum rent is given by a formula known as Hotelling's Rule.
In case you were wondering, Hotelling has nothing to do with hotels. Harold Hotelling was a mathematical statistician, also known for Hotelling's Law ("in many markets it is rational for producers to make their products as similar as possible") and Hotelling's Lemma (relates the supply of a good to the profit of the good's producer).
Wikipedia: Comparative Advantage Economic Rent, Hotelling's Law, Hotelling's Rule
REASC
Jean-Jacques Dubray (now with SAP) has posted an interesting SOA pattern on his blog. REASC: a pattern for constructing Composite Applications.
This pattern seems to assume a fairly simple event algebra - each event refers to a state-change of a single resource. This appears to restrict the pattern to atomic events.
How can the pattern be extended to support compound events? For example, in building an SOA to support the real-time business, I may want to create BI services that generate compound events. For example, an event may be triggered when the frequency of some transaction exceeds some threshold, or when some new pattern is detected in the data. These compound events might possibly be composed from atomic events, but this may not be the best way to specify them. In any case, I do not want to be forced to define compound (aggregate) resources that correspond to these compound events.
It is possible that JJ intends this kind of event algebra to be contained within the Coordinator. But I should prefer to elaborate the event itself to allow for event composition. This would also allow for amplification and attenuation (as found in Stafford Beer).
I am also interested in exploring the use of the REASC pattern for the service-based business, where resource perhaps equates to business asset. How might we interpret the Coordinator function in service-based B2B collaborations?
Technorati Tags: SOA service-oriented
This pattern seems to assume a fairly simple event algebra - each event refers to a state-change of a single resource. This appears to restrict the pattern to atomic events.
How can the pattern be extended to support compound events? For example, in building an SOA to support the real-time business, I may want to create BI services that generate compound events. For example, an event may be triggered when the frequency of some transaction exceeds some threshold, or when some new pattern is detected in the data. These compound events might possibly be composed from atomic events, but this may not be the best way to specify them. In any case, I do not want to be forced to define compound (aggregate) resources that correspond to these compound events.
It is possible that JJ intends this kind of event algebra to be contained within the Coordinator. But I should prefer to elaborate the event itself to allow for event composition. This would also allow for amplification and attenuation (as found in Stafford Beer).
I am also interested in exploring the use of the REASC pattern for the service-based business, where resource perhaps equates to business asset. How might we interpret the Coordinator function in service-based B2B collaborations?
Technorati Tags: SOA service-oriented
SOA Algebra
How do we reason about services and SOA? In particular, how do we reason about composition and decomposition?
One important type of reasoning involves the ability to think numerically. For example:
Algebra is the branch of mathematics that deals with structure, relation and quantity. The word Al-Jabr is the arabic for reunion - in other words composition. This is therefore the form of reasoning that formalizes and quantifies the composition and decomposition of separate elements. Algebra tells us when it is safe to use addition and multiplication, and when we need something more sophisticated.
For example, if artefact A is composed from components {A1, A2, ..., An}, then the reliability of A is some algebraic function of the reliability of the components A1 to An. Conversely, if you have a requirement for a given level of reliability, you need some algebraic function to decompose this requirement into an equivalent set of sub-requirements.
In an earlier post on Reliability and Availability, I took issue with a vendor white paper, which assumed that all you have to do is multiply the parts to get the whole. This is obviously only true if you make some fairly simplistic assumptions about the composition structure, and is not true in the general case.
Meanwhile, Jeff Schneider takes issue with a confusing formula of SOA costs proposed by Dave Linthicum, which apparently involves adding different kinds of complexity.
If people are getting into these difficulties, where is the SOA algebra to haul them back onto solid ground? I spoke to a friendly professor (Bashar Nuseibeh), who advised me that the formal theory of composition is not advanced enough to support precise quantification. He referred me to some of the work he has been doing with Michael Jackson and others on formalizing composition.
But if I can't have precise, I'll settle for approximate. Surely anything would be better than the algebraic fallacies and muddle currently in circulation?
R. Laney, L. Barroca, M. Jackson, and B. Nuseibeh , Composing Requirements Using Problem Frames , Proceedings of 12th IEEE International Requirements Engineering Conference (RE'04), Kyoto, Japan, 6-10 September 2004 (PDF).
Wikipedia: Algebra
Technorati Tags: composition SOA service-oriented
One important type of reasoning involves the ability to think numerically. For example:
- If service A costs $x to maintain, and service B costs $y to maintain, how much does it cost to maintain A+B?
- If service A has an availability of 95%, and service B has an availability of 98%, what is the availability of A+B?
Algebra is the branch of mathematics that deals with structure, relation and quantity. The word Al-Jabr is the arabic for reunion - in other words composition. This is therefore the form of reasoning that formalizes and quantifies the composition and decomposition of separate elements. Algebra tells us when it is safe to use addition and multiplication, and when we need something more sophisticated.
For example, if artefact A is composed from components {A1, A2, ..., An}, then the reliability of A is some algebraic function of the reliability of the components A1 to An. Conversely, if you have a requirement for a given level of reliability, you need some algebraic function to decompose this requirement into an equivalent set of sub-requirements.
In an earlier post on Reliability and Availability, I took issue with a vendor white paper, which assumed that all you have to do is multiply the parts to get the whole. This is obviously only true if you make some fairly simplistic assumptions about the composition structure, and is not true in the general case.
Meanwhile, Jeff Schneider takes issue with a confusing formula of SOA costs proposed by Dave Linthicum, which apparently involves adding different kinds of complexity.
If people are getting into these difficulties, where is the SOA algebra to haul them back onto solid ground? I spoke to a friendly professor (Bashar Nuseibeh), who advised me that the formal theory of composition is not advanced enough to support precise quantification. He referred me to some of the work he has been doing with Michael Jackson and others on formalizing composition.
But if I can't have precise, I'll settle for approximate. Surely anything would be better than the algebraic fallacies and muddle currently in circulation?
R. Laney, L. Barroca, M. Jackson, and B. Nuseibeh , Composing Requirements Using Problem Frames , Proceedings of 12th IEEE International Requirements Engineering Conference (RE'04), Kyoto, Japan, 6-10 September 2004 (PDF).
Wikipedia: Algebra
Technorati Tags: composition SOA service-oriented
Reliability and Availability
One of the pleasures of being an industry analyst is that you get to read a lot of vendor material.
Yesterday I came across the following statement in a white paper by Jonathan Purdy of Tangosol on Data Grids and SOA.
It is sometimes possible to orchestrate services in a way that that the reliability of the whole is greater than the reliability of the parts. Firstly, there may be underlying services that are not required for every transaction, so the reliability of these underlying services only partially impacts the reliability of the process. Secondly, there may be services that provide multiple or alternative provision of a given capability - alternative process paths can be defined to make the process more fault-tolerant.
That's not going to make the problem of reliability and availability go away of course, and there are undoubtedly some useful things that vendors such as Tangosol can offer in the physical implementation of SOA. And Purdy is right to warn his readers of the risks associated with complexity.
But what this raises for me is the general difficulty of reasoning about non-functional requirements. Do you add them, do you multiply them, do you average them, or do you need to perform a more complicated bit of algebra?
Technorati Tags: availability reliability SOA service-oriented Tangosol
Yesterday I came across the following statement in a white paper by Jonathan Purdy of Tangosol on Data Grids and SOA.
"As Business Services are integrated into increasingly complex work-flows, the added dependencies decrease availability. If a Business Process depends on a number of services, the availability of the process is actually the product of the weaknesses of all the composed services. For example, if a Business Process depends on six services, each of which achieves 99% uptime, then the Business Process itself will have a maximum of 94% uptime, corresponding to more than 500 hours of unplanned downtime each year."This might be true under certain circumstances, but it depends on how the business process is designed, and the degree of coupling within the process. A typical design objective is to compose services in a way that doesn't multiply the dependencies in this way. One of the principles of distributed systems has always been to avoid single points of failure, and this principle is surely inherited by SOA. When used intelligently, loose coupling and asynchrony should make a business more robust.
It is sometimes possible to orchestrate services in a way that that the reliability of the whole is greater than the reliability of the parts. Firstly, there may be underlying services that are not required for every transaction, so the reliability of these underlying services only partially impacts the reliability of the process. Secondly, there may be services that provide multiple or alternative provision of a given capability - alternative process paths can be defined to make the process more fault-tolerant.
That's not going to make the problem of reliability and availability go away of course, and there are undoubtedly some useful things that vendors such as Tangosol can offer in the physical implementation of SOA. And Purdy is right to warn his readers of the risks associated with complexity.
But what this raises for me is the general difficulty of reasoning about non-functional requirements. Do you add them, do you multiply them, do you average them, or do you need to perform a more complicated bit of algebra?
Technorati Tags: availability reliability SOA service-oriented Tangosol
