Enterprise Architecture

Moving Over To TypePad

2006-03-25T20:16:00.000-08:00

I decided that Blogger didn't offer the type of functionality I wanted without becoming a significant HTML hack, which I don't have time for. So....I just moved over to Typepad and my new blog is located at:

http://enterprisearchitect.typepad.com

Allthough I moved my archive over to the new site I will leave this one up indefinitely. And things will look a bit spartan on the new site while I get set up...hope to see all of you there!

UPDATE: The move over to Typepad is essentially complete, and I am disabling comments and trackbacks from this site. All of the posts here are over on the new site, so if you're linking or trackbacking, please use the new site and post URLs as I will be taking this site down in the next couple of months. Thanks!

The Real 'Two-Dot-Oh Community' We Live In

2006-03-21T18:30:00.000-08:00

James McGovern’s recent Ruby-on-Rails pieces touched off a range war between Ruby developers and himself, with some initial intermediation thrown in from certain industry analysts and EA bloggers. Unfortunately, predictably, and sadly, the ‘debate’ rapidly degenerated into ad hominem attacks on James in various contexts. Despite the fact that I found a good portion of James' commentary to be problematic, the personal attacks were unwarranted and uncalled for. Once commenters and certain bloggers started getting off topic and attacking James’ intelligence and experience wholesale, the whole thing decomposed into one gigantic shitstorm of little value to anyone. Sad.

The point of this post is not about Ruby, or any other programming language, design/development methodology, or technology component. Rather, I’m going to provide some perspective on this latest jihad and why the overblown two-dot-oh concept of ‘community’ needs a serious reality check because if the events of the last 4-5 days are a primary rationale of this so-called ‘community,’ I don’t want any part of it, and neither should you.

Religious wars over technology such as programming languages and operating systems are nothing new, and they’ve been going on for years. After the internet was invented, but before the web was, we had usenet newsgroups devoted to specific OS platforms, programming languages, and the like. Usenet and its newsgroups still exist today, albeit an extreme backwater to the blogosphere, discussion boards, and other modern forms of group communication over the internet.

The technical newsgroups were just as prone to insulting flame wars as blogs and blog comments are today. Some newsgroups self-corrected and righted themselves such that the signal-to-noise ratio returned to normal and constructive discourse eventually returned. Others never recovered and the key benchmark there was that only the combatants continued to post any material to the group, usually a content-free fusillade launched against the other side that was not worth reading; and all others had permanently abandoned the newsgroup. Eventually, even the combatants grew tired of their sneering nobody-wins-but-let-me-one-up-you-one-more-time game and moved on, most likely to other newsgroups to find another flame war to fight.

So, what’s changed about this ‘community’ between then and now? Other than the technologies, absolutely nothing. With rare exceptions, people act in their own self-interest, and they will continue to do so. When something outside of their worldview irks a group of folks, as James did with his Ruby posts (and yes, I found large portions of his content, while not hateful, ill-advised and not well thought-out), the chest-beating commenced, followed by the bullets whizzing by. Of particular interest to me were various disparaging comments in David Heinemeier Hansson's blog about architects and EA’s in general. To be expected? Sure. Can it ever be rectified to the satisfaction of all parties? Sadly, it would appear not.

Anne Zelenka warns us about fundamentalism in our approaches and I have blogged about holistic enterprise architecture viewpoints in the past. One cannot succeed as an architect by embracing one view of the world and showing disdain for all others. What gets me the most is all of the hue and cry over how we're building 'communities' (in the light of 'teaching the world to sing in perfect harmony' or 'Kum-By-Ah over the campfire') when the real motivation, as exposed in a recent Economist article appears to be primarily business-and-profit-oriented. Think of it this way, Oracle would not be trying to buy every open-source entity available if they didn't smell that sweet nectar of opportunity calling.

Here's a big lesson for the two-dot-oh crowd: if you can't have any sort of professional discourse over various internal technical issues, you won't be having them with paying customers either because you aren't capable of doing that. And perhaps the three-dot-oh mantra will be "We should listen better than we do now."

Don't know about all of the rest of you, but I'm not going to kill myself holding my breath waiting for that, because some things never change.

Open Source Gets a Bit of a Wedgie

2006-03-18T19:37:00.000-08:00

This week's issue of The Economist magazine is must read (note: registration and/or subscription required). The article 'Open, but not as Usual' describes how industries other than software are attempting to mimic the open source model. Industries as diverse as biotechology and pharmacuticals are mentioned.

The piece takes a pretty substantial whack at the entire concept. After describing the movement in the software arena, the limitations of open source organizations are explored. An example:

"But the biggest worry is that the great benefit of the open-source approach is also its great undoing. Its advantage is that anyone can contribute; the drawback is that sometimes just about anyone does. This leaves projects open to abuse, either by well-meaning dilettantes or intentional disrupters. Constant self-policing is required to ensure its quality."

Then we move on to Wikipedia as an example, which has recently been blasted by others (notably Nicholas Carr) for the quality of the entries, and has moved toward much tighter control over its content and who may alter or add to it:

"This lesson was brought home to Wikipedia last December, after a former American newspaper editor lambasted it for an entry about himself that had been written by a prankster. His denunciations spoke for many, who question how something built by the wisdom of crowds can become anything other than mob rule."

The piece continues on about quality, intellectual property issues, and makes some very astute observations about the real motivation and outcomes of open source software projects:

"One reason why open source is proving so successful is because its processes are not as quirky as they may first seem. In order to succeed, open-source projects have adopted management practices similar to those of the companies they vie to outdo. The contributors are typically motivated less by altruism than by self-interest. And far from being a wide-open community, projects often contain at their heart a small close-knit group.

With software, for instance, the code is written chiefly not by volunteers, but by employees sponsored for their efforts by companies that think they will in some way benefit from the project. Additionally, while the output is free, many companies are finding ways to make tidy sums from it. In other words, open source is starting to look much less like a curiosity of digital culture and more like an enterprise, with its own risks and rewards."

An example given directly afterward is MySQL, which freely gives out its source code, but the development is completely done by paid programming staff financed by the approximately 8000 'customers' that pay MySQL directly for support and maintenance. That looks a lot more like a business to me, and much less of a 'community.'

The article continues in-depth about open-source's strengths and weaknesses, but what's apparent to me is that a meshing of open source concepts and trditional corporate governance and control is taking place. It is beginning to appear that its not a choice of one or the other, it will wind up being a combination that, hopefully, gives us the best of both. The hue and cry of 'community' is being drown out by the hybridism with corporate interests that has emerged.

I think the same thing will happen to the various elements of Web 2.0 as well.

open source
Web 2.0
enterprise architecture

Lousy Businesses and Technology

2006-03-18T08:30:00.000-08:00

Nick Carr made some great insights into 'justification' of technology expenditures in a recent post. Of particular interest is his quotation of Charles Munger (Warren's Buffett's sidekick for many years) about investment in 'lousy businesses' - i.e. businesses with high competition and/or low profit margins: in essence, lousy businesses remain lousy despite technology investments, particularly if the technology is available to all players in a given market.

Quite a few of us work for or consult to such 'lousy' businesses. Oftentimes we (as IT) evangelize that the path away from lousy (short of selling out or closing down) is the implementation of new technology schemes. The mistake often made in these cases is that implementation and use of the technology itself will right the organization and lift it from lousy to good, or even great.

The mistake in thinking this is that these initiatives fail to provide the business with any competitive advantage in its markets or against its competitors. If IT is going to enable a shift away from lousy, any technology investment must:

lower the cost of doing business in a manner that cannot be easily duplicated by competitors. If every market player gets the same, nearly-immediate advantage from a technology implementation, what's the point in doing it and spending the money?
provide the basis for disruptive products and services that lift the business over its market and competition.
furnish the data and capabilities that drive the insights of marketing and sales functions to exploit market and competitive inefficiences to the organization's advantage over other players.

No amount of agility, SOA, SaaS, open-source anything, 'community,' Ajax, or Web 2.0 are going to vault lousy businesess over competitors unless it enables competitive differentiation or market disruption to the sole advantage of the business. The business rightly demands a fair return on its investments without passing all of the gains to customers, and the next time you hear a pitch that any technology du jour will provide substantial financial returns, ask yourself "...returns...to whom?"

And depending on the circumstances of your business and the marketplaces you serve, apply the above criteria liberally before issuing any purchase orders.

Relationship-Building for Architects

2006-03-09T18:40:00.000-08:00

If you haven't already done so, take a 'gut check' of where you stand personally and professionally with the following people in your organization (given in no particular order):

The CIO
The CTO
Your peer group of architects and senior IT management/technologists
The project managers and PMO if you have one
The development team leads and senior developers
The business unit heads and sponsors you routinely deal with
Members of non-IT business units you routinely deal professionally with

Why am I asking this? Well, as an enterprise architect, you're in a challenging spot because all of the above constituencies play a big role in you doing your job well. That is a lot of people to keep happy, but their overall satisfaction with you and your work is important because the loss or withholding of their support can put large roadblocks in the path of what you're trying to accomplish.

There are 2 direct ways to look at your business relationship with these stakeholders: what you get from them, and what you give to them. Here's an example of this with some of the constituancies listed above:

CIO - gives: Funding, Formal Approval, Strategic Direction; gets: Architecture

CTO - gives: Technology Strategic Direction/Standards,Thought Leadership; gets: Technical and Stategy Input/Advice

Project Managers - gives: Project-based Strategy, Leadership, Tracking/Status; gets: Buildable Architecture, Subject Matter Expertise

Development Team Leads - gives: Technical Leadership, Development Strategies, Development; gets: Functional Architecture Specifications, Interface/Data Specifications, Exchange Between Architecuture and Development Standards & Issues.

Business Sponsors/Unit Executives - gives: Funding, Approval, Requirements, Process Direction; gets: Working Software, Understandable Mapping of Technology Environment to Business Requirements and Processes

A couple of things to note before I continue - these are examples of stakeholder give & get relationships and if you attempt this exercise, the players roles and titles will most likely be different in name or function. And, I really wanted to put this in a table to make it clearer, but I didn't want to imbed a Word file in this post...if anybody has a good way to do this without a file and without changing blog sites, I'm all ears and grateful...just leave a comment to this post or e-mail me with my thanks in advance.

OK, back to our program. This exercise can also be enhanced by adding other data or intuitions such as 'perception of the IT organization,' 'perception of the EA function,' and others that are relevant and some kind of estimate low-to-high for each. Once you have completed this, then you can determine where to focus your energies with respect to building and maintaining/improving relationships and trust with the business and IT leadership and development.

I recommend doing a simple review like this at least once per year, and more often if there are signifcant areas for improvement. If you have, or can achieve, a relationship level with the stakeholders in which you are (or become) the trusted advisor or steward, it's going to make a number of aspects of your job as enterprise architect easier, and you can better achieve consensus or coalition-building to support initiatives you seek to develop and execute.

While this scheme may seem a bit simplistic, that's by design. You want simple truths like this so you can take appropriate action without much hand-wringing (and time-wasting) over what to measure, how to do it, and how often. We have more than enough complexity to deal with as it is, and getting a read on where we stand with stakeholders in a manner such as this provides quick benefits to the business, the IT organization, and ourselves.

Finally, this is not to be construed as a performance review, its more how the EA function (and by extension, you and your peer architects) are viewed by various stakeholders in the organization.

enterprise architecture

The Enterprise Architecture Definition Collection

2006-03-09T15:30:00.000-08:00

It's interesting, at least to me, to get a sense for all the different definitions of enterprise architecture out there. So, over time, I will post other people's definitions of enterprise architecture (and their sources) as I run across them in the literature, blogs, and websites. Updated March 9, 2006.

Good Enterprise Architecture:

Improves internal communications by providing a common language for describing how technology can support business initiatives.
Helps companies link business and IT priorities by creating road maps for decisionmaking about technology initiatives.
Helps reduce costs by encouraging technology standards throughout the organization, thus allowing IT to pinpoint trade-offs in project costs based on adherence to architectural requirements.
Improves the quality of technology initiatives for business by easily explaining plans to a broad range of constituents.

Poor Enterprise Architecture:

Enforces the use of technical terms and jargon that confuse both business and IT.
Creates such a high level of detail for defining technology initiatives that decisionmaking is paralyzed.
Requires a level of standardization that can potentially limit business-unit flexibility and speed to market.
Has unrealistic goals for transition to new corporate technologies.

CIO Insight magazine (website), "Enterprise Architecture Fact Sheet"

Enterprise Architecture is an infrastructure and a set of Machines constructed in order to manage a chaotic, dynamic, unpredictable, complex, organic, prone to error, frustrating, Enterprise IT, which has to support an ever increasing, dynamic portfolio of products and services, through constant "ASAP, Now, Right-Away" modifications of business processes.

Muli Koppel, Muli Koppel's Blog, published February 22, 2006

Enterprise architecture (EA) refers to the manner in which the operations, systems, and technology components of a business are organized and integrated. It defines many of the standards and structures of these components and is a critical aspect of allowing capabilities and their supporting applications to develop independently while all work together as part of an end-to-end solution. An EA consists of several compenent architectures which often go by different names. Some of the common ones are: business/functional architecture; data/information architecture; applications/systems architecture; infrastructure/technology architecture; operations and execution architecture.

John Schmidt, David Lyle, Integration Competency Center: An Implementation Methodology, 2005, Informatica Corporation. Posted January 29, 2006.

Enterprise Architecture

Monetizing Web 2.0?

2006-03-08T05:41:00.000-08:00

The blog traffic-generation site BlogExplosion is up for sale. The founders/owners sent an e-mail to members announcing their intentions, and the link in the 'for sale' text above provides plenty of additional and substantial details of their business.

Two things jumped out at me immediately from their disclosures: a) they appear to generate a ton of traffic to and through their site; and b) they generate very little revenue. Gross revenue over the 17 months reported where $74,194,50 USD with an monthly average of $4364.38. Their best revenue month was in December, 2004 with $5872.50 USD.

I'm not a financial analyst, and I'm not going to second-guess what the founders should and should not have done with respect to the site and its services or content, but a gross revenue stream like this doesn't sustain a business very long, not to mention that their gross revenues have been trending downward since June, 2005. All of this from a site that claims 50,000 members, signs up 100-150 members per day, 1 million listings on Google, and an Alexa ranking of 4174?

What's sticking in my craw right now is this: despite the impressive site and traffic/hit numbers, the revenue issue more than suggests that there isn't a sustainable business here, and if there is, what am I missing? What could the founders (or a buyer) do differently to drive up revenues?

And that's a question that needs answers in lots of cases like this, because it won't be the last...

Web 2.0

Thoughts on Process and Expectations

2006-03-07T21:53:00.000-08:00

I wanted to add my commentary to Scott Mark's excellent post about Agile in enterprises. I have participated and managed a few agile projects in the last few years and am sold on the concepts (as I am other methodologies) but retain my agnosticism as I strongly prefer to use different processes and methodologies as the situation dicates, and not dogmatically (which usually invites trouble in most cases).

The overall problems appear to me as attempts to strike a balance between results, process, and the expectations of users & the business. This balance is so tough to strike that few will truly succeed 100% in satisfiying all stakeholders, particularly on large projects.

Here are a few examples of perceptions/statements/mantras we all have been exposed to whether on-the-job or in publications and blogs:

The systems and applications IT develops cost too much and take to long to implement.
It costs too much to maintain operational IT systems (the 'keeping the lights on' part of the budget that is 70-80% of most IT expense).
It costs too much because of, as Scott aptly states "waste...being ceremonial or overhead activities that do not directly add value."
We can cut IT costs with an enterprise architecture (and indirectly, a SOA) that unifies and integrates applications and infrastructure
We develop one-offs and small-project apps that the business requires quickly but are difficult to maintain and support, don't interoperate with anything else we have in production or on the whiteboard, and have some development activities that are repetitive (particularly data and database-related) as we need to re-model or re-engineer the data access and movement/integration because of 'unique' business requirements.
We can implement and deploy faster if we continue using [fill in older technology here] rather than web services (and indirectly, a SOA), because that is what we know how to do without additional training or outside help.
We can service-enable anything faster and cheaper using REST rather than WS-* because the latter is cumbersome, too complex, and more difficult to implement.

I offer these as examples of statements and aspirations that compete with each other negatively in terms of overall business-technology approach and balance. The conundrum here is if we over-specify, over-document, or over-process (and that depends largely on which perception of 'over-' is being utilized), we take too long and are too spendy for the value received by the business. If we cut out too much, we run the risk of deploying systems marginally supportable that will need to be refactored (or in extreme cases, ditched and redone) because the inevitable changes to the system cost more than anticipated in time and money. This also lowers the value received.

What further muddies the water here is SOA itself. We're finding out that comprehensive SOAs are not built quickly and cheaply, and definitely not quick or inexpensive enough to satisfy urgent business needs. Well-built SOAs will pay back much more than they cost - eventually, and not fast enough to satisfy some parts of the business.

Finally, we get to the question of "value" of any process, activity, or sets of these. The question behind this is 'value to whom?' Business agility is one thing, IT agility is another, and overall costs to develop, deploy, and support are yet another. What kind of balance are we looking for here, and can we achieve some sort of consensus on this with the stakeholders?

I realize I'm asking a few more questions in this post than I'm answering, and much more thought and dialog is required. Scott and Cote' have begun a very interesting topic...much more to follow.

enterprise architecture

Various and Sundry Musings - 3/5/2006

2006-03-05T17:17:00.000-08:00

Things that have caught my attention lately, about anything...

Nick Carr thanks the Academy as he receives the award for "Best Performance in Curmudgeonly Role" for his sublime performance describing and commenting on the Web 2.0/Mashup-related "Camp Wars." Like Nick, I find it very difficult that certain individuals in the blogosphere wax rhapsodic about "community" with respect to mashups and Web 2.0 out of one side of their mouths and blast "competing" camps out of the other. Sad.

If you haven't seen the film "Walk the Line" about the late country music stars Johnny and June Carter Cash, by all means rent or buy it on DVD. Even if you aren't a fan of country music, the story is supurb and Johnny and June are wonderfully portrayed by Jaoquin Phoenix and Reese Witherspoon. A keeper.

Parts III and IV of my discussion about SOA and legacy systems will be out later this week. I apologize for the delay.

The continuing IT parody on the Fox TV show "24" (which Muli Koppel recently used to explain architecural points) continues to amuse me. I love the show and have watched it since the beginning 5 years ago, but the ability of the CTU organization to refactor systems and data, and alternatively to erase or lose it as the plot dictates, makes me laugh hard. Good thing its Hollywood and not truth...maybe...:)

The Keys to Good Enterprise Architecture

2006-03-04T07:51:00.000-08:00

I'm often asked what makes a good architect (enterprise, system, application, etc.) over other business and IT roles in an organization such as business analyst or developers. The key attribute an enterprise architect has is what I'll call balanced abstractionism.

As I mentioned previously, an enterprise architect's primary function is to look at both the business and IT (development in particular) in the abstract, a level or two or three away from the 'action' to ensure the following:

The organization gets its process needs objectively met through information technology
The resulting technical solutions are efficient, reliable, secure, cost-effective, and agile
Specific segments of the business-technology interface in a given organization, while using commonly available technologies, represent a core competency or value-add unique to the organization that provides competitive and profitable differentiation from rivals.

The hue and cry of certain parts of the business (and other IT pundits at times) is that all enterprise architecture produces is nifty-looking diagrams in geek-speak that don't deliver much value to the business. On the other hand, development teams resent EA being an 'ivory-tower' operation dictating standards and guidelines from on-high with no appreciation of development issues and goals.

Each camp wants EA to focus on their issues, usually exclusively. Both camps miss the point because EA is properly practiced with a balance of abstraction of and between both sides.

Why are we the middleman in all this, and why can't we (as an organization) remove an expensive cost-center from the mix and use the money not spent on EA for something else?

The answer is the same for both sides of the two camps: it's too easy to get bogged down in details on either side and partially or completely miss various opportunities to improve business processes or systems under development (or the aggregation of them) because our noses are at the grindstones of business/systems analysis, development, or project management exclusively. It skews perspectives so much that on one hand, business processes cannot be optimized, nor can systems or infrastructure because we're too close to the root issues of the matter to see what can and will go awry.

That's partially what thinking in abstract terms is about. Take SOA as an example - are we thinking about what types of loosely-coupled services we are going to offer the business, or are we thinking about how precisely they will be implemented down to the actual code? Actually, we are thinking about both, but at a level or two above what the groups responsible for more-specific definition and execution are thinking. This is a check-and-balance activity that yields great benefits to the business in terms of agility and response to business, cost/time-to-market, and technical issues.

Finally (for now) there is the issue of producing artifacts for these abstractions. As with the balance of abstration necessary between technology and business, enterprise architects must communicate the architecture(s), rationale, and benefits to a broad audience in terms that the specific audience understands. Too many architects rely on a single artifact (or tightly-coupled set of them) to address multiple constituencies. The usual result is that the works are too dry and technical for the business; and not detailed or nuanced enough for IT.

Before I get to a guideline list of artifacts useful for various constituancies that architects routinely encounter, allow me to point out that documentation should be, as Aglists put it 'good enough' to communicate what needs to be said, and nothing further. Striving for perfection in documents costs too much time for the little additional value added, and if you produce an artifact that nobody reads (or will read) except you, I would revisit the decision to produce that work.

Here's a sample of artifacts from various abstractions that could be produced:

The Business: Non-technical overview of the architecture with partcular emphasis on business initiative/process support, agility to meet changing demands, and efficiency both in time-to-market and costs.
Developers and Other Technical IT Folks: Technically oriented functional overview of the architecture with the understanding that you are describing "working softwares" with emphasis on the plural rather than singular. Individual project architectures and designs are part of the development team activities with EA support and input. Take great care to describe how all of this - applications and infrastructure - work together. Discussion of any singular project or system generally produces 'silo' thinking and discussion, which should be avoided.
Senior IT Management and Project Management/PMO: A combination of (1) and (2) above as necessary plus preliminary estimates of cost, time-to-market (design, develop, test, deploy) and resources necessary to execute.

Note that I don't specifically mention design technologies here like UML, MDA, RUP, and others. Use what you know and feel comfortable with. The tools aren't as important as the intent, communication, and results.

enterprise architecture

Why IT Can't and Won't Fix American Healthcare

2006-03-02T05:20:00.000-08:00

Going though my office mail today, the cover of the current issue of Business Integration Journal screams out: "Life Support For Healthcare! SOA To The Rescue!"

Uh huh. Next hype-ridden statement, please...

A lot of trees, ink, and electrons get expended in the US telling us as a profession that proper application of IT here, there, globally, best-practiced, ad nauseum will 'fix' the American healthcare system.

The current system isn't fixable, by IT or anyone/anything else, herewith are some important reasons why - Robert Samuelson's commentary in the Washington Post a few weeks back making a vital point why this won't happen (at least under present conditions):

Americans generally want their health-care system to do three things: (1) provide needed care to all people, regardless of income; (2) maintain our freedom to pick doctors and their freedom to recommend the best care for us; and (3) control costs. The trouble is that these laudable goals aren't compatible.

We can have any two of them, but not all three.

What Samuelson describes here we would call a mismatch of requirements - the goals that 'the business' wants in this case, while laudible, aren't compatible with each other, or in context to the problem being solved or the process under development. When we design IT systems to respond to situations like this, the systems become as flawed as the processes and decision-making being supported.

We have, as Samuelson puts it, quite a conundrum: IT can certainly be used (and to an extent, has been) in a number of contexts to reduce or control costs. But that won't matter much if the other two factors are still present because reducing process-related expense won't counteract the explosion in demand for all services paid for, of course, by other entities like the government or insurance companies.

I can't tell you how many times I look at the admin areas of my doctor and dentist's offices and see shelf-after-shelf of color-coded file folders and wonder what the cost-basis and savings would be of all of that paper were digitized. It probably could be and should, but in the end given the above three constraints, will it matter in terms of improving care and the access of all to it?

Until we get our house in order with respect to expectations, if we ever do, there are no silver bullets to 'fix' our healthcare problems, including IT. And claims in our trade and professional magazines to the contrary are the most dangerous forms of hype and illusion that I've come across.

Healthcare IT

Dot-Bomb 2.0

2006-03-01T18:28:00.000-08:00

Nicholas Carr penned a great review and commentary on Edgeio, the Web 2.0 tag-sale site, and Russell Beattie defines a new term for what's been going on hype- and business case-wise in the Web 2.0 world: WTF 2.0. I love it...:)

Once again folks, we have the same-old same-old, circa 1998. Lots of VC cash swimming around trying to find a home and winding up in some edgy tag-mashups that will attract 500 competitors within days if they have any success at all.

I guess some people don't learn that, as Nick put it 'Edgeio enters a crowded market with a ton of pizzazz and a gram of strategy. Sound familiar? It should. It's what's engaved on the virtual gravestones of hundreds of dot-coms.'

There's money to be made in Web 2.0-land, but this isn't it. As I stated previously, we're in a sea-change with all of this, but it appears that we gotta let these Webvan- and Pets.com-redoux sites have their 15 minutes in the limelight before they take their inevitable dirt nap. Maybe it will come to a head quicker and less painfully this time around...

Nahh....

Web 2.0

The Future of Lots of Things

2006-02-26T16:20:00.000-08:00

Muli Koppel's spot on post about EA and SOA resonated with me in a number of ways. He wrote:

Enterprise Architecture is an infrastructure and a set of Machines constructed in order to manage a chaotic, dynamic, unpredictable, complex, organic, prone to error, frustrating, Enterprise IT, which has to support an ever increasing, dynamic portfolio of products and services, through constant "ASAP, Now, Right-Away" modifications of business processes.

That is a formulation of my practical experience, as described in The Rise of the Machines: A new Approach to Enterprise Architecture. The outcome of an Enterprise Architecture is a Management Framework upon which, or rather, inside which, the Enterprise lives; physically.

Yet, if you consider the usual approach to Enterprise Architecture, you'd normally find committees, standards, governance, and smart people being in the confirmation pipe of business and technological initiatives.

Under this paradigm, the success of an EA Office is measured by the popularity of its staff members and of its guidelines and procedures.

I have always thought of EA in aggregate as the mapping of business processes and tactics to an optimal set of technology solutions that enable the business to carry out its initiatives. The value produced by EA 'done right' could be summarized as:

The business gets its process needs objectively met through information technology
The resulting technical solutions are efficient, reliable, cost-effective, and agile
Specific segments of the business-technology interface in a given organization, while using commonly available technologies, represent a core competency or value-add unique to the organization that provides competitive and profitable differentiation from rivals.

However, as Muli notes, EA has been bogged down with the issues of committees, standards, and governance. My consulting brethren have also contributed mightly to the weight through advocation of documentation-heavy 'best practices' that only produce, in large part, bookcases of 'shelfware' that will never be executed, if they were read seriously at all.

Now we have SOA, mashups, and SaaS on the near horizon as requisite environments, at a minimum a major portion of the technology environment in the next few years. I dismiss out-of-hand that any or all of these will replace EA, as none of these fully embrace the business-technology relationship in total. However, there is something lurking on the horizon related to these technologies that will become evident in the next few years, and the best term I can give to it now is the rise of what I will call 'Meta-Architectures.'

What's just over the horizon for all of us is the ability of users to precisely and individually define their applications look, feel, and results simply by clicking or drag-and-dropping combinations of SaaS'es or mashups in any manner that make sense to them and running it on their devices. Just like they do now with files and text/data objects in a desktop environment. It will not take the abilities or time of the IT organization to change applications, or data to meet this need, the users will just point to services they want, defined in terms they understand, combine them as they see fit, and the interoperability between what they select is a given as part of the services offered.

Moreover, given the current state of technologies such as .NET and the C# language, it is certainly possible that services, to a limited extent now but more so going forward, will be able to significantly refactor themselves logically per user directives and execute. As with the previous example, it will not take the time or talent of the IT organization to respond to changes like this - the capabilities will come with the systems and be user-inititiated without specific IT intervention or permission.

While we've heard a lot of this in the past in different contexts - things like COBOL and SQL being invented so users could easily write software or query databases and artificial intelligence able to mimic and potentially replace human expertise and decision-making, I think that the new paradigms have some real 'legs' in what I've described above, and I'm usually a noted skeptic about such things. I may be dead-wrong on this forecast, but I feel that these paradigms and techhnologies, properly deployed and managed, have the greatest potential over anything I've come across in my career.

Which leads to Muli's further discussion about EA evolving into this:

I believe these thoughts will finally change the perception of both EA and SOA: EA will shake off its image as a "documentation, procedures and guidelines" body, repositioning itself as a practical, implementation-oriented discipline aimed at the creation of an Enterprise Management Infrastructure, while SOA will be repositioned, no longer as an Integration/Interoperability architecture, but rather as an Enterprise Management architecture.

Corresponding to what I have described above, the type of EA shift Muli describes is precisely what EA must evolve into if it is to survive and add value to our organizations. One feature of the meta-architecures I described above is that they are controlled by the users, and not by IT. IT loses control of the combinations that make up the meta architectures once the underlying services are deployed because users build and execute off service combinations without the need for detailed requirements, extensive documentation, and development. IT will not have the resources or the expertise to 'police' user service combinations, merely the ability to monitor and track usage and respond to and solve fault conditions. IT manages the service environment and ensures the quality and security of services, creates new environments and services, and becomes much more process-oriented than in the past. It is the combinations (or in other words integration and interoperability) that it cedes to the business, thereby giving the organization the agility and flexibility so long sought after.

It is a facinating and important time to be in this business folks, and these concepts are still being fleshed out by others and myself. More to come on this soon.

enterprise architecture

Web 2.0

SOA

My Take on Resumes and Hiring

2006-02-25T17:00:00.000-08:00

Scott Mark started this topic, and James Tarbell has weighed in. Thought I'd take a whack at it also. Keep in mind that I'm a consultant, but I was an FTE for quite a few years before taking the plunge. Also, since I have more than one client, I 'interview' with prospective clients quite often, and selling my services to corporations and government entities is very similar to job interviews. I also vet candidates for clients from time-to-time, either screening resumes they get from HR or their recruiters or interviewing candidates they bring in along with their employees.

I'm going to take a different tack...if you're looking for a new position, or in certain extreme cases, paying work period, and all you have going is your resume on Dice, Monster, ComputerJobs, and the other sites, you're only going halfway, at best.

Why? Because I'm convinced that 99% of the great jobs out there aren't on these services or in the help wanted ads. They're only available from people you already know or need to meet. Yeah, that's right, you're going to need to get off your butt and start networking. Professional association meetings and conferences are great avenues for this. If you've kept in touch with folks from previous jobs and gigs, let them know you're looking (discreetly if you must, but let them know). Other professionals that you know socially are good too, as they may know someone else more aligned to what you're looking for and will make an introduction. It takes time and effort, but the payoffs can be huge compared to the job listings you and everybody else sees on the internet.

Scott and JT made lots of great points about resume mechanics and interviewing which I won't repeat here. I'm going to focus more on content of that document and interviewing:

A recruiter who placed me some years back that I remain friendly with remarked to me last month that 'everybody is an architect' these days: Data Architect (not modeler); Application Architect (not developer); System Architect (not development team lead); Network Architect (not network engineer). That, in my mind, is stretching things a bit. If I see 'Architect' on your resume I can guarantee that I will be asking you a lot of architecture-related questions and if the answers I receive lead me to believe that you haven't interfaced much with the business, negotiated requirements with users, or performed higher-level strategic and tactical design that functionally specifies things for developers and the data people, well, thanks, we'll get back to you...
I'm technology- and methodology-agnostic, and most good-to-great architects are also. I like to have a large toolkit at my disposal and select what fits well in any particular situation. If you start shouting fire-and-brimstone about any specific technology, 'best practices' (ugh), open source, or how Agile will save the world as we know it, I will politely decline the Kool-Aid that you're serving and move on to the next candidate. Resumes and interviews are not the time to get all evangelical. I'm more interested in learning how you think, how you handle setbacks, and how you get work done efficiently.
I don't care (and neither does anyone else) what you did 10 years ago, or 15 or 20 for that matter. You wouldn't believe the chronological resumes I've seen with experience listed all the way back to high school in the early 1970s. Stick to the last 5-8 years. If anything before that is possibly relevant, mention it last in a catch-all paragraph.
The same thing goes for your technical skills. The fact that you did batch COBOL programming only dates you as an older worker, and doesn't give you an additional arrow in your quiver to impress me that you can peform the work that you're interviewing for.
I see too many resumes where the experience description for the positions held list the projects and what the project tried to accomplish. I'm not much interested in the project you worked on. However, I'm very interested in what you did on those projects, not your team, your boss, or your group/organization.
I will ask you in an interview to describe a project you worked on or effort you made that failed - partially, completely, and miserably are all OK for response fodder. If you've worked for a substantial length of time in IT, you've been part of at least one of these turkeys. I will ask you what you did to correct the situation (even if it didn't work), what you learned from the experience, and how it influences your work and thoughts now. If you've got more than, say, 4-5 years of professional experience and you can't (or won't) describe an event like this, I'll deduce that either you're not being straight-up with me or you haven't done what your resume says you did somewhere. Your prognosis for the job I'm interviewing you for becomes fatal in that case.
If you currently (or have in the past) speak and/or write on professional topics, by all means, list it. Same goes for blogging if the blog is relevant, although I doubt that Mini-Microsoft will be listing his on a resume anytime soon...:) Speaking of Mini, anybody notice lately that his/her blog has basically degenerated into a bitch-fest for MS employees? Mini throws out the raw meat for a few sentences or paragraphs, and the piling on in the comments commences immediately. I read the comments more than his stuff now...lol
Beyond a BS or BA degree, your academic credentials are primarily important to HR, and perhaps senior management. Assuming that you have relevant paid professional experience, that matters more than your post-graduate education.
Although you may be (or have in the past) working in a complete hellhole, don't ever, ever, ever trash your current or former employer(s) in your resume or in interviews. You will immediately get deep-sixed without remorse or guilt on the part of the hiring authority. The time to reveal these 'learning experiences' is after you're hired and after you get the lay of the land in the new environment.
You've had 4 jobs in 4 years, and, as JT notes, you're not a consultant? You have a big problem. I don't know how you go about fixing it other than: a) become a consultant or contractor; or b) get in somewhere, somehow and stay put. And by all means, fix the problem that caused this situation, because it won't go away unless you do something to address it.
It is very, very easy and inexpensive to run background checks on anyone these days. As such, bald-face lies about employment, academic credentials, any criminal history, etc. will be outed in short order. Don't do it.
Treat your professional references like family. I'm very protective of mine. If you apply for a lot of contract positions, the time to reveal your references is not when the recruiter calls asking for your resume. It's when you've had an interview with the client, and then provide them directly to the client, not to the body shop. What happens if you reveal your references to every recruiter that contacts you is that they call your references, and its not to talk about you. Rather, its to drum up more business for them. That can get highly annoying to your references if you're on a substantial job search and they're getting 5-10 calls a day from recruiters dropping your name and asking for business. In fact, they usually become your ex-references shortly after such an experience. And don't fall for the crap recruiters give you that they can't present your resume without the references. They certainly can, and will if you're qualified, but will lie to you just to see if you'll capitulate. Legit recruiters never do this, and terminate conversations with those that do.
Same thing goes for your Social Security number and other personal information. The time to reveal all of that is when you get a serious, legit offer in writing. Not before.
Finally, what counts the most in a job search is getting in front of the people in direct position to hire you. That isn't HR, and never was. If you can get an 'in' within some organization (back to the networking again), you will be miles ahead of any competitors for the position. HR only can hire HR people, but some $12-per-hour HR clerk with no clue about what the work we do can screen you out so that the real hiring authority never sees your credentials. That's why networking and getting to know as many good professional folks as well as you can is so valuable.

I better stop before this gets too long-winded...and I hear a glass of wine calling my name and a quiet Saturday evening before next week's slog to enjoy...:)

The Walls of Sanity

2006-02-24T03:25:00.000-08:00

Dion Hinchcliffe recently defined the 5 "Walls of Confusion" about Web 2.0. Well done Dion, with my commentary on three of them:

The Wall of Hype: This seems to have calmed down a bit but it also might just be moving around. Web 2.0 hype does seem to have diminished in the face of some withering anti-hype and the hype cycle has moved more to Web 2.0-related developments like mashups and the latest round of Web 2.0 startups. Nevertheless, Web 2.0 promotion continues unabated in certain circles along with the anti-hype and if you're not following closely, you don't know what to believe:. Whether Web 2.0 is the next generation of the Web, or if it's snake oil. If it's the future of software, or just a marketing gimmick. I will give you my point of view one last time; Web 2.0 is real. And for that good reason, and some not so good ones, there is a lot of hype surrounding it.

Anything, technology-related or otherwise, is going to grab attention if it's talked about and promoted enough. This is no exception, and there are players who recognize an opportunity (or two or three) here. The hype will eventually sort itself out sooner rather than later, and like the dot-com bust, it will be 'put up or shut up' time in short order. If the whole thing doesn't survive, parts of it most certainly will, and those parts remain open to speculation at this point.

The Wall of Complexity: If you look at the Wired post above it has a particularly complex diagram in it. I actually drew that in order to create a pretty compehensive view of most of the moving parts in Web 2.0. There are a lot and it's hard to figure out where to start as a user, much less a software designer. The good news is that the good exemplars (Flickr and del.icio.us) and some of our approaches (like Ajax), actualy (sic) make it pretty obvious what you're supposed to do. But it's still very hard and what still not conveyed very well is the sense of balance and proportion required. In other words, you're not supposed to pile every single one of these Web 2.0 ingredients into the cake, bake it, and sell it to the nearest Web software giant. It doesn't work that way. There is a constant feedback loop with your users on the Web that guide you to in a close collaboration to add/remove features and capabilitgies (sic) while dynamically shaping and reshaping the product into what it needs to be at any given time.

Here is precisely where trouble lurks. Agility is a great and necessary capability and we must all move toward, if not embrace it. However, the enemies of agility are complexity and instability, both of which take many forms, and both of which can and have been successfully addressed in the past. As I have remarked previously, feedback loops that consist mostly of letting users test 'production' code hastily rushed onto sites and APIs will eventually (and probably quickly) lead to disaster.

Call me 'old school,' but I expect applications and websites that I work with to pretty much operate and look the same day-in and day-out. New features? Sure, but if you're radically changing my 'experience' every half-hour while I'm trying to get something important accomplished, I'm going to be a bit miffed if what you've pushed to your site or API interferes with or prevents what I'm trying to do or see or worse, causes all or parts of your site/API to fail. And if that continues, it will eventually drive me to one of your many competitors that will provide the level and consistency of service that I'm expecting (and, most likely, paying for).

Consider this alternative scenario: That's not counting what our developer buddy down the hallway is also pushing to our site, most likely without our knowledge. Since 'regression testing" doesn't seem to appear in the Web 2.0 lexicon (heck, 'testing' appears minimally), its a good bet that once we mash his changes with ours, our online stuff goes catatonic or takes a dirt nap until it occurs to either or both of us that the fast changes stepped on each other and/or the rest of our systems. Better go find last night's backup and pray we didn't lose another user to our competitors, and believe me, if we offer anything of significance to the community-at-large, there will be many to choose from...

The Wall of Significance. Is Web 2.0 a major new revolution in the way software is created and used? Probably. But there's a lot of stuff to learn, especially about the softer aspects of online systems like collaboration and social software. A lot of software developers, architects, and designers, more comfortable with the precise, exact parts that comprise software, are often pretty unhappy about this. Unfortunately for them, these aspects are probably here to stay, but they aren't sure. The competition for users, attention, and marketshare means you have to increasingly dangle the most effective engagement mechanisms or people will go elsewhere. And because we're human, there are few more powerful draws that building a sense of ownership and community. But in these early days, it's hard to tell if there really is a fundamental shift in first order software design, or just a passing wave of faddish affectation. Those of you who read this blog know where I stand, but it's hard for everyone to appreciate the significance of all this yet.

Well Dion, what hasn't changed a bit is that at a fundemental level, computers and networks still operate the same way they did in the past with respect to amplifying the imprecisions and mistakes humans routinely make in development, testing, and control of release to production. Ajax and web services can't help you there, my friend, and won't. What does help is a disciplined approach to all of these issues. While we have the capability to make changes on-the-fly (and looking back, always have although exercising it has been a choice dependent on circumstances), optimizing development and testing processes to shorten time-to-production and reach the ownership/community goals you seek are effective only in the context of providing consistent user experience, quality and security.

I agree that it's too early to tell whether or not we're in a total paradigm shift or a fad. I suspect at this point that it's a bit of both, and there is much more to observe and learn.

Web 2.0

Just Because Its Possible, Doesn't Mean Its Good

2006-02-15T19:56:00.000-08:00

Marc Hedlund posted a piece about Web 2.0 development techniques a few days back offering the blogosphere some new techniques that Web 2.0 developers are using that aren't found in traditional software development. Some are quite ingenious and good. Others, however, are frightening.

Marc wrote about software revisions as follows:

"Gone are the days of 1.0, 1.1, and 1.3.17b6. They have been replaced by the '20060210-1808:32 push'. For nearly all of these companies, a version number above 1.0 just isn't meaningful any more. If you are making revisions to your site and pushing them live, then doing it again a half hour later, what does a version number really mean? At several companies I've met, the developers were unsure how they would recreate the state of the application as it was a week ago -- and they were unsure why that even matters."

I'll echo a comment made directly to this post that the developers in question have never heard about change management controls and versioning. There are plenty of tools to accomplish revision control out there that work very well, and here's a real good reason why it's necessary: if somebody's 1/2-hour revision blows up and brings about an outage or severe loss of functionality for any reason, going back to the last-known-good revision in live production, and backing out the change is the first order of business in getting the system back on-line.

I also wonder how these folks regression test their systems before turning them live, and even with an automated test suite I doubt that any system of substance can be revised and completely tested in a half-hour.

That's assuming that these folks know what regression testing is in the first place, and why it matters.

Moving on to QA and testing:

"Developers -- and users -- do the quality assurance: More and more startups seem to be explicitly opting out of formalized quality assurance (QA) practices and departments. Rather than developers getting a bundle of features to a completed and integrated point, and handing them off to another group professionally adept at breaking those features, each developer is assigned to maintain their own features and respond to bug reports from users or other developers or employees. More than half of the companies I'm thinking of were perfectly fine with nearly all of the bug reports coming from customers. "If a customer doesn't see a problem, who am I to say the problem needs to be fixed?" one developer asked me. I responded, what if you see a problem that will lead to data corruption down the line? "Sure," he said, "but that doesn't happen. Either we get the report from a customer that data was lost, and we go get it off of a backup, or we don't worry about it." Some of these companies very likely are avoiding QA as a budget restraint measure -- they may turn to formal QA as they get larger. Others, though, are assertively and philosophically opposed. If the developer has a safety net of QA, one manager said, they'll be less cautious. Tell them that net is gone, he said, and you'll focus their energies on doing the right thing from the start. Others have opted away from QA and towards very aggressive and automated unit testing -- a sort of extreme-squared programming. But for all of them, the reports from customers matter more than anything an employee would ever find."

Wow, what a bad, bad move. Hope that they don't tell the CFO what types of risks they're taking betting the company like that. Lose the customer's data and get it off a backup? Fine. But what happens when the data was, inadvertently or otherwise, leaked to someone who isn't supposed to have it? Forget about some taggy Google Maps mashup...what about financial, credit, SSN, and medical records data? Forget about QA at this point after the government and trial lawyers pick over what's left of the carcasses of companies that 'test' this way.

Closer to the point, a guy named Boris Beizer wrote a software testing and QA book back in 1984 that, while it's out-of-print, has stood the test of time as a seminal, definitive work on software QA that is still applicable today, regardless of the technologies employed. In it, he stated one primary tenet of software testing: developers are poor resources to use in testing efforts, particularly of their own code, since they have an inherent bias towards their work that cannot be overcome nor relied upon to find substantial bugs in their code. This wasn't a slam on developers, as Beizer simply and correctly pointed out the natural biases we all have about our work, and software development isn't any different than other professions. And the Web as a computing platform doesn't change this tenet one iota.

In fact, the Web as a massive computing platform is no different than any other information system that came before it, with the requisite needs for adequate and substantial testing, revision and change management, and proper risk-mitigation techniques in production code and systems. Yes, the processes and techniques can certainly be optimized for speed and feature deployment, but thouroughness and the mitigation of outage, security, and data risks are still necessary no matter what new tools and techniques (or, as it appears, the lack of them) are engaged.

Finally, it appears that everything old is new again...we used to call 'eternal betas' prototypes back in the day....:)

Web 2.0
software testing

Converting Legacies to SOA: Part II

2006-02-15T03:20:00.000-08:00

In the second part of the process on whether to service-enable a legacy system or refactor or resdesign, we examine what it would take to refactor or re-design a legacy system from scratch (or nearly so) with service-enablement added. In Part III to follow, we'll compare what we've found in Parts I and II and get an idea about which approach is better from cost and time standpoints.

Some legacy systems are too large or obtuse to make a meaningful comparison in the manner I'm presenting here. Systems that fall into this category are usually third-party COTS packages (in which case one hopes the vendor has or is contemplating a service-enabled offering) or huge enough where the cost/time scope would be prohibitive - large ERP, supply chain, CRM, and HR packages come to mind. Again, since the majority of systems like this are third-party/outsourced, anything but the most trival services will most likely be provided by the vendors and their partners.

What we're doing in this portion of the exercise is to arrive at some reasonable estimates of what it takes in time and money to significantly refactor (or completely redesign) the legacy systems with service-enablement added. This exercise is important for the analysis in Part III that follows because we will be making comparisions between the labor, time, and money expended to redesign versus the cost of maintaining the legacy systems in their current state.

Yes, this is a bit of a project management exercise, but the style of project management (agile or process-oriented) isn't of interest here. How long and how much is what we're after. In all cases, whether you interate the development, testing, and deployment or engage in process-oriented techniques, the resulting system must have equivalent functionality and features as the ones it is replacing. As such, I haven't accounted for enhancements or alterations in the underlying business processes, which are separate issues.

The basics are as follows:

Develop a strawman refactoring or re-design of each legacy system with service enablement. If you decide to consolidate a number of legacies with respect to infrastructure (e.g. consolidating database servers) make sure to allocate time and costs proportionally across each system. The strawman architectures don't need to have large amounts of detail. They only need to be good enough to allow you to make reasonable estimates of time to implement and costs.
Once the strawman architecture is complete, then develop a preliminary task list that would be executed in sequence for each legacy system project.
Assign timeframes to each task or task group.
Estimate costs in labor, software licenses, and other project expenditures.

Now let's look at some specifics...

Strawman Architectures: It's easy to get bogged down and do a full-scale design of the legacy systems, and the analysis outcomes are going to be heavily influenced by the approach taken in this regard. My advice here is that you want answers to the overarching issues of legacy system retirement as soon as possible, so do enough design and modeling to get a good handle on what's going to be required with respect to time and cost. My take here is that this shouldn't take weeks per legacy system, but a few days to a week depending on complexity. Get help from your fellow architects and brainstorm it if you need to.

I'm assuming here that the SOA and other service-enablement issues have already been established or decided upon in your organization. If they haven't, it makes this entire exercise very nearly moot and you need to decide those issues and direction first before proceeding further.

Task Lists: Once the strawman architectures are complete, what it takes to implement and deploy the redesigned systems must be estimated from a task/work perspective. In project management-speak, this is known as developing a Work Breakdown Structure, or WBS. If your organization follows process-oriented project management styles, the WBS looks like a waterfall-model, start-to-finish single iteration. In an agile scenario, the iterations are mapped individually until you're comfortable that the proposed WBS retains the functionality being replaced or refactored. The amount of task detail that you develop is your decision, but the more comprehensive it is, the more sound your final analysis and estimates will be. Again, its easy to get bogged down in details and trivialities that sap time with little or no return on the final analysis, so I suggest that the task depth be kept managable and completed within a reasonable period of time.

An important thing to keep in mind at this step is that work breakdown structures are only concerned with tasks, not interdependencies or available resources. That comes later on and if you try to do two or three of these at once, you'll get bogged down with the complexity. Focus only on tasks at this point.

Timeframes: Now that you have a task structure for each legacy system, its time to make estimates of how long the tasks will take. For our purposes, time estimates take two forms:

The amount of labor to accomplish a task regardless of schedule. This is known in project management-speak as effort.
The calendar time required to perform the tasks. This is known as duration.

We're going to focus on effort at this point and touch on duration later in the process. For each task, you estimate how much effort (in time units of your choosing) it will take a reasonably competent person to perform the work. These estimates must be generic and not geared toward specific individuals or groups in your organization. The reason is straightforward: if the plan actually got executed, can you guarantee that these individuals are available to staff the projects? If the answer is no (and it usually is to some degree), then the estimates are automatically skewed and most likely wrong.

Cost Estimation: Now we're ready to get an estimate of the costs involved...

From your task time estimates in the previous step, take the total time that was assigned for each task, add them up, and multiply by the burdened labor cost factor you obtained in Part I. This will give you your esimated labor costs.
All software costs - licenses, support and maintenance contracts, etc. should be tallied up. I have not included hardware and network costs in this analysis because these issues are usually too broad to accurately address in a generic model. If you can comfortably estimate them and want to use them, then do so.
Any other project-related expenses unique or specific to your organization and situation must be included.

A caveat is necessary here regarding open-source software: open-source does not equal 'free of cost.' Any expenditure in labor, support, etc. must be accounted for in order to be as accurate as possible. Installing, maintaining, and supporting open-source software always incurs expenses in time and labor and must be accounted for.

After you've completed this section, total 1-3 above up for each system to arrive at the overall cost estimate.

Now that we've collected our data and performed preliminary design and scope, now we're ready to analyze what we have and where we should go in Part III to follow.

Converting Legacies to SOA: Don't Put Lipstick on a Pig

2006-02-13T05:30:00.000-08:00

Dave Linthcum has posted a few times on leveraging legacy systems through SOA. I agree, to a point. While we'd all love to wave our SOA wands and service-enable everything in sight, claiming 'Better, Faster, Cheaper' in the process, there needs to be some analysis done before committing time and resources to these efforts. This post gives you the start of an overall strategy to accomplish that with an eye to service-enabling legacy systems that benefit from it and scrapping and re-designing those that won't.

I'll be blunt: almost every IT organization has a few fragile, schizophrenic, resource-draining legacy systems in their shop. You know, the ones that always suffer outages, especially on weekends and holidays. The ones that take a battalion of developers and support people to maintain. The ones who have numerous and lengthy break-fix cycles. The ones that exist on hardware older than my teenage daughters. The ones that chew through IT expense budgets with high maintenance and support costs. And usually, they're still crucial to the business even though they should have been long ago retired and replaced with something else.

Well, sticking some type of service-enabled front-end on systems like this isn't going to make things any better. In fact, it could make things much worse operationally and financially for your IT shop because you're adding additional complexity to a system that already has numerous problems. While like many of you I find the siren song of agile, efficient, and cost-effective services very appealing, I have concerns that certain types of legacy systems are not suitable for service-enablement, never will be, and need refactoring or complete redesign. The trick here is to ferret these systems out before you attempt to service-enable them.

OK, I promised you strategy, and I'm pleased to give all of you one over the next few posts. To begin, you're going to need to get your hands on some data and facts, or 'educated' WAGs if such data isn't available or incomplete....

Overview of the Process

Part I: Legacy System Data or WAGs You'll Need

Part II: Refactoring/Redesign with Service-Enablement Project Data

Part III: Analysis

Part IV: Decision from Analysis

Part I: Data or WAGs You'll Need

For a given legacy system over the last 24 months (longer if you have the data, shorter if not...the more you have, the better the forecast/estimate will turn out):

Percentage of uptime over the period
Number (not percentage) of outages that had a severity of 'moderate' or higher (that is, subtract 'outages' that still allowed the majority of the system to function and be usable)
Amount of staff hours supporting system (help desk & other support)
Amount of staff hours maintaining system other than user support (break-fix work or developer/administrator maintenance that isn't user support)
Amount of staff hours designing and developing enhancements, new features and subsystems, refactoring existing code, porting, integrations, etc. These aren't related to break/fix and maintenance described above. Make sure to include all development cycle activites including requirements, specification, testing, etc.
Amount spent on annual maintenance agreements for all commercial software that the system utilzes. If a commercial software resource is shared with other systems, pro-rate this cost to each system as you choose and assign costs appropriately.
The burdened cost of labor for your organization or even better, for your IT department. This value is a figure representing what it costs per hour for your company to employ someone, including salary, benefits, employer-paid taxes, and overhead for facilities, office equipment, etc. For most large organizations, this value usually falls between $80 and $125 USD per hour. If you don't know this value, ask your financial people if they have it, or for an estimate. For the purposes of this exercise, use one generic value even if your financial people have this number broken down by job or location categories, as it will make the analysis much easier and give you nearly the same results.

You'll need this data in order to make comparisions in cost and time to service-enabling these systems. The next step is to make some baseline estimates of the service-enablement effort, and I'll post about that specifically later today. If you have questions, feel free to e-mail me or comment below and I'll get back to you.

Requiem for a Legacy Architect

2006-02-10T18:02:00.000-08:00

In many ways, this is a sad story to tell, but I'm sharing it with the blogosphere because clones of this man exist in a lot of IT organizations and need to be successfully dealt with...and that's not to mean downsized, right-sized, or whatever businesses are calling 'fired' these days.

Our man is in his mid-50s and has been in the IT business his entire career. He ascended through the programming ranks to lead teams and design major systems for his employer. His development experience was in structured programming languages such as FORTRAN and C on big old DEC hardware like VAX-11's (which, back in the day, were pretty awesome machines even though P4 PCs pack much more punch at 1/1000th the size...:)).

However, over the years as he gained more responsibility as a non-management-level architect, he grew resistant to newer technologies and ways of accomplishing development and testing. For him, it was as if time needed to stand still and the systems he knew well and had a hand in developing would stand the test of time and be in production forever.

More troubling is how he spent his working days, which largely consisted of going to meetings that in large part either consisted of the same topics over-and-over each week (what I call 'Groundhog Day' meetings) or full of technical minutae and trivialities. While the Groundhog-style meetings are largely indicative of his organization's culture, the latter set he needn't have attended at all, so his focus could be a bit more dealing with the lines of business and doing more architectural-related work (which his organization badly needed, and still does).

He also developed a nickname amongst the development staff: 'Dr. No,' after the villian in the James Bond movie. He liked to say 'no' a lot to new ideas and technologies. Web-based applications? No. Web services? No. Replace extremely aged, 30-year-old systems that, while they still work, are fragile and will permanently break any moment? No. Since he is an architect in the organization, management solicits and respects his opinions enough where he has enormous sway in decision-making on projects like this.

Lately, however, the organization hired a CTO who, after shaking his head at the state of affairs within the production architecture, is moving to upgrade and consolidate a large number of systems, servers, and networks. All of which freaked our man out to the point where he spends an inordinate amount of time with all of us blasting these decisions and proclaiming that the initiatives 'will never work' in addition to wasting large sums of money.

The reasons for his behavior became clear to me a couple of weeks ago when he returned from a week of training. I asked him what he went off-site for...

Me: How was the class?

Him: I went to a C# programming class. It didn't go very well.

Me: Why not?

Him: I never understood object-oriented programming, so I didn't get what was being presented in the course.

Then it hit me: the man is a control freak, and if he doesn't understand (or want to understand) something, it's bad because he feels that he loses any control he has if he can't or won't take the time to understand newer technology and issues.

Since 'structured programming' had more or less bit the dust some years ago and object-orientation is pretty much standard when using the terms 'programming' or 'programming languages,' you can tell how out-of-date (and out of touch) this man really is.

If architects and CTOs are going to succeed in their careers long-term, they must operate in what I call constant learning mode, which includes business as well as technologies. Another attribute successful architects and CTOs have is the ability to successfully refactor themselves career-wise when technology and business changes mandate them. The rest eventually get left in the dust, and unfortunately for our boy, he's recently been getting tuned-out by higher-ups and others are coming to the fore to drive technology development in the organization.

If you've got people like this in your organization, you're operating with a 100-pound weight strapped to your back. Even the time it takes to rebuff (or in extreme cases, usurp) people like this is much better spent getting some real work done. While time must always be allotted to playing organizational politics, the ROI on constantly dealing with guys like this is too low, but the price usually must be paid in any event until the situation comes to a head and a resolution is reached.

While it's sad to see this man flounder (or worse, go on another tirade) when topics like SOA and open-source come up, there is hope for guys like him if management and you play your cards right - and no, that doesn't necessarily mean giving him a pink slip. If these people have any worth left to the organization, have them oversee legacy operations and maintenance or even better, if they have excellent knowledge of your business, start dealing exclusively with the business as some sort of IT liason, with the decision-making they were formerly responsible for safely in the hands of you as enterprise architect, your systems & data architects, and your CTO if you have one.

And as the years pass by, don't let this happen to you.

Stuff I'm Reading - February 2006

2006-02-10T05:16:00.000-08:00

Here's my reading list for the month of February...

My system is short and sweet: I separate titles into geek and non-geek divisions and a simple rating system:

Yeah Baby! - Buy this and read ASAP.
Worth the $ - some minor quibbles, but take a good, long look
Get from Library - got a few ideas or nuggets, but don't buy
@$#)(&^! - author is entitled to his/her opinion, but I disagree with most if not all of the text
Crap - Self-explanatory. Avoid.

Non-Geek Tomes

Geoffrey Moore, "Dealing with Darwin: How Great Companies Innvoate at Every Phase of Their Evolution," Portfolio, 2005. Moore, the author of Inside the Tornado, Crossing the Chasm, and Living on the Fault Line, truly gets innovation (as does Clayton Christianson) and technology markets. In this volume, he makes and justifies Charles Darwin's theory of natural selection in terms of business and markets: there are complimentary processes in business that determine which survive and which don't. There are huge lessons in this book for business process and IT folks with respect to alignment and support of innovation.

Moore defines the word "core" (with respect to a business) as a concept used to describe differentiating innovation: "To succeed with core, you must take your value proposition to such an extreme that competitors either cannot or will not follow. That's what creates the separation you seek."

The case study is about Cisco, and Moore offers Cisco's leadership to followers market strategy as:

"Specifically, the other major players in the ecosystem must voluntarily embrace your platform. Knowing how much power this confers on another company, why would these companies ever do this? The answer is three-fold:

1. They get enormous productivity gains from leveraging your services.

2. They get access to a much broader marketplace.

3. They do not perceive the power you gain coming at their expense.

"Cisco's plan is to deliver on all three points....[It] seeks to leverage its own location advantage by providing services that are noncore to its major partners."

Hmmm...the entrepreneurs amongt us in the SaaS/Mashup spaces might want to pick this book up and digest thouroughly. Others involved with enterprise architecture and business processes will gain enormous insights also, particularly on how to organize tactical thoughts and plans against business strategies. Rating: Yeah, Baby!

Geek Tomes

John Schmidt, David Lyle, "Integration Competency Center: An Implementation Methodology," Informatica Corporation, 2005. Short (153 pages) but highly organized book on IT integration strategies and the context between enterprise architecture and the business as well as integration issues. Kind of a mish-mash between framework/methodologies and research notations from others. Very useful in that they go a step further off of frameworks and discuss implementation (including defining the dreaded 'integration hairball'), but not far enough for those who want huge amounts of detail. A well done piece of work, but I wonder why a software vendor published this (granted, with no plugs or ads) and not a mainstream publishing house. Rating: Worth the $

Working Outages

2006-02-07T07:11:00.000-08:00

I've been discussing SaaS and Mashup reliability for the past few posts, and I expressed concern about security and reliability of outside services, expecially in mission-critical applications.

Well, something has happened today to reinforce my points. Those of you that are frequent visitors to my blog know that I use blogrolling.com to list links to other IT and architecture blogs that I read and that you will hopefully find interesting also. They should appear on the left under the 'Links' header. I check my blogs every day as regular user before posting anything, and this morning, my blogroll disappeared inexplicably. Perhaps as you read this, it has returned, but at the time I posted this, it hasn't.

Here's what you get when entering blogrolling.com's URL at the current time:

Don't know what else to call this other than a hardcore outage. No explanation or redirection, no notice, no nothing. The basic services offered by this site are free, but the advanced ones aren't. What about the paying customers?

You might be thinking 'blogroll.com...so what?' Replace 'blogroll.com' with 'criticaldata.com' or 'customerservice.com' or 'taxreturn.com' or 'emergencymedicalrecords.com' and let's see how important reliability and security become as we move to the new paradigms.

No amount of agility or claims of 'working software' fixes the issues of reliability, security, and transparency. We have to design all of that in up front in architecture, regardless of what methods or practices we use.

More on Mashups and SaaS

2006-02-05T12:10:00.000-08:00

I've been reading a lot of blogs and other content on these topics over the weekend, and while I feel that we're taking some very exciting new turns, there are a few nagging thoughts as I digest all of this.

Mashups, SaaS, Web 2.0, whatever we're calling distributed & combined services this week, promotes agility and flexibility. Reconfigure or redesign, basically, at will. It makes the web one giant OS or platform with wiki-like ability to access and update data.

Flexibile and efficient. Who wouldn't want something like this as opposed to the monolithic apps we've built for decades? I sure would, and so would a lot of people I know.

And that very flexibility could be the great undoing of Web 2.0, if caution is thrown to the wind with respect to security and operational effectiveness. Right now, I've got a big problem with data security in schemes like this. For example, I'm not as much concerned about personal data flying over the ether as I am that it lands somewhere - in a database - where it shouldn't be. I'm just waiting for the first service or SaaS spoof incident to come forth. And that's not if folks, but when.

Finally, we get to a conundrum I've thought about but haven't had anything substantial crystallize as yet: we have always been taught that single-points-of-failure are bad, bad, bad. Made sense then for various reasons, and continues to make sense today. In a services scheme, SPF's come part-and-parcel because the uniqueness-of-service concept overrides wide distribution of a service. A basic or me-too service would most likely reside on multiple provider URLs and not be subject to SPF issues. However, as the recent salesforce.com outages reveals, dependent entities are largely dead-in-the-water if the dependency on services or SaaS apps is large enough to shut down significant portions of the subscriber's business. Not good at all.

Agility will never trump security and robustness as the latter two present hugely unacceptable risks to organizations. If we're going to do Web 2.0 right, these issues deserve as much consideration and effort as the agile mashing of services and the building of them do.

Mashups and Outsourced Services: Caution for Now

2006-02-04T23:41:00.000-08:00

Sandy Kemsley wrote a great post on mashups and corporate SOAs the other day and I'm on board with all of it. However...the following statement gave me pause:

'Dion Hinchcliffe thinks that 80% of enterprise applications could be provided by external services, which is a great equalizer for smaller businesses that don't have huge IT budgets, and could almost completely disconnect the issue of business agility from the size of your development team. I think that it's time for some hard introspection about what business that you're really in: if your organization is in the business of selling financial services, what are you doing writing software from scratch when you could be wiring it together using BPM and the global SOA that's out there?'

Wonderful scenario, but there have been a couple of events over the last few weeks that should give one pause before immediately going forth and downsizing the development teams: Salesforce.com's outages and subsequent commentary. In a nutshell, salesforce.com suffered a severe outage in their CRM site a few weeks ago...extending out to hours, and a minor one sometime after that.

While no system can exhibit 100% uptime, what's more troublesome is the "I'm peeing on your leg and telling you that its raining" response by Salesforce's management to the, as they characterized it, 'minor' outages suffered. Nick Carr covered the specifics of the outages and other commentary about them here.

This situation illustrates the primary concerns currently I have with SOA and SaaS, particularly when the major distribution of services comes from outside an organization: reliability, with secondary issues about orchestration and scale/performance.

Dion's scenario will occur eventually (and should), but we're on a curve right now where there are going to be repeated incidents of this type over the next few years. The trick will be what kinds of trust level and transparency ensues with outages and more importantly, how they're handled by SaaS suppliers. Anything else other than brutal honesty and better performance down the road smells only like hucksterism and hype.

Which are in large quantity at the moment in the SOA and SaaS segments of our industry.

Is it Scope Creep, or a Negotiation?

2006-02-02T05:43:00.000-08:00

A couple of bloggers have touched on requirements over the past few days. This is always a tricky subject, because success depends a lot on the personalities and motives involved on the IT and business sides and that of course varies across all organizations.

The standard colloquialism "gathering requirements" is a misnomer, as it gives the appearance that we scoop up or harvest requirements from the business much like a farmer reaps a harvest some length of time after planting and tending to his crops.

In real life, requirements are not often gathered, they're negotiated, and a good deal of those negotiations occur after the 'requirements gathering' phase of projects have passed. I have unscientifically polled students with IT backgrounds in my project management courses posing the question "How many of you have had any project in your career where 100% of the requirements were known up front and that's exactly what was delivered at the end of the project?"

The answer, of course, is zero. Requirements are always continuously refined and/or negotiated as development proceeds. Perhaps we should call the often-dreaded moniker 'scope creep' with what it actually is in reality: 'continuous negotiation.'

Google - SELL NOW!

2006-02-01T08:54:00.000-08:00

Google is a fine company, but even with the huge earnings gain Wall Street speculators were disappointed. The real problem is the stock and the market for it got waaaayyyy ahead of itself with hype and overblown expectations, just like Yahoo did in 1998-99.

If you've got a profit in this company's shares, sell. Now. If your greed glands were telling you that its a bargain around $390 and will certainly go well past $500 in a year, keep your money safe, because while I may end up being completely wrong, it ain't gonna happen anytime soon, if ever.

I don't own Google shares and am not a financial advisor. You take my opinion at your sole and complete risk...:)

Healthy Tension-building Between Project Managers and Architects

2006-01-30T06:00:00.000-08:00

A gentleman on my blogroll inquired about numerous topics, and since the issue of IT governance generally gives me a headache and temporarily causes me to rant, slobber, and develop unflattering facial tics, I'll hold my fire on that topic for another time and work the issues between architects and project managers. Specifically, said individual is asking about developing a healthy tension between architects and PMs. Here's my buck-and-a-half take:

Let's begin from a simple premise: architects specify and design, and PMs have to plan and execute projects from them. Although I have functioned in both roles, I don't to both simultaneously because its impossible to do both of them well in that mode. I also agree with other EA bloggers' assertions that good IT PMs primarily have an IT background with significant business-facing exposure as a side skill.

Now, let's look at this interaction of specify/design and plan/execute in a bit more detail. The architect has the following general responsibilities to the PM and project teams:

The design and specifications have to be tactical and buildable. By that, I mean that the design must be clearly communicated and understandable such that action can be taken by others - the PM and project team.
In whatever form architectural documentation takes, and this can range from prose, functional specs, UML models, tool-generated diagrams and documents, etc., the design and specifications must be clear and executable by project teams. I've seen too many 'architectures' that amounted to simply being 'plans to do more planning.' I blogged further on that particular malady in my Project Management weblog.
Architects make themselves available to PMs and team leads for review, further explanation, and in some cases, defense of the design and specifications prior to the majority of the project substantially starting work.
Designs and specifications adhere to existing standards within the organization and architects must petition for exceptions and approval from management when necessary. It is unwise to leave exception petitioning only to the PM, I'll explain that in a minute.
Architects avail themselves throughout projects executing their designs and specs to assist in scope changes or other technical issues that may arise. The interaction can vary widely depending on what PM methodologies are being used and the architect is assisting the project as a technical advisor, not as a development team member.
Architects understand that they are responsible for design and specification, not project execution or outcome.
Architects also understand that the PM is responsible for appropriate risk management and mitigation on the project, and assists with that (usually via design/specification) when necessary.

Those are the basics for the architect with respect to projects. Now let's look at what the project manager buys into in this deal:

The project manager engages the architect from the beginning, during the project charter phase, at a minimum as a subject matter expert (SME) and more often as the initial technical lead prior to development.
Project managers are up-front with architects with the architect's role in the project plan. No surprises or hidden agendas later on - PMs are factoring the architectural efforts into their formal plans or they don't and are up front with architects about it.
The PM engages the architects for all major exceptions to the architect's design and specifications that arise during project execution. PMs are not empowered to make these decisions on their own, but partner with architects and team leads to evaluate and present exceptions to established technical standards to whatever organizational body makes the final decision.
PMs utilize architects during the project for clarifications and review of project deliverables and other project work produced. Unless explicitly stated and agreed up-front, architects are not to be viewed or used as development or testing resources on projects.
PMs do not unilaterally alter, or authorize the alteration, of design and specification during project execution without the architect's input and recommendations.
PMs ensure that their leads and project teams understand, and are comfortable with, the architect's design and specifications before the majority of the project work begins. If they aren't, the PM facilitates review of the architect's work with the architect and project teams such that clarity to all is achieved.

From these two lists, the areas where architects and PMs intersect can breed a bit of tension, but the 'healthy' part of the tension is the intersection of accountibility between the two..design/specify (and in some cases, defend) for the architect, and understand/execute/deliver for the project manager. If the two set and hold proper expectations for each other, the correct tension that develops is that each excells further at what they're tasked to do, and can develop and maintain healthy respect for the other's work.

Lots more to say about this in future posts, but I'll throw this out to the blogosphere to start and for always-welcomed commentary.

Blogging-related Stuff

2006-01-29T06:27:00.000-08:00

I'm going to take a break from enterprise architecture blogging for a few hours and consult with the blogosphere on something I've really been starting to enjoy: blogging itself.

In addition to checking out various IT- and business-related blogs, I've been purusing and in some cases, creating accounts at the various 'blog services' sites over the past month. Serious bloggers will want to take a look at these sites if they haven't already and see what's in it for them.

I signed up on the big blog services (dare I make an acronym out of that...BS? :) ) like technorati so that I could get on their listings. I don't enjoy blogging just to hear myself think and write, and I think that most folks enjoy getting site traffic and commentary. I also signed up for various RSS feeds (google, yahoo, etc.) so those get picked up and hit the blogging search engines. On all of these sites, use of tagging is important so that the engines not only pick the blog up, but properly categorize it in user search results.

I'm amazed at all the different blogging services out there - for emailing posts to subscribers, link management, tagging, blog rolling, etc. Most provide basic functionality free, along with the usual AdSense ads. Some, especially me-too services, will not survive long term but its nice to have all of these choices.

I am interested in all commentary about what folks like, or don't like, about any specific blogging services.

Amplification

2006-01-28T07:37:00.000-08:00

James McGovern posted a blurb about a conference we all should be breaking down doors to attend...check it out.

Enterprise Architects, CTOs, and Coalitions

2006-01-27T11:50:00.000-08:00

I recently reviewed a CTO's handiwork explicitly framed by him as the "enterprise architecture" for the organization going forward. I'm using quotes because for various reasons, it clearly wasn't. In fairness, I can describe it as a roadmap of sorts to get to some sort of architecture, but the work suffered from numerous problems:

The artificact was packaged as a Powerpoint deck. Unfortunately, the deck appeared to be created by cutting-and-pasting portions of Word docs, PDFs, Website materials, and other stuff into the pages. Very tough to read, must less digest and understand, slides that were just many paragraphs of text.
The work was quite comprehensive, all the way down to explicit specifcations of laptops the organization will order for its employees from now forward.
The explicit infrastructure software bias - OS, enterprise messaging/EAI, and database servers were all from a certain extremely huge vendor located in Redmond, Washington. I wonder if that will tick their stock up a buck or two. Worse, there was no mention of how COTS-related dependencies (and they have them) with respect to other vendor's wares (i.e. Oracle) were to be handled, and the CIO mandated a few years ago that COTS solutions were to be purchased and implemented over custom development whenever feasible.
There was no unifying theme to the architecture - technical or business, and that was obvious when only base technologies were addressed and the organization's major applications and data stores went completely unmentioned.
Didn't see anything about the future, such as SOA, web services, etc. even though he muddled through a strawman to-be state that was inconsistent and in some cases, technically erroneous.
Standards for languages, coding, etc. were presented as 'guidelines' - that is, he's calling it a guideline but in reality, we'll do it his way and exceptions will be far and few between if they're granted at all.
Speaking of exceptions, if you aren't at the director/VP level in IT or above, you won't be presenting any to him.

Whoa. I haven't had that many warning buzzers go off in my head in such a short period of time for quite some years. It was clear that this guy doesn't have a good grip on architecture, much less presentation and political skills. Funny thing about this is, about half of this organization's IT folks thought it was great!

And I'll give you one guess who the other half was who thought it was crap - that's right, the organization's software developers. And they're right.

In my previous post about developer 'whining' about architects being placed into development teams, the point I was trying to make there is that enterprise architects need to strike some kind of balance and focus between the business and technical sides of the house. Yes, they need to interact directly with development efforts, but the reality is that they are not paid large sums of money to write (or even review) coding efforts. There just isn't enough time to prioritize that over architecture and business-focused work. Putting them on development teams solves nothing with respect to technical strategy and business-technology issues, and its the same for keeping enterprise architects in so-called 'ivory towers' - pretty soon, whatever they say or do is simply viewed as pontificating to development teams, and usually gets summarily dismissed or ignored.

The linkage I'm thinking about to address this problem focuses more on the interactions between enterprise architects and system architects/designers/development team leaders. Project managers also if the IT organization has that function. Getting all of these folks on the same page is critical if a to-be architecture will ever be realized. Mandates and "thou shalt/thou shalt not' edicts from EA's to the rest of IT don't cut it - never have, and never will. On the other hand, design-by-committee and endless consensus building doesn't get us anywhere either, at least not very quickly.

The enterprise architect in this situation has another avenue to pursue unity: building a coalition. The dictionary defines a coalition as "an alliance, especially a temporary one, of people, factions, parties, or nations." This is a great strategy for architects in getting agreement between the business and IT because temporarily getting the parties on board with respect to major issues trumps no agreement on anything (and thus, no progress) hands down.

True consensus is almost impossible to achieve on complex technical and business issues, but coalitions are relatively straightforward because all of the parties involved get something out of their membership and contribution, but not necessarily everything they're looking for.

Which is not a bad thing if you cannot achieve consensus and harmony (and very few can) on large, complex projects.

Spaceflight

2006-01-26T19:02:00.000-08:00

Hard for me to believe that it will be 20 years ago this Saturday that we lost Challenger in that awful accident. Time goes by too quickly.

The news is reporting on this anniversary, and NASA personnel are marking it also. NASA Adminsitrator Michael Griffin had this to say today as part of his prepared remarks:

"Spaceflight remains the pinnacle of human challenge, an endeavor just barely possible with today's technology," Griffin said in a statement Thursday. "It is an enormously difficult enterprise, made more so by the fact that we are human beings and flawed. The losses we commemorate today are a mute and terrible reminder of the sternness of the challenge, and of awful consequences of our flaws."

A good number of us work as architects, managers, and developers that don't deal with spaceflight, but with weighty issues such as the right to privacy, security of data, what we collect on customers and what we don't, and what we do with all of it after we have it. Not spaceflight mind you, but equally important to the customers that our businesses serve

I've had a wonderful time meeting all of the EA's and project managers through the blogosphere. I hope that our continued interactions with each other makes our individual and collective efforts the absolute best we can do for the people who depend on the products and services of our businesses.

More on Architects and Project Managers

2006-01-26T06:48:00.000-08:00

An interesting debate ensued between James McGovern (JM in this post) and James Tarbell (JT) concerning the roles and interactions between architects and project managers. Time for my two cents on the topic, and I feel uniquely qualified to comment since I have functioned successfully in both roles for years. I also teach PM at a midwestern university and maintain a separate PM blog more focused on that topic.

JM's rant about project management leaves me perplexed. He states that:

"Fundamental project management and those who practice it should minimally know what the status of any project is, how much the project has slipped from either original schedule or features, what are the impediments to delivery and what can/should be dropped in order to increase efficiency."

but then goes on to say:

"What would happen if enterprise architects started asking IT executives if project managers should only be providing value to management at the expense of the team? In my own travels, I have banged my head into my cubicle everytime a PMI certified project manager who hasn't written a line of code in their lifetime walk up to me and state: "I need the following information or I can't get my job done!". when did information become more important that the target of the information itself?"

Beyond JM's bias that only coders can succeed as architects, the simple fact is that project managers need to collect the information requested in the second citation in order to satisfy the requirements of the first cite. If the PM doesn't or can't know where JM's at with respect to project deliverables, he can't assess where the project is at with any degree of certainty, and those results are part of what was expected of him by management.

Unfortunately for JM, he can't have it both ways and there always needs to be accountibility for one's efforts on projects to PMs and management. The promise of 'working software' some months out into the future with no interim accountibility doesn't fly with the folks who pay the bills. Project reporting functions are present in every mainstream PM methodology, including agile. If he feels that his efforts are better spent on other work, then he hurts his project and team by not cooperating with the PM effort rather than help it, because it appears to others that the process discipline doesn't apply to him for some reason. When I've seen similar behavior on past projects, it labeled that person a malcontent, not a maverick.

JM goes on elsewhere in the post:

"During the bad days of my own career, folks within IT were given more "important" tasks that somehow were rationalized to be more important than actually delivering valuable working software to our business customers. Over time, the notion of project management instead of being something that is done as part of one's job became a full-time job for professional project managers. Of course this was justified in the name of increased productivity, but if you were to back-test this notion, did it really happen?"

While increased productivity may be one argument used to justify PM, it's a minor one. The two biggies are projects delivered on-time and on-budget, or as close to them as possible. IT has always had major credibility problems in the time and budget arena, which is a slam-dunk justification why the project management function exists. JT also correctly points out that 'PMs often get assigned to projects where a new solution or significant system changes is occurring. It is not their job to deliver on the promise of SOA or balance the impact of the changes to the IT portfolio--nor are they rewarded for such.' and '...Done correctly there is balance like Ying and Yang. Their priorities are just different. PMs need to get the function deliverables complete, on-time, with minimum number of resources and come under budget.'

Bingo. This is where the intersection between architects and PMs should be, and that they cooperate with each other acting in the best interests of the project - which is much more than delivering working software. I know a few PMs who are lousy leaders/managers, and every professional occupation has them. However, I know many more who are very good at what they do and deliver (with their teams) significant and complex IT projects within the necessary time and cost parameters with reasonably content project teams. JT's comment about 'balance' above is right on the mark, and successful enterprise architects cooperate with the project management function like they do all other parts of the business, not try to circumvent or usurp it.

Example of Vendor Hype About SOA Infecting The Business

2006-01-25T00:01:00.000-08:00

I previously posted about vendors infecting the business with SOA pitches - just drop one into your IT department and voila!, you've got profitablity and reduced costs!

Herewith is an e-mail I received for a webcast sponsored by Infoworld Magazine, a software vendor, and of course, an analyst firm touting how SOA will save call centers from the path to certain doom. I've taken out the names of the analyst firm and vendor to protect the guilty.

---------------------------------------------------------------------------------------------

Subject : Webcast: Using SOA to increase your call center's profits

Dear InfoWorld Reader,

The demands on your call center — to enhance customer loyalty,
reduce costs, react quickly to change, boost productivity —
can be overwhelming to both business and IT.

You're invited to learn how you can meet these demands and how
you can turn a major overhead expense into additional revenue.

LIVE WEBCAST -- THIS THURSDAY, January 26
Time: 10-11am PST/1-2pm EST

Register now to attend this live Webcast, presented by [VENDOR]:


Here is the agenda for this Webcast:
> 10:00 - 10:05 Introduction by InfoWorld editors.

> 10:05 - 10:20 [Analyst Firm] Analyst, [Analyst Name], will address
the impact SOA can make on your call center operations, by boosting
customer loyalty, improving cross-channel experiences, and reducing
customer service costs.

> 10:20 – 10:35 Learn real-world experiences on how SOA
best practices augment agent performance, enhance customer
loyalty, and improve call-handling time and resolution rates.

> 10:35 – 10:45 [Vendor Sales Monkey], Director of Industry Solutions,
[VENDOR], discusses how SOA makes it possible to overcome current
technology limitations to significantly reduce the cost of change.

> 10:45 – 10:50 Conclusion and comments from InfoWorld editors.

> 10:50 – 11:00 Q&A -- Get answers for your distinct call
center challenges directly from the experts.

By attending this Webcast, you'll discover how Service-Oriented
Architecture (SOA) can bring you many business benefits
while easing the load on IT. You'll also learn how you can
deliver a superior customer experience and quickly
adapt to constantly changing markets.

Register now to attend this live Webcast, presented by [VENDOR]


------------------------------------------------------------------------------------------------------------------------------

Don't think that someone in your business or IT department with an agenda and a checkbook isn't drinking this Kool-Aid? Think again...

Further Rants About Tool Jockeys

2006-01-24T04:24:00.000-08:00

My post yesterday on tool jockeys needs more exposition and yet another story on how they can make an architect's life miserable and what to do about it when it happens.

As I stated yesterday, tool jockeys are in love with software development suites, administration tools, metadata repositories, and other IT software that, by its very definition and presence in the enterprise, will 'make things better.' The tools involved usually do not discriminate between commercial or open-source, and all the better if it has a slick user interface reminicent of displays commonly seen on the Sci-Fi channel (hey, if it looks real slick, the suits will think it works and is doing something useful, right?).

What's usually the case beyond some misaligned hope that the tools will solve a wide variety of percieved or real problems in an IT organization is that experience in the tools will enhance one's resume. This is particularly true when tool jockeys in other organizations also have hiring decision input or authority - they're going to look for folks just like them, and the appearance of a big laundry list of tools in a job description is usually a dead giveaway that a TJ has hiring authority/input, or worse, that a Guild is present in the IT organization.

If not properly managed, TJs are a liability to architects in one area: dealing with the business. Too often, evangelism of this type can be fatal to projects or other IT initiatives if sold to the business as an ultimate solution and the business buys into it. As I stated yesterday, Guilds can be just as fatally effective in this regard, and I'll talk about them in more detail in a later post.

Here's a real world example of the damage a tool jockey can cause to architecture and projects:

A few years back I was counseling an organization on implementing a metadata management solution to address some serious problems this outfit had with data administration, data sourcing, and quality. While nothing had yet been presented to the business, the client had performed substantial research that resulted in a lengthy strawman proposal. I was given the document on day one of the gig and proceeded to digest it in my hotel room that evening.

It was, simply put, a disaster. After 5 pages explaining the overall issues, the remaining 50 pages waxed on and on about some vendor's metadata repository and toolset. Portions of it looked like it was copied verbatim from the vendor's marketing materials. The whole thing was skewed to tools so badly that most of it could not be salvaged for a business presentation.

The next morning I asked management who wrote the document. It turned out that their senior team lead developed most of it, and when I met him, he seemed very uninterested in the overall goals of the effort but much more engaged giving me a demo of the vendor's wares.

This proposed project had, all told, about a $2.5 million USD spend, so we weren't exactly talking spare change here, not to mention the organizational shifts that this effort would cause post-implementation. I informed the client management that from my perspective, what needed to be pitched to the business was the overall data admin/quality/management effort (and its benefits) and that while a metadata repository was certainly part of the solution, it was not the entire solution.

Unfortunately, the client decided to go with what they had into the management review, which turned out to be just as big a disaster as the document. This outfit's general management was very sharp and saw that they were getting pitched a proposal for a super-expensive product and not the solution to the issues. The proposal was declined by management and the effort to revive it floundered about for another month or so before being abandoned.

Architects and project managers need to pay close attention to situations like this or they court disaster even before the initiative gets off the ground. Anything that smells even slightly like a vendor pitch needs to be tossed, and if its stronger than that, the risks to the effort are too high to make bets like that.

Tool jockeys have their rightful place in any IT organization, particularly their deep expertise in day-to-day development and operational issues. However, architects need to think twice before incorporating this form of evangelism into designs or project proposals, particularly if such bias forms the majority of any major IT initiative.

The Best Enterprise Architecture Tool Ever Built

2006-01-23T06:30:00.000-08:00

I was in the office of a government client last week getting an impromptu demo of his new Popkin System Architect installation (Popkin was sold last year to Telelogic, which cleansed the Popkin name from the product but it's still currently referred to as Popkin in architecture trade). Popkin SA is a comprehensive software tool for architecture modeling that's particularly popular in the US federal government because it supports DoDAF, the Department of Defense Architectural Framework.

Anyway, he gave me the 30-minute demo of the latest release. Impressive software with an equally impressive price tag - about $30,000 USD per seat when you figure in the necessary training and support. Nice to know where our tax dollars are being spent.

However, lest you think this is a semi-disguised pitch for a software product, its not. I want to show you a very critical tool for enterprise architecture and system design that, depending on the size chosen, costs roughly $100-200 USD. Ready? Here it is...

Yep, a whiteboard. With a nice set of markers and an eraser. If you're really in the mood to spend, a bottle of whiteboard cleaner is a nice add-on, although I've found that repeated use of cleaners eventually takes the finish off of the boards and causes ink stains, but that's a minor quibble.

Nothing beats a whiteboard for a number of enterprise architecture activities, such as brainstorming, communicating ideas to others, and documenting designs and other activites. It's cheap, it's immediate, and it's highly effective. Sounds obvious and straightforward, right? Not to people who I refer to as 'tool jockeys' - who advance the notion that everything will work out great simply because we have a new tool for this or that. Mind you, tool jockeys are like children, today's tool is good for, well, today, and then they're immediately rhapsodizing about the next great tool that will dramatically change the enterprise for the better.

However, tool jockeys have a much more sinister role if they're congregated into a group, which I refer to as a Tool Guild, or simply, a Guild. Guilds can be dangerous to organizations because their cliquish behavior often is used to advance technical agendas at odds with architects and IT managers. These agendas are usually centered around the maintenance of the technical status quo, and thus their jobs. Here are two real examples from my experience:

An EAI group at a client designed the client's enterprise messaging system as a black box centered around their tools and view of the world, which meant that a lot of custom point-to-point interfaces got built instead of a common enterprise messaging system that the client was expecting. Objections to their work by others were routinely dismissed or quashed, and the client spent millions on the implementation of an underperforming, fragile messaging system that needs to be replaced only 2 years after it went into production.
I once sat in a meeting of a data modeling group attempting to come up with data modeling and administration standards for the rest of the IT organization. A common theme throughout the meeting was that the group were the only ones capable of doing data modeling correctly, primarily because they were the only ones who had (and knew how to use) the organization's data modeling tools and repository.

The primary focus of tool-centric Guilds is self-preservation, not technology and certainly not the best interests of the organization. In the example of the data modelers above, I mentioned that Microsoft Visio 2003 contains a very rich set of data modeling tools and templates that were available to all IT workers as part of their MS Office installations. The group immediately dismissed that because they claimed it didn't work with their tools and their repository. The Guild had spoken.

The looks of absolute horror on their faces when I showed them (via my laptop) that Visio did, in fact, easily interoperate file-wise with their toolsets was, in the vernacular of the Mastercard TV commercials, priceless. Wonder what they thought when some time later, their CTO mandated that application groups could do their own data modeling. Assuming, of course, that these folks are still employed there...

Again, nothing beats the good ol' low-tech whiteboard to develop and communicate ideas. For a quick and easy way to transmit whiteboard information to others that were not present (and no, its not copying it to paper longhand), click here.

Why is Open-Source Better?

2006-01-23T05:50:00.000-08:00

I'm not asking this question tongue-in-cheek, I'm asking it because I'm confused as to why open-source is 'better' than commercial systems in the business enterprise. I have no beef with open source systems - in fact, I have utilized OSS's such as the Apache HTTP Server and Tomcat routinely in architectural designs for years. I think these represent some of the best overall system designs ever developed, commercial or open-source.

However, I have a few issues:

As usual, the ugly heads of evangelism and zealotry arise within the architect and development community regarding open-source and commercially-supplied systems. Rather than sticking to the specific technical and architectural-fit issues, which should be driving discussion.
Some feel that open source = free, and worse, that's what they tell the business. Hate to burst their bubbles, but there isn't anything worthwhile that's completely free and this is no exception. Open-source may cost less to implement because you aren't paying vendors up front for software and code bases, but there are always costs involved in the installation, integration, and production support of any piece of software. Businesses like Red Hat would not exist, much less thrive, if that were not true.
As certain parts of software integration are commoditized, such as relational database and application servers, the open source alternatives present substantial benefits over commercial products. This is particularly true in the Linux and UNIX spaces where commercial software is much more expensive that open-source as opposed to the Windows market, where there are so many players that the pricing has always been much lower.
There have been numerous times throughout my career that code hacks and other 'workarounds' in open source code to fix problems wound up costing more than they should in labor and leaves the system vulnerable to other unforseen events as time passes. In retrospect, the fixes probably should have been made at another level of the system, but hey, we got the source code...piece of cake. Really.

So, I guess I'm back to my 'roadmap' approach to OSS: when there's a fork in the road, take the best path that leads to solving the problem, not introducing further dependencies, and controlling operational and support costs after the fact.

And the EA blogosphere thinks? :)

The Discipline of Enterprise Architecture

2006-01-22T06:42:00.000-08:00

James McGovern's recent musings about methodologies and responsibilities regarding EA led me to much thought about the topic. Since I also function at times as a project manager, my thinking is tempered along the lines of discipline, and not necessarily any particular methodology or body of knowledge.

I agree that most methodologies are sold by consulting/analyst firms, academics, and book authors as the Holy Grail that will cure most, if not all ills in an IT organization. That, simply put, is largely sales and marketing fluff. Then again, the main product these folks sell isn't services and software, its hope. And hope is not a strategy.

The architecture community is having a hard time in a number of areas due to the following:

Until we achieve some kind of definition or boundaries on what enterprise architecture is (and is not), it's going to be tough to come up with bodies of knowledge that directly drive process and thus drive discipline that leads to successful outcomes. Being too development/technology-centric or the same on the business side won't cut it. Frameworks are great, but knowledge about how to successfully use them in an economic and expedient manner is even more valuable.
Economic value to the business from our efforts must always be shown, not just technical value. Those who cannot or will not do this must also realize that the business has every right to question why the architecture function exists in the organization.
Knowledge and talent are separate from the ability to successfully apply them to business and technical situations in an organization. That usually requires discipline usually represented as processes and procedures. Does it really matter if such processes wre developed by a consulting firm or gleaned from a book, if they work well?

Remember the old saw that knowledge and skills are essentially useless unless they can be successfully applied and shared with others? Enterprise architecture isn't any different, in fact, it demands it even more than other lines of work because we can't build or refactor everything we specify by ourselves.

Enterprise architecture already has a wide body of knowledge, but it's very disorganized due to lack of agreement on definition, and with that shortcoming, a lack of discipline to practice it to the benefit of all. I think that we'll collectively get there eventually, but it will be a long march.

SOA: How Granular Would You Like Your Service Today?

2006-01-18T05:43:00.000-08:00

I had a very interesting conversation and whiteboard exercise with a collaborator yesterday regarding SOA. While I'm sold on the overall concept, including web services, I continue to fret about how a lot of these architectures are going to be actually built, and its not all the problem or fault of the hype-ridden vendor and consultant community.

It's easy to cut corners in just about anything, and EA is no exception. One of the issues I had was a basic tenet of SOA stating that appropriate interfaces are coarsely-grained and therefore useful in a wide variety of applications and interface scenarios. All well and good so far.

The issue with coarsely-grained interfaces is the temptation lots of folks have to regurgitate the distributed systems/client-server model of fine-grained interfaces, achieving tighter coupling. This is particularly prevalent in RPC/COM-model software with rigid APIs. My fear is that for various reasons, particularly time-pressure, developers will cut corners and code right down into huge complexities that are difficult and expensive to maintain after deployment, not to mention non-reusable elsewhere within the overall architecture.

Architects should pay particular attention to the fact that while building web services and establishing a SOA isn't particularly difficult from technical standponts, proper orchestration will be critical to overall success. While there will always be some needs for fine-grained services within an organization, one of the keys of good architecture is to carefully control when they are built, and how they are to be orchestrated and controlled in the production envronronment.

There are a number of WS-* standards that address policies, context and transaction management, and orchestration. From what I'm seeing in actual practice so far, lots of organizations are promoting and/or developing SOAs as a collection of web services without much work or thought into how these beasts are going to coordinate and function in live production environments. I think that we'll either be in for "SOA 2.0," that is, these architectures (and services, and code) will need to be automatically refactored one or more times to account for major coordination and granularity issues, or some organizations will abandon the concept because "it didn't work the way we expected it to"

Which are very good reasons that architects pay close attention to such matters and resist the temptation to relive their days as developers by writing code for some sundry accounting or inventory application because they need to 'get closer to the development teams."

Stuff I'm reading - January 2006

2006-01-14T10:32:00.000-08:00

I have always been a voracious reader since my youth - primarily non-fiction although I can be convinced to try a novel every so often. Enterprise architects generally tend to be a well-read lot (and quite a few are authors in their own right). However, only reading geek tomes shortens one's worldview. Not good if you plan to be effective with non-geeks, like the business.

Anyway, my system is short and sweet: I separate titles into geek and non-geek divisions and a simple rating system:

Yeah Baby! - Buy this and read ASAP.
Worth the $ - some minor quibbles, but take a good, long look
Get from Library - got a few ideas or nuggets, but don't buy
@$#)(&^! - author is entitled to his/her opinion, but I disagree with most if not all of the text
Crap - Self-explanatory. Avoid.

OK, here's the list of completed and/or current reads for this month:

Non-Geek Tomes

Ray Kurzweil, The Singularity is Near, Viking, 2005. You cannot help but admire this man and what he has accomplished in numerous areas, particularly in artificial intelligence. While I'm a big fan of his earlier work, The Age of Spiritual Machines, I've got a few issues with Singularity, particularly that he ignores cultural, economic, and political issues with respect to human-machine fusion and that the human race can always be bailed out of misery by good technical ideas. He also tends to be repetitive on topics he brilliantly covered in Spiritual Machines. Rating: Worth the $.

Geek Tomes

Thomas Erl, Service-Oriented Architecture - Concepts, Technology, and Design, Prentice-Hall PTR, 2005. A huge volume crammed with just about anything and everything you'd ever want to know about SOA. While I particularly liked the case studies and technical concepts explained in plain English (makes the book more marketable, I reckon), I'd have rather had those in some kind of supplement or separate volume because it distracted me a bit from absorbing the core material. Well written and thurough - a must have. Rating: Yeah Baby!

Dirk Kraftzig, Karl Banke, Dirk Slama, Enterprise SOA: Service Oriented Architecture Best Practices, Prentice-Hall PTR (Coad Series), 2004. Not as comprehensive as the Erl volume, but it's not aimed at developers and coders, but more at IT project leads and managers. The book is split into 3 sections addressing architectural concepts, organizational aspects and actual case studies. Very lucid descriptions and it's evident that the 3 authors have actually implemented SOA's with their clients. Rating: Worth the $.

Architecture Resources Repository

2006-01-12T10:38:00.000-08:00

I received the following e-mail from Jeff Tash concerning an IT architecture repository project he's been leading and thought that it would be of interest to the architecture-oriented blogosphere. Here's the e-mail unedited (I will fix the links after I publish this)...

---------------------------------------------------------------------------------------------------

Back in April 2005, IASA, the International Association of Software Architects, formed a working group focused on IT architectural "Foundations & Taxonomy" with the specific goal of charting the "largely uncharted" profession of IT architecture.

One objective of the F&T Workgroup is to locate, qualify, sort, coordinate, and explain resources relevant to IT architects. As a member of the workgroup team, I led an effort for creating a repository of architecture resources. You can access this Architecture 'Resources' Repository by clicking the following link:

http://www.itscout.org/itguide/login.cfm?rdtk=63C7707AB50A91D2607E9E2641CC48D8

Alternatively, you can access the repository by using the following login procedure:

URL:

http://www.ITscout.org/ITguide

Username:

architecture

Password:
itguide

The visual model used for organizing architecture information is a simple 2-shelf "bookcase" containing several sets of different colored books. I recommend first clicking the bookshelf itself (labeled "Architecture") and then exploring other sections by clicking on individual books. It's easy to submit suggestions for additional relevant resources offered by other organizations by simply clicking on the FEEDBACK icon located near the top right corner of each web page.

The primary goal of the F&T Workgroup is to fill-in missing "holes" in the existing resources that already exist by building on top of the IEEE 1471 standard. I encourage members of the ITscout community to make contributions to the Architecture 'Resources' Repository. Your assistance is greatly appreciated.

--------------------------------------------------------------------------------
Jeff Tash

CEO, Flashmap Systems, Inc.
http://www.FlashmapSystems.com
tash@flashmapsystems.com
617.332.3101

ITscout
http://www.ITscout.org
http://ITscout.blogspot.com
ITscout@ITscout.org
800.381.7515

SOA-ck It To Me Baby!

2006-01-10T17:10:00.000-08:00

Note to all software and network vendors out there: just because you wax rhapsodic about your wares being the definitive product to implement an SOA, doesn't mean that it is. Most of you took your existing product line and re-branded it. Smart marketing move for you, your shareholders, and investors. Dumb move for any IT flack who believes it. Then again, these are generally the same marks that bought ESB, EAI, and other 'holy grail' products hoping for that quick fix or silver bullet that could cure what ailed them. What they didn't realize is what ailed them then, and continues today, is bad processes and policy. A pity that we don't have a product line for that.

Since I last ranted about SOA (the entry with the magazine quip about selling it to the business) I find that the hype has now infected the business. I have sat through 4 meetings since the holidays ended where business execs and sponsors who know enough to be dangerous to IT-types belted out crescendo after cresendo on how we needed a SOA. Right now. When can we have this - tomorrow morning OK? One even went as far to have scheduled not one, but three vendor sales monkeys to give pitches! The CIO hasn't found out yet, but when she does, I really don't know how to categorize her response. Sometimes she holds her ground vs. the business, and sometimes not.

After the meeting, I had a private audience with the major business stakeholder and advised him that SOA isn't something you pick up at Best Buy and install along with the plasma TV and home theatre system. He's the very-direct type, so I anticipated his comeback by giving him a sanitized (read: de-technicalized) version of Muli Koppel's descriptions of SOA. What it is, and what it can and should be. I also put particular emphasis on the 'A' in SOA that effectively got him off the idea that he could just do the spend and make magic happen. Whew.

Business stakeholders like this - the ones that know just enough to be dangerous - are as bad for enterprise architects as poor development teams or crappy software. They have to be adroitly managed, which, at best, is a distraction from what we really should be concentrating on. At worst, then you're left with architecture-by-the-business, and that has never worked out well for anyone involved.

This is one reason I read the business press routinely. Once IT fads start hitting the mainstream press, work life can get pretty difficult for awhile.

Read Muli's stuff in any event. I like the way he presented what SOA should be and the potential architectural pitfalls involved. Solid commentary for all of us.

Take a Holistic View

2006-01-08T20:37:00.000-08:00

Vignesh Swaminathan blogged about Pain points of adopting enterprise architecture recently and made a number of comments about the evangelism displayed by various enterprise architects also describing some of the reasons that EA fails or is not taken seriously in organizations.

There are two fundemental problems in our line of work: a) as Vignesh states, enterprise architecture is sold to business and IT as a cure (along with the resultant expectations); and b) we try to be all things to all people - the business, IT development, IT operations, etc., and we realistically cannot.

Typically, these problems are addressed in the guise of evangelism - the pure view, the cure, and we (the enterprise architects) not only have the secret sauce, we are the only ones that can effectively use it. That is a recipe for large risk and eventual disaster, and always has been.

No wonder then, why many enterprise efforts fail. I suggest a different approach: taking a holistic view of architecture. Holism is defined in the dictionary as "the theory that living matter or reality is made up of organic or unified wholes that are greater than the simple sum of their parts." Why can't we take a similar stance with enterprise architectures?

Well, it has been tried to a degree, because that's how frameworks like Zachman, FEAF, and TOGAF evolved. The problem is evangelism crept in..slowly at first, then it became a crecendo. Some organizations that bought into enterprise architecture, got a strong dose of the 'religion,' and then had those efforts essentially fail turned their backs to it. With good reason, because while the frameworks are essentially sound, the approaches to them failed.

So, what to do about this? The simple answer (for a blog posting anyway) is to primarily concentrate on the business-technology interfaces (strategy-->processes-->technology) at a very high level with specific optimization goals that apply to that level only. Once that has been established, drilling down either way is not only encouraged, but necessary. The problem is that we get too bogged down on either side of the architcture spectrum and leave the other half wanting. That's why putting enterprise architects on development teams or aligning them too closely to the business doesn't work over the long-term.

This high level architecture shouldn't take months or years to develop, which is why its important to keep scope in-line and diligently focus on producing results in a short period of time. The process can be repeated over different swaths of the organization if one is dealing with large scope or a lot of complexity.

I've run into a lot of evangelists in my day - technical, religious, and political. With very limited exceptions, their message and efforts could never be sustained long-term. Then again, zealotry never gets anyone very far over the long haul either.

Blogoscopy

2006-01-06T06:37:00.000-08:00

James McGovern blogged the other day about a site named Squidoo that allows 'experts' to create mini-websites called 'lenses' that can contain lots of interesting stuff - RSS feeds, links to blogs, text, etc. I gave it a try yesterday, and out of approximately 11K lenses, the one I created for enterprise architecture ranks in the middle, right next to 'Attorney Referrals' and 'Art Quilts.'

Must be some kind of karma for enterprise architects being ranked adjacent to those types of categories...:)

The site looks and feels a lot like Blogger, which led me to believe that this is yet-another-Google-creation (along with the robotic hamburger cookers Mr. Gates referred to at the CES show the other day). My further investigation reveals that the site was started by Seth Godin, the marketing guy I referred to last week. His individual lens has some really interesting stuff in it along the lines of stories and convincing others of one's agenda that I posted last week.

The site is currently in beta, and as usual has all of the AdSense stuff in it. It's worth a look at the lenses folks have created if you're not interested in starting one.

Zachman Economics

2006-01-05T12:17:00.000-08:00

John Zachman states that we "can't cost-justify" architecture. I heard him state it verbatim to the Kool-Aid drinkers at a DAMA chapter meeting here in Portland last year. Before you get the idea that I think ill of the man and his work, let me state forthwith that I view his overall contributions to be seminal to what we now call enterprise architecture. Although I have more than a few quibbles with aspects of the Zachman Framework, its use, and his subsequent work after that, I have tremendous respect for him.

OK, that being said, I have a number of issues with the statement he made concerning cost-justification:

He doesn't tell us, at least not directly, whether he's referring to cost-justifying enterprise architecture efforts (i.e. the process and personnel EA requires) or the actual architecture produced (the as-is and to-be states).
If he's referring to the latter case in point (1) above, then cost-justification of the as-is state is a moot point since the architecture already exists. Last I checked, we don't cost-justify existing architecture that we already paid for with one exception...
...and that is it needs to be replaced or refactored. Which, of course, is the to-be state and would need to cost-justified to the business and IT management in any event. Why, then, is he telling us that we can't do that? That we shouldn't, or that it will never come out correctly (and exactly who's standard is being applied to that assessment)?

Now if he's talking about cost-justifying architecture efforts, that's another matter entirely, and something I'm going to leave to a posting next week, but suffice it to say his principle doesn't hold much water there either.

If I missed something in his message along those lines, anybody in the blogosphere should feel free to correct me.

Stories of Enterprise Architecture

2005-12-28T15:00:00.000-08:00

I've been holding off on commenting about Rebecca Parsons column in IEEE Software because I wanted to think about my approach to this issue for a period of time rather than being simply reactionary. Her column states that enterprise architects should become members of software development teams, and the primary rationale behind that assertion is that developers are often faced with the 'ivory-tower' approach to EA isn't often in touch with their technical work and burdens them with additonal requirements that must be realized in working software.

Point taken, and her premise misses the mark with respect to what the discipline of enterprise architecture is supposed to accomplish within organizations.

Enterprise architects are in a tough spot in the IT food chain. Enterprise architecture is sold to organizations as a means of getting their many needs met in an orderly, expedient, and cost-effective fashion. It's sold to CIOs, developers and IT operations as a, for lack of a better term at the moment, "unified theory" of IT that allows agility, robustness, and operational excellence of applications and supporting technology infrastructure.

Successfully straddling that line is a tall order for any professional, and I've searched for possible solutions and techniques for a long time. The first issue that comes to mind is how enterprise architects are supposed to get these seemingly divergent camps onto roughly the same page in addition to their own views and assertions about the various issues involved.

Part of the answer came to me on my flight back to Portland from my midwestern holiday: a marketing guy by the name of Seth Godin wrote a book titled "All Marketers are Liars" that I bought at the airport bookstore and thumbed through as the flight progressed.

Godin correctly asserts that humans pay far more attention to their wants as opposed to their needs, and drive their decisions and actions toward them. Simple enough, but this tenet (what Godin terms the worldview that all people have) plays a huge role in almost all decision-making, especially regarding purchases and other financial committments. I have no doubt that this extends to the workplace as well.

As such, Godin maintains that the old ways of marketing, and of being better, faster, and cheaper don't work anymore. The solution today is in the stories that we tell - both to ourselves and others. The kicker here is how the stories are interpreted by the listeners, complete with inherent wants and biases - in short, their worldview.

Does this mean that we tell lies? Not at all. Does this mean that folks hearing our messages discount things that don't align with their worldview, even in the face of solid data supporting our cause? In effect, lying to themselves? Most certainly.

What Godin means is that we have to become much better storytellers. In fact, those in the agile camp intimate to me that the better and more complete the user story is, the better the overall outcome of the agile project. Makes sense, but why doesn't it happen more often, especially on larger projects?

Parsons' article, deconstructed to its core element, is simply this: whining from one of the camps that enterprise architects deal with. In fairness, the business-side whines a lot too, but the continued mantra of "you can't do architecture if you don't write code" argument gets really old, really quickly. I could counter that with "you can't do architecture if you can't do business analysis," but the end result is the same.

But then again, they make (and have) the money. And Seth Godin is a marketer who tells stories and admonishes us to tell them also. It appears that we can learn a lot from him on successfully dealing with the stakeholders in our architecture efforts.

Wishing Everyone Happy Holidays!

2005-12-22T03:28:00.000-08:00

I'll be off-the-air for a few days to visit family in the Midwest for the holidays. I hope that all of you have a wonderful holiday season and I'll be back at it on December 26th! Cheers!

Vendor Claptrap: What vs. How

2005-12-21T01:17:00.000-08:00

Deep in my gut over the weekend, I knew this was coming as I perused a stack of professional and trade magazines before taking them to the recycling bin. The headline on the cover of a large-circulation IT rag screams: "Selling SOA to the Business!" Of course, it's laden with pithy vendor and analyst quotes on precisely how you get the biz to caugh up even more cash to store old wine in new bottles. And no benefits to actual business requirements and needs (except the standard admonishment to make sure to collect them) are ever explicitly stated, nor can they be.

If you're in the position as an EA where you have to hard sell technology-specific wares to your non-IT management, one or more of the following major problems exist in your organization:

IT (and probably all other departments) are financially micromanaged to the point that paper clips cannot be ordered without the CFO's approval. Forget technology here folks, organizations like this are in deep trouble across the board.
Line-of-business execs micromanage all technology buys, even if the company is flush with cash. This usually happens in organizations with weak and/or dissrespected IT management regardless of how good or awful the EA effort is.
The EA's, simply put, are not doing their job properly, or at all.

The solution to point (1) above is to find a position elsewhere as quickly as possible. The solution to point (2) would be a choice of the solution to point (1) or to hopefully have the CIO and/or major IT management shown the door and replaced by IT execs with a tad more competence in dealing with business executives.

It's the solution to point (3) that requires more explanation. An EA doing their job properly should never be in a position to have to sell any specific piece of technology to the business. The actual sale of an architecture is done in the context of meeting business issues and processes with the appropriate level of services and systems, regardless of how they are obtained and constructed, and within negotiated time and cost parameters.

In this example, to wax rhapsodic to the business about SOA as a primary solution completely misses the point about why we do enterprise architecture in the first place, because the technology-to-business mapping is either obfuscated or worse, missing entirely. EAs talk to the business about process and process enablement - what will be done for the business in terms that the business understands and not how its accomplished. Go into a meeting with your line executives and talk only about SOA, how great it is, what vendors you'll use, etc. Have a Powerpoint deck full of box-and-arrow diagrams using IT acronyms and dialect. The response will usually be glazed eyeballs and wristwatch or Blackberry checking after about 2 minutes (and you lose them completely) or ensuing, perhaps vicious, arguments about cost and time to deployment. I've seen both outcomes in cases like this, and they are never pretty.

Don't fall for magazine/vendor crap like this, it won't help you in any event. What helps is making that transformation from your technology base to the business strategies and processes you're supporting, and to discuss it only in those terms. You can accomodate anyone wanting a peek 'under the hood' offline - its a reasonable thing to do if certain managers are that interested in technical details. But if you've done your architectural duties properly, the decisions that you and your IT management make about architecture must and should stand up.

It just needs to be explained to the business in a different dialect, the translation of which EAs are also responsible for.

Consultants

2005-12-20T07:01:00.001-08:00

A good portion of the enterprise architect community consists of consultants, whether they work for a consulting firm or on an independent, freelance basis. I have done both for the majority of my IT career.

Like any other profession, we have our share of incompetents, hucksters, and snake-oil salesmen. However, I've run into more intelligent, hard-working, thoughtful EA consultants in my career than the former. I'm proud to call these folks friends as well as colleagues, even when we're competing for the same piece of business or disagree on professional and technical issues.

One of the reasons I became a consultant rather than an employee is that I've always had an independent streak about me, and that manifests itself in my work. This is particularly evident in my continuous challenging of the status quo, the ways "things have always been done" in organizations, and looking for new techniques, methods, and other things of value and interest.

The consulting business (both technical and business) exists for one reason: employees are usually not hired for what they know how to do, or what they have the potential of doing, or how great their grasp of technology or management is. They're hired because they will, or appear to, fit into the corporate culture of the organization. After they start working, they are assimilated into the corporate collective and lose a bit of perspective on the issues confronting them, their group, and their management. A lot of decision-making is made on the basis of being politically safe about one's position rather than sound technical or business principles.

Consultants are brought in by managers to, hopefully, rock the boat, not being constrained to the corporate culture. Of course, they are also brought in to advance other management agendas (the "two Bob's" consultants in the movie Office Space are a prime example). The ability to positively influence the status quo is a big reason I became a consultant.

I always recommend that an organization's EA always be managed and accounted for by employees. While consultants can be brought in to assist, evaluate, and initial drive EA efforts, the ongoing nature of the work makes it obvious that an employee of the organization run it - either right at the beginning or eventually.

Clients must also actively manage the relationship with EA consultants. Hiring them and then going off and leaving them to their own devices is the worst thing clients do. That doesn't mean they have to be actively managed, but the relationship is a two-way street, and clients are paying to have the expertise embedded into their employees and organization. If left on their own, good consultants will certainly give clients their money's worth, but it may not be exactly what the organization expected or needed.

Don't give short-shrift to knowledge transfer when the engagement is drawing to a close. Managers will want their employees assuming roles created and developed during the engagement, and formal transfer is the best way of insuring that knowledge transfer not only take place, but succeeds.

Finally, here is a list of characteristics clients can use to gauge successful consultants that they won't regret retaining:

Asks deeper and more probing questions as interviews and debriefs progress, in a non-judgemental manner.
Works well with client employees, especially with employees who are not amused that consultants are suddently present in their workplace.
Thinks of the client organization, people and goals, rather than the next billable hour. Avoid those that wish to remain on the engagement in perpetuity - its cheaper to hire them an employee...:)
Understands the limitations of corporate culture, budget constraints, and other factors that drive design, development, and procurement decisions. Works well with what's given to him or her without developing extensive strategies, goals, and wish-lists that are realistically unattainable.
Plans for the end; i.e. can show you in a short period of time what the end state looks like from the beginning. If they can't, then be aware that the consultants may think they will be camping out in your organization (and billing you for the 'privilege') for a very extended period of time.

New System Integration Blog

2005-12-19T11:28:00.000-08:00

I started a new blog that focuses on IT system integration issues from the perspective of the intersection of project management, enterprise architecture, and business strategy/process/analysis. I just set it up today, and will be writing on these topics over the holiday season and on into 2006.

Hope to see you there and have a great holiday!

SOA: Cures Whatever Ails You - Maybe

2005-12-16T11:21:00.000-08:00

The hysteria over Service Oriented Architecture (SOA) continues unabated. I haven't seen hype like this for an architecture since client-server went mainstream in the 1980s. Martin Fowler hit the nail on the head terming SOA "Service Oriented Ambiguity."

Of course, the vendors of software, services, training, books, and other essential elements of IT are all over this one with a vengence: "DO YOU HAVE A SOA STRATEGY??" screams one ad. Followed in another with: "IMPLEMENT SOA BETTER/FASTER/CHEAPER WITH OUR PRODUCTS!!" It's not surprising that vendors jump on fast moving bandwagons like SOA because, being vendors, they have to make money playing the tune du jour in the IT and development communities.

The problem with SOA, as Martin articulates, is that it means different things to different people, and that poses major problems when attempting to design enterprise and system architectures that are standard and robust. For example, I have run into more than a few situations where SOA simply means web services and little else. Or a refactoring and repackaging of data services previously handled as stored procedures in a database server termed SOA, in its entirety.

My advice is to skip the fad in terms of nomenclature and still keep selling EA to the business as EA. You may have an SOA, but so do your competitors and, it appears, everybody else...:)

Implications of the Word "Enterprise"

2005-10-27T03:27:00.000-07:00

Anyone following EA for any length of time knows that there are many definitions of the term 'Enterprise Architect." Some are very well-thought out, others are too skewed toward IT and in particular, IT developers. In the worst case, it is mangled by various software vendors to fit their wares into some marketing space to sell product, process and other EA functions be damned.

After reviewing and thinking about all of these definitions and explanations over the years, I have come to the following conclusion: when using the word "enterprise" with respect to any business or IT-related entity or issue, the usage must coincide with the business issues under discussion, not the underlying technology or any other systems-oriented focus. That would also mean that the word "system" would be appropriate when referencing technical and other IT-specific issues.

I favor an approach where the enterprise architect is tasked with developing and managing the business process-to-technology interface space, with substantial input into certain IT-centric issues such as infrastructure, data, IT project ROI/costs, and overall general architectures. On the IT side, a subclassification of an enterprise architect emerges: the system architect or system designer; who takes the broader view from EA and concentrates more on the technical mechanics of functional design, implementation, and production.

The hue and cry from certain technical camps demands that enterprise architects focus more on developer-centric activities. The fact of the matter is one cannot have it both ways: an EA cannot be all things to all parties, and I know of very few people who can assimilate and deal effectively with complex business issues and requirements along with learning and practicing the flavor-of-the-year programming environment or 'kewl' new development framework and databases. There's just too much there for one (or even 2 or 3) people to successfully handle in a medium-to-large environment.

So, just as there is an admittedly fuzzy demarcation line between an EA, the business, and IT, there also exists one between the EA and the next technical level within IT: the systems architects and technical project leads. The EA is tasked with successfully transmitting the overall IT architecture vision to the systems architects (and getting an understanding from them of the tradeoffs and issues/problems that they face without huge amounts of time spent in immersion of technical arcana. The system architects and technical project leads must work with the EA to get an understanding of the EA's broader architecture, what those implications mean to their projects, and deal with them effectively from development and implementation standpoints.

I know, easier said than done. It works best as a collaboration with the EA's vision treated as customer requirements would be. The EA must endeavor to rationalize the overall vision in terms of what is, and what realistically can be from a technical perspective. The two sides must rely on each other for overall success, not dictate terms to each other.

What Enterprise Architects Aren't

2005-09-24T22:52:00.000-07:00

There are many definitions of enterprise architecture, and about as many definitions or perhaps more of enterprise architect. To frame what an EA is or may be, here's some scope on three associated positions found in a typical organization:

Business System Analyst / Business Analyst - Typically a business process type. Intimately knows the segments of the business being dealt with, and either came from a technical background or acquired enough to be dangerous. Usually deals at the business / high-technical requirements level and may be involved in use case development and system testing.

System Architect / Software Architect - Analog of a business analyst on the technology side. Specifies software designs, hardware or network configurations, database schemas, and functional software specifications (of varying types) used by IT development teams. Usually materially participates in the development - writing software, data modeling, prototyping, testing, etc. as the development proceeds.

Project Manager / Project Lead - Individuals reponsible that a specific set of deliverables or outcomes are produced and delivered, hopefully on-time and on-budget. May control the above-mentioned personnel as subject-matter experts (SMEs) or, to use the project management term, as 'resources' on the project.

None of these people are enterprise architects, primarily because their scope is too narrowly defined looking at the dictionary term for 'enterprise.' They are primarily focused on their own work and projects, even if as often hppens, their work creates or perpetuates IT and process silos within a business.

Taking a look at the job descriptions for enterprise architects on major internet job boards leads to other descriptions that don't appear to merit the title:

Tool Jockeys - When the job description is overly loaded with technology buzzwords or vendor toolsets, the employer isn't looking for an enterprise architect, they are looking for someone intimately familiar with specific IT-related tools and don't appear to be overly concerned about what is produced with the technology and tools. Quite a few don't even list interacting with the business, much less understanding it, as a requirement of the position.

System/Software Architects - See above description. If the job description centers around writing code, doing functional specs and designs, agile development, or development in general, they aren't looking for an enterprise architect because the scope is too narrow. A number of agile and extreme software development advocates and authors make this erroneous distinction as well.

Expected to manage IT projects as well as "do architecture" - I've worked as a project manager and teach the subject at a midwestern university. Project management, done correctly, is full-time, difficult and complex work in its own right. Adding EA duties to PM work, or vice-versa, is usually granting a license to fail at either or both. Unless someone with both jobs can clone themselves, there isn't enough time in the day to address the unique characteristics and duties of both positions properly and completely.

So, you ask, what is enterprise architecture and what is an enterprise architect? We've examined what they aren't, and my next posts will examine what they are, and can successfully become.

Defining Enterprise Architecture

2005-09-02T04:30:00.000-07:00

Defining EA is a tall order, since there is little agreement on what exactly EA is. Some come from the technical-side of things, where everything is developer-centric and otherwise revolves around cool tools and subsystems to play with. Others take a more holistic approach, preferring to operate and a major system/subsystem-level, with still others taking it all the way to a business-process and IT integration level. Of course, we also have the various EA frameworks (Zachman, TOGAF, FEAF, etc.).

I'll share my definition (actually, definitions) of EA with you in an upcoming post. Before that, let's think about a few things:

The use of the word enterprise lends itself to an authority and scope that is much more than just IT, or just "the business."
Architects that operate at the developer/coder-level are not "enterprise" in scope, but more "system" or "application." So, I would call them System or Application Architects more than I would EAs.
System Analysts have similar issues, but they are more business-facing, so I would refer to them more as Business Analysts or, a term I fancy lately: "integration-ist." Yeah, I know it's not word...yet.

What one sees in various EA literature is the term bandied about for everything from systems analysis to development and the project management of development. That is particularly annoying and confusing to some because there are two major sides to EA: the technology side; and the business side. As we will see, you cannot skimp or gloss over one and concentrate on the other exclusively if an EA project is to succeed.

Here's a short list about what isn't EA:

Development and developer-centric philosophies that largely exclude the business-side. This stance goes a long way in explaining why technical people largely 'don't get it' in terms of the business their organization is in.
Business-centric philosphies that ignore the technology part of the architecture. This is a primary driver of the attitude of business-types that IT is just a cost center.
Ignorance of the "as-is" business and technology states of n organization and only concentrating on the 'to-be' or goal state.
Ignorance or oversight of business and technical processes - automated, workflow, compliance, financial, etc.

We've got a lot of ground to cover. Stay tuned.

Getting Set Up

2005-08-28T06:44:00.000-07:00

Getting things set up for my enterprise architecture and business process maangement (BPM) blog. In the meantime, feel free to check out my project mangagement blog at http://rjmtech.blogspot.com

Thanks and see you all soon!

URL:	http://www.ITscout.org/ITguide
Username:	architecture
Password:	itguide