Concerning Architecture and Complex Distributed Systems
What is architecture ?
Architecture in the context of IT systems and software has many meanings going from high level enterprise or business process or solution architectures through software & IT & system architectures and all the way down to hardware architectures. Hence with the term architecture we need to define very clear exactly what we are talking about. In this respect the comment from Martin Fowler is interesting: Architecture has become a very slippery word in the software business. It's hard to come up with any solid definition of what it means. I see it as a fundamentally subjective term - when people describe their software architecture they select the important parts of their systems, how these parts fit together, and the key decisions they made in designing their systems. Architecture is also seen as a technical issue, with the implication that the key decisions that need to be made are technical decisions.
In this section I would like over time to develop the ideas concerning architectures; it various facets; how they interrelate and practical examples of what I would consider successful, pragmatic architectures.
Pragmatic architectures - as illustrated very clearly by Michael Nygard in Release It!: Design and Deploy Production-Ready Software ...
- Two divergent sets of activities both fall under the term architecture. One type of architecture strives towards higher levels of abstraction that are more portable across platforms and less connected to the messy details of hardware, networks, electrons, and photons. The extreme form of this approach results in the "ivory tower" - a Kubrickesque clean room, inhabited by aloof gurus, decorated with boxes and arrows on every wall ...
- In contrast, another breed of architect rubs shoulders with the coders and might even be one. This kind of architect does not hesitate to peel back the lid on an abstraction or to jettison one if it does not fit ...
- When the ivory tower architect is done, the system will not admit any improvments; each part will be perfectly adapted to its role. Contrast this with the pragmatic architects creation, in which each component is good enough for the current stresses - and the architect knows which ones need to be replaced depending on how the stress factors change over time.
Adaptable enterprise architectures - again the thoughts of Michael Nygard ...
- Some archtiects aspire to create a modern day Colossus: The Forbin Project (or Tron if you're in my generation). They labor under a vision of the seamless enterprise, working as a united machine. Every part is a necessary piece of the whole, perfectly suited to its role. Programs and programmers dwell inside the machine, serving its needs. I regard such suptopians with deep suspicion.
- Such an architecture proceeds from the top down, usually beginning with a giant framework like Zachman or TOGRAF. They regard the architecture as an entity of its own, with mere systems filling in the cells of the enterprise architecture matrix. If this group has power, they will put projects on hold until the enterprise architecture is defined. If they do not have power, they gnaw on their own livers as project after nonconforming project rolls into production without the benefits of their architecture.
- This view rests atop two flawed assumptions. First, this assumes that the architecture can ever be finished. To state that the enterprise architecture is "in place" means an end of change. If the enterprise architecture stops changing, the organization will be frozen in time ... Second, the top-down approach assumes that theorganization can hold back time, denying change while the enterprise architecture is defined. This is costly in rel terms and in opportunity cost ...
- Mechanistic metaphors for the enterprise trouble me. Mechanical systems exhibit exactly those attributes we don't want our organizations to share; they are both rigid and fragile ...
- Real enterprises are always messier than the enterprise architecture would ever admit. New technologies never quite supplant old ones. A mishmash of integration technologies will be found, from flat file transfer with batch processing to publish/subscribe messaging. Any strategy formulated predicated on creating a monoculture - whether it isa a single integration technology or a single programming language - is doomed to a costly failure ...
- In fact, the most useful criterion for evaluating architectures is this: "Does it make IT better at responding to its users' needs ?" Most enterprise architectures are not constructed with this in mind.
Further definitions ...
See also definitions given by SEI (Software Engineering Institute) in
Software Architecture in Practice:
The software architecture of a program or computing system is the structure or structures of the system, which comprise software elements, the externally visible properties of those elements, and the relationships among them.
See also entries in Wikipedia and Grady Booch's Handbook of Software Architecture.
It is very difficult to find good books on IT architecture - see also section on SOA. The following I have collected over the years. Some still have relevance.
- The most important WIKI to read: 97 Things Every Software Architect Should Know
- Johannes Siedersleben: Moderne Software-Architektur. Umsichtig planen, robust bauen mit Quasar.
- A.-W.Scheer Architecture of Integrated Information Systems. Foundations of Enterprise- Modelling: Foundation of Enterprise Modelling
- Dan Woods: Enterprise Services Architecture (O'Reilly Field Guide to Enterprise Software)
- Gregor Hohpe, Bobby Woolf: Enterprise Integration Patterns: Designing, Building, and Deploying Messaging Solutions
- Martin Fowler: Patterns of Enterprise Application Architecture
- See also the rest of the series edited by Martin Fowler
- ...
I am not just interested here in the detailed architectures of individual software applications - but more specifically in enterprise or "gobal architectures" - large scale, globally deployed, distributed systems. Such systems present many engineering challenges - refer to the following for excellent discussions of the fundemental issues in real-life systems:
- Add simplicity: eBay's Dan Pritchett
- All Things Distributed: Amazon's Werner Vogel
- google's Gerald Hohpe
- ...
Topics ...
- Architectural Styles / Patterns
- Strategy
- Gartner: IT Architecture matters. Refer also to strategy section ...
- Engineering
- Keep it safe, simple (if I don't understand it, who will ?)
- Software is brittle, build in bulkheads
- Frameworks / Standard Applications
- Spring
- SAP / Netweaver
- ...
- Distributed Systems
- Fallacies of Distributed Computing - Peter Deutsch.
- Leslie Lamport (1978): "Time, clocks, and the ordering of events in a distributed system". Communications of the ACM 21
- Impossibility of distributed consensus with one faulty process
- The Byzantine Generals Problem
- Lazy Replication: Exploiting the Semantics of Distributed Services
- A Distributed Systems Reading List
- ...
- Transactions
- ...
- System Modelling / Description / Specification
- Security
- William Stallings: Network Security Essentials: Applications and Standards
- Security Research, Computer Laboratory, University of Cambridge
- ...
- Access + Identity Management
- ...
- Operational Manageability
- Michael Nygard: Release It!: Design and Deploy Production-Ready Software
- ebay's Dan Pritchard - Architecture Quality: Operational Manageability
- Forensic Architecture
- The process of describing the architecture of something after it has been built. It is a largely frustrating, cumbersome process ...
- Technoracle (a.k.a. "Duane's World")
- The art of ongoing System Documentation
- ...
- Release / Deployment processes
- ...
- Log files
- The importance of the humble log file is much neglected by developers. Most developers have not learned the difference between data and information. Sadly many logs files are verbose, lack information, are difficult to read or to parse.
- Monitoring
- ...
- Scalability / Performance
- Cal Henderson: Building Scalable Web Sites: Building, scaling, and optimizing the next generation of web applications
- Theo Schlossnagle: Scalable Internet Architectures
- Fault Tolerance / High Availability
- ...
- Other Related Topics for consideration ...
- Meta data
- Configuration, customization or programming - using hundreds of XML configuration files for configuration is also a form of programming and often leads to systems that are hard to maintain and understand.
- Virtualization
- Cloud computing
Books / References ...
- Further favorites ...
- Nati Shaloms Blog
- Duncan Cragg's Blog
- Michael Nygard: Release It !
- inoq: Tracking change and innovation in the enterprise software development community
- High Scalability | Building bigger, faster, more reliable websites.
- Web Performance Matters - Journal / Collected thoughts about software and Web performance management
- Website Infrastructure and Operations / Web Site Scalability and Systems Architecture
- Publications, Presentations ...
- Prostep Symposium 2004: plm2plm - Global Engineering Collaboration bei einem Zulieferer
- Prostep Workshop ECAD/MCAD-Integration July 12, 2005
- IEEE MultiMedia, December 1994 Supporting Cooperative Medicine: The Bermed Project - [1]
- OSF/DCE Focus Meeting, Frankfurt, April 1995 - [1] [2] [3]
- Open Systems in Medicine, (Studies in Health Technology and Informatics, Vol 8)
- "...clear overview by Martin Ohly ... The integrated hospital systems developed in the project and the telemedicine services are well described ... I ended by feeling more positive about the underlying strengths of the work described and admiring the achievement of delivering such a huge and broad-ranging set of demonstrations in clinical practice. The book is an important record of a major contribution in the evolution of health informatics.' - David Ingram, UCL Medical School, London, UK. Journal of Telemedicine and Telecare, volume 2, no. 2, 1996, p. 124"
- Aiming to improve quality in health care and diagnostics, this book discusses information processing and communication in medicine and presents a new approach.' Siemens Review, volume 62, no. 5, September/October 1995, p. 40 Abstracted in Biological Abstracts/RRM, BIOSIS, volume 47, issue 4, April 1994 Covered by Current Contents, Life Sciences (ISI), volume 38, no. 17, April 1995, p. 13-14 "
Reviews ...