Concerning Optimal Development and
Dedicated Colleagues everywhere I thank you all ! ...
You need just enough elbow space to move the mouse. Headphones help as well.
Topics - in time I would like to ellaborate on the following ...
- Organizational Structures + Teams + How to develop great software products - if nothing else then read about Microsoft ...
- Jim McCarthy: Dynamics of Software Development - dated but this is a classic - see below for more ...
- Steve Maquire: Debugging the Development Process: Practical Strategies for Staying Focused, Hitting Ship Dates, and Building Solid Teams
- Organizational structure at Microsoft, a typical Microsoft project has ...
- Project Lead - ultimately responsible for the code ...
- Technical Lead - the technical lead is the programmer on the team who whos the product's code better than anyone else ...
- Program manager - the program manager is responsible for coordinating product development with marketing, documentation, testing, and product support ...
- Alan Page, Ken Johnston: How We Test Software at Microsoft
- Methodologies - Agile manifesto revisited ...
- Individuals and interactions over processes and tools
- Working software over comprehensive documentation
- Customer collaboration over contract negotiation
- Responding to change over following a plan
- Balancing Agility + Disciplined approaches ...
Confused by the ever increasing range of methodologies - from agile to disciplined: XP, Scrum, V-Modell, CMMI, SPICE ... and any number inbetween (do we need another methodology ?) See the excellent book Barry Boehm, Richard Turner: Balancing Agility and Discipline: A Guide for the Perplexed. The top six conclusions of this book are especially pertinant:
- Neither agile nor plan-driven methods provide a silver bullet
- Agile and plan driven methods have home grounds where one clearly dominates the other
- Future trends are toward application developments that need both agility and discipline
- Some balancing methods are emerging
- It is better to build your method up than to tailor it down
- Methods are important, but potential silver bullets are more likely to be found in areas dealing with people, values, communication and expectations management
Unfortunately, software engineering is still struggling with a "separation of concerns" legacy that contends translating requirements into code is so hard that is must be accomplished in isolation from people concerns ...
- Learning from classical product development (how do the best build excellent products)...
- Software product development is also an engineering discipline
- Can we learn from other types of product or engineering development/projects ? See methods+processes of classical product development in my text concerning PLM - product life cycle management, classical PEP "Produktentstehungsprozess", Milestones, Start of production ...
- Optimized software product development: Milestones and iterations, clear gate criteria
- Development practices, read the following:
Jim McCarthy's: The Stages of the Game
- Opening moves include creating a reasonable market strategy, a product design, and a development plan and undertaking the initial development activities ...
- The middle Game is the (seemingly vast) expanse of time usually bounded at the early end by the first slip in the schedule and at the other by the onset of "Ship Mode" ...
- Ship Mode is what you must get into before you can get out of this hell ...
- Launch is that single climatic moment, the moment that will live on in the hearts and minds of yours customers forever - or until the next launch, whichever comes sooner ...
Key supporting processes ...
- Requirements management, Expectation management ...
- Estimatation (you can't estimate what you don't know)
- Writing specifications, documentation - a neglected skill !
- Quality Assurance, Software Testing, Test Automation - read Alan Page, Ken Johnston: How We Test Software at Microsoft®
- Software support processes - OSS and why SAP is so good
Coding ...
- Tools, tool supported development, automation
- Choosing the right tool (programming language, ...) for the job
- Version control / Config management / Subversion ...
- Issue / Defect management / Change management / JIRA ...
- Importance of code standards, consistency
- The Laws of Physics
- See Open collaboration within corporations using software forges...
- Software forges are tool platforms that originated in the open source community. Many corporations are improving and extending their software development practices by adopting forges internally.
- Software project conflict resultion strategies
- Diverse Teams
- Offshore Development / outsourcing ...
- Fred Brook's mythical man month revisited ...
- See how are the most successful effective ? How does google develop software ?
For example see Joe Beda's blog ...
- There is, by and large, only one code base at Google. This has many advantages. Most obvious is that it is really easy to look at and contribute to code in other projects without having to talk to anyone, get special permissions or fill out forms in triplicate. That is just the tip of the iceberg, though. Having one codebase means that there is a very high degree of code sharing. Need to base 64 encode/decode something? No problem, there is a standard Google routine for that. Found a bug? Just fix it and check it in after getting it code reviewed by a documented owner. One of the reasons that environments like Perl, Python, C#, Java, etc. flourish is that they have large and well through out libraries of useful code. For a variety of reasons, C++ has never had this. (I could theorize but that would be off topic.) Google has solved this problem by building up a large library of well documented and easy to integrate code. This not only lowers the bar for new projects but makes it easy to switch projects as you don't have to learn new conventions.
- Switching teams at Google is a very fluid process. An engineer can be 40% on one project, 40% on something completely different and 20% on his or her own thing. That mix can be adjusted as project requirements change. Switching groups should also not have an affect on your annual review score because of arbitrary team politics. Joining a new group is more about find a good mutual fit then going through HR and a formal interview loop. When there is an important project that needs to be staffed the groups and execs will evangelize that need and someone who is interested is bound to step up. If it is critical to the business and no one is stepping up (I don't know if this has occurred or not) then I imagine it would go come down to a personal appeal by the management team.
- The intranet in Google is super transparent. Teams are actively encouraged to share the most intimate details of their projects with the rest of the company. This happens through tech talks, design docs, lunch table conversations, etc. When two teams are doing similar things, people start with the assumption that they must have their reasons and that the situation will be worked out in time. There isn't a huge push to over optimize and have only one solution for each problem. This means that there isn't an adversarial relationship between teams that can lead to long standing animosities and information hiding.
- There is a big difference between pet projects being permitted and being encouraged. At Google it is actively encouraged for engineers to do a 20% project. This isn't a matter of doing something in your spare time, but more of actively making time for it. Heck, I don't have a good 20% project yet and I need one. If I don't come up with something I'm sure it could negatively impact my review.
- The intrapersonal environment at Google is very energizing. When someone comes up with a new idea, the most common response is excitement and a brainstorming session. Politics and who owns what area rarely enter into it. I don't think that I've seen anyone really raise their voice and get into a huge knockdown drag out fight since coming to Google.
Or from Steve Yegge's blog ...
- From a high level, Google's process probably does look like chaos to someone from a more traditional software development company. As a newcomer, some of the things that leap out at you include:
- There are managers, sort of, but most of them code at least half-time, making them more like tech leads.
- Developers can switch teams and/or projects any time they want, no questions asked; just say the word and the movers will show up the next day to put you in your new office with your new team.
- Google has a philosophy of not ever telling developers what to work on, and they take it pretty seriously.
- Developers are strongly encouraged to spend 20% of their time (and I mean their M-F, 8-5 time, not weekends or personal time) working on whatever they want, as long as it's not their main project.
- There aren't very many meetings. I'd say an average developer attends perhaps 3 meetings a week, including their 1:1 with their lead.
- It's quiet. Engineers are quietly focused on their work, as individuals or sometimes in little groups or 2 to 5.
- There aren't Gantt charts or date-task-owner spreadsheets or any other visible project-management artifacts in evidence, not that I've ever seen.
- Even during the relatively rare crunch periods, people still go get lunch and dinner, which are (famously) always free and tasty, and they don't work insane hours unless they want to.
- Software/System Deployment projects, Project managment, ...
And for the rest of us who do not work at google - this deserves a whole chapter on its own. Read Tom DeMarco, Peter Hruschka, Timothy Lister: Adrenaline Junkies and Template Zombies: Understanding Patterns of Project Behavior
- PM methodologies: PMBOK, PRINCE, ... (do we need another methodology ?)
- Project metrics, control + audit mechanisms (how much control to apply ?)
- The minimum to define: Specifications, Exclusion criteria, Customer obligations, Estimations, Work breakdown structures, Milestones, Acceptance criteria
- Your customer is your best: specification author, developer, tester, champion, salesperson ...
- Use of checklists ...
- Risk management (the unknowns will bring you down): identify the risks; eliminate the highest risks first; develop plan B
- And remember: always take meeting minutes !
- Improving Productivity
For example see Software development process is too slow: Henning Kagermann, chairman and CEO of enterprise software company SAP AG, believes the pace of the software development process lags the exponential speed of progress in other areas of computing. That must change if software aspires to be the engine of business innovation. "If you look to the efficiency in software development, we are still far behind Moore's Law, where productivity increases every 12 to 18 months, depending on the measure of the cycle. Someone gave the estimation that it takes six to seven years in software to double our productivity," Kagermann said. "That's not very good." Cutting that production cycle in half can only be achieved by what Kagermann calls the "industrialization of software." This is a fundamental shift akin to the automation of the car industry, where a standard platform paved the way for custom design and innovation. Shortening the cycle of the software development process also depends on involving the end user "much earlier," he said, "so you can iterate faster."
Ken Schwaber: Agile Project Management with Scrum: the heart of Scrum lies in the iteration. The team takes a look at the requirements, considers the available technology and evaluates its own skills and capabilities. It then collectively determines how to build the functionality, modifiying the approach daily as it encounters new complexities, difficulties, and surprises. The tesm figures out what needs to be done and selects the best way to do it. The creative process is the heart of the Scrum's productivity.
Testing ...
Software forges / Open Source Tools
Staffing ...
Books / References ...
I have been buying and reading all sorts of computing books since 1975 - I have literally hundreds stacked up in the basement. Here is just a selection of my favorites - concerning software development, engineering, computer science - those with a place on my desk. Those that get read again and again.
- Robert N. Britcher: The Limits of Software: People, Projects, and Perspectives
- Robert L. Glass: Facts and Fallacies of Software Engineering
- Fact 2: The best programmers are up to 28 times better than the worst programmers - I know this - has anyone demanded that you provide estimatations to an accuracy of < 1% ?
- Fact 3: Adding people to a late project makes it later. - see also Fred Brooks ...
- Fact 10: Software estimation is usually done by the wrong people.
- Fact 23: One of the two most common causes of runaway projects is unstable requirements
- Fact 34: Test tools are essential, but rarely used
- Fact 41: Maintenance typically consumes 40 to 80 percent of software costs. It is probably the most important life cycle phase of software
- Fallacy 2: You can manage quality into a software product
- Fallacy 5: Software needs more methodologies
- ...
- Jim McCarthy: Dynamics of Software Development
- Rule 31: Beware of a guy in a room
- Rule 32: If you build, it will ship
- Rule 33: Get to a known state and stay there
- See the following for a summary 21 Rules of Thumb – How Microsoft develops its Software
- and also The McCarthy Show
- ...
- Software Engineering Classics (the books and articles I started reading in 1981 ...)
- Niklaus Wirth: Algorithms + Data Structures = Programs
- Donald E. Knuth: Art of Computer Programming, Volume 1: Fundamental Algorithms
- Brian W. Kernighan, P. J. Plauger: Software Tools
- Alfred V. Aho, Ravi Sethi, Jeffrey D. Ullman: Compilers: Principles, Techniques, and Tools
- Alfred V. Aho, Brian W. Kernighan, Peter J. Weinberger: The AWK Programming Language
- Andrew S. Tanenbaum: Operating Systems: Design and Implementation
- Andrew S. Tanenbaum: Computer Networks
- Bertrand Meyer: Object-Oriented Software Construction
- ...
- D.L. Parnas: On the Criteria To Be Used in Decomposing Systems into Modules
- E. F. Codd: A Relational Model of Data for Large Shared Data Banks, Communications of the ACM, Vol. 13, No. 6, June 1970
- Butler W. Lampson: Principles for Computer System Design, 1993 ACM Turing Award Lecture
- J.H. Saltzer, D.P. Reed and D.D. Clark: END-TO-END ARGUMENTS IN SYSTEM DESIGN, M.I.T. Laboratory for Computer Science
- Jim Gray: What Next? A Few Remaining Problems in Information Technology, 1998 ACM Turing Award Lecture
- C.A.R.Hoare: The Emporer's Old Clothes, 1980 ACM Turing Award Lecture
- ...
- Modelling
- James Rumbaugh et al: Object-Oriented Modelling and Design - The classic, the predessor to UML.
- Ivar Jaocbson: The object advantage - business process reengineering with object technology - Another classic and predessor to UML.
- Philippe Kruchten: The Rational Unified Process: An Introduction
- Martin Fowler: UML Distilled: A Brief Guide to the Standard Object Modeling Language
- Grady Booch: Managing the Object-Oriented Project
- Scott W. Ambler, Ron Jeffries: Agile Modeling: Effective Practices for Extreme Programming and the Unified Process
- ...
- Software Development ...
- Steve McConnell: Code Complete: A Practical Handbook of Software Construction
- Steve McConnell: Rapid Development: Taming Wild Software Schedules
- Steve Maquire: Debugging the Development Process: Practical Strategies for Staying Focused, Hitting Ship Dates, and Building Solid Teams
- Steve Maquire: Writing Solid Code
- Scott Meyers: Effective C++: 55 (originally 50) Specific Ways to Improve Your Programs and Designs
- Brian W. Kernighan, Dennis M. Ritchie: C Programming Language
- Bjarne Stroustrup: The C++ Programming Language
- Ken Arnold, James Gosling: Java(TM) Programming Language
- Joshua Bloch: Effective Java
- ...
- Project Management / SW Development Processes / Methodologies - an overview of my favorites
- The basic wisdom from Jim Mccarthy: Remember the triangle. There are only three things that you are working with as a development manager: resources (people and money), features and the schedule. Changing one has an impact on at least one other axis, usually two. It is a simple enough matter to mentally run through the sides of the triangle, or force others to do so, when discussing any part of it. Since the people, the product or the schedule is almost always what you’re discussing, this means that you must constantly envision the triangle. This leads to the most fruitful line of thought.
- My current favorite - this book presents 86 patterns of project bevavior - most of which I can identify with: Tom DeMarco, Peter Hruschka, Timothy Lister: Adrenaline Junkies and Template Zombies: Understanding Patterns of Project Behavior
- Fred P. Brooks: The Mythical Man-Month
- Tom DeMarco: The Deadline: A Novel About Project Management
- Watts S. Humphrey: Introduction to the Personal Software Process
- Barry Boehm, Richard Turner: Balancing Agility and Discipline: A Guide for the Perplexed
- Kent Beck: Extreme Programming Explained: Embrace Change
- Ken Schwaber: Agile Project Management with Scrum
- Scott Berkun: The Art of Project Management
- Klaus Hörmann, Lars Dittmann, Bernd Hindel, Markus Müller: SPICE in der Praxis: Interpretationshilfe für Anwender und Assessoren
- Mary Beth Chrissis, Mike Konrad, Sandy Shrum: CMMI: Guidelines for Process Integration and Product Improvement (Sei Series in Software Engineering)
- ...
- Specification / Estimation / Requirements Management
- Software Testing / Quality Assurance
- Software Project Risks and Failures ...
This is a neglected classic. I have read it several times. Maybe too melanchonic for many. For me it is pure beauty. Britcher uses as example his experiences from the massive FAA Advanced Automation System (AAS) project, an attempt at an improved air traffic control system that ran into trouble in the early 90s after an estimated 6.5 US$ billion was spent. He describes this project as " ... the greatest debacle in the history of organized work... we learned nothing from it ..." Concerning the demise: ... it was the software of course. But not in the sense we all want to believe. It was in the nature of software, its roots tangled in the mathematics of contradiction and logic and symbol production, plied by young tradesmen schooled in the tools of the latest process, its promise intensified by greed and ease, its complexity beyond imagining, man mingling with machine to a degree that cannot be described, felt to be immune to the laws of size - these computer programs, easy to miss within the great facilities that house them, amidst the spandrels and joists and treelike arches and great enclosures, decorated for art's sake ... What chance do we have ?
Robert Glass is one of my favorite authors in the realm of computing. Maybe there is little new in Facts and Fallacies. But it is certainly reassuring, refreshing to find a no-nonsense, pragmatic, knowlegable summary of the realities of software development. Here are some of my favorites:
An excellent insight into the reality of the development of "shrink-wrapped" software - direct from Microsoft. Introduces the power of the daily build and famous for the 54 rules, such as:
This topic interests me most - and hopefully this is an area that I can expand further upon. Software projects suffer failure more often than any other type of project. Why is this ? Have we learned the lessons ? Are we improving ? Latest example: Heathrow Terminal 5 baggage handling system - don't we ever learn ? do you also have this feeling of deja vu ?