We get called in occasionally because a company’s leaders don’t feel that their product development is happening rapidly enough. They recall how fast the product evolved when the company was first started and they want that pace again. There are many reasons for the pace of development to have slowed. Certainly one of the more popular catch phrases that people use is technical debt, which is a metaphor to explain the eventual consequences of fast paced development. As you incur technical debt, your pace of development slows.

I think there is another factor that is equally or possibly more responsible for slowing the pace of development, interruptions. Engineers need large blocks of uninterrupted time to think, design, plan, code, and test. Disrupting an engineer during these tasks often require a wholesale reset of their thought process. There have been lots of studies that support this one such study found that when tasks were interrupted people require upwards of 27% more time to complete, commit twice the number of errors, and experience twice the increase in anxiety as compared to uninterrrupted tasks. And, as a recent CNN article explained, this problem of disruptions affecting our productivity gets worse as we get older.

So what is interrupting engineers? I’d wager it’s predominantly meetings. While communicating, coordinating, interviewing, etc are all very important for engineers to participate in, doing so in a haphazard manner can be devasting to productivity. In this competitive hiring environment, interruptions might just be driving your engineers out the door. Try a few of these suggestions to reduce interruptions for engineers:

• Have at least one day per week where meetings are not allowed
• Only allow meetings at the beginning or end of the day
• Require all meetings to have agendas and goals
• Question standing meetings to ensure all participants are necessary

While measuring productivity is incredibly difficult most organizations can feel when the pace of development has slowed. Reduce the interruptions of your engineers and see if this doesn’t help increase the pace again.

We’ve completed the first draft of our new book “Scalability Rules – 50 Principles For Scaling Web Sites” and wanted to share the table of contents with everyone. We have a terrific team of technical editors who are reviewing every rule in detail but would also like to offer this opportunity to anyone else so inclined. Our publisher, Addison-Wesley Professional, has posted the draft versions of Chapters 1-5 (Rules 1-19) on line at Safari Rough Cuts and should have a couple more chapters available soon. If you’re interested in a sneak preview or would like to provide feedback, sign up and take a look. Below is the book’s table of contents.

Chapter 1 – Reduce the Equation

• Rule 1 Don’t Over Engineer The Solution
• Rule 2 Design Scale Into the Solution (D-I-D Process)
• Rule 3 Simplify the Solution 3 Times Over
• Rule 4 Reduce DNS Lookups
• Rule 5 Reduce Objects Where Possible
• Rule 6 Use Homogenous Networks

Chapter 2 – Distribute Your Work

• Rule 7 Design to Split Reads and Writes (X axis)
• Rule 8 Design to Split Different Things (Y axis)
• Rule 9 Design to Split Similar Things   (Z axis)

Chapter 3 – Design to Scale Out Horizontally

• Rule 10 Design Your Solution to Scale Out – Not Just Up
• Rule 11 Use Commodity Systems (Goldfish not Thoroughbreds)
• Rule 12 Scale Out Your Data Centers
• Rule 13 Design to Leverage the Cloud

Chapter 4 – Use The Right Tools

• Rule 14 Use Databases Appropriately
• Rule 15 Firewalls, Firewalls, Everywhere!
• Rule 16 Actively Use Log Files

Chapter 5 – Don’t Duplicate Your Work (Nov 30th)

• Rule 17 Don’t Check Your Work
• Rule 18 Stop Redirecting Traffic
• Rule 19 Relax Temporal Constraints

Chapter 6 – Use Caching Aggressively

• Rule 20 Leverage CDNs
• Rule 21 Use Expires Headers
• Rule 22 Cache Ajax Calls
• Rule 23 Leverage Page Caches
• Rule 24 Utilize Application Caches
• Rule 25 Make Use of Object Caches
• Rule 26 Put Object Caches on Their Own “Tier”

Chapter 7 – Learn From Your Mistakes

• Rule 27 Learn Aggressively
• Rule 28 Don’t Rely on QA to Find Mistakes
• Rule 29 Failing to Design for Rollback Is Designing to Fail
• Rule 30 Discuss and Learn from Failures

Chapter 8 – Database Rules

• Rule 31 Be Aware of Costly Relationships
• Rule 32 Use the Right Type of Database Locks
• Rule 33 Pass on Using Multi-phase Commits
• Rule 34 Try Not to Use “Select For Update”
• Rule 35 Don’t Select Everything

Chapter 9 – Design for Fault Tolerance and Graceful Failure

• Rule 36 Design Using Fault Isolative “Swim Lanes”
• Rule 37 Never Trust Single Points of Failure
• Rule 38 Avoid Putting Systems in Series
• Rule 39 Ensure You Can Wire On and Off Functions

Chapter 10 – Avoid or Distribute State

• Rule 40 Strive For Statelessness
• Rule 41 Maintain Sessions in the Browser When Possible
• Rule 42 Make Use of a Distributed Cache For States

Chapter 11 – Asynchronous Communication and Message Buses

• Rule 43 Communicate Asynchronously As Much As Possible
• Rule 44 Ensure Your Message Bus Can Scale
• Rule 45 Avoid Overcrowding Your Message Bus

Chapter 12 – Miscellaneous Rules

• Rule 46 Be Wary of Scaling Through 3rd Parties
• Rule 47 Purge, Archive, and Cost-justify Storage
• Rule 48 Remove Business Intelligence from Transaction Processing
• Rule 49 Design Your Application to Be Monitored

Chapter 13 – Rule Review and Prioritization

The Paleolithic Era (Old Stone Age) lasted roughly from 2.5M to 10,000 years ago. During this time humans moved around in small bands as hunter/gatherers. Sometime around the Neolithic Age (New Stone Age) humans invented or discovered farming. While turning unedible crops like wheat into food is impressive, what’s even more impressive is that humans separately invented farming at least three times and possibly as many as seven times. Different civilizations from Eastern Mediterranean to China to Mexico all came up with the idea of farming, presumably without sharing this knowledge in any way.

While the discovery of farming might seem an evolutionary necessity for long term survival the coincidental simultaneous invention by disparate individuals is apparently not uncommon at all.  In 1611, sun spots were discovered at least four different times, in 1869 both Cros and du Hauron invented color photography, and one that you might be more familiar with the invention of the phone by Bell, Gray, and la Cour to name a few of the individuals involved.  Napier and Briggs are credited with logarithms but Burgi also invented them a few years earlier.  Another popular one is the theory of natural selection being developed independently but simultaneously by Wallace and Darwin. There are so many of these simultaneous discoveries or inventions that William F. Ogburn and Dorothy Thomas published a paper “Are Inventions Inevitable? A Note on Social Evolution” in 1922 that documented 148 of these simultaneous discoveries.

No one is really sure why this happens. Some believe in a sort of efficient-market hypothesis, which in financial markets means that information is ubiquitous and therefore you cannot consistently beat the market because everyone knows the same information almost simultaneously. Ogburn and Thomas postulated in their paper that because there are very few completely new discoveries, most inventions are inevitable.  Inventions are built on top of other inventions such as the steam boat being dependent on boats and steam engines being invented prior.

While a curiosity, you’re probably wondering how this applies to hyper growth startups. The key takeaway is that while you’re coming up with a great idea so is everyone else. The ability to iterate quickly on ideas is more critical than ever. Combine this absolute need for quick iterations with the requirement for measuring results of effort, lest it be completely wasted and you have A/B testing on features that are launched in weekly sprints. SaaS companies have no excuse for not releasing in very short sprints (if not continuously), watching user behavior to learn what works and what doesn’t, then iterating again.

Despite the plethora of articles and books to the contrary, there are very few million dollar ideas, just million dollar executions of ideas. If investors are looking for key attributes about a team that make them more likely to succeed or not, I’d suggest looking for a team that can deliver quickly and knows the importance of measuring success.

97 Things Every Programmer Should Know is the third O’Reilly’s 97 Things series.  Editor and contributor Kevlin Henney has done a nice job bringing together some insights from some of today’s most experienced and respected practioners. The book is a compilation of short essays ranging on topics as diverse as Bugs, Error Handling, Customers, Refactoring, and Expertise.

Having tossed my hat into the ring of debate of whether software development is a craft or an engineering discipline, I was thrilled to see essays such as Neal Ford’s “Testing Is the Engineering Rigor of Software Development” that states “Compared to ‘hard’ engineering, the software development world is at about the same place the bridge builders were when the common strategy was to build a bridge and then roll something heavy over it.”  Along this vein I was intrigued by how many times the authors used terms such as ‘simple’ or ‘beautiful’. Peter Sommerlad suggest that we treat our code like “a poem, an essay, a public blog…” while Linda Rising uses the phrase “…a beautiful piece of code this is.” In fact there is an entire collection of essays around the theme of “Simplicity”.  This brings up my one and only annoyance with the book, the organization of the essay’s by title. There is an index of the contributions by category but the actual essays are laid out in alphabetical order. I would have much preferred to read all of the essays on a particular theme together without having to flip through to find them.

The purpose of the short essay is not to answer all your questions or be a definitive guide to programming. Rather the purpose is to provide a starting point for a conversation. To this end, I think a practical way to use this book whether in academia or a development team would be to assign groups of essays to be read ahead of time to stimulate classroom or team meeting discussions.

To highlight a couple of my other favorite essays, I particularly enjoyed Marcus Baker’s whimsical treatment of convincing someone to install your software in “Install Me” and being a script junkie myself, I found myself nodding in agreement with Cay Horstmann’s “Step Back and Automate, Automate, Automate.” As a conversation starter, thought provoker, or small collection of wisdom’s, you will likely enjoy many of these essays yourself.

Many authors have previously described the relationship between business and war and we believe that the most successful businesses approach their operations as would General Douglas MacArthur when he claimed that “In war, there is no substitute for victory”.

Carl von Clausewitz offered several tenets of war in his essay “Principles of War” and later expanded upon those in his book “On War”.  Many armed forces throughout the world have taken portions of these tenets and adopted them for their own use.  This post is the first in a two part series relating the 9 US Armed Forces Principles of War to your everyday business activities, strategy and tactics.  The 9 US Principles of War are Objective, Offensive, Mass, Economy of Force, Manuever, Unity of Command, Security, Surprise and Simplicity.  We will discuss the first 5 in this post and the next 4 in a subsequent post.

Objective.  The US Armed Forces definition is to direct every military operation toward a clearly defined, decisive and attainable objective.  We think this is pretty self explanatory and includes concepts about which we’ve previously blogged such as the need to set aggressive but achievable goals.  The most important aspects of “Objective” as applied to your business are for your goals to be clearly defined, well understood, measurable and attainable.

Offensive.  The military definition is to seize, retain and exploit the initiative.  The business definition here is found by looking at what Offensive implies – specifically that it’s all about time to market and getting the right features, products and services out and adopted first.  Being first offers the best chance at achieving virility within the market, and creating a viral marketplace or product is the military equivalent of seizing the high ground.

Mass.  The military definition is to mass the overwhelming effects of combat power at the decisive place and time.  Mass here in military terms is different from the concentration of forces which may not be desirable.  Combat power refers to all the aspects of military power from infantry and armor, to field artillery and other combat multipliers. The business equivalent is to ensure that your business units are aligned with your greater business objective and that they are contributing to it properly.  Your technology, product, marketing and finance teams should all realize and be contributing to the core objectives necessary to win your business battle.  If you wish to win quickly, they cannot be marching to separate agendas and they should not be fighting with each other.

Economy of Force.  This one can be confusing, but within the military definition is a reference to “No part of the force should be left without a purpose”.  The military definition also hints that every part of the force should be used in the most effective way possible.  Goals and objectives are again part of this, but more importantly you should be able to answer the question of whether you are using the right team for the job at hand.  Not only should you ensure that every organization has a purpose directly relating to your most important initiatives, you need to ensure that they are the best team to have those specific goals and objectives.  Client Services and Customer Support teams might be useful in helping to QA new products but allocating them 100% to such an endeavor is probably not the most leveraged use of their time.  Conversely, forgetting to include Customer Support or Client Services in any product rollout is a failure to employ a very important part of your “combat power” in achieving product success.  While its useful for engineers to understand customer needs and complaints, allowing more than 5 to 10% of their time to be taken up by such activities is a costly endeavor relative to your future product needs.

Maneuver. Place the enemy in a position of disadvantage through the flexible application of combat power.  This one relates to how flexible you are in your product delivery lifecycle, and whether you are set up to respond to your competitors actions in the marketplace.  This IS NOT an argument that you should abandon products in flight and constantly change your strategy.  Constant change in strategy is a clear indication of a management team incapable of defining a winning path and it’s a early indication of likely future failure.  You should be flexible, and changing features or making course corrections a few times a year is appropriate.  Ensuring that your product delivery processes allow you the flexibility to change (with the additional cost that implies) is critical to success.  But constant change is not a strategy – it’s a recipe for disaster.