I’ve said recently that your opinion doesn’t matter and now I’m going to add insult to injury. Not only does your opinion not matter but your ideas don’t matter either. If you think that you’re just one great idea away from becoming a tech-legend, think again. There are two reason for my statements. The first is simultaneous discovery and the second is non-disclosure agreements.

As I’ve written before about simultaneous discovery, it happens all the time. Perhaps the earliest example is farming. Sometime around the Neolithic Age (New Stone Age) about 10,000 years ago, humans separately invented farming at least three times and possibly as many as seven times. Different civilizations from Eastern Mediterranean to China to Mexico all came up with the idea of farming, presumably without sharing this knowledge in any way. More recent examples include, in 1611 the discovery of sun spots at least four different times, in 1869 both Cros and du Hauron invented color photography, and Bell, Gray, and la Cour just to name a few invented the phone at nearly the exact same time. The list of simultaneous discoveries or inventions is so extensive that William F. Ogburn and Dorothy Thomas documented over 148 of them in 1922. The likely cause of this is that inventions are built on top of other inventions which makes them “ripe” for discovery by anyone with that combination of information.

Regarding non-disclosure agreements and how they prove that your idea doesn’t matter. If you’ve ever pitched your idea to a VC you know that they don’t sign NDAs before they hear your idea. There are several reasons for this, many documented well in this post. I would sum it up into three major reasons. The first is that they are likely to hear the same (or similar) idea from several teams. Second, VC’s know that the idea doesn’t matter nearly as much as the execution of the idea. And, the third reason is that no idea stands the test of interaction with the customer, meaning if your company succeeds it is likely going to be with some offshoot idea and not the original one.

So, if your idea doesn’t matter and your opinion doesn’t matter…what does? Execution matters. That you can build a great team, provide them leadership, establish a culture of learning and deliver products to customers all matter. Don’t wait for the perfect idea. Pick an interesting problem that needs to be solved and get the solution into the hands of your customers quickly in order to learn.

We define risk as being comprised of three components: severity of impact, the probability of occurrence, and the ability to detect the event. The combination of these provide an overall risk assessment of a failure mode or a manner in which something can fail. We often apply this formula to code releases or for a more granular approach, we apply it to individual features within a release. By identifying the ways (failure modes) in which a feature could fail in production and giving that a score based on these three factors, we can quantify how much risk we have. We often teach this technique as a FMEA (failure mode effects analysis).

Risk = severity * probability * inability to detect

The most typical approach to risk mitigation has been an attempt to reduce the risk by reducing the probability. This is often done through rigorous requirements definition, thorough design reviews, and most often, lots of testing. The problem with this approach is that no matter how good we are, bugs will slip through. Users have different configurations than we have in our test environments or they use the product differently than we expect. A more recent approach to reducing the probability has been to deploy smaller changes. This was done by reducing development cycles or sprints to 1-2 weeks or in the extreme, employing continous deployment. The theory being the smaller the release, the fewer changes, and thus the less probability of failure.

A different approach to risk reduction and mitigation is by attempting to reduce the severity factor. There are several ways in which we can attempt to do this. The first is through the use of monitoring. By monitoring business metrics (checkouts, listings, signups, etc) we can quickly identify if there is a problem. Continuous deployment requires a rigorous approach to monitoring in order to quickly identify the problem and rollback the changes, thus reducing the severity or impact of the problem to your customers.

Another approach to reducing the severity is by pushing code changes to a small set of your users. Ideally this should be done through “swim lanes” but it can also be accomplished manually. In a process that we call “incremental rollout”, you would deploy new code to a small set of your servers (1-5%) and watch for issues. Once you’re satisfied with that there are no issues, roll to a larger set of your servers. Continue this “roll, pause, and observe” cycle until you have the release completely deployed. Teams that employ this strategy often take days to deploye code changes but by doing so they have much less risks of a customer impacting problem.

There are lots of ways to approach risk mitigation and we should continue to add these approaches to our toolkits. Some approaches work better than others based on the team culture and product or service being offered.

One of our most viewed blog entries is PDLC or SDLC, while we don’t know definitively why we suspect that technology leaders are looking for ways to improve their organization’s performance e.g. better quality, faster development. etc. Additionally, we often get asked the question “what is the best software development methodology?” or “should we change PDLC’s?” The simple answer is that there is no “best” and changing methodologies is not likely to fix organizational or process problems. What I’d like to do in this posts is 1) give you a very brief overview of the different methodologies – consider this a primer, a refresher, or feel free to skip and 2) provide a framework for considering which methodology is right for your organization.

The waterfall model is an often used software development processes that occurs in a sequential set of steps. Progress is seen as flowing steadily downwards (like a waterfall) through the phases such as Idea, Analysis, Design, Development, Testing, Implementation and Maintenance. The waterfall development model originated in the hardware manufacturing arena where after-the-fact changes are prohibitively costly. Since no formal software development methodologies existed at the time, this hardware-oriented model was adapted for software development. There are many variations on the waterfall methodology that changes the phases but all have the similar quality of a sequential set of steps. Some waterfall variations include those from Six Sigma (DMAIC, DMADV).

Agile software development is a group of software development methodologies based on iterative and incremental development. The requirements and ultimate products or services that get delivered evolve through collaboration between self-organizing, cross-functional teams. This methodology promotes adaptive planning, evolutionary development and delivery. The Agile Manifesto was introduced in 2001 and introduced a conceptual framework that promotes foreseen interactions throughout the development cycle. The time boxed iterative approach encourages rapid and flexible response to change. There are many variations of Agile including XP, Scrum, FDD, DSDM, and RUP.

With so many different methodologies available how do you decide which is right for your team? Here are a few questions that will help guide you through the decision.

1) Is the business willing to be involved in the entire product development cycle? This involvement takes the form of dedicated resources (no split roles such as running a P&L by day and being a product manager by night), everyone goes through training, and joint ownership of the product / service (no blaming technology for slow delivery or quality problems).
YES – Consider any of the agile methodologies.
NO – Bypass all forms of agile. All of these require commitment and involvement by the business in order to be successful.

2) What is the size of your development teams? Is the entire development team less than 10 people or have you divided the code into small components / services that teams of less than 10 people own and support?
YES – Consider XP or Scrum flavors of the agile methodology.
NO – Consider FDD and DSDM which are capable of scaling up to 100 developers. If you team is even larger consider RUP. Note that with agile, when the development team gets larger so does the amount of documentation and communication and this tends to make the project less agile.

3) Are your teams located in the same location?
YES – Consider any flavor of agile.
NO – While remote teams can and do follow agile methodologies it is much more difficult. If the business owners are not collocated with the developers I would highly recommend sticking with a waterfall methodology.

4) Are you hiring a lot of developers?
YES – Consider the more popular forms of agile or waterfall to minimize the ramp up time of new developers coming on board. If you really want an agile methodology, consider XP which includes paired programming as a concept, and is a good way to bring new developers up to speed quickly.
NO – Any methodology is fine.

A last important idea is that it isn’t important to follow a pure flavor of any methodology. Purist or zealots of process or technology are dangerous because a single tool in your tool box doesn’t provide the flexibility needed in the real world. Feel free to mix or alter concepts of any methodology to make it fit better in your organization or the service being provided.

There are of course counter examples to every one of these questions, in fact I can probably give the examples from our client list. These questions/answers are not definitive but they should provide a starting point or framework for how you can determine your team’s development methodology.

Below is our most recent newsletter. If you would like to subscribe and have it delivered to your inbox, you can do so here.

Trends
We’re privy to meeting and talking with hundreds of high tech, hyper growth companies and therefore we have a unique opportunity to see trends that are taking place in the industry. Here are a few that we thought would be interesting to share with our friends.

NoSQL Skills – Many companies are experimenting with NoSQL solutions including Membase, Hadoop, Cassandra, and many others but are finding that employees with skills in these are hard to come by. Even in areas rich in talent such as the Silicon Valley, Boston, Austin, Seattle and NY the demand for this rather nascent skill set is higher than the current supply. Many companies are relying on on-the-job training for these types of open source solutions. Most importantly, if you’ve properly fault isolated your architecture you can tolerate the risk associated with small segments going down during beta releases while you iron out the kinks.

Oracle’s NoSQL – Oracle recently announced their entry into NoSQL with announcements around Hadoop integration and their own NoSQL key-value solution. The Hadoop integration isn’t much more than a conduit to allow data to and from an Oracle database and a Hadoop cluster. This technology has been around for quite a while in other relational database solutions such as GreenPlum and Asterdata. It was also recently announced by Microsoft that SQL Server would support this Hadoop connection as well.

Oracle’s NoSQL is a distributed, replicated key-value store and is new and exciting even though it has attributes similar to other product offerings including their acquisition, Berkeley DB. Given trend #1 above, it may sound like an interesting alternative to adopting an enterprise supported NoSQL alternative to open source solutions but beware. The marketing materials for Oracle’s NoSQL solution include the BASE acronym (Basically Available, Soft State, Eventaully Consistent) but unlike other NoSQL products such as Dynamo, SimpleDB, or Cassandra, the Oracle NoSQL Database does not support eventual consistency. Oracle’s solution to eventual consistency appears to be by not accepting writes when the primary node for that key is down.

ORM – many companies start off using an object relational mapping solution such as Hibernate or Active Record but we are seeing many of them having difficulty scaling with them. The solution for several companies has been to use ORMs for simple queries but resort to ODBC or their own Data Access Layer for handling more complex queries. Be wary of using a solution to handle your query development as we’ve had a number of clients with incredibly complex and costly queries bring down their platforms for extended periods of time.

Enterprise Monitoring Frameworks – Until very recently, we had not seen a proprietary third party (non open-sourced) monitoring solution at a customer for at least 2 years and that includes our large Fortune 500 clients. Many of our clients have adopted innovative new monitoring solutions from Wilytech and Coradiant (now CA and BMC respectively) that look for and help diagnose patterns “on the wire” – whether it be from the browser to their servers or from app servers to databases. While these are interesting and potentially worth some of your attention, our very best clients design their systems to be monitored from the ground-up – ensuring that their software helps identify performance problems as they happen.

Distributed File Systems – Take your pick of implementations, but many of our clients are eschewing traditional NAS/NFS devices for distributed storage pools the likes of Gluster, MogileFS and Ceph. Nearly every case has resulted in significant savings relative to proprietary systems with few reported impacts to availability or response times. Of course, as with any other architectural change you need to ensure that you are properly managing your risk through pods or swim lanes.

No matter how many lines of code we’ve coded or architectures we’ve designed, we occasionally get push back on changes we suggest. Usually this comes in the form of a statement such as “oh, that’s just a simple matter of programming, huh?” For those uninitiated, a Simple Matter of Programming (SMOP) is a phrase used to ironically indicate that a suggested feature or design change would in fact require a great deal of effort implying that the person proposing the feature underestimates its cost. We occasionally joke with teams about how long a change will take but we admittedly don’t know the system well enough after a couple days to provide estimates. Even teams with great engineers get estimates wrong more often than not. However, what we do know from lots of personal experience and having worked with over two hundred companies, are orders of magnitude and what things should be considered difficult and what things should be easy.

It is common for a company to think that splitting their systems into swim lanes or partitioning by the Y or Z axes would be incredibly difficult until we walk them through the logic of how to make the changes. Certainly these logical steps are a far cry from actual code changes but most of the time companies “get it” and before the day is over start agreeing that the changes are reasonable.

An exception to this occurred recently where a non-technical manager made the obligatory SMOP comment and then continued to resist our suggestions on how the changes might be accomplished. The engineers in the room “got it” and you could see them nodding in agreement about how changes could be made but the business leader refused to acquiesce. In the end this organization will not make the necessary changes to scale and will likely limit their future value because of this.

To me this makes the point about mindset. I learned during college that I could convince myself that a subject matter was easy and I could learn it quickly (programming) or that it was impossible and I was no good at it (foreign language). The difference wasn’t the difficulty of the subject matter but rather my mindset regarding each topic. Your organization is the same way and if you’re the leader, your teams take their cues from you about how easy or hard something is going to be. This isn’t a recommendation that non-technical business leaders make estimates for feature development but rather that you maintain an open mindset about situations and challenges. Sometimes the greatest revelations come from the darkest hours and what seemed impossible was a key differentiator that made all the difference.

Resist the urge to SMOP someone’s idea and be open about changes. Approaching things as an individual and an organization with an open mindset will benefit you immensely.

You may not remember the 1988 movie The Dead Pool but you probably know the simile that Clint Eastwood made famous “Opinions are like a**holes. Everybody’s got one and everyone thinks everyone else’s stinks.” In this post we’re going to talk about opinions and what you should think of not only other people’s but your own as well.

According to Wikipedia, experts are widely recognized as “a reliable source of technique or skill” and have “extensive knowledge or ability based on research, experience, or occupation” in a domain. It might see odd to use Wikipedia, a crowd sourced knowledge repository, to define “expert” since there is a theory known as the “wisdom of crowds” made popular by James Surowiecki in his book of the same name, that postulates that decisions are better made by a group than a single person, even an expert.

A different perspective on experts comes from another popularization of research by journalist Malcolm Gladwell in his book Outliers. He states “In fact, researchers have settled on what they believe is the magic number for true expertise: ten thousand hours.” Yes, 10,000 hours of practicing something can make one an expert at which point presumably you become a reliable source of knowledge. Gladwell goes on to explain how work or practice influences one’s success.

Once a musician has enough ability to get into a top music school, the thing that distinguishes one performer from another is how hard he or she works. That’s it. And what’s more, the people at the very top don’t work just harder or even much harder than everyone else. They work much, much harder.

While I agree with Sun Tzu’s “the more you sweat in peace, the less you bleed in war” and Louis Pasteur’s “chance favors the prepared mind”, my own experience has proven that opinions can be and often are wrong. An example comes from the time I spent at an online advertising company. Several of the senior exec team were discussing the look and feel of advertisements on publishers’ pages. We were all of the “expert opinion” that ads formatted a certain way would lead to improved performance. Given my six sigma background, I decided to run a quick test to “prove” our intuition. The results were startling in that they were the exact opposite of what we were sure was going to be correct. This resulted in us taking advantage of this insight to improve our advertisements performance and it also taught most of us a great lesson, test your assumptions as well as the unknowns.

There are so many easy-to-use A/B testing frameworks that you can use, there really is no excuse not to do so. Despite some designer’s resistance to use data to drive the look and feel of sites, this is how the largest corporations turn basis point improvements into millions of additional dollars in revenue. If you want to be an “expert”, besides practicing for 10,000 hours, you should adopt Socrates’ mindset of “the more I learn, the more I learn how little I know.”

I know I’ll remember where I was and what I was doing when Steve Jobs passed away.  You simply can’t do what we do for a living – what we’ve done for a good portion of our lives – and not have had Steve touch a significant portion of your life.

It’s not just about what Mr. Jobs did at Apple, or about his insane product genius, or about his seemingly super-human ability to bring innovative products to market at just the right time (or almost the right time in the case of the Newton).    While those things alone would absolutely be enough to take a few minutes and remember him, his impact reached far beyond Apple or the direct users of his products.

Steve, though his genius, created new market segments and made existing and dominant players in existing segments even better.  While the Windows interface has been much maligned (in many cases rightfully so) for years, imagine where it would be without Apple and the Mac to chase in usability.  How much longer would we have been fumbling around with CD and Tape Walkmans without the iPod and how much worse would our workouts be because of it?  How many more years would we be lamenting the existing tablet computer industry had it not been for the iPad?  When would computer mice have gone mainstream if not for the Mac?  Would we still primarily be using dot matrix printers without Apple finding a way to introduce the laser printer to the mass market?

You went way too early Steve.  Thanks for everything that you’ve given us – both the things that you and your companies created as well as the standard of excellence you set for your competitors.

All modern humans were hunter-gatherers until around 10,000 years ago when we invented farming at least three separate times. Since that time most of the world’s societies have become agricultural, growing and farming for subsistence. We’ve also specialized allowing fewer people to produce food for everyone. Our behavior with information has some interesting parrallels to this evolutionary progression.

For a particular subject area we start off as information-gatherers, reading blogs, articles, and boards looking for information. Then if we have the ability, we become information-farmers, producing information on a particular topic by writing posts or contributing to threads. Finally, we start to specialize and people or blogs or boards become authoritative about topics and the rest of us allow them to feed us.

No doubt the Internet has democratized information but I think we’re starting to see a specialization in the production of content. Like our ancestors who sat around the campfire telling stories, some people created them and others retold them. Continuing the food analogy, a few people create the food but we all buy, cook, and serve it to our families and friends. Today, we are still participants in the process of information delivery but most of us are doing it by retelling the stories. We do this by sharing, “liking”, and +1 information that we enjoy.

Besides the parallels between our evolution from hunter-gatherers to agricultural or pastoral based societies the other interesting aspect of information is how this specialization requires us to build and maintain reputations. In the past reputations as experts were denoted by rank, class, or title that were bestowed upon people by a select few. In today’s world we get to decide on who is an expert and who isn’t.

Bringing this back to something practical, we should all be thinking about this idea of expertise for our careers and our organizations. The world gets to denote who is an expert and who isn’t. If you work hard and earn the reputation as an expert in an area you’ll likely be sought out and rewarded for it.

As authors, we are always interested in what our readers find interesting or useful.  Online we can see general article level interest through hit rates, etc.  In the print world, we can gauge interest based on book sales.  But neither of these approaches allows us to see what within an article might spark interest or be seen as useful by our readers.    Amazon’s Highlight utility within their Kindle format gives us that opportunity.  We thought we would share the most highlighted pieces of The Art of Scalability with you here.  You can see these quotes as well as new ones and changes in number of highlights over time here

 

You can delegate anything you would like, but you can never delegate the accountability for results.  Highlighted by 30 Kindle users

Management means measurement and a failure to measure is a failure to manage.  Highlighted by 28 Kindle users

As your resource pool dwindles, the tendency is to favor short-term customer facing features over longer-term scalability projects. That tendency helps meet quarterly goals at the expense of long-term platform viability.  Highlighted by 24 Kindle users

Bad behaviors are as good a reason for removing a person from the team as not having the requisite skills, because bad behavior in any team member creates a vicious cycle of plummeting morale and productivity.  Highlighted by 21 Kindle users

An organization that does not foster the creation, distribution, and acceptance of standards in coding, documentation, specifications, and deployment is sure to decrease efficiency, reduce quality, and increase the risk of significant production issues.   Highlighted by 19 Kindle users

These usually consist of teams responsible for the overall architecture of the product (architecture), the software engineering of the product (engineering), the monitoring and production handling of the product (operations), design and deployment of hardware for the product (infrastructure engineering), and the testing of the product (quality assurance).  Highlighted by 18 Kindle users

By having a single person or organization responsible, you are abiding by the “one back to pat and one throat to choke” rule. A gentler way of saying this is that distributed ownership is ownership by no one.  Highlighted by 17 Kindle users

Leadership is important to scale as it not only sets the direction (mission) and destination (vision) but it inspires people and organizations to achieve that destination.  Highlighted by 16 Kindle users

The tool we most often use is called RASCI. It is a responsibility assignment chart and the acronym stands for Responsible, Accountable, Supportive, Consulted, and Informed. Highlighted by 16 Kindle users

Although there are always exceptions even to this broad range of choices, our low boundary for team size is six and our upper boundary is 15.  Highlighted by 15 Kindle users

Motivated employees are great but inspired employees are even better. You might not have thought much about the difference between motivation and inspiration but it could be well worth a few minutes to do so. For the formal definitions we turn to the dictionary:
inspire – to exert an animating, enlivening, or exalting influence on
motivate – to provide with a motive (something that causes a person to act)

While not exceptionally helpful these definitions do offer some insight into the differences. Inspiration is described as an influence that is “animated, enlivening, or exalting” while motivation is described as just something that causes one to act. Lots of things can cause someone to act including fear, disgust, hatred, love, hunger, etc. Would you rather have an employee who is acting out of fear or because of an exalting influence? I know which one I would prefer.

Before we tackle the question of how to inspire someone, let’s discuss how people try to motivate employees. As a young manager, I was taught that people are motivated by one of five things: achievement, respect, responsibility, advancement, and growth. However this list isn’t at all complete. As we discussed above, people are motivated by lots of other negative things as well as the positive things in this list. More importantly, motivations are short lived. The excitement over a potential pay increase, a better title, or even fear of losing your job can only last so long before we become immune to them. Think of walking into a room and smelling a strong aroma but in a few minutes our olfactory sense becomes accustom to the smell and we stop being aware of it. The same goes for motivations. Inspirations on the other hand are much longer lasting.

When an employee is inspired, they are working for a cause, they believe in the mission, they have a higher calling. The word inspiration from its origin means “breathed upon.” In Greek mythology there are muses responsible for inspiration but there are no such equivalents for motivation. The way we inspire employees is to make them feel part of something that is bigger than themselves. Treat your employees like team members, tell them why the work they do is important, and share with them your passion. Work on inspiring your employees instead of just motivating them and you’ll have better results that last longer.