AKF Partners

Abbott, Keeven & Fisher PartnersPartners In Hyper Growth

Archives from author » dave

Achieving Maximum Availability in the Cloud

We often hear clients confidently tell us that Amazon’s SLA of 99.95% for EC2 instances provides them with plenty of availability. They believe that the combination of auto scaling compute instances with a persistent data store such as RDS provides all the scalability and availability that they will ever need. This unfortunately isn’t always the case. In this article we will focus on AWS services but this applies to any IaaS or PaaS provider.

The SLA of 99.95% is not the guaranteed availability of your services. It’s the availability of the EC2 for an entire region in AWS. From the AWS EC2 SLA agreement,

“Monthly Uptime Percentage” is calculated by subtracting from 100% the percentage of minutes during the month in which Amazon EC2 or Amazon EBS, as applicable, was in the state of “Region Unavailable.”

Even this guaranteed limited downtime by Amazon of 21.56 minutes per month is not always achieved. Over the course of the last few years there have been several outages that lasted much longer. Additionally, your services availability can be impacted from other third parties services used in your product and an even more likely from simple human error by your engineering team.

To combat and reduce the likely of a customer-impacting event or performance degradation, we recommend various deployment patterns that will help.

  • Calls made across Regions should be done so asynchronously. This reduces latency and the likelihood of a failure.
  • A given Service should be completely deployed within multiple Availability Zone.
  • Use abstraction layers with AWS Managed Services so that your product is not tied to such services. Today these managed services are at different stages of maturity and an abstraction layer will allow for a migration to more robust solutions later.
  • Services, or microservices depending on your architecture, should have a dedicated data store. Allowing app server pools to communicate to multiple data stores reduces availability.
  • To protect against region failures deploy all services needed for a product into a second region and run active / active designating a home for each user.

We have come across clients who have invested in migrating from one cloud provider to another because of what seemed like infrastructure outages in a particular region. Before making such an investment of time and money make sure you are not the root cause of these outages because the way your product is architected. If you are concerned about your architecture or deploying your product in the cloud, contact us and we can provide you with some options.


Comments Off on Achieving Maximum Availability in the Cloud

5-95 Rule

Previously, I wrote about mitigating risk in the face of uncertainty. I suggested that an agile development model was one way successful companies have been able to mitigate risk. In that post I compared the similarities between organizing into “Scrums” to how Army Special Forces organized in order to succeed in uncertain operating conditions. In addition to the way a company is organized, the way a company approaches project planning can mitigate risk. One of the most often overlooked aspects of project management is contingency planning.

Army Green Berets are fond of the saying, “No plan survives contact with the enemy,” a quote attributed to the famous Prussian General Helmut Von Moltke. Von Moltke and his Prussian contemporaries were brilliant strategists and sought to perfect the “Theory of War.” Their ideas still serve as basis for our doctrine today. Moltke understood that battles can be broken down to a near infinite set of complex options, each of them in turn having still more options depending upon the enemy’s response. A plan only goes so far, at which point the enemy ultimately casts his vote. Von Moltke provided his subordinate commanders with his intentions (a method still taught today in military basic leadership courses), and held them responsible for extensively preparing for all plausible contingencies.
von_moltke

As a Special Forces Detachment Commander, once my team received our orders, we immediately began to plan. Along with the mission, we were given a specific deadline by which we needed to brief our commander. Regardless of the deadline, whether it was 6 hours or a week, we spent approximately 5% of the time coming up with a solid, practical, and safe plan. We spent the other 95% of the time “war-gaming” the plan. During the “war-gaming” portion, we would go through every step of the plan from the time we left our barracks to the time we got back. We thought of every possible thing that could go wrong. What if only two helicopters showed up and not four? What if the helicopter couldn’t land where we wanted it? What if someone got injured? What if more bad guys were on the objective than we had originally thought? We would break our mission down into distinct pieces, or milestones, and would work in teams to come up with solutions to every possible contingency. We would collaborate with our partners, the pilots for example, and let the subject matter experts take the lead in their scope of responsibility. Time permitting, we built mock-ups of the buildings we anticipated operating in, and conducted detailed rehearsals. The rehearsals revealed other contingencies we hadn’t planned for, such as equipment that was being carried by the wrong guy or individuals who needed to work together that weren’t located near each other. We did this because we knew the battlefield was going to be fluid and dynamic. Being prepared not only enabled us to be successful, it saved lives.

While the tech industry doesn’t deal in life or death, although it certainly might feel like that at times, Von Moltke’s wisdom still applies, especially in complex project development. What if equipment doesn’t arrive on time during a data center build? A critical engineer or developer gets sick during a launch or gets hired away? AKF Partners is a company heavily influenced by veterans and our collective experience on and off the battlefield. We encourage our clients to practice the AKF “5-95 Rule” in their Agile methodology. Having a solid plan is a great start but understanding the possible variations in the project’s execution will help ensure the projected is delivered on time and within the budget. Remember the AKF “5-95 Rule” and spend 5 percent of your time planning and 95 percent of your time developing contingencies.


1 comment

A-Teams

In his book, The Lean Start-Up, Eric Ries defines a start-up as a “human institution designed to deliver a new product or service under conditions of extreme uncertainty.” Operating in uncertainty is inherently risky. What makes many start-ups successful, among other things, is their ability to mitigate risk. One way a company can remain adaptive and responsive during uncertainty, and therefore reduce risk, is employing an agile development model. A good example of such an organization is Spotify, which continues to embrace, practice, and improve on agile fundamentals even after growing to over 30 development teams in three different locations.

There are striking similarities between the way Spotify organizes their teams and the way U.S. Army Special Forces organizes their Operational Detachments-Alpha, aka “A-Teams.” It makes sense that the two would be similar because the A-Team was in fact an innovation designed to deliver a “service” under conditions of extreme uncertainty. In this case, the “service” was military assistance to the struggling Republic of Vietnam, and later to guerrilla forces and resistance movements waging unconventional warfare against the North Vietnamese. The U.S. Army had never undertaken a mission similar in objective and scope. They did not know the conditions, the support, or the communication capabilities that their soldiers would find. The U.S. Army needed a self-sustaining team with all the skills necessary to complete a mission with a broad and evolving scope. Moreover, the team would have to operate autonomously and would have to be trusted to make decisions normally left to upper echelons.

HALO

The U.S. Army assessed and selected twelve of their most seasoned veterans, and then trained them each in one of six specialties: medicine, engineering, communications, weapons and tactics, or intelligence. Having redundancy in each position was desired in case one soldier got injured, and it also gave the team the ability to pair a new member with a more senior member of the same specialty. Redundancy also facilitated the ascension of the team’s most senior soldier to the role of Operations Sergeant. The A-Team Operations Sergeant is comparable to the role of Spotify’s Agile Coach, or sometimes called a “Scrum Master” in other organizations. He no longer fills a specialty function on the A-team. He is the team’s mentor and draws on his experience to coach the team. He wants his team to be efficient and safe, which equates to a quality product in their realm. It is the also the Operations Sergeant’s job to advise and informally train the A-team’s commander on the team’s competences and potential with respect to their assigned mission. He ensures that the commander does not over commit the A-team to the stakeholders.

The A-Team was designed to operate deep behind enemy lines. They would not maintain frequent contact to their command. Because the complexity of the mission at hand, the A-Team worked not only for their command, but on the on behalf of many other stakeholders, such as the U.S. State Department, CIA, and other government agencies. In the absence of guidance from their command, and other key stakeholders, it is crucial that the commander, with input from the team, manage the team’s priorities with respect to the mission’s goals. Just like Spotify’s Product Owner, it is the A-team’s commander who ensures the team is on track to accomplish the stakeholders’ desired end state by prioritizing projects and associated tasks.

The way that Spotify tweaked the idea of a Matrix Org to facilitate product-centered teams while managing professional development and adherence to standards across functional areas resembles the method used by A-teams. Spotify organizes their development teams or “Squads,” into “Tribes” of 100 contributors of varying disciplines such as software engineering, hardware engineering, DBAs, etc. Each Tribe has a “Chapter” in which contributors with the same competencies are grouped. Chapters are led by “Chapter Leads” who are responsible for the quality of work produced by his or her chapter. Special Forces A-Teams are organized into “Companies” of six multi-disciplinary teams (around 70 people). A-Team “Companies” have a headquarters unit that is responsible for the professional development and standards of each specialty on the team. As an example, the medics on each team practice medicine under the license of a medical doctor assigned to the headquarters unit. The doctor’s function is similar to that of a Chapter Lead because he is responsible for each medic’s continued professional growth, refresher training, and the overall standard of care that is provided by his chapter.

The A-Team model’s structure has not significantly changed since their inception. During the Vietnam War, and most recently in Iraq and Afghanistan, Special Forces A-Teams have produced tangible results where larger, conventional military units have not. Their self-sufficient and decentralized approach allows the team to focus on mission accomplishment. The conventional Army’s centralized and hierarchical structure doesn’t allow for the flexibility needed to mitigate risk in uncertain environments. Results often become secondary to strict adherence to protocol for protocol’s sake, and the approval of long reporting chains. Similarly, in the technology industry, companies like Spotify who adhere to agile fundamentals are able to scale and grow successfully amid the uncertainty, while larger, less “agile” organizations become bogged down in bureaucracy, and can not navigate the evolving and uncertain market.


Comments Off on A-Teams

Seizing the Initiative

As a young Army Lieutenant leading a Platoon in Iraq in 2006, I was faced with a crisis. Our platoon was continually getting ambushed during our patrols in sector. The uncertainty of waiting to be attacked, the physical, mental and emotional strain on the men, not to mention the damage to our vehicles and equipment, started to take its toll. The ensuing battles would generally last for hours. We would stay out in sector for upwards of 16 hours. This would only leave 8 hours to come back to the forward operating base, refit and conduct maintenance on our vehicles. This type of operation couldn’t be sustained. Our morale was starting to fall, tensions were starting to rise between my platoon and the maintenance crews servicing our vehicles. We’d bring in leaking, limping, bullet ridden trucks and expect to jump back in them and roll out in a matter of hours. In military terms, we had “lost the initiative.” That is to say, the enemy was dictating where and when and how the battle would go. We were only reacting. We were compelled to react to the enemy’s strengths and capabilities. We were treading water trying to stay afloat. In this crisis, my men depended on me to come up with a plan to take back control, or “seize the initiative.” The importance of having a plan is emphasized in Sun Tzu’s famous quote, “ Victorious warriors win first and then go to war, while defeated warriors go to war first and then seek to win.” My platoon and I were at war, and trying to win.

sun_tzu_general

The crisis I experienced in Iraq is similar to the challenges many executives face with regard to scaling. Granted, no one is shooting at them, but the enemy can represent any or all of the variables and uncertainty in the industry; the customer, the competition, the software, the hardware, the market, etc. Some executives find they are compelled to react to daily incidents, i.e. “lets just make it another 24 hrs without an outage,” and it’s a struggle to maintain, let alone scale effectively. Just like with my platoon, this type of operation takes a toll on an organization. Aside from being bad for business, it sows the seeds of despair, brings down morale, and possibly creates tension between the engineers and the business folks similar to that between my troops and our maintenance crews. The leader needs to take a step back, understand what is going on and figure out a plan to get his or her team out of it.

For me, the solution was counter-intuitive at first. I knew we needed to take time to regroup, but I hesitated to not keep pressing the enemy. I feared that we’d lose what little ground we had. Then, I remembered something my instructor told me while learning to navigate in the woods with a map and a compass. He warned us, “If you don’t know where you are going, don’t walk as fast. You’ll only get lost in a hurry.” In our eagerness to get out into sector and fight the enemy we were getting lost in a hurry. We had gone to war, and were now trying to win. It is a difficult decision to halt operations, or take a product offline. Aggressive leaders are often very successful because they persevere in the face of adversity. However, there is a fine line between perseverance and stubbornness. I made the call to postpone operations. I needed time to develop an effective sustainable strategy.

“Seizing the initiative” requires leaders to make sound decisions under uncertain conditions and to anticipate events so their organization can recognize and leverage opportunities. The key to anticipating events is to understand the problem. In my case, we really did not understand the enemy. From an institutional standpoint, we had been training to fight the soviets for the last 40 years, and here we were faced against an adaptive and evolving enemy. We scaled our operations back at first, focused on reconnaissance and understanding the enemy. In doing so we were better able to anticipate events, and began setting the conditions of the fight. We were engaging the enemy when and where we wanted. We had seized the initiative. Regardless of your type of organization, if you are struggling just to maintain, if you have lost the initiative, the best solution is to take the necessary pause in operations, understand the problem, and develop a strategy for a sustainable effective approach.


Comments Off on Seizing the Initiative

A False Sense Of Security and Complacency = Revenue Loss

Its Monday morning and past Saturday evening issues in one of your datacenters triggered a failover to your second data center for service restoration. In other words, all customer traffic has been routed to a single datacenter. The failover was executed flawlessly and the team went back to bed waiting for Monday morning to permanently fix the issue so traffic could once again run out of both datacenters. On Monday morning, we are expecting a flash sale and will make close to $8000 a minute at peak. All is well and there is nothing to worry about. Right?

Hopefully you cringed at the above scenario. What if the data center you are running out of suffers from a failure? Or what if the only data center and its components that is now live for all of your traffic simply wasn’t sized correctly for acceptable performance during a traffic spike?

If it hasn’t happened yet, it will. If that were the case, your business would stand to lose significant revenue. We see it over and over again with many clients and have also experienced it in practice. Multiple datacenters can serve as a false sense of security and teams can become complacent. Remember, assume everything will fail as a monolith. If you are only running out of a single data center and the other is unable to take traffic, you now have a SPOF and as a whole the DC is a monolith. As a tech ops leader you have to drive the right sense of urgency and lead your team to have the right mindset. Restoring service with a failover is perfectly acceptable. However, the team cannot stop there. They must quickly diagnose the problem and return the site to normal service, which means you are once again running out of two datacenters. Don’t let the false sense of security slip into your ops teams. If you spot it, call it out and explain why.

To help combat complacency from setting in, we recommend considering the following:

  1. Run a Morning Ops meeting with your business and review issues from the past 24 hours. Determine which issues need to undergo a postmortem. See one of our earlier blogs for more information: http://akfpartners.com/techblog/2010/08/29/morning-operations-meeting/
  2. Communicate to your team and your business on the failure and what is being done about it.
  3. Run a postmortem determine multiple causes and actions and owners to address the causes: http://akfpartners.com/techblog/2009/09/03/a-lightweight-post-mortem-process/
  4. Always restore your systems to normal service as quickly as possible. If you have split your architecture along the Y or Z axis and one of the swim lanes fails or an entire datacenter fails, you need to bring it back up as quickly as possible. See one of our past blogs for more details on splitting your architecture: http://akfpartners.com/techblog/2008/05/30/fault-isolative-architectures-or-“swimlaning”/

 


Comments Off on A False Sense Of Security and Complacency = Revenue Loss

Common Cloud Misconceptions

Over the course of the last year, we have seen several of our clients either start exploring or make plans to move their SaaS products to the “Cloud” or an IaaS provider. We thought we would share some of the misconceptions we sometimes see and our advice.

– I can finally focus on product development and software engineering and not worry about this infrastructure stuff.
The notion that IaaS providers like Amazon have eliminated your worries about infrastructure is only partially true. You may not need to understand everything about designing an infrastructure with bare metal but you need to make sure you understand how your virtual configuration in the Cloud will affect your product. IaaS helps us to quickly deploy infrastructure for our products but it doesn’t eliminate the need for good high availability and fault tolerance design. There are several levers you can pull within an IaaS console and design decisions that will impact your products performance. To ensure good design and configuration, its ultra important that your SaaS product engineering team is made up of talent that has expertise in distributed application architecture design, infrastructure, security, and networking. Having this knowledge will help you design a high performing, fault isolated product for your business.

– Going to the cloud pretty much guarantees me high availability because of auto scaling.
Going to the cloud will provide you with the ability to scale quickly as load increases but it will not provide you with high availability. For example, if you have a monolithic code base that you deployed and you are pushing to production on a regular basis, there is a pretty good chance you will introduce a defect at some point that impacts the availability of your entire service and business. We advise our clients to split their applications appropriately, deploy the services to separate instances, and, assuming you are using Amazon, configure them to run across multiple zones within a region at a minimum (preferably across regions). This allows you to focus dedicated teams to the individual services and reduce the likelihood of introducing a defect that takes down your system.

– The Cloud will be cheaper than a collocation or managed hosting provider.
There are several factors that need to be considered before you can confirm that is cost effective. You should look closely at load on your servers. If your servers are not serving traffic around the clock, it may be better from a cost perspective for you to buy and maintain your own infrastructure in a collocation or in an existing data center you may have. The economics of this decision is changing rapidly as IaaS pricing is declining due to the competition in the industry. A simple spreadsheet exercise will help determine if the move to Cloud would be cost effective for your business.

– The Cloud isn’t secure so we better not use it.
The cloud isn’t necessarily what makes or breaks security around your SaaS product. Many believe that public cloud services like Amazon’s EC2 service isn’t secure. First off, you are far more likely to experience a security breach because of an employee’s actions (either intentionally or unintentionally) than caused by an infrastructure provider. Secondly, Amazon likely has invested much more in security at various layers, including their physical data centers, than most companies we see who have their own data centers. Amazon has designed the infrastructure to isolate customer instances and you can also choose to take advantage of Amazon Virtual Private Cloud that can be configured to create an isolated network. There are various options for encrypting all of your data as well. This only touches the surface for security design options you have and they continue to be enhanced everyday. You can see why it’s important to staff your team with an engineer who has experience in this space.

If you are looking to move to Cloud, don’t rush into the decision. Do your homework and make sure it’s right for your business. Make sure you have the talent that has experience with the technology that will get you there and run your operations. Once you make the leap, you will have to live with it for a while.


Comments Off on Common Cloud Misconceptions

Driving Change

Driving big changes can be difficult in an organization and it can be especially difficult for new managers. Change is necessary and we should never be happy with status quo as our competitors and the overall market is changing around us. The larger the organization, the harder change can be. We touched on this in an earlier blog titled Five Leadership Lessons From Former Bosses but decided to dive into a little more detail this time. As a former manager of engineers and other talented technologists, I recall in my early days how I simply wanted to identify where we were headed with my team without giving them time to collaborate and provide input. On one occasion, we were in the midst of changing our PDLC to Agile. Agile was fairly new at the time so many didn’t understand the basis of it and in turn were resistant to using it. That problem is easily solved, right? I could just tell the team this is where we are going and this is how we are going to do it, now make sure you execute. And in a couple weeks time, poof, the change would be made and life would be good. After all, that would have been easy, right?

Although I didn’t say the words “get onboard,” I might as well have. I can tell you this doesn’t work well. My team was zapped. At that point they were no longer empowered and didn’t have any ownership of the change and when that happens a team’s performance will be suboptimal. Yeah, I could use the excuse that my senior management dictated what they wanted and I could drum up a bunch of other excuses but the fact of the matter is I should have raised a flag with my leadership and encouraged better collaboration for input from the engineers, QA testers, product managers, and others.

There were two key learning points that I was able to take away from my experience. Technology leaders should ask themselves two simple questions before they start to drive change in their organizations.

1) How do I make sure that my team has input and ownership of the change we are about to make?

Sit down with the team and work with them on coming up with solutions and their recommended timing. Engineers love to problem solve and they are very good at it. Provide the team with the problem that needs solved and any constraints and let them determine the solution. For example, in the situation I described above, we needed to improve our time to market. Many products and features across the organization were being delivered late, payloads for deployments were too big, and the business was frustrated. As on overall company we decided that we would adopt Agile utilizing Scrum practices but there were still many problems that needed to be solved. Those included determining a branching strategy, determining a deployment strategy, identifying how to measure a sprints performance, and many others. By providing the context of why changes need to be made, the team will recognize the significance of their input and by allowing them to come up with solutions they will feel empowered and have ownership in the game. All of this will help you to create a higher performing engineering team.

2) What do I need to do as a leader to establish guardrails in my organization and sustain the change as my company grows?

Once the team has determined how to solve some of the problems, you will want to make sure that you sustain them in your organization. To do this, work with them to establish documented guardrails or principles that be used as the what. For example, the Agile Manifesto essentially identifies principles that you can use. One principle could be “Business people and developers must work together daily throughout the project.” This describes a behavior that’s needed but doesn’t identify how. The team can decide the specifics on how they will achieve the principle. We at AKF believe in establishing architectural and scaling principles that are essentially guardrails as well. (Visit our earlier blog on architectural principles for more details.) For example, Rule Number 39 – Ensure You Can Wire on and Off Functions in our book Scalability Rules identifies the need to have a framework where you can quickly disable and enable features to minimize impact to customers when a failure occurs. You can solve this many ways and your engineers can determine what the solution should be. Once your principles are established you should monitor them to make sure they are being upheld and reward results.

If you ask yourself these two simple questions before diving into a big change your road to success will be much easier. Now that you have empowered your team to determine how to solve the problem and you have documented your principles and guardrails, the team should be able to operate within them and make decisions on their own. In turn, you will find that they feel empowered and you will have helped to create and maintain a higher performing team.


1 comment

Signs That You May Be Disconnected From Your Business

We as engineers love to problem solve. In fact, we love to explore new technologies and debate with our colleagues as to what may be the best to use. Should we use Python? Should we use PHP? Should we use Java? Should we try Google’s App Engine and code in GO? Should we use Amazon Web Services exclusively for our product? What database technology should we use? All of these decisions are important and factors such as skillsets, security, performance, and cost should be considered. We love to code and see the product in action as it provides us with a sense of accomplishment. Once we are done with a project and its deployed in production we typically celebrate all of the hard work that went into the solution and we move on to the next project. Many times after diving deep into the technical aspect of the solution, for weeks and maybe even months, we wake up and we discover we are not in touch with the business like we should be. All of us have seen this in practice and many of our clients face this challenge.

What are some of the signs that you may not be aligned closely enough with the business and its performance and what should you do about it?

1) Not understanding feature impact – New features are introduced into your product without an understanding of the business impact.

You should never introduce new features without understanding the impact it is supposed to have on your business. In other words, establish a business goal for new features. For example, a new feature that allows for one click purchase is expected to improve conversion rates by 15% within 2 months. Remember all goals should be SMART goals (per chapter 1 in The Art of Scalability – Specific, Measurable, Attainable, Realistic, and Time constrained).

2) Celebrating launch and not success – Upon deployment of your product and confirmation that everything is working as expected, fireworks go off, confetti falls from the ceiling, and the gourmet lunch is ordered.

While recognizing your team for their efforts to launch a feature can be important, you really should celebrate when you have reached the business goal that is associated with that feature (see item #1 above). This might be achieving a revenue target, increasing the number of active accounts, or reaching a conversion target. If you deploy a feature or a product, and your business targets are not met as expected, you have more work to do. This should not be a surprise. Agile development’s basic premise is that we do not know what the final state of the feature and thus we must iterate.

3) No business metric monitoring – Your DevOps team is alerted that something is wrong with one of your services but you rely on your customer support or service desk to tell you if customers are impacted.

This is something that many of our clients struggle with. We believe its critical to detect a problem before you customers have to tell you or you have to ask them. You should be able determine if your business is being impacted without asking your customers.

By monitoring real time business metrics and comparing the actual data to a historical curve you can more quickly detect if there is a problem and avoid sifting trough alerting and monitoring white noise that your systems will inevitability produce. For more details on our preferred strategy, visit one of our earlier blogs. You can also read more about the Design To Be Monitored principle in our book Scalability Rules.

As you company grows, make sure that your product, your engineering team, and your technical operations are closely aligned with your business. We firmly believe that we as technologists must stay aligned with our business for success. In fact, we really are the business. Our solutions enable the company to earn revenue.


1 comment

Dealing with Shared Services

In our latest Newsletter, we wrote about the importance of aligning your agile teams to the architecture of the system and the trend we are seeing as our clients move towards PODS. We believe and teach that designing your architecture is only the first step in building an organization that can scale in support of your product. Remember, agile autonomous teams are able to act more efficiently which results in a speedier TTM. Ideally, they should almost be able to behave like mini-startups.

Aligning teams to swim lanes is pretty straightforward but what do you do with a team that’s central to multiple services?

We often see clients who have this challenge. They have a shared service or feature that the other clearly split autonomous teams depend on. For example, there are several sites that bring consumers together with businesses in their local area. These sites often have categories or verticals that need search functionality. Asking each team to design its own search functionality would be wasteful as you would end up with engineers designing redundant functionality which would exponentially cost more to operate. It is absolutely feasible to create a team that would focus on search and be used by other teams in this situation. We do recommend that you minimize the existence of these types of teams when possible, as there is always risk that they could slow down TTM.

Great! All of the other teams are going to bombard the shared service team with new development requests. So what do you do to mitigate the risk of over allocating your engineers with such a team?

This risk is real as the other teams will make requests for enhancements or functionality to support their services and they will want it quickly. To mitigate this risk we suggest thinking of the team almost like you would an open source project. That doesn’t mean you simply open up the search code base to all of your engineers and let them have at it. Rather, it means you put mechanisms in place to help control the quality and design for your business. An open source project often has its own repo and typically only allows trusted developers to commit. In our search example, you could designate a couple trusted and experienced engineers in the other PODS that can code and commit to the search repo. Engineers on the search team can be focused on making sure new functionality aligns with architectural and design principles that your company has established. This approach should help to mitigate the potential bottleneck such a team could create.

OK, now that you have spread out the development of search, who really owns it?

Remember, ownership by many is ownership by none. In our example, the search team ultimately owns the search feature and code base. As other developers commit new code to the repo, the search team should conduct code, design, and architectural reviews. Just as the other PODS will deploy new features to production, the search team will also own deployment of search. Overall, all of your teams should have objectives that align with a few key business success metrics.

Remember, whatever mechanisms you put in place, your shared service or tools team should be a gas pedal and not a break for TTM. Good luck scaling your architecture and your organization. We would love to hear about some of the experiences from those of you that have tried this or other approaches.


6 comments

Preparing For a Financing Round or an Acquisition

We spend a good portion of our time performing due diligence on companies that are either looking for financing or preparing to be acquired. We thought it would be interesting if we shared some of our observations on what makes the process go smoothly and what causes problems. In our experience and through working with our clients we believe it’s important that you consider the following:

1) Secret Sauce – First off, you need to figure out your secret sauce. Simply stating that you have the world’s largest online bicycle sharing service is only a start. While a service like bicycle sharing might be a great concept, there should also be an algorithm or business method that isn’t necessarily visible to you and I from the outside. For example, the company may have a pricing algorithm that factors in various demand variables for bicycle rentals on a given day and adjust prices accordingly. Look for something similar with your venture. Figuring out your secret sauce will help to raise your valuation.

2) Open Source – We’re a huge fan of open source tools but you need to be diligent about what you are using. Specifically, keep track of the open source products that you use and be aware of the license under which you are using it. There are many different licenses in use today. Below are a few of the most popular and our, non-attorney, interpretations of them. Note, that there is disagreement among much more erudite individuals than us on the terms of each of these so be sure to check with your legal representative before you use any of them.

  • GPL – GNU Public License – You are allowed to use, redistribute and change the software, but any changes you make must also be licensed under the GPL. So that means you have to give everyone else the same rights as you got.
  • LGPL – Lesser General Public License – If you modify the software, you still have to give back the source code, but you are allowed to link to it (compile with it) without giving the source code to everything you built back.
  • BSD license – Berkley Software Distribution – You can take the code and turn it into a proprietary application and you don’t have to give it back.
  • Apache license – Allows the use, modification, and distribution of the software for any purpose but does not require modified versions of the software to be distributed using the same license.

3) Patents – One method of demonstrating value and uniqueness is to build up a patent portfolio. To be valid, your patent must be new, useful, non-obvious, and of a subject matter defined as patentable. Machines, processes, methods, and compositions of matter are examples of patentable ideas. Algorithms and laws of nature (e.g. E=mc2) are examples of subject matter than cannot be patented.

Remember your secret sauce? You should not file a patent on this because as soon as you do it becomes publicly disclosed. Instead this is a trade secret which is information that derives independent economic value from not being generally known. A trade secret can be an algorithm that you have developed and resides in your codebase. In our bicycle example, the renting pricing algorithm is a trade secret.

4) Patent Trolls – As soon as you announce an acquisition or any other news that indicates your company is highly valued, someone is likely to file a lawsuit for patent infringement. Facebook was sued five times in the first two months alone after announcing they were going public. Microsoft recently introduced the “Live Tiles” feature as part of Windows 8. They are being sued by a small operating system company for allegedly stealing the patent on “tiles.” Such a lawsuit will tie up resources in your organization and run up legal fees. Make sure you prepare for this inevitable cost of doing business.

Obviously, none of this guarantees success with fund raising or getting acquired but these are items that we often see stop a process or delay it. Make sure you discuss all of this with your advisers, directors, and attorneys.


1 comment