Archive for the ‘Engineering’ Category

« Older Entries

How To Say “No”

Monday, July 26th, 2010

It’s not uncommon to hear from our clients that they have multiple versions of their application running, many of them dedicated to a single “special customer”. While such implementations destroy the economics of a SaaS company, sometimes we must bend a few rules and accept certain risks early in our company’s lives to stay afloat. A startup that is desperate for cash will do just about anything to attract and retain their bigger customers, especially when this customer interaction is lead by sales staff unfamiliar with the hidden costs of customizations.

As technologist, maintaining multiple versions of our SaaS software makes us cringe. We all know that this is a recipe for a maintenance nightmare. Multiple versions require – bug fixes, patches, and features to be tested and deployed on different versions as well as dedicated hardware or virtual instances to run the different versions. The cost of this can quickly double a technology organization’s budget when additional QA, developers, and operations staff are accounted for as well as the environments. In fact, when running a SaaS platform with dedicated instances (remember the original ASP days?) one can quickly create the worst of both the hosted and packaged software worlds and sink a company.  So how do you know when to say “No” and how do you do it?

The way to make decisions, whether build vs buy or feature prioritization or customizations, is to gather the appropriate data to establish a well reasoned argument without exhibiting “analysis paralysis”. For customizations this general includes the upside in terms of future revenue, the downside in terms of development, environment, and maintenance costs, and the risk. No matter what the sales reps are saying there is no guarantee that with the customization you will win the business or without it you won’t. This is the reason for adding a risk variable (beta) that attempts to quantify the likelihood of success or failure.

If the decision is to say yes, then spend some time thinking about what level of customization will occur and how you can possibly reuse the customization. According to authors Pine and Gilmore in “The Experience Economy” there are four types of customizations, transparent, collaborative, adaptive, and cosmetic. Transparent is when the customer isn’t aware that their version is different from others, such as internationalization. Collaborative is when the product or service works with the customer to achieve a custom solution. Adaptive is allowing the customers to change the product or service themselves, such as choosing personal feeds. Cosmetic is re-skinning the product without modifying the functionality.

If you’ve decided to say no to the customer’s request for customization don’t just say “no”. Think about how you can use a carrot and/or stick approach to guide the customer to where you would like them. Sticks are things like additional costs. If a customer wants to stay on a previous version and you’ve decided that’s not acceptable to you consider charging them progressively higher rates until they either leave or make the upgrade. If they are going to leave any way then the temporary additional revenue will help offset the loss. A carrot approach might be refusing to add new features to a custom solution. Eventually the customer, hopefully, will want the newest features and acquiesce on the customization. Instead of just saying “no” emply carrots and sticks in an attempt to move your customer to where you both will be happy.

Tags: ,
Posted in Engineering | 1 Comment »

Moving from Packaged Software to SaaS

Monday, July 19th, 2010

It’s probably no surprise to our readers that many old packaged software companies are attempting to take their software and hence their business models “online”.  And why not?  The model is attractive and benefits accrue to both the providers of service through software and those who outsource portions of what was once bothersome internally hosted software.  The providers benefit from economies of scale in hosting that generate attractive profits for the provider and savings for the customer, lower maintenance costs resulting from custom customer deployments, predictable revenue streams fostered through closer customer contact, more frequent and smaller releases that reduce risk and faster implementation times that result in faster profit recognition.  Customers benefit from outsourcing non-core IT functions, providers who specialize in delivering specific services, lower capital expenditures and faster deployment times.  SaaS is both a desert topping and a floor wax!  It’s the cure for cancer and the answer to the riddle of life!

But what many of these companies don’t realize is that the way one architects a product and runs a company focused on service delivery is simply different than the approach of a company focused on delivering software.  Customers expect that you are going to give them higher availability and fewer headaches.  Software alone simply won’t meet this goal; it is imperative that one design SaaS systems holistically which in turn requires skills in both infrastructure and software architecture (or “systems” architecture).   The cost leverage necessary to both increase profit margins and decrease customer cost typically requires multi-tenancy which has its own share of headaches.  Fault isolation and rollback capabilities are a must to minimize customer impact and mitigate rapid deployment risks.

It is not enough to simply bundle up an application in a hosted fashion and label yourself a “SaaS” company.  If you don’t work aggressively to increase availability and decrease your cost of operations, someone with greater experience will come along and simply put you out of business.  After all, your business is now about SERVICE – not SOFTWARE.  This is a fundamental mind-shift that some companies simply can’t overcome or maybe simply don’t recognize.  This isn’t to say that a good engineer or product manager can’t be equally good at developing packaged and SaaS applications, but it does mean that the approach is completely different.

Stop trying to figure out how to leverage your existing assets with minimal work and start thinking about having two different products.  Or, determine which business you want and kill the other one off.  If you decide to keep both products alive, you can share services and code between these platforms, but you should not do so at the expense of optimizing your SaaS solution.  Attempting to satisfy both with a single architecture will likely result in you failing at both.

Tags: , , , , ,
Posted in Engineering | 2 Comments »

Evolving Architecture And Software

Monday, July 5th, 2010

When asked by a team what they should prepare for an engagement with us, we usually tell them to not prepare anything. Instead of PowerPoint slides showing the architecture, network, etc we prefer for people to jump to the white board and draw. One of the primary reasons is that we often find people debating how the architecture actually exists. How does your architecture diagrams or institutional knowledge reflect reality of the software?

In the May issue of Computer, there is an article, “Evolving Software Architecture Descriptions of Critical Systems”, by Tom Mens, Jeff Magee, and Bernhard Rumpe, in which the authors’ state:

An explicit architecture description is important but not sufficient to manage the complexity of developing, maintaining, and evolving a critical software-intensive system.

The authors continue explaining that the architecture description must be accurate and traceably linked to the actual implementation in software so that changes in the architecture are reflected in implementation and vice versa.

If your team has spent a bunch of time creating an architecture that will scale, all that effort is wasted when the software implementation doesn’t abide by the architecture. Because of the ever evolving nature of complex software systems it is admittedly difficult to keep the architecture description and software artifacts aligned.  The authors of the article suggest that evolving architecture descriptions requires co-evolution of different viewpoints such as the structural and behavioral. To this I completely agree but they address the solution to this issue from the aspect of Architecture Description Languages (ADLs). The problem with this approach is that I don’t know of many, if any, SaaS companies using ADLs. Therefore, in order to accomplish this co-eveolution of software artifacts and architecture descriptions we have to seek a different solution.

To ensure that architectural changes are reflected in the software we typically suggest that companies rely on architecture principles. We’ve dedicated an entire chapter in The Art of Scalability to this subject but I’ll try to summarize it here. Architectural principles are a set of ideas that the team has determined when used as guidelines during the design and development of the software will yield a scalable, available, and cost effective system. Principles should help influence the behavior and the culture of the team. We often use the SMART acronym to describe good principles as being Specific, Measurable, Achievable, Realistic, and Testable.

So how about the other direction, how do we ensure the architecture description accurately reflects the software? By using JAD and ARB processes, which we’ve covered in detail before on this blog as well as in the book, we can help ensure that software artifacts that deviate from the established architecture are discussed and noted by the appropriate individuals and teams.

Remember that the co-evolution of the software as well as the architecture design is critical in order to manage the development and maintenance of complex, critical software systems. Implement simple but efficient processes to ensure these remain synchronized.

Tags: , , , , , ,
Posted in Engineering | 1 Comment »

Scalability Warning Signs

Monday, June 7th, 2010

Unless you’re one of the incredibly lucky sites where the traffic spikes 100x overnight, scalability problems don’t sneak up on you. They give you warning signs that if you are able to recognize and react to appropriately, allow you to stay ahead of the issues. However, we’re often so head down getting the next release out the door that we don’t take the time to realize we’re experiencing warning signs until they become huge problems staring us in the face.  Here are a few of the warnings that we’ve heard teams talk about in the past couple of months that were clearly signs of problems on the horizon.

Not wanting to make changes – If you find yourself denying request for changes to certain parts of your system, this might be a warning sign that you have scalability issues with that component. A completely scalable system has components that can fail without catastrophic impact to customers. If you’re avoiding changes to a component because of the risk of problems this is a warning sign that you need to re-architect to eliminate or at least mitigate the risk.

Performance creep – If after each release you need to add hardware to a subsystem or you accept a performance degradation in a service you could have a scaling issue approaching quickly. Consistently increasing consumption of CPU or memory resources in a service with each release will lead you into an unsustainable situation. If today you’re comfortably sitting at 40% CPU utilization and you allow a modest 10% degradation in each release you have less than nine releases before you are well above 100% but the reality is you won’t get close to that without significant issues.

Investigating larger hardware – If you’ve started asking your vendors or VAR about bigger hardware you’re heading down the path of scalability problems. The scale of more computational resources per dollar is not linear, it’s closer to cubic or even exponential scales. Purchasing more expensive hardware might seem like the economical way out when you compare the cost of the first hardware upgrade versus developer time but run the calculation out several iterations. When you get to a Sun Fire™ E25K Server with Oracle Database 10g at a $6M price tag you might feel differently about the decision.

Asking vendors for advanced features – When you start exploring advanced options of your vendor’s software you’re likely heading down the path of increased complexity and this is a clear warning sign of scalability problems in your future. Besides potentially locking you into a vendor which lowers your negotiating power it puts the fate of your company in someone else’s hands, which wouldn’t make me sleep very well at night. See our post on using vendor features to scale for more information.

Watch out for these or similar warning signs that scalability problems are looming on the horizon. Dealing with the problems today while you have time to plan properly might not get you an award for being a firefighter but you’ll more likely deliver a quality product without costly interruption.

Tags: , , ,
Posted in Engineering | 1 Comment »

Agile Architects

Wednesday, June 2nd, 2010

We recently posted what agile is not, where we outlined questions that we often hear about agile development. Another question that is often raised is how to combine the seemingly long-term process of architecture with the short-term nature of agile development. We believe that architecture is not at odds with agile development and that the two can not only coexist but complement each other. To ensure your architecture standards are being integrated with each sprint, resulting in a scalable and available architecture, we rely on the Joint Architecture Development (JAD).

We’ve covered JAD before but as a recap, this is the process by which features are designed in a series of meetings where developers, architects, and tech operations come together to create the design. This multi-function representation early in the development process ensures that individuals are aware of standards, there is buy-in from all concerned parties, and that the design is benefitted by the knowledge that exists in different technical fields.

In the IEEE Software article “Architects as Service Providers,” Roland Faber says that “the architect role is to champion system qualities and stability, while the developer role is to champion the system functions and change.” The agile architect interacts frequently and flexibly with the developers, building a trust relationship with them.

The JAD is ideal for flexible interaction that can happen in short bursts of effort that correspond to sprints. The agile architect must understand that, because of the nature of agile development, architecture must be dynamic and not static. Architects must rely on personal interaction with developers not documentation to understand the requirements.

Faber continues in his article describing two phases of the architecture process as preparation and support. During preparation the architect engages in processes such as prepare rules, frameworks, and structures. During support the architect helps resolve conflicts, engages in firefighting, and stimulates architecture communication. He makes a point that if the developers don’t believe the architects will provide support they won’t tell them when they are breaking the rules.

Tags: , , , ,
Posted in Engineering | 1 Comment »

« Older Entries