As a side practice in our scalability and availability engagements we often work with companies on the performance of their SaaS offerings by attempting to speed up their web page load times. Citing a Google white paper, “Speed Matters for Google Web Search” by Jake Brutlag, we point to the fact that even tenths or hundredths of a second matter. Brutlag states that through experiments they have shown that increasing web search latency from 100 to 400 ms reduces the daily number of searches per user by upwards of 0.6%. Given that we are attempting to become practitioner-scholars, in order to bridge the gap between academia and practice, we decided to dive into this subject area a little deeper. Our goal was to understand what other research had been done and if there was anything more practitioners could learn besides “speed up your pages!”

Research in computer system delay has been taking place for decades and has shown that excessive computer system delay results in negative responses such as anxiety (Guynes, 1988) and satisfaction with the system itself (Rushinek & Rushinek, 1986). However, the research does not support the relationships between increases in delay and the attitude toward the company (Rose & Straub, 2001). It was shown that increases in delay treatments from near 0 to 15 seconds did not correlate with a reduction in satisfaction measures such as ease of use or content appeal (Otto et al., 2000). It was also found that increases in delay treatments did not consistently predict likelihood of future patronage (Rajala and Hantula, 2000).

So from all this research we have the notion that delays cause frustration, even anxiety, but yet they don’t appear to cause a decrease in satisfaction or even predict continued usage. Why is this?

Attribution theory, which deals with how individuals infer causality between events (Kelley and Michela, 1980), would explain this phenomenon as the customers assigning blame for the delay to something or someone other than the SaaS provider. This theory has also been used to show that the presence of a self-serving attribution bias and an actor–observer attribution bias in entrepreneurs’ representations of events (Rogoff et al. 2004) but we’ll save that for another post. It turns out that perceived wait time is much more critical than the actual wait time (Baker & Cameron, 1996).

Rose, et al. (2005) content that “it may be less important to reduce objective delay than it is to create a system where users will be less likely to attribute the delay to the retailer.” An example would be to give the user the option of selecting a low or high graphic site in order to provide the users with the control. Users will likely perceive this as an active effort on the part of the SaaS provider to minimize download time and thus attribute delays to themselves, their computer, their ISP, etc but not the site.

References:

Rushinek, A., & Rushinek, S. (1986). “What makes users happy?” Communications of the ACM, 29, 594–598.

We’ve written about trying to use vendor features to scale but given how often we run across companies that have been convinced by vendors to rely on them, this topic is worth revisiting. To state it as directly as possible, every major SaaS company that has relied on a vendor, software or hardware, to scale them through hyper-growth has failed and had to solve the scale problem themselves.

Since Oracle World took place recently I’ve decided to use Oracle RDBMS as an example of failing to scale with vendor features. We have nothing against using Oracle as an RDBMS, even though there are open source options that can scale just as well, but let’s use one of their scalability features, Real Application Clusters (RAC), as an example. In Oracle’s own words RAC “…enables a single database to run across a cluster of servers, providing unbeatable fault tolerance, performance, and scalability with no application changes necessary.” A nice concept – to scale with “no application changes” – but this isn’t realistic with hyper-growth companies. One large reason is that RAC does not scale across multiple data centers, which is a requirement for hyper-growth companies since everything fails eventually including data centers. Even with the “Extended Distance Clusters” for RAC nodes, they only extend to 25 kilometers using Dark Fiber (DWDM or CWM) technology.

The use of RAC for increased availability is fine but you should review our post on the downside of using vendor features and how to negotiate with vendors. In particular you should be aware that by using this feature you have weakened your position during renewal negotiations. If you think your sales person is being nice by throwing in the RAC feature for a low price, think again. As soon as you start using this feature they have the upper hand in negotiations.

Enough of the RAC rant, especially since this is just one example of many that are out there. Hardware vendors, both servers and storage, are just as guilty of trying to convince SaaS companies to rely on them for scalability. Keep your destiny in your own hands and resist relying on short term solutions to long term problems.

If you are a technology organization within a Software as a Service (SaaS) company, there is no such thing as an “internal customer”.  We often see this anachronistic IT phrase thrown around in web X.0 companies by executives and engineers who simply have not adopted the new SaaS mindset.  Do you think you’ll hear the left offensive tackle of an NFL team refer to the quarterback as his “internal customer”?  The quarterback consumes services (energy to block opponents) of the left tackle – so why wouldn’t he be a customer?  The answer is simple – because the notion of a customer relationship is different than the notion of a relationship within teammates.

The first reason why your teammate isn’t your customer is because he or she is, well, your TEAMMATE.  Customers are someone for whom you produce a service or product and teammates are someone with whom you work to accomplish a goal.  The difference between working FOR someone and working WITH someone is HUGE.  This difference creates a contextually activated identity that forces you to think about customers in a different light than you would a teammate.  Very often, as we’ve written before, this can result in affective (role based or bad) conflict between teams.  Affective conflict is bad and it destroys shareholder value.  Working as a team is important and customers aren’t part of your team.

The next reason that your teammate isn’t your customer is that the customer is always right.  Your teammate isn’t always right.  You need to debate certain points as a team to come to better solutions.  This isn’t affective conflict, it is cognitive conflict and if handled properly it is good and helps to create shareholder value.

The most important reason there isn’t a customer relationship here is that your teammate isn’t paying!   “Servicing” your teammate (uggh…that’s an ugly term) doesn’t create shareholder value.  Working as  a team to delivery a service or product to your  “real” customer is what creates shareholder value.  One design, one approach, one ruthless drive as a team to get across the goal line is what is necessary to thrive and succeed.

So stop using the ugly “internal” C word in your SaaS company.  It doesn’t have a place there.  Let the old world, internal IT folks continue to provide services to their internal customers.  Start acting like a team, designing and building services rather than software.

I’ve noticed lately that more companies are putting up signs in hallways and cube farms requesting that people avoid having conversations in these areas. While having a nice quiet work environment makes sense to me as a developer, doesn’t this completely void having people work beside each other? The ad hoc/hallway/water cooler/coffe machine conversations or ones overheard when cube mates are chatting about a new feature are one of the primary benefits of having people work in small open environments.

I haven’t done any sort of scientific study but it seems that these sort of “please be quiet” signs are more prevalent at larger companies. These are the same ones that are trying to mimic the small startup with agile development processes or open work spaces to compete in a fast moving SaaS marketplace. Imitating the actions without understanding the purpose or allowing old school corporate policies to overrule are surefire ways to tank the initiative.

A parallel to the “please be quiet” sign is allowing corporate IT to dictate the architecture of the SaaS offering based on a corporate standard that works for the ERP system. Running Oracle ERP on a 16-way system might be the vendor recommended, preferred approach but for scaling a SaaS offering this is a quick way to run up the costs and ensure lower availability. We often use the analogy of goldfish and thoroughbreds for comparing small, cheap 1U servers with large, expensive multi-processor boxes with lots of memory. The goldfish (small, cheap servers) are inexpensive to purchase and replaceable while the thoroughbred (large servers) are expensive to purchase/maintain and cause big impacts when they go down.

The take away to all of this is that if your part of a corporate initiative to run an internal startup or deliver a Software as a Service from inside a larger organization, don’t allow corporate policies to prevent your success. The differences in approaches, architectures, organizations, and offices have a purpose and should not be discounted as non-critical to the success of your initiative.

It’s probably no surprise to our readers that many old packaged software companies are attempting to take their software and hence their business models “online”.  And why not?  The model is attractive and benefits accrue to both the providers of service through software and those who outsource portions of what was once bothersome internally hosted software.  The providers benefit from economies of scale in hosting that generate attractive profits for the provider and savings for the customer, lower maintenance costs resulting from custom customer deployments, predictable revenue streams fostered through closer customer contact, more frequent and smaller releases that reduce risk and faster implementation times that result in faster profit recognition.  Customers benefit from outsourcing non-core IT functions, providers who specialize in delivering specific services, lower capital expenditures and faster deployment times.  SaaS is both a desert topping and a floor wax!  It’s the cure for cancer and the answer to the riddle of life!

But what many of these companies don’t realize is that the way one architects a product and runs a company focused on service delivery is simply different than the approach of a company focused on delivering software.  Customers expect that you are going to give them higher availability and fewer headaches.  Software alone simply won’t meet this goal; it is imperative that one design SaaS systems holistically which in turn requires skills in both infrastructure and software architecture (or “systems” architecture).   The cost leverage necessary to both increase profit margins and decrease customer cost typically requires multi-tenancy which has its own share of headaches.  Fault isolation and rollback capabilities are a must to minimize customer impact and mitigate rapid deployment risks.

It is not enough to simply bundle up an application in a hosted fashion and label yourself a “SaaS” company.  If you don’t work aggressively to increase availability and decrease your cost of operations, someone with greater experience will come along and simply put you out of business.  After all, your business is now about SERVICE – not SOFTWARE.  This is a fundamental mind-shift that some companies simply can’t overcome or maybe simply don’t recognize.  This isn’t to say that a good engineer or product manager can’t be equally good at developing packaged and SaaS applications, but it does mean that the approach is completely different.

Stop trying to figure out how to leverage your existing assets with minimal work and start thinking about having two different products.  Or, determine which business you want and kill the other one off.  If you decide to keep both products alive, you can share services and code between these platforms, but you should not do so at the expense of optimizing your SaaS solution.  Attempting to satisfy both with a single architecture will likely result in you failing at both.