Posts Tagged ‘software engineering’

Evolving Architecture And Software

Monday, July 5th, 2010

When asked by a team what they should prepare for an engagement with us, we usually tell them to not prepare anything. Instead of PowerPoint slides showing the architecture, network, etc we prefer for people to jump to the white board and draw. One of the primary reasons is that we often find people debating how the architecture actually exists. How does your architecture diagrams or institutional knowledge reflect reality of the software?

In the May issue of Computer, there is an article, “Evolving Software Architecture Descriptions of Critical Systems”, by Tom Mens, Jeff Magee, and Bernhard Rumpe, in which the authors’ state:

An explicit architecture description is important but not sufficient to manage the complexity of developing, maintaining, and evolving a critical software-intensive system.

The authors continue explaining that the architecture description must be accurate and traceably linked to the actual implementation in software so that changes in the architecture are reflected in implementation and vice versa.

If your team has spent a bunch of time creating an architecture that will scale, all that effort is wasted when the software implementation doesn’t abide by the architecture. Because of the ever evolving nature of complex software systems it is admittedly difficult to keep the architecture description and software artifacts aligned.  The authors of the article suggest that evolving architecture descriptions requires co-evolution of different viewpoints such as the structural and behavioral. To this I completely agree but they address the solution to this issue from the aspect of Architecture Description Languages (ADLs). The problem with this approach is that I don’t know of many, if any, SaaS companies using ADLs. Therefore, in order to accomplish this co-eveolution of software artifacts and architecture descriptions we have to seek a different solution.

To ensure that architectural changes are reflected in the software we typically suggest that companies rely on architecture principles. We’ve dedicated an entire chapter in The Art of Scalability to this subject but I’ll try to summarize it here. Architectural principles are a set of ideas that the team has determined when used as guidelines during the design and development of the software will yield a scalable, available, and cost effective system. Principles should help influence the behavior and the culture of the team. We often use the SMART acronym to describe good principles as being Specific, Measurable, Achievable, Realistic, and Testable.

So how about the other direction, how do we ensure the architecture description accurately reflects the software? By using JAD and ARB processes, which we’ve covered in detail before on this blog as well as in the book, we can help ensure that software artifacts that deviate from the established architecture are discussed and noted by the appropriate individuals and teams.

Remember that the co-evolution of the software as well as the architecture design is critical in order to manage the development and maintenance of complex, critical software systems. Implement simple but efficient processes to ensure these remain synchronized.

Agile Architects

Wednesday, June 2nd, 2010

We recently posted what agile is not, where we outlined questions that we often hear about agile development. Another question that is often raised is how to combine the seemingly long-term process of architecture with the short-term nature of agile development. We believe that architecture is not at odds with agile development and that the two can not only coexist but complement each other. To ensure your architecture standards are being integrated with each sprint, resulting in a scalable and available architecture, we rely on the Joint Architecture Development (JAD).

We’ve covered JAD before but as a recap, this is the process by which features are designed in a series of meetings where developers, architects, and tech operations come together to create the design. This multi-function representation early in the development process ensures that individuals are aware of standards, there is buy-in from all concerned parties, and that the design is benefitted by the knowledge that exists in different technical fields.

In the IEEE Software article “Architects as Service Providers,” Roland Faber says that “the architect role is to champion system qualities and stability, while the developer role is to champion the system functions and change.” The agile architect interacts frequently and flexibly with the developers, building a trust relationship with them.

The JAD is ideal for flexible interaction that can happen in short bursts of effort that correspond to sprints. The agile architect must understand that, because of the nature of agile development, architecture must be dynamic and not static. Architects must rely on personal interaction with developers not documentation to understand the requirements.

Faber continues in his article describing two phases of the architecture process as preparation and support. During preparation the architect engages in processes such as prepare rules, frameworks, and structures. During support the architect helps resolve conflicts, engages in firefighting, and stimulates architecture communication. He makes a point that if the developers don’t believe the architects will provide support they won’t tell them when they are breaking the rules.

Data Access Layers

Wednesday, April 21st, 2010

Ways of abstracting the storage of data have been around for a long time.  In data warehouses engineers abstract data into business or domain objects that can be manipulated in reports.  For object oriented programming, engineers can use the active record pattern to create a wrapper class representing a table and methods for inserting, updating, or deleting. Thus the manipulation of the database rows are abstracted into the object oriented parlance of classes and methods. This layer of computing is known as a Data Access Layer (DAL) and hides the complexity of the underlying data store from engineers who do not need to be bothered with those details.

There are many frameworks or object-relational mapping (ORM) tools for creating DAL’s for different programming languages such as Active Record for Ruby and Hibernate for Java.  We often hear from development teams that have adopted an ORM that the pros include: shorter development time, better designed code and reduction in amount of code.

However, a quick search will show you that not everyone is sold on the benefits of a DAL. The reduction in code is debatable when constructing complex queries or considering that Hibernate is over 800 KLOC of Java and XML.  There are also concerns about the ability to scale effectively when using DALs.  While it is possible with an ORM to scale on the X-axis such as with MySQL master-slave replication, the Y and Z axes splits can become much more complicated.

I am a fan of DAL’s for their centralization of data objects and their abstraction of relational data into objects. To me these advantages speed development and testing time and improve quality. Additionally, given the sophistication and open source of ORM’s today, I think it makes sense to consider using one as a framework. However, if you choose to do so, you need to consider ahead of time how you would shard your data along other axes. The time for those considerations is moved up when using an ORM. Think of the D-I-D approach where the cost to make a change during the Design phase is negligible compared to changes made after Deployment.

PDLC or SDLC

Sunday, January 3rd, 2010

As a frequent technology writer I often find myself referring to the method or process that teams use to produce software. The two terms that are usually given for this are software development life cycle (SDLC) and product development life cycle (PDLC). The question that I have is are these really interchangeable? I don’t think so and here’s why.

Wikipedia, our collective intelligence, doesn’t have an entry for PDLC, but explains that the product life cycle has to do with the life of a product in the market and involves many professional disciplines. According to this definition the stages include market introduction, growth, mature, and saturation. This really isn’t the PDLC that I’m interested in. Under new product development (NDP) we find a defintion more akin to PDLC that includes the complete process of bringing a new product to market and includes the following steps: idea generation, idea screening, concept development, business analysis, beta testing, technical implementation, commercialization, and pricing.

Under SDLC, Wikipedia doesn’t let us down and explains it as a structure imposed on the development of software products. In the article are references to multiple different models including the classic waterfall as well as agile, RAD, and Scrum and others.

In my mind the PDLC is the overarching process of product development that includes the business units. The SDLC is the specific steps within the PDLC that are completed by the technical organization (product managers included). An image on HBSC’s site that doesn’t seem to have any accompanying explanation depicts this very well graphically.

Another way to explain the way I think of them is to me all professional software projects are products but not all product development includes software development.  See the Venn diagram below. The upfront (bus analysis, competitive analysis, etc) and back end work (infrastructure, support, depreciation, etc) are part of the PDLC and are essential to get the software project created in the SDLC out the door successfully.  There are non-software related products that still require a PDLC to develop.

Do you use them interchangeably?  What do you think the differences are?

From Technician to Engineer

Monday, October 5th, 2009

We’ve had a couple posts on this topic of engineer vs craftsman vs technician but I found myself in the past couple of days discussing this topic in two different settings and thought it would be fun to revisit on the blog. I started the conversation with a post that quoted Tom DeMarco concluding that “software engineering is an idea whose time has come and gone.” Also quoted was Jeff Atwood stating that “If you want to move your project forward, the only reliable way to do that is to cultivate a deep sense of software craftsmanship and professionalism around it.” Marty picked up the conversation in another post stating that:

All of this goes to say that software developers rarely “engineer” anything – at least not anymore and not in the sense defined by other engineering disciplines. Most software developers are closer to the technicians within other engineering disciplines; they have a knowledge that certain approaches work but do not necessarily understand the “physics” behind the approach. In fact, such “physics” (or the equivalent) rarely exist. Many no longer even understand how compilers and interpreters do their jobs (or care to do so).

I think it is possible that we are seeing the evolution of our discipline as it struggles to determine what its final form will take. Computer science, information technology, software engineering, and other related disciplines are all relatively new fields of study. With a new discipline it should be expected that definitions and themes will need to be stretched or reconsidered.

Software engineers, similar to other engineering disciplines who are taught more “true” laws such as Newton’s or Faraday’s, also undergo something of an apprenticeship once their degree is conferred. Mechanical and electrical engineers work beside senior engineers who help them transition from the theoretical to the practical. Software engineers are often apprenticed in the same manner by more senior engineers. If the practical implementation of one discipline is considered engineering because it is based upon laws and principles, I would argue that the principles of architecture for scalability are of a similar nature. This in fact I think is a strong differentiator between technicians and engineers within the software development discipline. A technician can write code, setup a database, or administer a server. An engineer can architect a database or system or pool of servers such that it can scale. We’ve written several posts about the principles and patterns of scalability and a large part of our book is dedicated to these principles. Are these sufficiently established to be called a principle as defined in Wikipedia?

A principle is one of several things: (a) a descriptive comprehensive and fundamental law, doctrine, or assumption; (b) a normative rule or code of conduct, and (c) a law or fact of nature underlying the working of an artificial device

I still like the idea of software developers as craftsmen and -women but as I concluded in the other post, that discussion for me is as much about organizational size and control as it is anything else. The technician vs engineer discussion I think is best held in the light of are they or are they not applying laws or principles. As the American Engineers’ Council for Professional Development defines engineering “The creative application of scientific principles to design or develop…” Have we as a discipline, especially in terms of scalability, advanced enough to call what we use “principles”? Let us know what you think.