Scientists, Engineers, and Technicians
What is, or perhaps should be, the difference between a Computer Scientist, Data Scientist, Software Engineer, and Programmer? What should your expectations be of each? How should they work together?
To answer these questions, we’ll look at the differences between scientists, engineers, and technicians in more mature disciplines, apply them to our domain, and offer suggestions as to expectations.
Science and Scientists
The primary purpose of science is “to know". Knowing, or the creation of knowledge, is enabled through discovery and the practice of the scientific method. Scientists seek to know “why” something is and “how” that something works. Once this understanding of “why and how” are generally accepted, ideally they are codified within theories that are continually tested for validity over time.
To be successful, scientists must practice both induction and deduction. Induction seeks to find relationships for the purposes of forming hypotheses. Deduction seeks to test those hypotheses for validity.
In the physical world (or in physical, non-biological, and non-behavioral sciences), scientists are most often physicists and chemists. They create the knowledge, relationships, and theories upon which aerospace, chemical, civil, electrical, mechanical, and nuclear engineers rely.
For our domain, scientists are mathematicians, computer scientists – and more recently – data scientists. Each seeks to find relationships and approaches useful for engineers and technicians to apply for business purposes. The scientist “finds”, the engineer “applies”. Perhaps the most interesting new field here is that of data science. Whereas most science is focused on broad discovery, data science is useful within a business as it is meant to find relationships between various independent variables and desirable outcomes or dependent variables. These may be for the purposes of general business insights, or something specific like understanding what items (or SKUs) we should display to different individuals to influence purchase decisions.
True scientists tend to have doctorates, the doctoral degree being the “certification” that one knows how to properly perform research. There are of course many examples of scientists without doctoral degrees – but these days that is rare in any case other than data scientists (here the stakes are typically lower).
Engineering and Engineers
The primary purpose of engineering is to “create” or to “do”. Engineers start with an understanding (created by scientists) of “why” things work, and “how” they work (scientific theories – often incorrectly called “laws” by engineers) and apply them for the purposes of creating complex solutions or products. Mechanical engineers rely on classical physics, electrical engineers rely on modern physics. Understanding both the “why” and “how” are important to be able to create highly reliable solutions, especially in new or unique situations. For instance, it is important for electrical engineers to understand field generation for micro-circuitry and how those fields will affect the operation of the device in question. Civil and mechanical engineers need to understand the notion of harmonic resonance in order to avoid disasters like the Tacoma Narrows bridge failure.
The domain of “software engineering” is much more confusing. Unlike traditional engineering domains, software engineering is ill-defined and suffers from an overuse of the term in practice. If such a domain truly existed, it should follow the models of other engineering disciplines. As such, we should expect that software engineers have a deep understanding of the “whys and hows” derived from the appropriate sciences: computer science and mathematics. For instance, computer scientists would identify and derive interesting algorithms and suggest applications, whereas engineers would determine how and when to apply the algorithms for maximum effect. Computer scientists identify unique scenarios (e.g. the dining philosophers problem) that broadly define a class of problems (mutual exclusion in concurrency) and suggest approaches to fix them. Engineers put those approaches into practice with the constraints of the system they are developing in mind.
Following this logic, we should also expect our engineers to understand how the systems upon which they apply their trade (computers) truly function – the relationship between processors, memory, storage, etc. Further, they should understand how an operating system works, how interpreters and compilers work, and how networks work. They should be able to apply all these things to come up with simple designs that limit the blast radius of failure, ensure low latency response, and are cost effective to produce and maintain. They should understand the depth of components necessary to deliver a service to a customer – not just the solitary component upon which they work (the code itself).
Very often, to be a successful engineer, one must have at least a bachelor’s degree in an engineering domain. The degree at least indicates that one has proven to some certifying body that they understand the math, the theories, and the application of those theories to the domain in question. Interestingly, as a further example of the difference between engineering and science, some countries issue engineering degrees under the degree of “Bachelors of Applied Science”.
There are of course many famous examples of engineers without degrees – for instance, the Wright Brothers. The true test isn’t the degree itself, but whether someone understands the depth and breadth of math and science necessary to apply science in highly complex situations. Degrees are simply one way of ensuring an individual has at least once proven an understanding of the domain.
Practical Applications and Technicians
Not everything we produce requires an engineer’s deep understanding of both why and how something works. Sometimes, the application of a high-level understanding of “how” is sufficient. As such, in many domains technicians augment engineers. These technicians are responsible for creating solutions out of reusable building blocks and a basic understanding of how things work. Electricians for instance are technicians within the domain of electrical engineering. Plumbers are a very specific application of civil engineering. HVAC technicians apply fluid mechanics from mechanical engineering.
These trades take a very technical skill, implementation specific tradecraft, and a set of heuristics to design, implement, and troubleshoot systems that are created time and time again. Electricians, for instance, design the power infrastructure of homes and offices – potentially reviewed by an electrical engineer. They then implement the design (wire the building) and are also responsible for troubleshooting any flaws. The same is true for HVAC technicians and plumbers.
Programmers are the technicians for software engineers (engineering domain) and computer and data scientists (science domain). Not everything we develop needs a “true engineer”. Very often, as is the case with wiring a house, the solution is straight forward and can be accomplished with the toolset one gains with several weeks of training. The key difference between an engineer and a programmer is again the depth and breadth of knowledge.
Technicians are either trained through apprenticeship or through trade schools. Electricians can come from either approach. Broadly speaking, it makes sense to use technicians over using an engineer for at least 2 reasons:
- Some things don’t require an engineer, and the cost of an engineer makes no sense for these applications.
- The supply of engineers is very low relative to demand within the US across all domains. During the great recession, engineers were one of the only disciplines at or near economic full employment – a clear indication of the supply/demand imbalance.
Implications and Takeaways
Several best practices follow the preceding definitions and roles:
- Don’t mix Science and Engineering teams or goals: Science, and the approach to answering questions using the scientific method, are different animals and require different skills and different expectations from engineering. The process of scientific discovery is difficult to apply time constraints; sometimes answers exist and are easy to find, sometimes they are hard to find, and sometimes they simply do not exist. If you want to have effective analytics efforts, choose people trained to approach your Big Data needs appropriately, and put them in an organization dedicated to “science” activities (discovery). They may need to be paired with programmers or engineers to accomplish their work – but they should not be confused with a typical product team. “Ah-Ha!” moments are something you must allocate time against – but not something for which you can expect an answer in a defined time interval.
- Find the right ratio of engineers to programmers: Most companies don’t need all their technical folks to be engineers. Frankly, we don’t “mint” enough engineers anyway – roughly 80K to 100K per year since 1945 with roughly 18K of those being Computer Science graduates who go on to become “engineers” in practice. Augment your teams with folks who have attended trade schools or specialized boot camps to learn how to program.
- Ensure you hire capable engineers: You should both pay more for and expect more from the folks on your team who are doing engineering related tasks. Do not allow them to approach solutions with just a “software” focus; expect them to understand how everything works together to ensure that you build the most reliable services possible.