Discussion: Black boxes
Alexandre D'Amours - 2021-01-21
1. Introduction
Most structural engineering software on the market can be described as black box software. There is no way to access the inner workings (code) to validate how it works. You simply input data and receive outputs.
From the point of view of software developers, giving access to the code would be financial suicide. Anyone with access to the code can easily recreate the software for free. This business model, in which the code is the product, is the simplest and most widespread in the software market at large.
This situation is far from ideal for engineers. In this article, we will discuss why and what can be done about it.
2. Alignment problem
In most countries, engineering is a protected profession. The use of the title ("engineer") is reserved for those who have shown, with study and practice, that they are qualified to do their job. It also comes with personal liability with regards to public advice and actions.
To engineer is to create models of the world and use them for future design. For example, if you want to build a skyscraper, you need to understand what forces will act upon it (statics). You must understand how different materials and sections react to these external forces (mechanics). You need to understand how the forces can be safely transferred to the surrounding soil (geotechnical engineering). Furthermore, engineers are not only responsible for understanding and using these models, but also their inherent limitations. The difference between the real world and the model must be known and taken into account by engineers (the map is not the territory).
From a legal perspective, it's quite simple: engineers will be held responsible for the failure of their designs. Whether they misunderstood the model (miss-read the map) or failed to see that the model didn't apply (mistook the map for the territory), the end result is the same.
As previously stated, most structural engineering software is "black box" software. By definition, the mathematical and logical models used for calculations are hidden to users. Structural engineers are responsible for understanding their models and their inherent limitations, but they can't access the software code. They are left using whatever data is available in software reports to reverse engineer how the code works. This trial and error detective work should be repeated as many times as there are edge cases, but in practice, engineers often only back check a couple of known examples. For experienced engineers, this might be acceptable, they have experience to understand when the results are completely out of line. For younger engineers under pressure to get results fast, back checking and software validation is a step that is easy to skip. And, as stated in the post on human error, there are more and more young unsupervised engineers, which might mean more risk in the future.
In short, engineers need to be able to thoroughly understand their models. Current structural engineering software developers do not fully disclose how their software works. Since the business model does not allow them to give their clients a necessary feature (easy to validate software and code), we have a fundamental alignment problem.
3. Open source "white box" software
The simplest solution to the alignment problem, in our view, is to move to a new business model. Code should be verified and understood by each any user that wishes to do so. It should be evolving with the most up-to-date norms and regulations. You should be able to drill down into your software all the way to the code. This means that the code itself cannot be the product.
Open-source software functionality is much more in line with the spirit of engineering. In the open-source world, you often do not pay for access to the software, but you also accept that the software comes with no guarantees and that you hold the full responsibility over its use. There is no gray zone as to who is responsible for mistakes in the software.
And here lies the rub. Black box software is very popular because, for some, it *feels* like the software is already complete and does not require further testing. It seems like any mistakes in the software would have already been detected by other, more rigorous professionals. As previously discussed, from a legal perspective, you are kidding yourself. When push comes to shove, you will be held liable for not having validated the software you used for your design.
4. Conclusion
For structural engineers in particular, failure of design can be catastrophic. For this reason, practice must change in slow and mindful ways. New ways of doing business must take into account past hard-earned lessons as well as the unforgiving nature of the work.
The market for engineering software is stuck in a black box business model that was burrowed from software engineering. The model itself is detrimental to its users and is holding back the profession.
No software is free of bugs, and, in engineering, end-users will be held responsible for any problems that that might cause. In our view, the best remedy is to have as many people as possible validating the work, and that means having "white box" software. Let us move past the black box and start to explore the full potential of the structural engineering community.