Dave Farley's book is really good, and I'd highly recommend it.
However, I do have an issue with his focus on applying science in the definition, even though it doesn't come across in his final conclusions. In the history of engineering, that's a relatively new development. It started, to some extent, in Europe in the 1700s, but it really took off in the 1920s and exploded in the United States after World War II (1940s-1950s). It culminated with the Report of the Committee on Evaluation of Engineering Education (the Grinter Report) in the mid-1950s.
This isn't to say that science isn't important to engineering, since it absolutely is. Science provides knowledge used to better understand the world being changed by engineering. But there are also plenty of examples of engineering going ahead of science - the steam engine, the airplane, generative AI. We didn't fully understand the governing rules before the technology existed.
Any definition of engineering needs to be broader. Ferguson (Engineering and the Mind's Eye), Florman (several works, but primarily The Existential Pleasures of Engineering), and Vincenti (What Engineers Know and How They Know It) all explore ways in which engineering can't rely on science alone. I think Koen (Discussion of the Method) puts it best, where the application of science is one heuristic that engineers may choose to draw upon.
Having a bottleneck with AI-generated code is probably a good thing.
Consider before AI. In cases where code was written by a trusted human, you had confidence that they were doing the right stuff - the people understood the context and requirements, the changes were based on those requirements, and tests were updated. You could rely on some basic automated checks to run a linter and static analyzer, execute the tests, and check test coverage. You can defer longer-running tests to a later stage, but people can merge their work in and collaborate. You can have blocking reviews for untrusted outside contributors or new employees and review the rest later.
After AI, the input is a mess. I haven't personally seen the worst of it, but I've read about people generating code before any requirements development or analysis. Code is generated without understanding what it is supposed to do, which also prevents good testing. Even with good requirements, people generate solutions that aren't right, and the generation step may remove failing tests or make unintended changes that aren't covered by tests. So much code is generated from untrusted sources and rubber-stamped. Introducing a bottleneck where a human can slow the process down is a good thing here.
Maybe if we can get people to understand how to use AI tools responsibly and have humans take full accountability for the output and changes, we can get back to the old way. Trusted humans and an AI agent assistant can make changes that are solid enough to merge. Independent control over the pipeline and basic automated checks would give just enough confidence in the change, and delayed review before deploying or enabling the feature would work. But not until trusted humans actually maintain oversight over their tools.
In practice, the frameworks don't work out of the box. Take Scrum, for example. The framework described today has had all the original technical practices used by the first Scrum Teams stripped out, with no new technical practices added, and it's been generalized to fit many organizations so it's marketable. There are many questions to ask. Why do I need to have a Daily Scrum every day? If I'm mobbing, do I need a Daily Scrum at all? Why a maximum timebox of a month for a Sprint instead of 5 or 6 weeks? What technical practices do I need to deliver software every day or week? All of these are important questions that would likely result in changes.
The culture is the big thing.
I've only worked in very large enterprises, but I've never had to deal with a dickhead CEO because they push decisions down. I haven't even had to deal with a dickhead business unit VP or director. Decisions are actively made 3 or 4 levels below the CEO. That's almost always at the product level, which could be one level up from a team if multiple teams work on a single product. I've also rarely had to deal with people being shoehorned in because we strive to eliminate single points of failure. That means having at least two people with (or working toward) a given skill set, cross-training, and being willing to work outside their comfort zone. People who stay in their niche and don't work with others often don't last long, by their choice.
So, yes. These things can work. I've done it. But it means you can't follow someone else's script or a canned framework and set of tools. It means having a deep understanding of why a framework has made certain choices around roles and events and what to change to build something that works for you. But it's also cultural change in the organization because agile ways of working are inherently different than predictive methods - they need different relationships and inputs and create different outputs.
Retros: Retros are about being more effective. That usually means talking about and solving problems. If too many problems are beyond the team's control, that's indicative of a "faux agile" or "dark agile" environment. If your "CEO being a dickhead" is the biggest problem in anything but the smallest organizations, your CEO is not pushing decisions down to the lowest possible level where they can be made and isn't trusting and respecting the people with the knowledge and skills to get the work done. This is the kind of cultural shift needed to make agility work. If most decisions are made two or three (or more) levels above the team, then agility falls apart. When it comes to successes, it's not about patting yourselves on the back, but about finding ways to repeat those successes again and again in the future.
Standups: Even a "quick morning briefing'" isn't the intent of a standup. It's a planning session. If you have a bunch of people who barely work together, you won't get value from a daily standup. But when you have highly collaborative teams and are planning things like who is going to pair or mob today, what work will be integrated and would be ready for review, what unexpected problems came up, how to account for someone's unplanned time off, or how to plan for someone's upcoming time off, then you're getting value. In some cases, daily is even a bit much. Two or three times a week could work, especially if the team communicates regularly and well outside of this meeting.
Agile coaches: Unfortunately, companies hiring unqualified coaches has turned this role into nothing more than a team secretary or admin. Looking at Lyssa Adkins and Michael Spayd's Agile Coach Competency Framework shows that an effective agile coach needs many skills: lean and agile practice, teaching and mentoring, coaching and facilitating, and mastery in at least one of technical, business/product, or organizational development and transformation. Instead of hiring people who could actually teach and coach their teams and organizations on how to reorganize themselves and apply new practices, they hired people who took a two-day course without any practical experience in a lean or agile organization and who can't teach the people doing the work. Even if you do get someone with the right skills, the organization needs to trust and respect this person and be willing to make the changes needed to support agility.
Sprint / cycle planning: There are many models that emphasize continuous flow. However, it's not so much about the planning but the continuous heartbeat of the entire iteration. Sometimes, it's easier for key stakeholders to interact with the team on a regular, pre-defined cadence. When you establish that interaction, regular planning comes naturally since you want to plan what you'll do between the end of one interaction and the start of the next. If you're getting constant feedback, though, the value of this decreases. I would say that estimation, especially at the unit-of-work level, is unnecessary, and there are plenty of resources on using historical data on work completion to plan upcoming work.
Grooming: No single person can make the right work on their own. Investing a little time from the whole team to review the work and develop a shared understanding greatly reduces the time needed to plan the work. You can also think about many perspectives - what various external stakeholders need or want, the architecture of the system, how you'll verify and validate the work, how to make sure it's deployable and operatable (and monitorable) in production. You'll find ways to reorder the work because doing X before Y will reduce the risk of Y or you'll find that the team needs to prototype and explore. Without some kind of refinement activity, I'm not sure how work can be clearly defined for the team to execute.
Trusting people to do their jobs is the heart of lean and agile methods. But, at the same time, you need structures and patterns to coordinate people, both internally on a team and externally with stakeholders. The problems arise when there is no trust, and people are forced to work in ways that simply aren't appropriate for their context.
I graduated from a software engineering program accredited by ABET as engineering, rather than as a computing program, which would include CS, data science, information systems, and IT. Even with that, I didn't have much, if any, coursework covering the topics needed to pass the FE exam.
The other issue is when your state has requirements for work experience under a PE. At the time, I was working in aerospace/defense, and we had almost no PEs in any discipline. I knew of two or three PEs, one of whom was a project manager and no longer doing technical work. None of them were in the software organization. Even if people could take and pass the FE exam, it would be hard to get the experience. This one is on the states, though, since they create the requirements for becoming a PE.
What makes it worse is that my undergrad degree almost exactly mapped to the SE PE exam, which was very heavily based on SWEBOK. Before I graduated and before the PE exam was a thing, I started down the path of the IEEE CS certifications, which covered much of the same topics. I stopped that when they revamped their certification pathways.
The AGPLv3 doesn't protect against cleanroom engineering.
The full passage from Clause 13 is: "...if you modify the Program, your modified version must prominently offer all users interacting with it remotely through a computer network (if your version supports such interaction) an opportunity to receive the Corresponding Source of your version by providing access to the Corresponding Source from a network server at no charge, through some standard or customary means of facilitating copying of software".
The output of cleanroom engineering is not considered a modified version of the Program. Therefore, users do not need an opportunity to receive the Corresponding Source. The wording about a "standard or customary means of facilitating copying software" is designed to be inclusive of various delivery mechanisms, such as sending a self-addressed stamped envelope to the developer to receive a flash drive or CD, or hosting the source code on GitHub, or a website that allows users to download compressed files.
All that said, there are interesting questions about whether using AI tools to create specifications and then build from those specifications is a form of cleanroom engineering. That's probably a worthwhile avenue to pursue, especially demonstrating that AI tools offer the provable isolation of developing the specification and developing the implementation.
I also checked out the Malus website, and at least one of the claims is a bit questionable. A service like Malus doesn't really solve license compliance overhead. Organizations that require extensive legal reviews and audits to use components will likely also have robust vendor management processes in place. The burden would shift from reviewing and auditing the component to reviewing and auditing Malus. These organizations also tend to be rather risk-averse. Claims about "full legal indemnification" in an "offshore subsidiary in a jurisdiction that doesn't recognize software copyright" would raise legal flags.
I'm also interested in what models tools like Malus use, especially since many are trained on open-source projects to begin with. So far, the training piece has been considered fair use in a handful of US cases. The output, though, is still to be determined. This is why GitHub has integrated a public code search for the output of its AI agents and organizations are investing in static code analysis that searches public code repositories for similar code. AI tools that reproduce open-source code may trigger those licenses, and I don't see any mention of Malus assessing the output against open-source projects.
There are still plenty of open questions, but we're far from the death of open-source software.
> You absolutely can; the model is quite capable of reproducing works it was trained on, if not perfectly then at least close enough to infringe copyright. The only thing stopping it from doing so is filters put in place by services to attempt to dodge the question.
The model doesn't reproduce anything. It's a mathematical representation of the training data. Software that uses the model generates the output. The same model can be used across multiple software applications for different purposes. If I were to go to https://huggingface.co/deepseek-ai/DeepSeek-V3.2/tree/main (for example) and download those files, I wouldn't be able to reverse-engineer the training data without building more software.
Compare that to a search database, which needs the full text in an indexable format, directly associated with the document it came from. Although you can encrypt the database, at some point, it needs to have the text mapped to documents, which would make it much easier to reconstruct the complete original documents.
> That's not something you can handle via guardrails. If you read a piece of code, and then produce something substantially similar in expression (not just in algorithm and comparable functional details), you've still created a derivative work. There is no well-defined threshold for "how similar", the fundamental question is whether you derived from the other code or not.
The threshold of originality defines whether something can be protected by copyright. There are plenty of small snippets of code that can't be protected. But there are still questions about these small snippets that were consumed in the context of a larger, protected work, especially when there are only so many ways to express the same concept in a given language. It's definitely easier in written text than code to reason about.
Point 2 misses the distinction between AI models and their outputs.
Let's assume for a moment that training AI (or, in other words, creating an AI model) is not fair use. That means that all of the license restrictions must be adhered to.
For the MIT license, the requirement is to include the copyright notice and permission notice "in all copies or substantial portions of the Software". If we're going to argue that the model is a substantial portion of the software, then only the model would need to carry the notices. And we've already settled on accessing over a server doesn't trigger these clauses.
Something like the AGPL is more interesting. Again, if we accept that the model is a derivative work of the content it was trained on, then the AGPL's viral nature would require that the model be released under an appropriate license. However, it still says nothing about the output. In fact, the GPL family licenses don't require the output of software under one of those licenses to be open, so I suspect that would also be true for content.
So far, though, in the US, it seems courts are beginning to recognize AI model training as fair use. Honestly, I'm not surprised, given that it was seen as fair use to build a searchable database of copyright-protected text. The AI model is an even more transformative use, since (from my understanding) you can't reverse engineer the training data out of a model.
But there is still the ethical question of disclosing the training material. Plagiarism still exists, even for content in the public domain. So attributing the complete set of training material would probably fall into this form of ethical question, rather than the legal questions around intellectual property and licensing agreements. How you go about obtaining the training material is also a relevant discussion, since even fair use doesn't allow you to pirate material, and you must still legally obtain it - fair use only allows you to use it once you've obtained it.
There are still questions for output, but those are, in my opinion, less interesting. If you have a searchable copy of your training material, you can do a fuzzy search of that material to return potential cases where the model returned something close to the original content. GitHub already does something similar with GitHub Copilot and finding public code that matches AI responses, but there are still questions there, too. It's more around matches that may not be in the training data or how much duplicated code needs to be attributed. But once you find the original content, working with licensing becomes easier. There are also questions about guardrails and how much is necessary to prevent exact reproduction of copyright protected material that, even if licensed for training, isn't licensed for redistribution.
The premise of this whole post is incorrect. If an organization is building an AI product or offering an AI service, then a SOC 2 report, or at least a SOC 2 Type 2 report, should answer these questions.
"What happens if someone tries to extract training data?" CC6.7 covers data loss and data transfer restrictions. I've typically included controls related to monitoring data transfer, including flagging and highlighting potential breaches. Documented procedures on what happens if data loss or unauthorized data transfer occurs. These can be reviewed, but may be hard for the auditor to test unless they were executed and there's evidence that they were executed as written.
"Can this agent be manipulated into accessing data it shouldn't? How do you test for adversarial attacks?" I'm struggling to understand the difference between these questions. It seems like part of the answer likely overlaps with controls to address CC6.7 and data loss or data transfer restrictions. CC8.1 discusses testing the product or service.
"How do you prevent prompt injection?" This may be a bit specific for a SOC 2 Type 2 report, since it really gets into requirements, architecture, and design decisions rather than controls over the requirements, architecture, and design. That is, you can essentially not require preventing prompt injection and follow all of your controls related to, for example, CC8.1. CC8.1 talks about managing, authorizing, executing, and documenting changes. You can do all of these things well without that requirement in place.
"What guardrails are in place, and have they been validated?" This is the entire SOC 2 Type 2 report. It lists all evaluated criteria, describes the organization's controls, and provides an audit of those controls. It's up to the organization being audited, however, to think about what controls are necessary for their context. The controls that should be in scope of the audit will differ for an AI product or service than for something else. The recipient of the SOC 2 report can review the controls and ask questions.
Part of the burden is on the organization getting the SOC 2 audit report to think about what controls they need. But there's also a burden on the organization reviewing the audit report not just to see that there are no exceptions, but to review the controls described to make sure the controls are in place for the given product or service. And this detailed information about the controls is what makes something like the SOC 2 audit report a whole lot more useful than something like an ISO 27001 certificate, which says that whatever policies and procedures are in place meet the requirements of the standard and doesn't offer details on how those requirements are met.
However, I do have an issue with his focus on applying science in the definition, even though it doesn't come across in his final conclusions. In the history of engineering, that's a relatively new development. It started, to some extent, in Europe in the 1700s, but it really took off in the 1920s and exploded in the United States after World War II (1940s-1950s). It culminated with the Report of the Committee on Evaluation of Engineering Education (the Grinter Report) in the mid-1950s.
This isn't to say that science isn't important to engineering, since it absolutely is. Science provides knowledge used to better understand the world being changed by engineering. But there are also plenty of examples of engineering going ahead of science - the steam engine, the airplane, generative AI. We didn't fully understand the governing rules before the technology existed.
Any definition of engineering needs to be broader. Ferguson (Engineering and the Mind's Eye), Florman (several works, but primarily The Existential Pleasures of Engineering), and Vincenti (What Engineers Know and How They Know It) all explore ways in which engineering can't rely on science alone. I think Koen (Discussion of the Method) puts it best, where the application of science is one heuristic that engineers may choose to draw upon.